Streaming

Set "stream": true and the reply arrives as server-sent events: one data: {...} line per chunk, then data: [DONE]. Every model with text output streams.

curl

curl -N https://api.thetoken.dev/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/glm-5.3-flash",
    "messages": [{"role": "user", "content": "Count from 1 to 5."}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

Python

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.thetoken.dev/v1", api_key=os.environ["TOKEN_API_KEY"])

stream = client.chat.completions.create(
    model="zai-org/glm-5.3-flash",
    messages=[{"role": "user", "content": "Count from 1 to 5."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

TypeScript

import OpenAI from 'openai'

const client = new OpenAI({ baseURL: 'https://api.thetoken.dev/v1', apiKey: process.env.TOKEN_API_KEY })

const stream = await client.chat.completions.create({
  model: 'zai-org/glm-5.3-flash',
  messages: [{ role: 'user', content: 'Count from 1 to 5.' }],
  stream: true,
  stream_options: { include_usage: true },
})
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '')
  if (chunk.usage) console.log('\n', chunk.usage)
}

The stream looks like this (shortened):

data: {"id":"chatcmpl-3b9e","object":"chat.completion.chunk","created":1761379200,"model":"zai-org/glm-5.3-flash","choices":[{"index":0,"delta":{"role":"assistant","content":"1"},"finish_reason":null}]}

data: {"id":"chatcmpl-3b9e","object":"chat.completion.chunk","created":1761379200,"model":"zai-org/glm-5.3-flash","choices":[{"index":0,"delta":{"content":", 2, 3, 4, 5"},"finish_reason":null}]}

data: {"id":"chatcmpl-3b9e","object":"chat.completion.chunk","created":1761379200,"model":"zai-org/glm-5.3-flash","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-3b9e","object":"chat.completion.chunk","created":1761379200,"model":"zai-org/glm-5.3-flash","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":187,"total_tokens":201}}

data: [DONE]

Usage in streams

• The last chunk before [DONE] carries usage, with an empty choices list. TOKEN sends it unless you set stream_options.include_usage to false; setting it to true, as above, also makes OpenAI’s SDKs expect it.
• Reasoning models may stream their reasoning first, in delta.reasoning_content, before the answer in delta.content.
• You are billed for the tokens in that final usage. If you close the connection early, you are billed for the text generated until then (TOKEN counts it when the provider’s own count never arrives).

Errors during a stream

Errors found before the reply starts (an invalid key, the balance, a rate limit) come as a normal JSON error with an HTTP status; see Errors and rate limits. If a provider fails before sending anything, TOKEN retries on the next provider, so the stream simply starts later. Once text has been sent, a failure cannot be retried without repeating it, so the stream ends early; send the request again.

Other formats

• Responses API (/v1/responses with "stream": true): events such as response.output_text.delta and response.completed, as in OpenAI’s API.
• Anthropic Messages (/v1/messages with "stream": true): Anthropic’s events (message_start, content_block_delta, …, message_stop).