Streaming

Sätt stream: true så levereras svaret token-för-token via server-sent events (SSE) — standard OpenAI-format, så SDK:ernas inbyggda streaming fungerar direkt.

from openai import OpenAI

client = OpenAI(api_key="sk-st-your-key", base_url="https://api.staik.se/v1")

stream = client.chat.completions.create(
    model="qwen3.6:35b-a3b",
    messages=[{"role": "user", "content": "Skriv en haiku om Stockholm."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Chunk-format

Varje SSE-event är en data:-rad med en JSON-chunk; strömmen avslutas med data: [DONE]:

data: {"choices":[{"delta":{"content":"Snö"},"index":0}], ...}
data: {"choices":[{"delta":{"content":" faller"},"index":0}], ...}
data: [DONE]

• Inkrementell text kommer i choices[0].delta.content
• Med thinking-aliasen streamas resonemanget separat i choices[0].delta.reasoning_content
• Tool calls streamas som delta.tool_calls-fragment

Anthropic-endpointen

/v1/messages streamar i Anthropic-format i stället: message_start, content_block_delta, message_stop.