Streaming Responses

For longer responses, the wait until generation completes can be noticeable. Streaming solves this: instead of waiting for the finished response, the server delivers individual tokens to the client immediately after they’re generated, via Server-Sent Events (SSE). This produces the familiar “typing in real time” effect, and users see the first words after mere fractions of a second.

Streaming is enabled by setting "stream": true in the request body. All other parameters remain unchanged compared to a normal chat completion request.

curl

curl -X POST https://ai.noris.de/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -N \
  -d '{
    "model": "vllm/release/gpt-oss-120b",
    "messages": [{"role": "user", "content": "Tell me a story"}],
    "stream": true
  }'

The -N flag disables buffering so each chunk appears in the terminal immediately.

SSE Format

The response consists of a sequence of lines, each prefixed with data: . Behind that is a JSON object with a partial fragment (delta) of the message. At the end of the transmission, the server sends a final line data: [DONE]. Typical output:

data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Once"},"finish_reason":null}]}

data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" upon"},"finish_reason":null}]}

...

data: [DONE]

Important: unlike the non-streaming variant, the content is not in message.content but distributed across multiple chunks in choices[0].delta.content. Concatenate all delta.content fragments to obtain the full text. The usage and finish_reason fields are delivered exclusively in the last data chunk before [DONE], earlier chunks contain null for these.

Python

The openai SDK fully encapsulates SSE parsing. Pass stream=True and iterate over the returned stream instance:

from openai import OpenAI

client = OpenAI(base_url="https://ai.noris.de/v1", api_key="YOUR_API_KEY")

stream = client.chat.completions.create(
    model="vllm/release/gpt-oss-120b",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Because of end="", each part is output immediately without the console waiting for a newline. If you need token metrics, access chunk.usage on the final chunk (provided the provider supports stream_options={"include_usage": True}).

JavaScript

In the browser or Node.js, the ReadableStream interface of fetch is a good fit. Parse the data: lines manually and stop at [DONE]:

const response = await fetch("https://ai.noris.de/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY"
  },
  body: JSON.stringify({
    model: "vllm/release/gpt-oss-120b",
    messages: [{role: "user", content: "Tell me a story"}],
    stream: true
  })
});

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const {done, value} = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, {stream: true});

  const lines = buffer.split("\n");
  buffer = lines.pop();

  for (const line of lines) {
    const trimmed = line.trim();
    if (!trimmed.startsWith("data:")) continue;
    const payload = trimmed.slice(5).trim();
    if (payload === "[DONE]") return;
    const json = JSON.parse(payload);
    const delta = json.choices[0]?.delta?.content;
    if (delta) process.stdout.write(delta);
  }
}

Notes

• usage and finish_reason appear exclusively in the last chunk before [DONE]. Only evaluate these fields after the stream has ended.
• On network interruptions during streaming, the client must decide whether to restart the call or continue using the partial history, an automatic resume mechanism is not part of the protocol.
• Empty delta.content fields can be ignored; they occasionally occur at the start (on the role token) or between segments.