OneMux/Docs/Streaming Output

Streaming Output

Server-Sent Events (SSE) streaming returns AI responses one character at a time, like a typewriter, improving the user experience. This page explains how to enable streaming, parse SSE data chunks, and handle streaming connections. It's suitable for chat conversations, content generation, and other scenarios.

Why Use Streaming?

  • Large language models can take time to respond, so users need immediate feedback
  • Creates a typewriter effect similar to ChatGPT
  • Improves the user experience

Enable Streaming

Set "stream": true in your request

from openai import OpenAI

client = OpenAI(
    api_key="sk-xxx",
    base_url="https://api.onemux.net/v1"
)

response = client.chat.completions.create(
    model="gpt-6.1-sol",
    messages=[{"role": "user", "content": "Write a poem"}],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

SSE Response Format

Streaming responses use the Server-Sent Events (SSE) format. Each data chunk starts with data::

data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"You"}}]}

data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":" there"}}]}

data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"!"}}]}

data: [DONE]