# Streaming

https://modelportal.ai/docs/streaming

Streaming returns output as the model generates it, which suits long answers and interfaces that want the first token fast.

| Protocol | How to enable | End marker |
| --- | --- | --- |
| OpenAI-compatible | Add `"stream": true` to the body | `data: [DONE]` |
| Anthropic-compatible | Add `"stream": true` to the body | The `message_stop` event |
| Gemini-compatible | Call `:streamGenerateContent?alt=sse` | End of stream; without `alt=sse` you get a JSON array |

```bash
curl -N https://modelportal.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "MODEL_ID", "stream": true, "stream_options": {"include_usage": true}, "messages": [{"role": "user", "content": "Write a short poem"}]}'
```

```python
from openai import OpenAI

client = OpenAI(base_url="https://modelportal.ai/v1", api_key="YOUR_API_KEY")
stream = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Write a short poem"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

## Usage

- In the OpenAI format, add `"stream_options": {"include_usage": true}` to get `usage` in the last chunk; the platform does not add it to requests that did not ask.
- Usage in the stream is the billed usage and matches your bill. The last event with usage also carries `provider`, `attempts` and `cost` in the usage object (the charge, in USD); every OpenAI Chat chunk carries `provider`. See the [API overview](/docs/api-overview).

## Timeouts and interruptions

- The first content event may take up to 600 seconds, shared by all attempts. Once output starts, a gap of more than 120 seconds between events closes the stream, and one output can run for at most 60 minutes.
- While waiting for the first content event, the platform sends an SSE comment `: keepalive` every 15 seconds so that clients and proxies do not drop the idle connection. SSE clients ignore comments; in the Gemini JSON array (without `alt=sse`) the keepalive is a newline.
- Headers go out when the first content event arrives or keepalives start, and always include `X-ModelPortal-Request-Id`.
- A failure before the headers go out returns an ordinary HTTP error. A failure after that, including when every attempt fails after only keepalives were sent, ends the stream with an SSE error event. An interruption after output starts is billed for the usage produced so far; when exact usage is unavailable, an estimate is billed and marked Estimated on your bill.