SSE streaming and WebSocket realtime connections

Beginner: run one unbuffered stream

curl -N disables client output buffering. This Chat Completions example requires a model that the marketplace shows as compatible with the endpoint.

Copycurl -N "$BASE_URL/v1/chat/completions"   -H "Authorization: Bearer $API_KEY"   -H "Content-Type: application/json"   -d '{"model":"<MODEL_ID>","stream":true,"messages":[{"role":"user","content":"Explain SSE in three sentences."}]}'

Receiving text is not completion. Chat terminal markers, typed Responses events, Claude content-block events, and Gemini SSE have different shapes. Use a parser for the selected protocol.

Parse SSE correctly

Split events on blank lines, combine consecutive data: fields, ignore comment heartbeats, and dispatch using the event name or payload type. Never parse by TCP packet, one read(), or one newline. A JSON value may span reads and one read may contain several events. Bound event size and keep consuming to prevent a slow client from holding connections and memory.

Completion, errors, and cancellation

Declare success only after the defined terminal event and after all tool or media parts are complete. An error can arrive after HTTP 200. On cancellation, close the body and propagate Context or AbortSignal. Whether generation or billing already happened must be checked through logs and usage; disconnecting does not prove zero cost.

Proxy and deployment checks

Disable unnecessary response buffering and aggregation, configure an adequate idle timeout, and allow timely flushes. Measure DNS, connection, time to first byte, time to first token, full duration, disconnect position, and terminal event. Corporate proxies can cut long-lived connections, and browser use also requires correct CORS.

WebSocket is not SSE

/v1/realtime is a bidirectional WebSocket with session state, concurrent input, server events, audio frames, and close codes. EventSource cannot consume it. Enforce authentication, message-size limits, heartbeat, idle and total duration, and backpressure. Reauthorize high-risk tool actions on the server.

Expert recovery boundary

Assign a business idempotency key and retain Request-ID, model, event sequence, and committed tool operations. Partial text can be displayable, but database writes, payments, and outbound messages need tool-layer idempotency. When the protocol has no recovery cursor, mark a disconnected result as unknown and let the business choose replay, review, or abandonment.