Server-Sent Events in production

HOW IT FITS TOGETHER

KEY TERMS

Wire format

UTF-8 text, one field per line as name: value, events separated by a blank line. data lines accumulate (joined with newlines), event sets the type (default message), id sets the last event id, retry sets the reconnect delay in milliseconds and is ignored unless it is all digits. Lines starting with a colon are comments. An id containing NULL is ignored; an empty id resets the cursor.

Reconnect and resume

After a network error EventSource waits the reconnection time and retries, sending Last-Event-ID if it has one. A non-200 status or a Content-Type other than text/event-stream fails the connection permanently (readyState CLOSED, no retry); servers use 204 to tell a client to stop. Resume semantics are yours: ids must map to positions in a log with known retention, plus a reset event for cursors that fell off the end.

Connection limits and HTTP versions

Over HTTP/1.1 browsers allow about six connections per origin, shared by every tab; Chrome and Firefox marked raising it as won't fix. Five open tabs with an EventSource each leave one connection for the whole site. HTTP/2 and HTTP/3 multiplex streams over one connection (the server advertises the limit, commonly 100), so terminate HTTP/2 at the edge; a SharedWorker can also share one stream across tabs.

Buffering, compression and timeouts

Anything that waits for more bytes before forwarding breaks streaming: nginx proxy buffering, gzip middleware, some PaaS edges. Turn buffering off per response with X-Accel-Buffering: no, exclude text/event-stream from compression, and send a comment heartbeat well inside the shortest idle timeout in the path (nginx 60 s read timeout, Cloudflare 125 s, ALB 60 s idle by default).

Authentication

EventSource cannot send Authorization headers, so same-origin cookies (HttpOnly, SameSite) are the clean option; cross-origin needs withCredentials plus CORS with an exact Access-Control-Allow-Origin and Allow-Credentials. Tokens in the query string end up in access logs, browser history and proxies: if you must, use a short-lived single-use ticket minted by an authenticated call. Recheck authorization on reconnect, and end the stream when the session is revoked.

Scaling and fan-out

Each client holds one response open on one instance, so state is per connection: memory, a file descriptor, a heartbeat timer. Instances subscribe to a shared pub/sub or log (Redis, NATS, Kafka, Postgres LISTEN/NOTIFY) and write matching events to their local connections. Because reconnects can land anywhere, replay must read from the shared log, never from instance memory.

LLM token streaming

The dominant modern use. The Claude Messages API with stream: true sends named events (message_start, content_block_delta, message_delta, message_stop, plus ping and error); OpenAI's Responses API sends typed events such as response.output_text.delta and response.completed. Both are POST requests, so browsers consume them with fetch, usually through your own backend so API keys stay server-side. Errors can arrive mid-stream as an event after a 200.

IN YOUR STACK

app.get("/events", async (req, res) => {
  res.set({ "Content-Type": "text/event-stream", "Cache-Control": "no-cache, no-transform", "X-Accel-Buffering": "no" });
  res.flushHeaders(); // EventSource fires "open" now, not at the first event
  res.write("retry: 3000\n\n");

  let last = Number(req.get("Last-Event-ID") ?? req.query.after ?? 0); // header on auto-reconnect, query on manual
  const send = (e) => {
    if (e.id <= last) return; // seen in replay already
    last = e.id;
    res.write(`id: ${e.id}\nevent: ${e.type}\ndata: ${JSON.stringify(e.data)}\n\n`);
  };
  // Subscribe before reading history, and buffer, so the gap between them loses nothing.
  let replaying = true;
  const pending = [];
  const unsubscribe = eventLog.subscribe((e) => (replaying ? pending.push(e) : send(e)));
  for (const e of await eventLog.readAfter(last)) send(e);
  replaying = false;
  pending.forEach(send);

  const ping = setInterval(() => res.write(": ping\n\n"), 15_000);
  req.on("close", () => { clearInterval(ping); unsubscribe(); });
});

import { Hono } from "hono";
import { streamSSE } from "hono/streaming";

const app = new Hono();

// POST endpoint that relays model tokens; the browser reads it with fetch.
app.post("/chat", async (c) => {
  const { messages } = await c.req.json();
  return streamSSE(c, async (stream) => {
    const upstream = new AbortController();
    stream.onAbort(() => upstream.abort()); // user closed the tab: stop paying for tokens
    let seq = 0;
    try {
      for await (const token of generateTokens(messages, upstream.signal)) {
        await stream.writeSSE({ event: "token", id: String(++seq), data: JSON.stringify({ token }) });
      }
      await stream.writeSSE({ event: "done", data: "[DONE]" });
    } catch (err) {
      console.error(err);
      // Headers are already sent, so the failure must travel as an event.
      if (!upstream.signal.aborted) await stream.writeSSE({ event: "error", data: JSON.stringify({ message: "generation failed" }) });
    }
  });
});

export default app;

func events(w http.ResponseWriter, r *http.Request) {
	rc := http.NewResponseController(w)
	_ = rc.SetWriteDeadline(time.Time{}) // lift Server.WriteTimeout for this stream only
	w.Header().Set("Content-Type", "text/event-stream")
	w.Header().Set("Cache-Control", "no-cache")
	w.Header().Set("X-Accel-Buffering", "no")
	sub := broker.SubscribeAfter(r.Header.Get("Last-Event-ID")) // replay, then live
	defer sub.Close()
	ping := time.NewTicker(15 * time.Second)
	defer ping.Stop()
	fmt.Fprint(w, "retry: 3000\n\n")
	for {
		if err := rc.Flush(); err != nil {
			return // write failed or this ResponseWriter cannot stream
		}
		select {
		case <-r.Context().Done(): // client went away
			return
		case e := <-sub.C:
			fmt.Fprintf(w, "id: %d\nevent: %s\ndata: %s\n\n", e.ID, e.Type, e.JSON)
		case <-ping.C:
			fmt.Fprint(w, ": ping\n\n")
		}
	}
}

from collections.abc import AsyncIterable
from typing import Annotated

from fastapi import FastAPI, Header
from fastapi.sse import EventSourceResponse, ServerSentEvent

app = FastAPI()

@app.get("/events", response_class=EventSourceResponse)
async def events(
    last_event_id: Annotated[int | None, Header()] = None,
) -> AsyncIterable[ServerSentEvent]:
    yield ServerSentEvent(comment="connected", retry=3000)
    # event_log.tail: replay entries after the cursor, then follow live ones.
    async for entry in event_log.tail(after=last_event_id or 0):
        yield ServerSentEvent(data=entry.payload, event=entry.type, id=str(entry.id))

@app.post("/chat/stream", response_class=EventSourceResponse)
async def chat(prompt: Prompt) -> AsyncIterable[ServerSentEvent]:
    async for token in llm.stream(prompt.text):
        yield ServerSentEvent(data={"token": token}, event="token")
    yield ServerSentEvent(raw_data="[DONE]", event="done")

let lastApplied = 0;
let source;

function connect() {
  source?.close(); // never run two streams (visibility and error handlers can both call this)
  // A new EventSource starts with an empty cursor, so pass ours explicitly.
  source = new EventSource(`/events?after=${lastApplied}`, { withCredentials: true });

  source.addEventListener("order", (e) => {
    const id = Number(e.lastEventId);
    if (id <= lastApplied) return; // duplicate after a replay
    lastApplied = id;
    render(JSON.parse(e.data));
  });
  source.addEventListener("reset", () => reloadSnapshot()); // cursor older than retention

  source.addEventListener("error", () => {
    if (source.readyState === EventSource.CONNECTING) return; // browser is retrying itself
    // CLOSED: non-200, wrong Content-Type or 204. The browser will not retry.
    source.close();
    setTimeout(connect, 2_000 + Math.random() * 8_000);
  });
}

connect();
document.addEventListener("visibilitychange", () => { if (document.hidden) source.close(); else connect(); });

async function* postSSE(url, body, { headers = {}, signal } = {}) {
  const res = await fetch(url, { method: "POST", signal, body: JSON.stringify(body),
    headers: { "Content-Type": "application/json", Accept: "text/event-stream", ...headers } });
  if (!res.ok) throw new Error(`HTTP ${res.status}: ${await res.text()}`);
  const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
  let buf = "", end;
  for (;;) {
    const { value, done } = await reader.read();
    if (done) return;
    buf = (buf + value).replace(/\r\n|\r(?!$)/g, "\n"); // hold a trailing CR until its LF arrives
    while ((end = buf.indexOf("\n\n")) !== -1) {
      const block = buf.slice(0, end);
      buf = buf.slice(end + 2);
      const ev = { event: "message", id: undefined, data: [] };
      for (const line of block.split("\n")) {
        if (line.startsWith(":")) continue; // comment / heartbeat
        const i = line.indexOf(":"), field = i < 0 ? line : line.slice(0, i);
        const val = i < 0 ? "" : line.slice(i + 1).replace(/^ /, "");
        if (field === "data") ev.data.push(val);
        else if (field === "event" || field === "id") ev[field] = val;
      }
      if (ev.data.length) yield { ...ev, data: ev.data.join("\n") };
    }
  }
}

server {
    listen 443 ssl;
    http2 on;                         # browsers multiplex SSE streams over one connection

    location /events {
        proxy_pass http://app_upstream;
        proxy_http_version 1.1;       # default only since nginx 1.29.7
        proxy_set_header Connection "";
        proxy_buffering off;          # or let the app send X-Accel-Buffering: no
        proxy_cache off;
        gzip off;                     # never compress-and-buffer the stream
        proxy_read_timeout 1h;        # default 60s between two reads from upstream
        proxy_send_timeout 1h;
    }
}

WHEN TO REACH FOR WHAT

WHERE IT BITES

• Testing locally, then deploying behind nginx or a PaaS edge that buffers: events arrive all at once when the stream ends. Check with curl -N through the real edge, and set X-Accel-Buffering: no and exclude text/event-stream from compression.
• Serving SSE over HTTP/1.1: a user with a handful of tabs open exhausts the per-origin connection pool and the whole site stops loading. Terminate HTTP/2 or HTTP/3 at the edge, or share one stream across tabs with a SharedWorker or BroadcastChannel leader.
• Using in-process counters as event ids. After a deploy or a reconnect to another instance, Last-Event-ID points at nothing and events are silently skipped. Ids must be positions in a shared log, with a reset event for cursors past retention.
• Returning 401 or 500 to a stream and expecting a retry: any non-200 closes EventSource for good. Use that deliberately (204 to stop, 401 to force re-login) and handle readyState CLOSED in the client.
• Leaving the upstream work running after the client disconnects. Listen for close/abort and cancel subscriptions, database cursors and especially LLM calls, which keep consuming tokens.
• Putting bearer tokens in the EventSource URL. They land in access logs, CDN logs and browser history. Prefer HttpOnly cookies, or a single-use ticket that expires in seconds.

WHEN IT BREAKS IN PRODUCTION

• Streamed events arrive in bursts or only at the end
• The app hangs once users open a few tabs
• A logged-out user keeps receiving live updates
• A request that does minutes of work times out, then runs twice
• Requests fail with 504 while the server keeps working after the client gave up
• Code works locally but fails in production
• Memory balloons when producers outpace consumers

SOURCES

• HTML Standard: Server-sent events (opens in new tab)Checked 2026-11-06
• MDN: EventSource (opens in new tab)Checked 2026-11-06
• NGINX: ngx_http_proxy_module (opens in new tab)Checked 2026-11-06
• Cloudflare: Connection limits (opens in new tab)Checked 2026-11-06
• Claude API: Streaming messages (opens in new tab)Checked 2026-11-06
• OpenAI: Streaming API responses (opens in new tab)Checked 2026-11-06