Running WebSockets and SSE for many users

HOW IT FITS TOGETHER

KEY TERMS

Connection capacity

Each open connection costs a file descriptor, kernel socket buffers, TLS state and your per-connection objects, often tens of KB before application data; permessage-deflate can add far more. Raise the process fd limit (LimitNOFILE, ulimit -n) well above expected connections, size socket buffers and accept backlogs deliberately, and remember a proxy talking to one upstream address is limited by its ephemeral port range per upstream IP and port. Load-test with realistic idle-to-active ratios: idle sockets are cheap, broadcasts are not.

Load balancing and stickiness

Once upgraded, a WebSocket stays on one server, so stickiness only matters for multi-request transports: Socket.IO's HTTP long-polling fallback requires sticky sessions or it returns 400s. Prefer stateless connection servers: any server can accept any client because session state lives in a shared store and messages arrive through the backplane. Balance on active connections, not requests, or new servers stay empty after a deploy while old ones stay full.

Backplane

The bus between publishers and connection servers. Redis pub/sub and core NATS are at-most-once: whatever a server misses while disconnected is gone. Redis Streams, JetStream and Kafka keep a log you can read from an offset, which is what makes resume possible. Many teams run both: a fast pub/sub path for live delivery and a log for replay on reconnect.

Fan-out shape and presence

Topics usually map to rooms or channels (room:42) and per-user topics (user:7, for notifications across devices). Subscribe a server to a topic only while it has a local subscriber, or every server receives every message. A room with a million members is a celebrity fan-out problem: batch or coalesce, sample, or move it to a broadcast tier. Presence is soft state: heartbeat-refreshed entries with a TTL (Redis keys or sorted sets), published as diffs, never derived from close events alone because crashes send none.

Ordering, delivery and slow consumers

You only get ordering per topic from one publisher through one ordered path; across topics and reconnects, assign a per-topic sequence number and let clients dedupe and detect gaps. Delivery to a browser is at most once unless the client acks or resumes from a cursor. A client reading slower than you publish grows its send queue (ws bufferedAmount, NATS pending limits) until memory runs out, so pick a policy per message type: drop or coalesce for ephemeral state like cursors and prices, buffer up to a cap for important events, then disconnect so the client resyncs from a snapshot.

Auth on long-lived connections

A socket authenticated at connect time outlives the token that opened it. Store the session id and expiry on the connection, re-check on a timer or on sensitive actions, let clients send a fresh token in-band before expiry, and subscribe each server to a revocation topic so logout, password change or role removal closes affected sockets (WebSocket close 1008) within seconds. Re-check topic permissions on resubscribe after every reconnect.

Deploys, draining and observability

Every deploy disconnects every client on the replaced servers. Fail readiness first, then close connections gradually with 1012 (service restart) or 1001, and make clients reconnect with exponential backoff and full jitter so the fleet is not hit by a synchronized storm. Watch open connections per server, connects and disconnects per second by close code, backplane lag (publish to socket write), send-queue depth, dropped and slow-consumer counts, and replay versus snapshot ratios.

IN YOUR STACK

import { createServer } from "node:http";
import { Server } from "socket.io";
import { createClient } from "redis";
import { createAdapter } from "@socket.io/redis-streams-adapter";

const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();

const httpServer = createServer();
const io = new Server(httpServer, {
  adapter: createAdapter(redis, { maxLen: 100_000 }), // replay window shared by all nodes
  connectionStateRecovery: { maxDisconnectionDuration: 2 * 60_000, skipMiddlewares: false }, // re-auth on resume
});
io.use(async (socket, next) => {
  const user = await verifyToken(socket.handshake.auth.token);
  if (!user) return next(new Error("unauthorized"));
  socket.data.user = user;
  next();
});
io.on("connection", (socket) => {
  // Recovered sockets get their rooms, data and missed packets back; register handlers either way.
  if (!socket.recovered) socket.join([`user:${socket.data.user.id}`, `org:${socket.data.user.orgId}`]);
});
export const notifyOrg = (orgId: string, event: unknown) => io.to(`org:${orgId}`).emit("event", event);
httpServer.listen(3000);

import { WebSocketServer } from "ws";
import { createClient } from "redis";

const sub = await createClient({ url: process.env.REDIS_URL }).connect();
const wss = new WebSocketServer({ port: 8080, maxPayload: 64 * 1024 });
const rooms = new Map(); // room -> Set<WebSocket>, local to this node only

async function join(ws, room) {
  if (!rooms.has(room)) {
    rooms.set(room, new Set());
    await sub.subscribe(`room:${room}`, (msg) => fanOut(room, msg)); // first local member
  }
  rooms.get(room).add(ws);
  ws.once("close", () => leave(ws, room));
}
async function leave(ws, room) {
  const members = rooms.get(room);
  if (members?.delete(ws) && members.size === 0) { rooms.delete(room); await sub.unsubscribe(`room:${room}`); }
}
function fanOut(room, msg) {
  for (const ws of rooms.get(room) ?? []) // over 1 MiB queued: disconnect, client resyncs
    ws.bufferedAmount > 1 << 20 ? ws.close(1013, "slow consumer") : ws.send(msg);
}
wss.on("connection", (ws, req) => join(ws, roomFor(req))); // roomFor: authenticated room lookup

nc, err := nats.Connect(os.Getenv("NATS_URL"),
	nats.MaxReconnects(-1),
	nats.ReconnectWait(2*time.Second),
	nats.ErrorHandler(func(_ *nats.Conn, _ *nats.Subscription, err error) {
		if errors.Is(err, nats.ErrSlowConsumer) {
			slowConsumer.Inc() // this node dropped backplane messages
		}
	}),
)
if err != nil {
	log.Fatal(err)
}
defer nc.Drain()

sub, err := nc.Subscribe("user.*.events", func(m *nats.Msg) {
	userID := strings.Split(m.Subject, ".")[1]
	for _, c := range hub.Conns(userID) { // this node's sockets for that user
		select {
		case c.Out <- m.Data: // bounded per-connection channel drained by a writer goroutine
		default:
			c.Close("slow consumer") // never block the NATS callback
		}
	}
})
sub.SetPendingLimits(100_000, 64<<20)

import { DurableObject } from "cloudflare:workers";

export class Room extends DurableObject {
  async fetch(req: Request) {
    if (req.headers.get("Upgrade") !== "websocket") return new Response("Expected WebSocket", { status: 426 });
    const user = req.headers.get("X-User-Id")!; // set by the Worker after it verified the session
    const { 0: client, 1: server } = new WebSocketPair();
    this.ctx.acceptWebSocket(server, [`user:${user}`]); // hibernatable; tags find sockets later
    server.serializeAttachment({ user }); // survives hibernation, max 16 KiB
    return new Response(null, { status: 101, webSocket: client });
  }

  async webSocketMessage(ws: WebSocket, message: string | ArrayBuffer) {
    const { user } = ws.deserializeAttachment();
    const seq = ((await this.ctx.storage.get<number>("seq")) ?? 0) + 1;
    await this.ctx.storage.put("seq", seq);
    const out = JSON.stringify({ seq, user, body: String(message) });
    for (const peer of this.ctx.getWebSockets()) peer.send(out);
  }

  async kick(userId: string) { // RPC from the Worker on logout or revocation
    for (const ws of this.ctx.getWebSockets(`user:${userId}`)) ws.close(1008, "session revoked");
  }
}

import { createServer } from "node:http";
import { WebSocketServer } from "ws";

let draining = false;
const server = createServer((req, res) => {
  if (req.url === "/readyz") return res.writeHead(draining ? 503 : 200).end();
  res.writeHead(404).end();
});
const wss = new WebSocketServer({ server });
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

process.once("SIGTERM", async () => {
  draining = true;  // readiness fails: the LB stops sending new upgrades here
  await sleep(10_000); // longer than health-check interval x failure threshold
  server.close();   // stop accepting; upgraded sockets are not closed by this
  const spread = 30_000;
  for (const ws of wss.clients) {
    // Random order and time, so the rest of the fleet sees a ramp, not a spike.
    setTimeout(() => ws.close(1012, "service restart"), Math.random() * spread);
  }
  await sleep(spread + 5_000);
  for (const ws of wss.clients) ws.terminate(); // peers that never finished the close handshake
  process.exit(0);
});
server.listen(8080);

let attempt = 0, lastSeq = 0, resumeToken = "";

async function connect() {
  let ticket;
  try { ticket = await fetchTicket(); } // short-lived, single-use, from an authenticated HTTP call
  catch { return retry(); }
  const ws = new WebSocket(`wss://rt.example.com/ws?ticket=${ticket}`);
  ws.onopen = () => { attempt = 0; ws.send(JSON.stringify({ type: "resume", token: resumeToken, after: lastSeq })); };
  ws.onmessage = ({ data }) => {
    const msg = JSON.parse(data);
    if (msg.type === "session") resumeToken = msg.token;
    else if (msg.type === "reset") reloadSnapshot(); // server cannot replay from lastSeq
    else if (msg.seq > lastSeq) { lastSeq = msg.seq; apply(msg); } // drop duplicates
  };
  ws.onclose = (e) => (e.code === 1008 ? showSignedOut() : retry()); // revoked: do not hammer
}

function retry() {
  // Full jitter: uniform in [0, min(cap, base * 2^attempt)].
  const delay = Math.random() * Math.min(30_000, 500 * 2 ** attempt++);
  setTimeout(() => (navigator.onLine ? connect() : addEventListener("online", connect, { once: true })), delay);
}
connect();

WHEN TO REACH FOR WHAT

WHERE IT BITES

• Broadcasting from in-process memory: works on one node, silently misses clients on the others once you scale out. Every cross-client message must go through the backplane.
• Unbounded send queues. One slow mobile client can buffer megabytes per socket; at scale this is an outage. Cap queue depth and choose drop, coalesce or disconnect per message type.
• Deploys that close every socket at once, and clients that reconnect immediately or with fixed delays. The synchronized storm overloads auth, the backplane and replay. Drain gradually and use full-jitter backoff.
• Authenticating only at connect time. Revoked users keep receiving data for as long as the socket lives. Track expiry per connection and subscribe to a revocation topic.
• Treating an at-most-once backplane as reliable. Redis pub/sub and core NATS drop messages during reconnects or slow consumption; without sequence numbers and replay, clients drift from the truth.
• Hitting platform limits in production: process fd limits, LB idle timeouts (ALB defaults to 60 s and ignores HTTP/2 PINGs), API Gateway's 2-hour cap and 10-minute idle timeout. Heartbeat below the shortest one and expect forced reconnects.

WHEN IT BREAKS IN PRODUCTION

• Real-time messages reach only some users after scaling out
• Every deploy causes a spike as all clients reconnect at once
• A logged-out user keeps receiving live updates
• Live connections drop after a minute of quiet
• WebSockets disconnect, duplicate, or lose events
• One big account posts and everyone's feed lags
• Memory balloons when producers outpace consumers
• A brief blip turns into a long outage
• A contract upgrade corrupts state or breaks the proxy

SOURCES

• Redis: Pub/Sub (opens in new tab)Checked 2026-11-06
• Socket.IO: Connection state recovery (opens in new tab)Checked 2026-11-06
• NATS: Core NATS (opens in new tab)Checked 2026-11-06
• Cloudflare: Durable Objects state API (opens in new tab)Checked 2026-11-06
• AWS: API Gateway WebSocket quotas (opens in new tab)Checked 2026-11-06
• Ably: Connection state and recovery (opens in new tab)Checked 2026-11-06