← Back to the library
System designApplied · 8 min

Memory balloons when producers outpace consumers

“During a traffic spike or a slow downstream, memory climbs in a straight line, latency grows with queue length, and the process is OOM-killed; after restart the backlog is gone, and so is the work.”

ERROR CODES YOU MAY SEE

A code is a clue. Use its meaning and surrounding evidence to narrow the cause.

137Container exit codes · Kubernetes · platform

The container process was killed with SIGKILL (128 + 9). With reason OOMKilled, the kernel killed it for exceeding its cgroup memory limit.

Check lastState.terminated.reason in kubectl describe pod. OOMKilled means the memory limit; 137 without it can also be a kill after the termination grace period ran out.

Official reference for 137 (opens in new tab)Code definition reviewed
503HTTP status codes · RFC 9110 · standard

The server is temporarily unable to handle the request due to overload or maintenance. It may send Retry-After.

Treat it as transient: honor Retry-After when present, otherwise back off with jitter and cap total attempts.

Official reference for 503 (opens in new tab)Code definition reviewed

THE PRINCIPLE

A queue absorbs bursts, not sustained overload. If the producer is never told to slow down, memory becomes the queue.

FIRST MOVES

  1. Find the unbounded buffer: in-process arrays used as queues, Promise.all over an unbounded list, stream writes ignoring write()'s return, unbounded executor queues, client-side batching.
  2. Bound it, and pick a full-queue policy: block or pause the producer (backpressure), reject cheaply (429/503 or nack), or drop the oldest work that no one will use.
  3. Limit concurrency with a fixed pool or semaphore instead of starting a task per item.
  4. Use pull-based or flow-controlled plumbing: stream.pipeline, async iterators, Reactive Streams request(n), broker prefetch limits.
  5. Alert on queue age, not only depth, and bound how long a request may wait; work whose caller has given up should be dropped, not done.

TOOLS: THEN → NOW

readable.on('data', (c) => writable.write(c))stream.pipeline() / async iteration, which pause the source when the sink is full
Unbounded in-memory job arraysBounded queues or a durable broker with prefetch / max in-flight limits
Let the queue absorb overloadLoad shedding with a queue-time bound and request criticality

PATTERN SNAPSHOT

import { pipeline } from "node:stream/promises";

// Anti-pattern: ignores write() === false, so a slow sink buffers everything.
// source.on("data", (row) => sink.write(row));

// pipeline pulls the next chunk only when the sink has room.
await pipeline(
  source, // e.g. a database cursor stream
  async function* (rows) {
    for await (const row of rows) yield JSON.stringify(row) + "\n";
  },
  sink, // e.g. an upload or HTTP response stream
);

CLOSE THE AI. EXPLAIN THIS.

A consumer handles 800 msg/s and the producer sends 1,000 msg/s for an hour. What happens with an unbounded queue, and what are your three options once the queue is bounded?

HOW IT WORKS UNDERNEATH

SOURCES

Guide reviewed