← Back to the library
BackendApplied · 8 min

Every deploy causes a burst of 502s

“Each rollout produces a short spike of 502s, 503s, or connection resets at the load balancer, even though the new version is healthy and the old one had no bugs.”

ERROR CODES YOU MAY SEE

A code is a clue. Use its meaning and surrounding evidence to narrow the cause.

502HTTP status codes · RFC 9110 · standard

A gateway or proxy received an invalid response from the upstream server it contacted, including a connection that was reset or closed mid-request.

If 502s cluster around deploys, check whether instances exit before the load balancer stops routing to them, and whether the app's keep-alive timeout is shorter than the proxy's idle timeout.

Official reference for 502 (opens in new tab)Code definition reviewed
503HTTP status codes · RFC 9110 · standard

The server is temporarily unable to handle the request due to overload or maintenance. It may send Retry-After.

Treat it as transient: honor Retry-After when present, otherwise back off with jitter and cap total attempts.

Official reference for 503 (opens in new tab)Code definition reviewed
ECONNRESETNode.js system errors · platform

The peer forcibly closed the connection.

Correlate client, proxy, and server logs. A reset alone does not reveal whether an operation completed before the connection ended.

Official reference for ECONNRESET (opens in new tab)Code definition reviewed

THE PRINCIPLE

Shutdown is a protocol: stop advertising, wait for routers to notice, finish in-flight work, close connections, exit before the hard kill.

FIRST MOVES

  1. Handle SIGTERM in the app process itself (PID 1 in the container, or behind an init that forwards signals); otherwise it is killed after the grace period with requests in flight.
  2. On SIGTERM, fail readiness and keep serving for a few seconds: in Kubernetes, endpoint removal happens in parallel with SIGTERM, not before it.
  3. Then call server.close() to stop accepting connections and finish in-flight requests, with a hard deadline below terminationGracePeriodSeconds (default 30s).
  4. Set the app's keep-alive timeout longer than the load balancer's idle timeout (ALB defaults to 60s; Node's keepAliveTimeout defaults to 5s).
  5. Gate new instances on a readiness probe that checks they can actually serve, and align the load balancer's deregistration delay with your drain time.

TOOLS: THEN → NOW

Kill and replace instancesReadiness probes, preStop delays, and connection draining or deregistration delay at the load balancer
Default server keep-alive settingsApp keep-alive tuned above the proxy idle timeout

PATTERN SNAPSHOT

const server = app.listen(8080);
server.keepAliveTimeout = 65_000; // must exceed the LB idle timeout (ALB default 60s)

let draining = false;
app.get("/readyz", (_req, res) => res.sendStatus(draining ? 503 : 200));

process.once("SIGTERM", () => {
  draining = true; // readiness fails; keep serving while endpoints update
  setTimeout(() => {
    server.close(() => process.exit(0)); // stop accepting, finish in-flight requests
    setTimeout(() => process.exit(1), 15_000).unref(); // stay under the grace period
  }, 10_000);
});

CLOSE THE AI. EXPLAIN THIS.

Kubernetes sends SIGTERM and removes the pod from endpoints at the same moment. Why does exiting immediately on SIGTERM still drop requests?

HOW IT WORKS UNDERNEATH

SOURCES

Guide reviewed