← Back to the library
BackendApplied · 8 min

Large uploads crash the server or time out

“Small files upload fine; a 2 GB video makes the API's memory spike until the process is killed, or the request dies with 413 or 504 somewhere between the browser and the app.”

ERROR CODES YOU MAY SEE

A code is a clue. Use its meaning and surrounding evidence to narrow the cause.

413HTTP status codes · RFC 9110 · standard

The server refuses to process the request because its content is larger than it is willing or able to process (Content Too Large).

Find which hop rejected it: a proxy body limit (nginx client_max_body_size defaults to 1m), the load balancer, a serverless payload cap, or the app's body parser. Raising every limit is rarely the fix for large files; upload directly to object storage instead.

Official reference for 413 (opens in new tab)Code definition reviewed
504HTTP status codes · RFC 9110 · standard

A gateway or proxy did not receive a timely response from the upstream server it needed to complete the request.

Compare the proxy's timeout with the upstream's real latency. The upstream may still finish the work after the client saw 504, so only retry idempotent requests.

Official reference for 504 (opens in new tab)Code definition reviewed

THE PRINCIPLE

Your API should authorize and record uploads, not carry the bytes, unless it has to transform them.

FIRST MOVES

  1. Find the hop that fails: proxy body limits (nginx client_max_body_size defaults to 1m), load balancer or serverless payload caps, body-parser limits, or memory in the app.
  2. If bytes pass through the app, stream them with backpressure (stream.pipeline in Node) instead of reading the body into a buffer or a base64 JSON field.
  3. Prefer direct-to-storage uploads: the API issues a presigned request with a short expiry and a server-chosen key; use a presigned POST when you need a size limit, since its policy can enforce content-length-range.
  4. For files over about 100 MB, use multipart upload so parts upload in parallel and retry individually; add a lifecycle rule that aborts incomplete uploads.
  5. Treat the upload as untrusted until processed: confirm completion server-side, check size and type, scan if needed, then mark it usable.

TOOLS: THEN → NOW

multipart/form-data parsed fully into memory by the appStreaming parsers with limits, or presigned direct uploads to S3-compatible storage
One giant PUT that restarts from zero on failureS3 multipart upload with resumable parts, or resumable protocols such as tus

PATTERN SNAPSHOT

import { S3Client } from "@aws-sdk/client-s3";
import { createPresignedPost } from "@aws-sdk/s3-presigned-post";

const s3 = new S3Client({});

// The API authorizes the upload; the file goes straight to S3.
export async function createUpload(userId: string) {
  return createPresignedPost(s3, {
    Bucket: process.env.UPLOAD_BUCKET!,
    Key: `incoming/${userId}/${crypto.randomUUID()}`, // server-chosen key
    Conditions: [["content-length-range", 1, 500 * 1024 * 1024]], // 1 B to 500 MiB
    Expires: 300, // seconds
  }); // => { url, fields }: the client POSTs fields + file as multipart/form-data
}

CLOSE THE AI. EXPLAIN THIS.

Why does raising client_max_body_size and the app's body limit to 5 GB make the system less reliable rather than more?

HOW IT WORKS UNDERNEATH

SOURCES

Guide reviewed