← Back to the library
OperationsApplied · 8 min

Rolling back a deploy fails because the schema already changed

“A release misbehaves, you roll back the app, and the previous version crashes with "column does not exist" or writes bad data, because the migration that shipped with the release already ran.”

ERROR CODES YOU MAY SEE

A code is a clue. Use its meaning and surrounding evidence to narrow the cause.

42703PostgreSQL SQLSTATE · standard

A query referenced a column that does not exist (undefined_column), for example: column "x" does not exist.

If the query worked yesterday, someone renamed or dropped the column. Find every reader of that table, including other services, reports, and ETL jobs, before changing it back.

Official reference for 42703 (opens in new tab)Code definition reviewed

THE PRINCIPLE

Every schema change must work with both the current and the previous app version. Ship schema changes separately from the code that depends on them, and make destructive steps a later, separate release.

FIRST MOVES

  1. Stop the bleeding forward: if the old code can't read the new schema, deploy a fix or re-add the column instead of running a down migration on live data.
  2. Make migrations additive by default: new nullable or defaulted columns, new tables, and new indexes built concurrently. Renames and drops become expand, migrate, contract over several releases.
  3. During the transition, write to both the old and new structures so that reverting the app loses nothing. Switch reads only once the backfill is verified.
  4. Run migrations as their own pipeline step before the code deploy. Contract (drop) only after the previous version can no longer be redeployed.
  5. Add a CI check that runs the previous release's tests against the new schema.

TOOLS: THEN → NOW

Paired up/down migrations shipped with each deployForward-only, backward-compatible migrations with expand/contract steps
Migrations run on app boot by every replicaA single migration job (a CI step or Kubernetes Job) that runs ahead of the rollout

PATTERN SNAPSHOT

-- Rename users.fullname -> display_name without breaking rollback.
-- Release 1 (expand): additive only; old code ignores the column.
ALTER TABLE users ADD COLUMN display_name text;
-- App v2 writes BOTH columns and still reads fullname.
-- Backfill in batches; repeat until it updates 0 rows:
UPDATE users SET display_name = fullname
WHERE id IN (SELECT id FROM users
             WHERE display_name IS NULL AND fullname IS NOT NULL LIMIT 5000);
-- App v3 reads display_name and keeps dual writes (rollback to v2 is safe).
-- App v4 stops writing fullname. Contract once v3 can never return:
ALTER TABLE users DROP COLUMN fullname;

CLOSE THE AI. EXPLAIN THIS.

Which app versions must be able to run against a schema at the moment its migration finishes, and what does that imply for a column rename?

HOW IT WORKS UNDERNEATH

SOURCES

Guide reviewed