← Back to the library
System designApplied · 8 min

Search shows deleted or outdated items

“A product deleted an hour ago still appears in search, clicking it 404s, and some results show last week's price even though the database is correct.”

ERROR CODES YOU MAY SEE

A code is a clue. Use its meaning and surrounding evidence to narrow the cause.

409Elasticsearch document APIs · platform

version_conflict_engine_exception: with version_type=external, the stored document version is greater than or equal to the version in the request, so the write was rejected.

In an index-sync consumer this is usually the expected outcome of a late or replayed event; drop it. If it fires for every write, the version you send is not increasing (e.g. a timestamp from a skewed clock).

Official reference for 409 (opens in new tab)Code definition reviewed

THE PRINCIPLE

The database is the source of truth; the index is a cache with a version. Never let an older change overwrite a newer one, and always be able to rebuild.

FIRST MOVES

  1. Compare one stale document's indexed version or updated_at with the database row; tells you whether the event was lost, applied late, or overwritten by an older one.
  2. Feed the index from the database's change stream (CDC or a transactional outbox), not from best-effort writes in request handlers.
  3. Upsert with external versioning (Elasticsearch/OpenSearch version_type=external, using a row version or commit LSN), so late and replayed events are rejected with 409.
  4. Send deletes through the same versioned path; a deleted document's version is kept only briefly (index.gc_deletes), so long-delayed events can resurrect it unless you use a filtered tombstone or a rebuild.
  5. Rebuild into a new index, replay changes since the snapshot, then swap an alias atomically; in the UI, filter results against the source on click and treat "just changed" as eventually visible.

TOOLS: THEN → NOW

Dual write to the database and the index in the requestCDC or outbox (e.g. Debezium) feeding an idempotent indexer
Nightly full reindex to hide driftVersioned upserts plus periodic reconciliation, with alias-swapped rebuilds
refresh=true on every writeDefault refresh interval; refresh=wait_for only where a user must see their own write

PATTERN SNAPSHOT

// CDC consumer: the row version decides, not arrival order.
async function apply(change: ProductChange) {
  const common = { index: "products", id: change.id, version: change.version, version_type: "external" as const };
  try {
    if (change.op === "delete") await es.delete(common);
    else await es.index({ ...common, document: toSearchDoc(change.after) });
  } catch (err) {
    if (isVersionConflict(err)) return; // equal or newer version already indexed
    throw err; // transient: retry; repeated failures go to a DLQ
  }
}

CLOSE THE AI. EXPLAIN THIS.

An update with version 7 and a delete with version 8 arrive in the order delete, update. What does each indexing strategy you know leave in search?

SOURCES

Guide reviewed