Disaster Recovery
Plan to rebuild application memory from a source you control, prove deleted data stays deleted, and rehearse the replay before you need it.
Set objectives you can defend
Your recovery time depends on how fast you can replay from a source you control. Estimate it now, with your own numbers, rather than assuming a rebuild is instant.
Compare that number with the recovery time your product actually promises. If the gap is uncomfortable, the fix is a smaller authoritative dataset, a faster replay path, or a lower stated objective — not optimism.
Rebuild in a safe sequence
- 01Freeze
Name a decision owner, stop destructive jobs, and record the scope being rebuilt.
- 02Isolate
Create a separate recovery project so a partial rebuild is never visible to users.
- 03Exclude
Load your deletion list first, so nothing deliberately removed can be replayed.
- 04Replay
Ingest in deterministic batches with durable checkpoints and bounded concurrency.
- 05Reconcile
Compare counts, re-check known deletions, and run a fixed set of retrieval questions.
- 06Cut over
Switch application traffic only after reconciliation passes and the owner approves.
Deletion safety is non-negotiable
A rebuild that resurrects deleted data is a privacy incident, not a recovery. Keep a durable record of every deletion request your application honors, apply it before any replay, and verify a known-deleted fixture is still absent before you allow user traffic.