Production / Client resilience
High Availability
Availability your users feel is decided in your client: deadlines, which responses you retry, and what the product does when memory is unavailable.
Client policy
Pick the response, see the required behavior
What it means
You are sending more than the limit allows.
Do this
Reduce offered load. Wait for the interval indicated by Retry-After when present, and add jitter so instances do not retry together.
Retry:Yes, after waiting
Order of sacrifice
Degrade in a fixed order
Decide this order before an incident, so the choice under pressure is already made.
- 1Drop optional enrichment calls that only improve a response.
- 2Serve the journey with recent local context your application already holds.
- 3Serve the journey with no memory and label the experience honestly.
- 4Queue non-interactive writes for later, only when replaying them is safe.
- 5Fail the request — last, and only when correctness requires it.
Enforce in code
Guardrails worth enforcing in code
- Deadline
- A total request deadline, not just a socket timeout.
- Attempt cap
- A maximum attempt count with jittered backoff between tries.
- Concurrency cap
- A ceiling on in-flight memory calls per process.
- Idempotency
- An application-level rule for what a repeated write means.
Was this page helpful?