Hot Cache

• Repeat queries embed once per hour instead of every call.
• Popular recall results return without touching the index or the durable store.
• Project-scope and tenant-principal lookups stay sub-millisecond on the hot path.

On a hit, the cached value is returned in well under 10 ms and no downstream call happens. On a miss, the platform performs the full path (embedding API, index call, datastore read) and writes the result back with the configured TTL — but only if the call succeeded. Failed work is not cached.

1. The cache client logs a single warning at INFO (deliberately not ERROR — alert spam is worse than the outage itself).
2. The client disables itself in-process.
3. Reads return None, writes are silently dropped.
4. A lazy reconnect attempt runs on the next cache call — not on a timer, to avoid thundering herd on a restarting server.
5. Throughput drops; correctness does not.

• Adding, updating, or deleting a memory invalidates the recall result keys for that user.
• Updating a project invalidates proj:{project_id}.
• Rotating a tenant key invalidates tenant:{tenant_id}.
• Embedding cache entries are not actively purged; old ones expire on TTL.

• Connection pool size 100, socket timeout 0.5 s, connection timeout 0.5 s — chosen so a slow cache cannot stall a request loop.
• Cache URL is read from an environment variable; if it is empty, caching is disabled at boot and every call becomes a no-op.
• Custom TTLs are supported per write — pass ttl_seconds on set to override the default.

• Anything that needs to survive a process restart on the cache server.
• Anything that must be transactional with the durable store — there is no two-phase commit between them.
• Anything larger than tens of kilobytes — cache entries are sized for hot paths, not bulk payloads.