Horizontal Scaling

The platform separates compute into two independent scaling groups that respond to different signals:

• Request-serving capacity. Handles synchronous traffic to POST /memory/add, POST /memory/query, and the management endpoints. Capacity tracks live request rate and pipeline-stage latency.
• Background work capacity. Handles asynchronous jobs such as tier transitions, retention sweeps, compaction, re-evaluation, summarisation, webhook delivery, and audit forwarding. Capacity tracks queue depth, not request rate.

Both groups scale independently. A read-heavy workload grows request-serving capacity without changing background capacity; a heavy ingestion job grows background capacity without inflating the API tier. Customers do not configure either group — sizing is automatic.

Capacity is added and removed continuously in response to load signals. The platform optimises for two competing goals: latency stays inside the published budget, and capacity removal never thrashes (so traffic dips don’t cause cold starts on the next spike).

Every instance serving your traffic is interchangeable. The platform does not require sticky sessions, and your application should never assume two consecutive requests land on the same instance:

• Authentication is per-request. Each call carries its own API key. There is no server-side session that needs to be replicated.
• Caching is shared. Cache reads/writes go through the platform’s shared cache layer. A cache hit on one instance is visible to every other instance immediately.
• Uploads are durable. Files uploaded to extraction or DSR export endpoints are written to shared durable storage before the response is returned. The next instance reading them sees a consistent view.
• No client-side affinity headers. Your callers do not need to send affinity cookies or pin to a specific edge. Standard HTTP load-balancing applies and is invisible to the integrator.

Multi-tenant isolation is enforced at every layer so that a spike on one organization’s account never affects another’s latency:

• Tenant resolution. Every request is routed to the correct organization via the API key and X-Project-ID header. The tenant identity is established before any business logic runs, and the request can never cross into another tenant’s data.
• Per-key rate limiting. Rate limits apply at the API-key level. One tenant exceeding its rate limit produces 429 responses on that tenant’s traffic only — every other tenant continues unaffected.
• Per-tenant failure isolation. Upstream provider failures (model errors, embedding outages) trip a circuit breaker for the affected tenant only. Other organizations on the same platform keep operating normally.
• Fair-share scheduling. Background queues are fair-scheduled across tenants. A heavy backfill on one organization cannot starve another organization’s sweeps.

As background capacity scales, multiple workers may process the same job type at the same time. The platform guarantees they never step on each other:

• Lock-and-skip claims. Sweep jobs (retention, tier transitions, re-evaluation, compaction) claim candidates with row-level locks that skip already-claimed work. Two workers never process the same record.
• Atomic state transitions. Each state change (tier promote/demote, memory compaction merge, soft-delete) is wrapped in a single transaction. Either it commits in full or rolls back — partial transitions are impossible.
• No starvation. Records skipped because another worker held the lock are picked up on the very next sweep cycle. The sweep cadence guarantees every eligible record is processed within a bounded time window.
• Bounded batch size. Each sweep processes a fixed batch so that a single worker never monopolises shared resources, even during a backfill.

Your plan governs how much elastic capacity you have access to. The platform never blocks individual requests for capacity reasons — it scales to meet legitimate traffic — but it enforces the plan quotas you signed up for:

If you expect to exceed your plan’s monthly quota or per-key rate limit, contact billing in advance. The platform will not throw errors that surprise your application — the only failure mode for plan-quota exhaustion is the documented silent-degradation path.