Worker Tuning
A large amount of work happens on your tenant outside the request path: tier transitions, retention enforcement, compaction, re-evaluation, summarisation, webhook delivery, and audit forwarding. The platform schedules, scales, and recovers all of this on your behalf — there is no scheduler to operate. This page describes what the background system does for you, what visibility you have into it, and the few controls that surface to your dashboard.
What Runs in the Background
The background system handles every long-running and periodic task that supports the API surface. The headline categories:
| Category | Purpose |
|---|---|
| Memory lifecycle | Tier transitions (hot → warm → cold), compaction of duplicate clusters, re-evaluation of importance scores, and forgetting-policy enforcement. |
| Retention enforcement | Purging audit entries, soft-deleted memories, and expired tombstones in accordance with your tenant retention configuration. |
| Outbound integrations | Webhook delivery, SIEM forwarding, scheduled integration syncs, and DSR export packaging. |
| Operational housekeeping | Key rotation checks, session cleanup, usage rollups for billing, and recovery of any work interrupted by a deploy. |
All of this work is invisible to your application code by design. Your code calls the API; the platform takes care of everything else over time.
Scheduling Guarantees
The background scheduler provides a set of guarantees you can plan against, even without configuring anything:
- Single-execution per cycle. Each scheduled job runs at most one instance per tenant per cycle. If a previous run is still in progress, the next one is skipped rather than queued, so the platform never piles up duplicate work.
- Catch-up after deploys. Missed runs during deploys or recovery are coalesced into a single follow-up execution. Long maintenance windows do not produce a thundering-herd burst.
- Bounded latency for time-sensitive work. Webhook delivery, SIEM forwarding, and usage rollups have published cadence guarantees on the order of minutes. Maintenance work (retention, audit cleanup) runs daily during a low-traffic window.
- UTC-aligned schedules. Any cadence visible to you in the dashboard is expressed in UTC. There are no daylight-savings shifts.
Fair-Share Across Tenants
Because background capacity is shared by all tenants on the platform, the scheduler enforces fair-share to keep one organization’s heavy work from delaying another’s:
- Per-tenant concurrency caps. Each tenant has a maximum number of concurrent background jobs in flight at any moment. Once the cap is hit, further work for that tenant queues until earlier jobs finish.
- Bounded batch size. Sweep jobs (retention, tier transitions, compaction, re-evaluation) process a fixed batch per cycle so no single tenant monopolises the shared worker pool.
- Priority lanes. Time-sensitive jobs (webhook delivery, DSR export, retention) take priority over best-effort sweeps. Heavy backfills never delay a user-visible event.
Retry & Dead-Letter Handling
Transient failures are retried automatically. Persistent failures are isolated so they don’t consume capacity indefinitely:
- Exponential backoff. Each failed job is retried with growing intervals so a temporarily-down dependency is given time to recover without amplifying the failure.
- Bounded retries. After a small, fixed number of retries (typically 3), the job is moved to a dead-letter state. The original payload is preserved for inspection.
- Per-tenant counters. The dashboard surfaces a dead-letter counter per category (summarisation, webhook delivery, integration sync). A growing counter is the canonical signal that an upstream dependency owned by your team is misbehaving — typically a webhook endpoint that has changed URL or stopped accepting traffic.
- Stuck-job recovery. Jobs that exceed their timeout window are reclaimed by another worker and resumed automatically. Customers do not see partial progress; either the job completes or it goes to dead-letter.
Visibility & Webhooks
The background system is observable through the dashboard and through webhooks:
| Surface | What you see |
|---|---|
| Dashboard – Activity | A rolling timeline of significant background events for your tenant: tier transitions, compaction merges, retention purges, DSR completions, and webhook delivery health. |
| Dashboard – Webhooks | Per-endpoint delivery stats: success rate, last delivery, recent failures, and the dead-letter counter. Use this to spot a receiver that stopped accepting events. |
| Dashboard – Audit Trail | Every state change applied by a background job is recorded in the audit trail with the system actor, the resource affected, and the reason. |
| Webhook events | Outbound notifications fire when notable background outcomes occur: DSR completed, compaction merged, retention purge ran on a project, webhook endpoint moved to dead-letter. |
What You Control
There is no scheduler configuration surface, no batch-size tuning, and no manual job triggering for customer-facing roles. The handful of inputs that genuinely affect background behavior are all on the dashboard:
- Retention windows. Per-project retention configuration determines how long memories, audit entries, and DSR records are kept before they are eligible for the retention sweep.
- Tier thresholds. Per-project hot/warm/cold age thresholds drive how aggressively memories move down the tier hierarchy.
- Webhook endpoints. The set of HTTPS URLs that receive outbound events, and the secret used for signature verification.
- Outbound integrations. Connection settings for SIEM forwarders and scheduled integration syncs (GitHub, Notion, etc.) including the sync cadence per integration.
If you have a workload where the default background cadence is not a good fit — e.g. a project that needs more aggressive compaction than the default, or a slower retention sweep — contact support and the cadence can be adjusted for your tenant.