How It Works

Pipelines

This page traces a single POST /memory/add from the moment it hits the API server to the moment the row is searchable. It is the canonical write path — every other ingestion source (integrations, SDK helpers) ultimately funnels through this same service.

The seven stages of one ingestion

1Authenticate. The middleware stack resolves the credential to a principal and attaches tenant_id, project_id, environment.
2Schema validation. The body is parsed as MemoryCreateRequest — text must be ≥10 characters after stripping whitespace; importance if present must be in [0.0, 1.0]; tags are deduped.
3Low-value detection. The validator rejects pure filler ("hi", "ok", "thanks") with LowValueInputSkipped — the API returns 2xx with status="skipped" so the caller does not retry.
4Encryption. The plaintext is encrypted with the user's per-user key using authenticated encryption. The ciphertext goes into memories.text_ciphertext; plaintext is never persisted.
5Embedding. The ingestion service calls the embedding manager synchronously. The cache is checked first; on miss, the embedding service is called, and the resulting vector is cached for future identical inputs. The vector is held in memory.
6Database commit. One transaction inserts the memories row with retention_status="active", tier="hot", the resolved scope keys, the encrypted text, the metadata, the importance, and the embedding version stamp.
7Vector upsert. The vector is written to the per-user namespace in the vector index with metadata (memory_id, tier, retention_status, environment, project_id, tags, source) so query-time filtering works without a database round-trip.

What the request body looks like

HTTP

POST /memory/add HTTP/1.1
Authorization: Bearer <jwt>
X-Project-ID: proj_a1b2c3d4e5f6a7b8
Content-Type: application/json

{
  "text": "User Alice prefers concise replies and uses dark mode.",
  "tags": ["preference", "ui"],
  "source": "support-chat",
  "event_type": "preference.set",
  "importance": 0.7,
  "metadata": { "channel": "web", "agent_id": "ag_42" }
}

What the response looks like

JSON

{
  "id": 18421,
  "vector_id": "v_18421_e7c9",
  "content": "User Alice prefers concise replies and uses dark mode.",
  "tier": "hot",
  "retention_status": "active",
  "embedding_version": "v3",
  "importance": 0.7,
  "tags": ["preference", "ui"],
  "source": "support-chat",
  "created_at": "2026-05-04T10:14:32Z"
}

What can cause a write to <em>skip</em> instead of insert

Reason	Detection point	Caller sees
Pure filler text	Validator: pattern + length check.	`200` with `status="skipped"`, `reason="low_value"`.
Near-duplicate of an existing memory (≥80% word overlap)	Semantic validator after schema parse.	`200` with `status="skipped"`, `reason="redundant"`, plus the existing memory id.
Non-content placeholder ("This is a file", "Untitled")	Pattern matcher.	`200` with `status="skipped"`, `reason="placeholder"`.

Why embedding runs <em>during</em> the request, not after

Two reasons. First, latency contract: the API guarantees that a memory written by one request is queryable by the very next request — that is only possible if the vector exists by the time the response returns. Second, error reporting: an embedding failure is something the caller wants to know about, not a silent background problem.

When embedding is deferred

In soft-degradation mode (embedding service rate-limited, cache miss), the row still commits and a background task retries the embedding. The response carries status="pending_embedding" so the caller knows the memory will appear in queries soon, not yet.

What runs after the response is sent

memory_events row written with event_type="created".
Audit row written and (if SIEM forwarding is enabled) queued for forwarding.
Intelligence gatekeeper schedules a scoring pass — see Intelligence Pipeline.
If any registered webhook subscribes to memory.created, a webhook_deliveries row is queued.

Three things to check when an add misbehaves

Did vector_id come back null? The embedding step degraded — the row is in the database but not yet queryable. The pending task will fill it in.
Did status="skipped" come back? The validator chose to drop the input. Read reason on the response — that tells you which rule fired.
Did the scope keys match? A successful add still returning empty queries usually means the query is hitting a different project_id or environment than the add did.

← Previous

Failure Modes & Retries

Memory Query Flow