Storage Layers

The primary datastore is the system of record. Every other store — vector index, hot cache, cold archive — is rebuildable from it. This page walks through the tables that matter for memory traffic and the rules every read and write follows.

The memories table

This is where every POST /memory/add ultimately lands. The full column inventory lives on Record Structure; here is the shape from a storage perspective:

Primary key id (integer, auto-increment).
Hard scope columns: user_id, environment, retention_status — all indexed.
Soft scope column: project_id — indexed, nullable, set-null on cascade.
Encrypted payload columns: text_ciphertext, summary_ciphertext.
Vector backup column: vector (array of floats), plus the index handle vector_id.
Lifecycle columns: retention_status, retention_expires_at, archive_at, hard_delete_at, plus the four transition timestamps.

Tables that move in lockstep with memories

Table	Purpose	Cascade behaviour
`memory_relationships`	Edges in the relationship graph (typed: supports, extends, contradiction, …).	Cascade deletes when either endpoint memory is deleted.
`memory_events`	Audit trail of every state transition (created, updated, accessed, exported, deleted).	Retained even after the memory row is purged.
`memory_clusters`	Clusters and their centroid vectors. Centroids are loaded in bulk by the recall ranker.	Cluster row deletes when its last member memory is purged.
`memory_candidates`	Extraction-pipeline candidates that may or may not have become memories.	Independent of the memory row — survives even when the candidate was discarded.

Lifecycle and deletion tables

deletion_progress — checkpoint table tracking the ordered hard-delete steps. Used to recover from worker crashes mid-deletion.
deletion_tombstone — resurrection-prevention markers so a re-asserted memory after a deletion does not silently come back.
deletion_receipt — write-once compliance proof of deletion. Carries the user id, request date, completion date, and reason.
retention_audit_log — every transition in the lifecycle state machine, with old status, new status, actor, and reason.

Encryption at rest

text_ciphertext and summary_ciphertext are stored as authenticated-encryption ciphertexts. The database never sees plaintext.
Each tenant has a Data Encryption Key (DEK) stored in tenant_encryption_keys; the master key wrapping it lives in an external KMS, never in the datastore.
Key rotation is online: a rotated DEK is marked active, the previous DEK stays read-only for 30 days, and a background re-encryption pass rewrites old rows to the new key during that window.

How tenancy is enforced at the storage layer

The service layer never trusts a body-supplied tenant_id. Every read is built with the resolved tenant scope from the auth principal, and every write inherits it. There is no global "select all memories" path; even admin tools enumerate per-tenant.

Indexes and constraints that matter

environment is a check-constrained string — only development | staging | production are valid.
vector_id is unique across the table, so the index handle is globally unambiguous.
Indexes on user_id, project_id, environment, tier, retention_status, created_at, retention_expires_at, archive_at, hard_delete_at support every common scan the recall and retention paths run.

Schema migrations

Schema changes ship as ordered, idempotent migration files. Each migration handles one logical change and is applied sequentially on deployment. Downgrade migrations are deliberately not provided — rolling forward with a fix is the supported pattern.

← Previous

Edge Cases

Vector Index