Getting Started
LiveKit Voice Memory
Long-term memory for LiveKit Agents voice AI — callers your agent remembers across calls, injected under a hard latency budget so a reply is never late. Background prefetch makes the common case a zero-network cache hit, and both sides of every conversation persist with idempotency guarantees.
What the package provides
| Piece | What it does |
|---|---|
MemorySyncMemory (composition) | Attach memory to YOUR Agent subclass: one call in on_user_turn_completed injects recalled context under a hard budget (default 1.2s); attach(session) captures both roles as items finalize. No inheritance demanded. |
MemorySyncAgent (drop-in) | An Agent subclass with recall and capture pre-wired, for greenfield agents. |
create_memory_search_tool | A function_tool the LLM calls to search memory on demand — the right recall path for speech-to-speech realtime models. Errors return as readable strings, never raise into the model. |
| Failure contract | Slow backend → reply proceeds without memories. Dead backend, quota exhaustion → same. A memory outage can never stall or break a call, by test. |
| Mem0 | Supermemory | Zep | MemorySync | |
|---|---|---|---|---|
| LiveKit integration | △ docs recipe, no package | ✗ none | ✓ zep-livekit package | ✓ livekit-memorysync package |
| Recall latency budget | ✗ unbounded await in the reply path | — | ✗ unbounded await | ✓ hard timeout, default 1.2s, tested against a 5s-slow backend |
| Prefetch | ✗ | — | ✗ | ✓ next recall warmed in the background — common case is a cache hit |
| Captures both speakers | ✗ user turns only | — | ✓ | ✓ with interruption metadata |
| Composition or inheritance | copy-paste recipe | — | must subclass ZepUserAgent | ✓ composable engine — keep your own Agent class (drop-in also available) |
| Injected context re-stored as memory | unguarded | — | unguarded | ✓ injection is turn-only and capture-excluded, tested |
Install and wire it in
pip install livekit-memorysync
Prefer zero wiring? MemorySyncAgent(instructions=..., api_key=..., user_id=...) is the same engine pre-attached. Every stored turn carries a deterministic idempotency seed, so reconnects and retries never duplicate memories server-side.
What runs when
| Moment | What happens |
|---|---|
| User turn completes | Prefetched context is taken if fresh; otherwise recall runs under the hard budget. The memory block lands in the turn context with a guard line — background information, not instructions. |
| LLM reply | Proceeds on time regardless: with memories when recall answered, without them when it did not. |
| Conversation item finalizes | User and assistant turns persist in the background with role fidelity and interrupted metadata. Injected memory blocks are excluded — recalled context never re-enters storage. |
| After each turn | The next recall is prefetched in the background, so the next injection is usually a zero-network cache hit. |
| Backend slow / dead / over quota | Empty context, silent skip — the call continues. Free-tier quota exhaustion is silent by design; evaluation keys surface strict 429s instead. |
Configuration
| Parameter | Default | Meaning |
|---|---|---|
api_key | MEMORYSYNC_API_KEY env | MemorySync API key |
user_id | required | Stable end-user identity — memories follow this id across calls |
thread_id | none | Scope the transcript to one room or call thread |
recall_timeout | 1.2 | Hard budget in seconds for recall injection |
top_k | 5 | Memories injected per turn |
persist_injection | False | True writes the memory block into the session context instead of turn-only |
prefetch | True | Warm the next recall in the background |
Supported versions
| Surface | Requires | Verified on |
|---|---|---|
livekit-memorysync 1.0.0 | livekit-agents 1.x, Python 3.10+ | 16 CI checks driven through real AgentSession machinery on the latest livekit-agents: budget enforcement against a 5s-slow backend, prefetch instant-hit, injection exclusion, both-role capture, idempotency seeds, thread scoping, quota modes, dead-server survival |
Where to go next
Was this page helpful?