MemorySync
Integrations

Pipecat Voice Memory

Long-term memory for Pipecat voice pipelines — one FrameProcessor between your context aggregator and LLM enriches every context frame under a hard latency budget and sends only new user turns for fact extraction. Built on the current frame API and tested through Pipecat’s own harness.

Pipecat enriches each LLMContextFrame before the LLM, then sends only new user messages after the push.

What the package provides

PieceWhat it does
MemorySyncMemoryServiceA FrameProcessor you place before your LLM service. Each LLMContextFrame is enriched with relevant memories (as a system message) and its new user turns are sent to fact extraction — then pushed on, enriched or not, on time.
Budgeted enrichmentRecall runs under a hard timeout (default 1.2s). A slow or dead memory backend means an unenriched frame, never a stalled voice reply.
Delta-only captureOnly user messages not seen before are sent, tracked by deterministic idempotency seeds — O(new), not O(entire context) per frame. Assistant replies are not sent.
Lifecycle safetyOn EndFrame, queued sends get a bounded window to land so the caller’s last words are never lost; CancelFrame pushes first, then briefly salvages.
Mem0SupermemoryZepMemorySync
Pipecat integration✓ in-tree service△ exists but imports a removed frame API — cannot load on Pipecat 1.x✗ none✓ pipecat-memorysync package
Recall latency budget✗ unbounded await before every LLM push——✓ hard timeout, default 1.2s, tested against a 5s-slow backend
Write amplification✗ re-stores the ENTIRE context every frame — O(n²) writes per call——✓ delta-only with idempotency seeds — only new user messages are sent, proven by test
Injected context re-storedunguarded——✓ the injected block is capture-excluded, tested
End-of-call data losspending stores can be torn down——✓ graceful EndFrame flush, tested
Tested via the official harness△ partial✗—✓ every test runs through pipecat.tests.utils.run_test

Requirements

You needWhere to get itUsed for
MemorySync API keyDashboard → API Keys (read + write scopes)Recall and capture
Pipecat 1.0+pip install "pipecat-ai[webrtc]" (plus the service extras you use, e.g. deepgram, openai, cartesia)The pipeline runtime
STT / LLM / TTS keysYour chosen providers — the processor is provider-agnosticYOUR pipeline’s voice loop. Not used by MemorySync.
Python 3.10+python.orgRuntime
BASH
# your pipeline's own services (LLM/STT/TTS) - never sent to MemorySync
OPENAI_API_KEY=sk-...
DEEPGRAM_API_KEY=...
CARTESIA_API_KEY=...
# the only key MemorySync needs
MEMORYSYNC_API_KEY=ms_...

No MemorySync-side setup beyond the key — tenants, embeddings and fact extraction are provisioned automatically on first write.

Install and place it

BASH
pip install pipecat-memorysync
PYTHON
from pipecat.pipeline.pipeline import Pipeline
from pipecat_memorysync import MemorySyncMemoryService
memory = MemorySyncMemoryService(
api_key="ms_...", # or MEMORYSYNC_API_KEY env var
user_id="caller-42", # stable end-user id
session_id="call-123", # optional: scope to this call
)
pipeline = Pipeline([
transport.input(),
stt,
context_aggregator.user(),
memory, # ← between aggregator and LLM
llm,
tts,
transport.output(),
context_aggregator.assistant(),
])
BASH
# What capture performs on the wire — one call per NEW user message only.
# The server extracts the durable facts in it and stores only those,
# grouped under the pipecat:: session scope. Assistant replies are not sent.
curl --request POST https://api.memorysync.io/v1/memory/add_turn \
--header "X-API-Key: $MEMORYSYNC_API_KEY" \
--header "Content-Type: application/json" \
--data '{"tenant_id":"acme","user_id":"caller-42","source":"pipecat","text":"book the same hotel as last time","role":"user","speaker":"human@pipecat::call-123#h<content-hash>","metadata":{"session_id":"pipecat::call-123"}}'

Placement matters: after context_aggregator.user() so the frame carries the full context, before llm so enrichment reaches the model. The service is reusable across pipeline runs; call await memory.aclose() at application shutdown for an explicit flush.

Complete runnable example

import os
from dotenv import load_dotenv
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.processors.aggregators.llm_context import LLMContextAggregatorPair
from pipecat.runner.types import RunnerArguments
from pipecat.transports.base_transport import TransportParams
from pipecat_memorysync import MemorySyncMemoryService
load_dotenv()
async def bot(runner_args: RunnerArguments):
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
llm = OpenAILLMService(model="your-model")
tts = CartesiaTTSService(api_key=os.getenv("CARTESIA_API_KEY"))
memory = MemorySyncMemoryService(
user_id=os.getenv("MEMORYSYNC_USER_ID", "caller-42"),
session_id="call-123",
)
context = LLMContextAggregatorPair()
transport = ... # your transport (see any Pipecat foundational example)
pipeline = Pipeline([
transport.input(),
stt,
context.user(),
memory, # ← the only new line
llm,
tts,
transport.output(),
context.assistant(),
])
runner = PipelineRunner()
await runner.run(PipelineTask(pipeline))

Say "My name is Alex and my favorite color is teal", disconnect, refresh the page, reconnect — the bot greets you with what it remembers. The processor is one added line in any of Pipecat’s foundational examples; everything else is a stock pipeline.

Understanding user_id and session_id

FieldWhat it scopesChoose it like this
user_idThe person. All recall and all facts attach to this id — it is what makes the caller recognized across calls and across every MemorySync integration.A stable id from YOUR system: account id, CRM id, or the caller’s number. Never a random per-run value.
session_idThe conversation. Groups the facts distilled from this call under one session scope (pipecat::<session>).The call or room id. Optional — omit it and the facts still store under the user.

What runs when

MomentWhat happens
LLMContextFrame arrivesRecall runs under the budget; relevant memories land as a system message (position configurable). The frame is pushed on time either way.
Same frame, before it is pushed onNew user messages — and only new ones — are queued in the background for fact extraction. Assistant replies are not sent, and the injected memory block is excluded.
EndFrame (graceful end)Queued sends get a bounded 3s window to land before the pipeline stops — the caller’s last words are not lost.
CancelFrame (abort)The frame is pushed first; sends get a brief salvage window.
Backend slow / dead / over quotaUnenriched frame, silent skip — the pipeline is never affected. Free-tier quota exhaustion is silent by design; evaluation keys surface strict 429s instead.

Configuration

ParameterDefaultMeaning
top_k6Memories injected per turn
recall_timeout1.2Hard recall budget in seconds
system_promptguarded headerPrefix line for the injected block; doubles as the capture-exclusion marker
add_as_system_messageTrueInject as a system message (else as a user message)
position1Index in the message list where the memory block is inserted (0 puts it first; the default lands right after your system prompt)
min_prompt_chars8Skip recall for trivial prompts

Example: what the caller experiences

Call 1 — caller"Hi! My name is Alex and my favorite color is teal."
Bot"Great to meet you, Alex! Teal it is. What can I do for you?"
*(server, seconds later)*Extracted facts land: *Name is Alex* · *Favorite color is teal* — not the caller’s words, no "ok"/"hmm" junk, nothing from the bot’s reply
Call 2 — caller"Do you remember anything about me?"
Bot"Of course — you’re Alex, and your favorite color is teal!"

What happens server-side

ConcernBehaviour
What is storedThe caller’s turns go to fact extraction, and only the durable facts in them are stored — never the turn text, and nothing from the assistant’s replies. It is the pipeline every MemorySync integration shares. Speech-pause fragments ("and", "dinner") and filler turns store nothing.
Capture timingEvery new user message is in flight the moment its frame passes — BEFORE the LLM replies — so a browser disconnect can never lose the caller’s last (usually most important) sentence.
BillingEach sent user turn counts one add request. Each enrichment is a recall: one retrieval request, plus one more when it returns no context and the plain semantic query fallback runs. A frame repeating the last user message is not recalled again. Deletes are free.
Over quotaProduction keys degrade silently — sends are accepted and skipped (processing_status: "skipped"), recalls answer empty — a billing state is never spoken to a caller. Evaluation keys get a strict 429 instead.
DuplicatesDeterministic idempotency seeds — reconnects and frame replays are recognised server-side and extracted once.

Verify it is working

BASH
# list what the bot remembers about a caller
curl "https://api.memorysync.io/v1/memory/acme/caller-42/list?limit=20" \
--header "X-API-Key: $MEMORYSYNC_API_KEY"

Or open Memory Explorer in the dashboard and filter by the user id — facts appear within seconds of each turn (extraction is asynchronous, typically under 10s).

Troubleshooting

SymptomCause and fix
Bot never remembers anythingAlmost always a user_id that changes between runs — pin it. Second check: the key needs write scope.
Remembers within a call, not across callsExtraction is asynchronous — give it ~10s after the call before reconnecting. If facts exist in Memory Explorer but recall misses them, raise top_k.
Replies feel slow on a home networkRaise recall_timeout (e.g. 6.0) during development — the 1.2s default is tuned for datacenter latency and skips injection rather than wait.
Reconnect hangs in the local playgroundRefresh the page (F5) between disconnect and reconnect — a quirk of the dev client, not of the pipeline or the memory service.
Nothing stores and nothing errorsQuota exhausted — production keys degrade silently by design. Check the Usage page.

Privacy and data safety

Only extracted facts persist — the caller’s turns are processed and discarded, and assistant replies are never sent, so chit-chat and mishears never become permanent records. The injected memory block is capture-excluded and can never re-enter storage. Deletion is a first-class API (/memory/forget) and dashboard action, per end user. The memory engine sees text only — audio never reaches MemorySync.

Supported versions

SurfaceRequiresVerified on
pipecat-memorysync 1.2.1pipecat-ai 1.0+, Python 3.10+19 CI checks driven through pipecat.tests.utils.run_test — the framework’s official harness — on the latest pipecat-ai: budget enforcement, immediate loss-proof capture (including the exact disconnect sequence), delta-only proof, injection exclusion, user-turn-only sending (replies never sent), quota modes

Where to go next

Was this page helpful?