ElevenLabs Agent Memory System
Give your ElevenLabs agents a memory.
ElevenLabs voice agents forget every caller the moment a call ends. ELAMS sits behind the three native webhook touchpoints and turns every finished call into durable caller memory — so returning callers hear a greeting that remembers them, instantly.
One-time purchase. Full git repository. Deploy on your own infrastructure.
Agent · first words on connect
“Welcome back! Last time we worked on your LinkedIn message to your HRIS network — did the CHRO outreach land any replies?”
How it works
Four moving parts, wired to the three webhook touchpoints ElevenLabs already exposes. No audio access, no unofficial APIs.
Capture every call
The post-call webhook delivers each finished conversation. ELAMS verifies the HMAC signature, parses the full payload, and archives the raw JSON untouched — nothing is ever discarded.
hook post_call · HMAC-verified · raw JSON archivedSynthesize memory
A background job sends the call summary plus the caller's existing profile to your LLM key — Anthropic or OpenAI — and gets back an updated consolidated profile and a personalized greeting for the next call.
async job · your API key · profile + next-call greetingGreet instantly
When the caller rings again, the conversation-initiation webhook serves the pre-staged greeting and history variables in milliseconds. No LLM call happens while the caller waits.
hook conversation_initiation · pre-staged · ms responseRecall mid-call
Webhook tools let the live agent query the caller's history, profile, and past conversations mid-sentence — memory the agent can reach for, not just a greeting.
hook webhook_tools · history / profile / past callsFeatures
Everything in the box is in service of one outcome: a caller who never has to re-introduce themselves to your own agent.
F-01
Instant caller recognition
Returning callers are matched by phone number the moment a call connects — before the agent says a word.
F-02
Pre-staged greetings
Personalized greetings are generated after each call, so the next one opens with memory and zero added latency.
F-03
Mid-call memory tools
Webhook tools give the live agent on-demand access to caller history, profile, and past conversations.
F-04
Nothing dropped, ever
Every webhook payload is archived verbatim. Parsing can evolve; the original record never changes.
F-05
Evolving caller profiles
Each call refines a consolidated profile — goals, context, open threads — instead of piling up disconnected notes.
F-06
Historical import
Bring existing conversations in via the ElevenLabs API or JSON upload, deduplicated against what's already stored.
Admin console
A relational console your non-technical staff can run. Light, plain, fast — it looks like this because the app actually looks like this.
+1 (612) 508-2017
+1 (415) 776-0903
+1 (208) 314-5561
Consolidated profile
Next-call greeting
“Welcome back! How did the callbacks from your HRIS outreach go — ready to prep that first interview?”
Sample caller shown with the console's actual surface tokens — white and slate, hairline borders, one cyan accent.
Callers
Search by number, read and edit the consolidated profile, delete with cascade warnings that show exactly what's affected.
Conversations
Every parsed field, evaluation criteria, a rendered transcript — and a view-raw-JSON escape hatch for the verbatim payload.
Greetings
Pending, consumed, and superseded greetings in one queue. Edit the copy or regenerate before the next call lands.
Flagged items
Ingestion failures surface here for review and replay — never silently lost.
Self-hosted, by design
Your callers' memory never leaves your infrastructure. That's not a feature toggle — it's the architecture.
Runs anywhere Docker runs
A Dockerized deployment kit ships in the repository. VPS, bare metal, your existing cluster — if it runs Docker, it runs ELAMS.
One database, yours
Everything — callers, conversations, greetings, jobs, the raw payload archive — lives in a single PostgreSQL database you control.
Verified webhooks, encrypted keys
Every inbound webhook is HMAC-verified; your LLM and ElevenLabs credentials are encrypted at rest.
Full source, no black box
You're buying engineering you can read and audit — a standard Rails 8 + React 19 codebase, not an opaque binary.
Integrates with · built on
Pricing
One license, one business, no subscription. You own your copy of the code from day one.
ELAMS source license
- Complete git repository
- All three webhook endpoints
- LLM synthesis engine — bring your own key
- Full admin console
- Historical importer
- Dockerized deployment kit
- ElevenLabs wiring guide
FAQ
Questions, answered.
What exactly am I buying? +
Access to the complete ELAMS git repository under a single-business source license. You clone it, configure your credentials, and deploy it on your own infrastructure. There is no hosted version and no subscription.
What do I need to run it? +
A Docker-capable host, a PostgreSQL-friendly budget (it ships in the compose file), an ElevenLabs account with at least one phone-enabled agent, and an Anthropic or OpenAI API key for memory synthesis.
How does it connect to ElevenLabs? +
You paste three webhook URLs into your ElevenLabs dashboard: conversation initiation, webhook tools, and post-call. The included wiring guide walks through each one. No audio access, no unofficial APIs.
Is caller data ever sent to a third party? +
Only the transcript summary and existing profile go to the LLM provider you configure with your own key, during synthesis. Everything else stays in your PostgreSQL database on your hardware.
Can my non-technical team use it? +
Yes — the admin console is built for staff: search callers, read rendered transcripts, edit profiles and greetings, and delete records with cascade warnings that show exactly what is affected.
Can I customize the code? +
It is a standard Rails + React codebase and you own your copy. Extend the synthesis prompts, add fields, restyle the console — the license covers your business's own use.