Every conversation with an AI agent starts from zero. You explain your project, your preferences, your stack, your coding conventions — and by the next session, all of it is gone. The agent does not remember that you use Vitest, not Jest. It does not remember that you deprecated that API last week. It does not remember the architectural decision you spent an hour explaining. You are paying in tokens — and time — to re-teach the same context, over and over, every single session. This is not a bug. It is the default architecture of every LLM-based agent: stateless inference, no persistent storage, a context window that resets when the conversation ends.
The fix is a memory layer: a system that sits between the agent and the model, captures what the agent learns during a session, and injects the right piece at the right time in the next one. This is not RAG (RAG retrieves from a fixed document store at query time; agent memory accumulates and updates what the agent has learned — corrections, preferences, decisions — over time). It is also not fine-tuning (fine-tuning bakes facts into model weights you cannot read, correct, or delete; memory is instant, reversible, and inspectable). A memory layer is the persistent substrate that makes an agent get better the more you use it, instead of resetting to blank each time.
Large language models are stateless by design. Each inference call takes a prompt, produces a token, and moves on. There is no persistence between calls — the “memory” you experience in a single conversation is just the growing context window, and when that window overflows or the session ends, the content is gone. Expanding context windows (128K, 200K, 1M tokens) does not solve this: longer windows mean higher costs and degraded attention — the model retrieves less accurately from a larger context — but the content still disappears when the session closes. A 2026 survey of 12 agent memory systems found that “no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck” (arXiv:2606.24775, June 2026).
The research is clear: the problem is not “context is too short” — it is that there is no system to persist, organize, and retrieve what matters across sessions. That system is what a memory layer provides.
A memory layer performs four operations:
The simplest option is the memory feature built into your agent’s platform. Claude Code has two mechanisms: CLAUDE.md files (persistent instructions you write) and auto memory (notes Claude writes itself based on your corrections and preferences). Both load at the start of every conversation (docs.anthropic.com). ChatGPT has a similar memory feature — it stores facts about you across conversations.
Good for: zero setup, works immediately. Limitation: memory is locked to one platform. You cannot carry it to a different agent. You cannot inspect what it stored in a structured format. You cannot export it. If you switch from Claude to GPT, you start over.
If you want memory that is not locked to one provider, open-source memory engines are the next step.
Good for: structured memory that is not locked to a single model provider, with APIs you control. Limitation: the memory format under the hood is a vector store, agent state blocks, or a knowledge graph — not a human-readable file you can open, edit, and diff. “Open source” does not mean “open format.”
The most recent option is memory that is both open-source and open-format, exposed via the Model Context Protocol — an open protocol (JSON-RPC 2.0, specification 2025-11-25) that standardizes how LLM applications connect to external tools and data sources (modelcontextprotocol.io). MCP means the same memory server works across Claude Code, Hermes, OpenClaw, Cursor, and any other MCP-compatible runtime.
PLUR (Apache-2.0, github.com/plur-ai/plur) is one example. Each memory — an “engram” — is a human-readable YAML entry with an id, statement, type, domain, scope, confidence, and provenance. You can open it in a text editor, put it under version control, correct a single fact mid-conversation (no retraining), and delete an entry with provable erasure (remove the entry — a git diff proves the deletion). It is local-first: your data, your infra, no vendor lock-in.
Good for: agents that operate across multiple tools, models, and providers — where memory must be portable, inspectable, and correctable. Limitation: newer and less battle-tested than Mem0 or Letta at production scale.
| Your situation | Start with |
|---|---|
| I use one agent (Claude Code) and want zero setup | CLAUDE.md + auto memory |
| I want structured memory with a simple API, no infra | Mem0 (cloud) |
| I want a self-managing stateful agent | Letta |
| I need time-aware facts (what is true now) | Zep / Graphiti |
| I need memory that is inspectable, correctable, and portable across runtimes | PLUR via MCP |
The underlying question is not “which tool is best” — it is “what do I need to own?” If you only ever use one agent platform, built-in memory is fine. If you use multiple agents, or if you need to audit what your agent knows, correct it mid-conversation, or prove what it forgot — you need a memory layer that is external to the model, in a format you control.
Why does my AI agent forget everything between sessions? Because LLMs are stateless — each inference call takes a prompt and produces output with no persistence between calls. The “memory” within a conversation is just the context window, and when the session ends, that context is gone. You need a memory layer — a system that captures, stores, and retrieves what the agent learned across sessions.
Is this the same as RAG? No. RAG retrieves from a fixed document store at query time. Agent memory accumulates and updates what the agent has learned — corrections, preferences, decisions — over time. RAG answers “find me a document about X”; agent memory answers “remember that we decided to use Vitest instead of Jest last week.”
Is fine-tuning better than memory for this? No. Fine-tuning bakes facts into model weights you cannot read, correct, or delete. Memory is instant (store now, use now), reversible (update or delete a single fact), and inspectable (you can see what the agent knows). Fine-tuning is for changing how the model behaves; memory is for what the model knows.
Can I add memory to Claude Code? Yes. Claude Code has built-in CLAUDE.md files and auto memory. For cross-runtime memory, you can connect an MCP-compatible memory server (like PLUR) that works across Claude Code, Hermes, OpenClaw, and other MCP-compatible agents.
What is the cheapest way to stop my agent forgetting? CLAUDE.md files are free — you write persistent instructions in a markdown file that loads at the start of every conversation. For structured, accumulating memory that does not require you to write every instruction manually, an open-source memory layer like Mem0 (free self-host) or PLUR (free, local-first) is the next step.