RAG (Retrieval-Augmented Generation) retrieves relevant passages from a fixed document collection at query time and pastes them into the prompt. Agent memory stores what the agent has learned from interactions — corrections, preferences, decisions, behavioral patterns — and updates that knowledge over time. The distinction is not subtle: RAG is read-only retrieval from external corpora; agent memory is read-write learning with feedback loops, forgetting, and consolidation. RAG answers “find me a document about X”; agent memory answers “remember that we decided to use Vitest instead of Jest last week, and that you corrected me on the API endpoint last Tuesday.”
| Property | RAG | Agent Memory |
|---|---|---|
| What it stores | Documents, passages, chunks from external corpora | What the agent learned from interactions — corrections, preferences, decisions |
| Read / write | Read-only at query time (the corpus is static) | Read-write (the agent writes new memories, updates, and deletes old ones) |
| Updates | Corpus changes when you add/remove documents; no learned updates | Memories are created, reinforced, decayed, and forgotten based on feedback |
| Persistence | Documents persist; retrieval results do not accumulate | Memories accumulate across sessions and improve over time |
| Learning | No learning — same query retrieves the same passage | Learns from corrections, feedback signals, and usage patterns |
| Forgetting | No forgetting mechanism — all documents remain equally retrievable | Supported by design — several systems apply time-based decay (ACT-R-inspired) so outdated memories lose retrieval strength |
| Provenance | Source document is the provenance | Each memory has provenance — who said it, when, in what context |
| Format | Vector embeddings in a vector store | Varies: vector store, knowledge graph, YAML files, agent state blocks |
The distinction maps to a well-known one in cognitive science: semantic memory (general knowledge, facts — what RAG provides) vs episodic and procedural memory (what happened to you, what you learned to do — what agent memory provides). RAG gives the model access to a library. Agent memory gives it a notebook it writes in.
RAG was formalized by Lewis et al. in 2020 (“Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” NeurIPS 2020, arXiv:2005.11401). The architecture is simple: given a query, retrieve the top-k relevant passages from a document index, prepend them to the prompt, and let the model generate an answer conditioned on both the query and the retrieved context. A 2023 survey by Gao et al. (arXiv:2312.10997) catalogued the explosion of RAG variants — pre-retrieval, post-retrieval, modular pipelines — and confirmed the core pattern: retrieve, augment, generate.
RAG is excellent for question answering over known documents. If you have a corpus of PDFs, a product manual, a codebase, or a wiki — RAG lets the model cite specific passages from that corpus. It is the standard approach for “chat with your documents” use cases.
RAG has four structural limitations for agent use cases:
RAG retrieves from a static corpus. If you correct the model — “actually, we use Vitest, not Jest” — the retrieval pipeline does not update. Next session, the model retrieves the same documents and may make the same mistake. The corpus is fixed; the model’s behavioral knowledge does not accumulate. A 2024 survey of LLM-based agent memory mechanisms (Zhang et al., “A Survey on the Memory Mechanism of Large Language Model based Agents,” arXiv:2404.13501) identified this as the fundamental gap: “memory designs are scattered across different papers” because RAG addresses retrieval, not learning.
RAG cannot remember that you prefer tabs over spaces, that your database is PostgreSQL, or that you deprecated an API last week. These are not documents in a corpus — they are facts the agent learned from interacting with you. RAG has nowhere to put them.
In RAG, all documents are equally retrievable forever. There is no mechanism to say “this fact is outdated” or “this preference changed.” If you migrated from Jest to Vitest, both the old and new documentation sit in the index with equal standing. Agent memory systems can model forgetting explicitly — several projects adapt the ACT-R cognitive theory, applying time-based decay so outdated memories lose retrieval strength while recently-reinforced ones stay sharp (implementations vary: not every memory engine ships decay, but the architecture supports it, which RAG’s does not).
RAG retrieval is one-shot: query, retrieve, generate. There is no signal that says “that retrieval was helpful” or “that retrieval was irrelevant.” Agent memory systems can close this loop: feedback signals (positive, negative, neutral) train the injection pipeline so recall quality improves with use. Not every engine implements feedback, but where it exists, the more you use it, the better it gets at surfacing the right memory at the right time.
Agent memory systems perform four operations that RAG does not:
Capture — Extract facts, corrections, preferences, and decisions from the conversation as they happen. Not a document dump; structured memory items (the agent learned you prefer tabs, that the database is PostgreSQL, that the auth flow uses JWT).
Store — Persist those items outside the model in a format that survives session end. Formats vary: Mem0 uses a vector store, Letta uses agent state blocks, Zep uses a temporal knowledge graph, PLUR uses human-readable YAML files.
Retrieve (with context) — At the start of the next session, surface the right memories for the current context. Not all of them — that would overflow the window. The relevant ones, selected by hybrid search (BM25 + embeddings), activation strength, and feedback signals.
Update and forget — When a fact changes, update the memory. When a fact is wrong, correct it. When you want something deleted, delete it. Memories decay over time if not reinforced, so the system does not accumulate stale noise.
This is not theoretical. MemGPT (Packer et al., 2023, “MemGPT: Towards LLMs as Operating Systems,” arXiv:2310.08560) demonstrated that treating memory like an OS manages memory tiers — core memory in the context window, archival memory retrieved on demand — dramatically improves agent performance on multi-session tasks. The key insight: agents need managed memory, not just more context.
Yes, and many agent architectures do. RAG and agent memory serve different layers:
A coding agent might use RAG to search your repository for a function definition, and use agent memory to remember that you prefer functional style over OOP. The two systems are complementary, not competitive. The mistake is thinking RAG alone gives your agent memory — it gives your agent a library, but no notebook.
| Project | RAG? | Agent Memory? | Format |
|---|---|---|---|
| LangChain RAG | Yes (core feature) | Partial (memory modules, but basic) | Various module types |
| LlamaIndex | Yes (core feature) | Partial (chat history buffers) | Buffer, summary, vector |
| Mem0 | No | Yes (memory layer) | Vector store |
| Letta (formerly MemGPT) | No | Yes (stateful agent OS) | Agent state blocks |
| Zep / Graphiti | No | Yes (temporal knowledge graph) | Knowledge graph |
| Cognee | Partial | Yes (graph + vector + relational) | Own data model |
| PLUR | No | Yes (engram engine + MCP) | Human-readable YAML (open format) |
The key differentiator within agent memory is format openness. Mem0 and Letta are open-source (Apache-2.0) but store memories in opaque formats — vector embeddings or agent state blocks you cannot read in a text editor. PLUR (Apache-2.0, github.com/plur-ai/plur) stores memories as YAML files you can open, edit, diff, and version-control. The MCP (Model Context Protocol, specification 2025-11-25) makes the transport layer open; the memory format is the differentiator.
| If you need… | Use |
|---|---|
| Search and cite a fixed document corpus | RAG |
| Answer questions over your codebase, docs, or wiki | RAG |
| Remember user preferences across sessions | Agent memory |
| Learn from corrections and not repeat mistakes | Agent memory |
| Track how facts change over time | Agent memory (Zep for temporal, PLUR for decay-based) |
| Inspect, edit, and correct what the agent knows | Agent memory with open format (PLUR) |
| Both (search documents AND remember interactions) | RAG + agent memory (complementary) |
Is agent memory just RAG with extra steps? No. RAG retrieves from a static document collection at query time. Agent memory stores what the agent learned from interactions, updates it with feedback, and decays outdated entries. RAG is a read-only retrieval pipeline; agent memory is a read-write learning system with forgetting and consolidation.
Can RAG replace agent memory? Only if your agent never needs to remember preferences, corrections, or decisions across sessions. If your agent works in a single session or only queries fixed documents, RAG is sufficient. If your agent accumulates knowledge over time — learning your coding style, remembering architectural decisions, not repeating corrected mistakes — you need agent memory.
Can you use RAG and agent memory together? Yes. They serve different layers: RAG handles the knowledge layer (documents, code, reference material), agent memory handles the experience layer (what the agent learned from working with you). Many agent architectures combine both.
Is agent memory slower than RAG? Not necessarily. Both involve a retrieval step at query time. Agent memory adds a write step (capturing memories during the conversation) and a decay/feedback step (updating memory strength), but these are background operations that do not add latency to the inference path. Zep reports sub-200ms retrieval for its temporal knowledge graph (docs.getzep.com).
What is the ACT-R decay model in agent memory? ACT-R is a cognitive architecture from cognitive science that models how human memory decays over time. Several agent memory systems adapt it: memories that are not accessed lose retrieval strength, while memories that are reinforced (accessed or given positive feedback) gain strength. This prevents the memory store from accumulating stale noise — something RAG cannot do because all documents remain equally retrievable.