Memory

Large language models do not retain memory across independent calls. Between requests, model weights remain completely static: nothing you type alters internal parameters, and each new API call executes in total isolation from the last. When a conversational application greets you by name or remembers your preferences from last week, an external software layer fetched those details from a database and injected them directly into the prompt payload.

In generative AI engineering, refers specifically to text recorded outside the model and assembled back into the prompt during later interactions. It is fundamentally an external data storage and prompt-assembly pattern managed by the application harness, not an innate capability of the model itself.

Distinguishing session memory from persistent memory

Architectures separate conversational context into two distinct operational lifetimes:

• represents the active conversation transcript. When turn 12 references an idea from turn 3, it succeeds solely because turns 1 through 11 were bundled and resent in the payload on turn 12. Session memory requires no custom database schema; it is simply the message list within the current window. However, it is strictly ephemeral: as soon as the session closes or the terminal process exits, that context vanishes.
• bridges separate sessions. It consists of an external, durable data store maintained specifically to record durable facts about the user, project conventions, or past decisions. At the start of a subsequent conversation, the harness reads these notes and prepends them to the context window.

Session memory is an ephemeral transcript that resets between sessions. Persistent memory is a durable layer underneath that carries key facts across conversational boundaries.

Designing a persistent memory system requires answering three fundamental questions:

1. What to record: Durable facts, confirmed user preferences, architectural decisions, and key project commands. Content must remain concise, because every saved memory consumes permanent token budget in future requests.
2. Who controls writes: The system can allow the model to trigger updates via tool calls, enable users to register facts explicitly via slash commands, or use programmatic heuristics to extract profile details. Model-driven writes require minimal user effort, but they risk saving irrelevant trivia or misinterpreting transient comments as permanent rules.
3. When to recall: In most architectures, stored memories are loaded unconditionally at session initialization before the user submits their first query. This process—known as —differs fundamentally from query-conditional search.

How memory differs from retrieval

Because both mechanisms inject stored text into prompts, memory and retrieval are frequently confused. In practice, they serve different purposes and operate on different scales:

Dimension	Retrieval (RAG)	Persistent Memory
Author and source	External authors (documentation, knowledge bases, manuals)	The application or agent, summarizing user actions or preferences
Data volume	Gigabytes or millions of tokens; far too large for any window	Kilobytes; typically a compact set of bullet points
Access trigger	Conditional: queried only when a user prompt matches relevance	Unconditional: loaded automatically during session startup
Primary objective	Grounding the model in external facts and domain knowledge	Maintaining personalization and continuity across sessions

As memory systems grow larger—for example, when an assistant accumulates hundreds of historical notes—they eventually incorporate vector search to select relevant memories conditionally. At that point, the underlying mechanism becomes retrieval under the hood, even if the product interface labels it as memory.

Storage patterns for persistent memory

Production systems typically store persistent memory in one of three environments:

• Local project files: A markdown document stored within the repository root (such as a developer guidelines file, a scratchpad, or notes managed by the agent). This format is plain text, easily inspected, and trackable in git. Because it lives alongside the code, any developer cloning the repository shares the same baseline instructions.
• Application database records: A relational database or key-value store managed by the host application. A customer support platform, for instance, might store user subscription tiers, past issue summaries, and communication preferences in an operational database, injecting them during prompt assembly.
• Vendor-managed memory stores: Hosted cloud services that automatically extract user facts and inject them into API requests behind the scenes. While convenient, vendor-managed storage limits visibility, making it harder to audit what information has been retained or purge incorrect entries.

In all three architectures, performing a simply updates a record in external storage. The underlying model weights remain entirely untouched.

Common operational pitfalls

While persistent memory creates a seamless user experience, poorly governed memory stores introduce distinct failure modes:

• Data staleness: If a memory written in January notes that "the test suite runs via ./scripts/test.sh", and the team migrates to pytest in March, the obsolete memory will continue to be injected at the top of every session. The assistant will confidently execute a non-existent script. Without automated expiry or periodic human review, stale memories degrade model performance.
• Confidently retained misconceptions: If a user says "format this file with tabs" during an isolated edge case, an eager memory system might record "User prefers tabs over spaces". From that point forward, the assistant will reformat all code across the codebase until corrected.
• Privacy leaks across conversational boundaries: Because memories are recalled automatically without user review, sensitive data recorded in one context can unexpectedly surface elsewhere. An API key, a customer name, or an internal credential noted during a private debugging session might be exposed in a subsequent chat shared on a screen.

What to write to persistent memory

Write a fact only if it will still be true next month.

Write these:

• A stated preference, in the user's own words.
• A decision and the date it was made.
• A correction the user made to the agent's behaviour.
• A stable fact about the project, such as the build command.

Do not write these:

• A secret, a credential, or a token.
• Personal data about anybody other than the user.
• The contents of a file. Write the path instead.
• A conclusion the agent inferred but the user never confirmed.
• Anything specific to one task that is now finished.

Date every entry. Delete an entry as soon as it is wrong.

Treating persistent memory as a transparent, user-editable text file rather than an opaque black box helps teams catch errors, prune obsolete guidelines, and maintain consistent agent behavior over time.

Terms introduced

• Memory: text written outside the model and put back into the context on a later call, so that something appears to carry over. The model itself retains nothing.
• Session memory: the conversation history, resent in full on every turn and discarded when the session ends.
• Persistent memory: a small store that survives the end of a session and is read back into the context at the start of the next one.
• Recall: reading a memory store back into the context, usually at the start of a session and usually without a query.
• Memory write: an update to a memory store during a session, decided by the model, the user, or a rule in the application.

---

How providers do it

The two platforms answer this page's three questions very differently. One ships an explicit memory feature and leaves the storage to the caller. The other ships durable conversation state and no fact store at all.

Question	Anthropic	OpenAI
A memory feature in the API	Yes, a memory tool the model drives	None found on the docs read today
Where the data sits	The caller's own storage, or a provider-hosted memory store for Managed Agents	The caller's own database or vector store
What survives a session by default	A local instruction file, plus auto memory in the coding agent	Response data for at least 30 days; Conversation objects with no expiry
Who decides what is written	The model, through tool calls the caller executes	Nobody. There is no extraction step to decide
Is recall free	No. Recalled text is input tokens like anything else	No. Chained turns rebill every earlier input token

The Anthropic rows are confirmed. The OpenAI first row is an absence rather than a stated "no", and consumer memory behaviour is left open in both tabs.

The distinction to carry away is who owns the store. A memory tool moves the decision about what to write into the model and leaves the storage, the retention and the access control with the caller. Durable conversation state moves the storage to the provider and leaves the decision about what matters unanswered.

Anthropic

What this maps to: four separate things that all get called memory. Two are API features, one is a coding tool's file convention, and one is a consumer product setting. They store data in different places and different people can see it.

Question	Answer	Status
Is there a memory feature in the API?	Yes. A memory tool, {"type": "memory_20250818", "name": "memory"}, available on Claude 4 and later with no beta header	confirmed
Where does it store data?	Nowhere on the provider's side. The model only requests file operations such as view, create, str_replace, insert, delete and rename under a /memories prefix, and the calling application executes them against storage it controls	confirmed
What does the API add?	It injects a memory protocol instruction into the system prompt automatically	confirmed
What does the caller have to build?	The storage, and path traversal protection. The docs call that out explicitly	confirmed
Is there a provider-hosted store?	Yes, for Managed Agents: memory stores under /v1/memory_stores, behind the agent-memory-2026-07-22 beta header. Workspace-scoped, mounted into the session sandbox, 100 kB per memory, 10,000 memories per store, immutable versions with a 30 day audit trail and a redact endpoint	confirmed
How does the coding agent do it?	Two mechanisms, both machine-local files. CLAUDE.md, read from managed policy then the user directory then the project, concatenated in that order. And auto memory, on by default, written under the user's project directory with a MEMORY.md index whose first 200 lines or 25 KB are loaded each session	confirmed
Is the coding agent's memory shared between machines?	No	confirmed
Who can see a consumer memory?	Memories are saved as per-topic files, encrypted at rest, and users view, edit and delete them in settings. Owners cannot view or edit an individual user's memories, though entries appear in organisation compliance exports	confirmed
Is consumer memory on by default?	On for Free, Pro and Max. Off for Team and Enterprise until an owner enables it	confirmed
Does each project get its own memory?	Yes, projects have separate memory spaces	confirmed

Their vocabulary

Standard term	Their term
Persistent memory, caller-hosted	Memory tool (memory_20250818)
Persistent memory, provider-hosted	Memory store, memory, memory version
Instruction file loaded every session	CLAUDE.md
Model-written notes across sessions	Auto memory, indexed by MEMORY.md

Where to look

For the API memory tool, the store is the caller's own filesystem, so what was written is whatever is in it. For the coding agent, /memory lists what is loaded and CLAUDE_CODE_DISABLE_AUTO_MEMORY=1 turns the automatic part off. For consumer accounts, the memory settings screen is the record.

Last verified: 2026-09-09 against the memory tool docs (https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool), the Managed Agents memory docs (https://platform.claude.com/docs/en/managed-agents/memory), the Claude Code memory docs (https://code.claude.com/docs/en/memory) and the consumer memory support article (https://support.claude.com/en/articles/11817273-use-claude-s-chat-search-and-memory-to-build-on-previous-context).

OpenAI

What this maps to: durable conversation state rather than a memory product. The API keeps responses and conversations; it does not maintain a per-user store of extracted facts.

Question	Answer	Status
Is there a persistent memory feature in the API?	None was found on the API docs read today. The nearest equivalents are the Conversations API and vector stores. This is an absence rather than a vendor statement	unconfirmed
What survives a session?	Response data, stored for at least 30 days when store is true. Conversation objects and the items in them are explicitly not subject to that 30 day expiry	confirmed
Does that count as memory?	It is session memory made durable. Reading it back still costs input tokens: with previous_response_id, all previous input tokens in the chain are billed again	confirmed
Where would a fact store live?	In the caller's own database, or in a vector store queried per session. Both are the caller's to build	confirmed
Is API data used for training?	The docs state that data sent to the API is not used to train or improve models unless the customer explicitly opts in	confirmed
How long are abuse monitoring logs kept?	Up to 30 days	confirmed
How long do stored objects live?	Assistants, threads and vector stores persist until deleted, and are removed 30 days after a delete request	confirmed
Can storage be limited by geography?	Data residency is offered across the US, Europe, Australia, Canada, Japan, India, Singapore, South Korea, the UK and the UAE. Modified abuse monitoring and zero data retention are available to approved customers	confirmed
How does the consumer product's memory behave?	Not answered. The support and marketing pages returned 403 to the fetches made today, so nothing about it is asserted here	unconfirmed

Their vocabulary

Standard term	Their term
Session memory held by the provider	Conversation state, store, previous_response_id
A durable conversation identifier	Conversation object
Turning off retention	Zero data retention, modified abuse monitoring

Where to look

The dashboard's data controls page carries the retention and residency settings. For anything resembling persistent memory, the design question is where the caller's own store lives, because the API does not provide one.

Last verified: 2026-09-09 against the conversation state guide (https://developers.openai.com/api/docs/guides/conversation-state), the Responses create reference (https://developers.openai.com/api/reference/cli/resources/responses/methods/create) and the data controls guide (https://developers.openai.com/api/docs/guides/your-data).

---

Check your understanding

0 of 4 answered

1. A chat product greets a returning user by name. What made that possible?

   Something outside the model read the name from a store and put it in the text sent with the question.The model updated its weights during the previous conversation.The provider keeps the previous conversation loaded on the server between visits.The model recognises the user from the pattern of their writing.
2. Why can turn 12 of a conversation refer to something said at turn 3?

   Turns 1 to 11 are resent in full as part of turn 12's request.The model keeps a short-term buffer of the current session.The provider stores the session and looks up earlier turns as needed.A memory write happened automatically at turn 3.
3. Which of these belongs in persistent memory?

   The build command for this project, which has not changed in a year.The contents of the config file the agent read this morning.The API token the user pasted in to unblock a test.The list of files changed in the task that just finished.
4. What most clearly distinguishes memory from retrieval?

   Memory is a small store the system wrote about this user or this work, and it is read back unconditionally at the start of a session.Memory is stored inside the model, and retrieval is stored outside it.Memory uses embeddings and retrieval uses keyword search.Memory is written by the user and retrieval is written by the model.