Ask an engineer who has deployed an AI agent and one limitation arrives early: agents forget.

This is not a metaphor. A language model works within a context window: a finite amount of information available during a request. When earlier material leaves that window, the model cannot use it unless another system stores and retrieves it (Packer et al., 2023).

For long tasks, this is not forgetfulness in the charming human sense. It is missing state.

The Three Layers of Memory

“Memory” usually refers to three different mechanisms, bundled under one familiar word. Research taxonomies use finer distinctions, but the practical divide between working context, external stores, and learned parameters is a useful place to begin (Sumers et al., 2024).

In-context memory is the information currently available to the model: recent conversation, tool results, and task state. It is immediate and limited. When the context changes, so does the available past.

External memory stores information in a database, document collection, or knowledge graph. The agent retrieves pieces when needed, an approach formalized in retrieval-augmented generation and extended by later agent architectures (Lewis et al., 2020). This extends its working history, but retrieval can be slow, incomplete, or confidently wrong about what matters.

Fine-tuned memory is the convenient but imperfect name for knowledge and behavior encoded in model weights. It does not recall an episode. It changes how the model responds, and changing it again is not equivalent to deleting a database row; it may require further training, model editing, or machine unlearning (Yao et al., 2024).

Why This Matters for Agency

A capable agent may use all three. Context carries the present task. External stores carry project history and preferences. Model weights carry learned patterns and domain knowledge.

The hard part is orchestration: deciding what to store, what to retrieve, what to forget, and who may inspect or erase the record. A system that remembers your preferred writing style may also preserve an obsolete assumption, a private detail, or last Tuesday’s mistake. Researchers have recovered memorized training examples, including personal information, from language models (Carlini et al., 2021). Memory keeps useful things. That is not all it keeps.

Continuity can make agents more capable and easier to trust. It also gives errors a longer life. Remembering is not understanding.

Sources