# Agent Memory: What an AI Agent Should Remember, and Where

> Short-term context, long-term memory and task state are different problems. How to store each, keep context windows small, and avoid remembering the wrong things.

- Source: https://akshaythakare.com/notes/agentic-ai/agent-memory
- Topic: Agentic AI
- Level: intermediate
- Reading time: 3 min
- Tags: agents, memory, llm, agentic-ai
- Updated: 2026-10-11

## Models are stateless

An LLM remembers nothing between calls. Everything an agent "knows" during a step is what you put into that step's context window. Memory is therefore an application design problem.

| Kind | What it holds | Lifetime | Typical storage |
| --- | --- | --- | --- |
| Working memory | Current conversation, recent tool results | One task or session | The prompt itself |
| Task state | Plan, completed steps, intermediate results | Until the task finishes | A database row or graph state |
| Long-term memory | User preferences, facts learned, past outcomes | Across sessions | Database or vector store |

## Working memory: keep it small

Context windows are large but not free: long prompts cost more, run slower and can bury the important parts. Common techniques:

- Keep the last few turns verbatim and **summarize older turns**.
- Store large tool outputs outside the prompt and include only the relevant excerpt.
- Drop tool results once the step that needed them is complete.

## Task state: make it explicit

Don't rely on the model to remember where it is in a multi-step job. Persist state in code:

```python
state = {
    "goal": "Reconcile yesterday's payments",
    "steps_done": ["fetched_payments", "fetched_ledger"],
    "mismatches": [{"payment_id": "P102", "reason": "amount differs"}],
}
```

Frameworks such as LangGraph model this as a state object passed between nodes, with checkpoints so a failed run can resume instead of starting over.

```mermaid
flowchart LR
  accTitle: Agent memory layers
  accDescr: Each step builds its prompt from working memory, persisted task state and relevant long-term memories retrieved from storage.
  L[(Long-term store)] -- retrieve relevant --> P[Prompt for this step]
  S[(Task state)] --> P
  W[Recent turns] --> P
  P --> M[Model] --> A[Action / answer]
  A -- update --> S
  A -- write durable facts --> L
```

## Long-term memory: be selective

Write only durable, useful facts ("prefers SQL examples in Snowflake syntax"), not every message. Retrieve by relevance — often with embeddings — and include only a handful per step.

> **Warning: Memory is user data**
>
> Long-term memory is personal data: let users see and delete it, scope it per user, and never let one user's memories reach another user's context.

## A simple rule

Put in the prompt only what the next step needs, persist everything required to resume, and remember long-term only what you would be comfortable showing the user.