Memory: threads & checkpoints
An agent has to remember what happened and decide what to do next. Both fit inside a limited context window.
Planning
For a long task, an agent first writes a plan: a short list of steps. It carries the plan out one step at a time and revises it when a step fails. Planning turns one large, vague goal into small steps that can each be checked.
Memory
Short-term memory is whatever sits in the context window. It is limited, so something must decide what stays. Common choices: drop the oldest turns, summarise them, or store facts outside the window and retrieve the relevant ones when needed.
In the lab, a user mentions a fact early in a conversation and asks about it at the end. Change the window size and the memory strategy, then see whether the agent can still answer.
In LangChain
Short-term memory is the message history of one conversation, saved by a checkpointer and keyed by a thread_id. Reusing a thread continues the conversation; a new thread starts empty. Middleware can trim or summarise long histories before each model call, and facts meant to outlive a conversation belong in a separate long-term store.
from langgraph.checkpoint.memory import InMemorySaver
agent = create_agent(model="provider:model-name", tools=[], checkpointer=InMemorySaver())
config = {"configurable": {"thread_id": "user-42"}}
agent.invoke({"messages": [{"role": "user", "content": "My dog is called Biscuit."}]}, config)
agent.invoke({"messages": [{"role": "user", "content": "What is my dog called?"}]}, config)
# same thread_id, so the second call sees the first messageA worked example
With a 40-word window and the oldest turns dropped, the first turn about the dog no longer fits, so the agent cannot answer “What is my dog’s name?”. Summarising keeps the topics (a trip, a dog-friendly hotel) but loses the name. Retrieval stores every turn and pulls back the one that matches the question, so the name returns even with a small window.
In practice
Agents usually combine all three: recent turns verbatim, a running summary of older ones, and a searchable store of facts and documents. For long tasks they also keep the plan itself in memory and update it as steps succeed or fail, so the model does not lose track of where it is.
Easy to confuse
- A long context window is not memory across sessions; once the conversation ends, nothing persists unless the application stores it.
- Retrieved memories are text the model reads again, so they can be wrong or out of date like any other input.
An agent keeps only the last 40 words of the conversation. A user told it their dog’s name 200 words ago. What happens?
A model only reads what is inside its context window. Anything that falls out is gone unless something copies it into longer-term memory.
What is the weakness of summarising old turns to save space?
A summary is lossy. If it leaves out a name or a number, that fact cannot be recovered from the summary.
Why plan before acting?
A written plan gives the agent something to follow and revisit, and makes it easy to notice when a step failed and the plan needs to change.
Read more in the books
- LangChain and LangGraph documentation, LangChain: LangChain: Short-term memory; LangGraph: Persistence and Memory.
- AI Engineering, Chip Huyen: Ch. 6, “RAG and Agents”: agents (planning) and memory.
- Artificial Intelligence: A Modern Approach (4th ed.), Russell, Norvig: Ch. 11, “Automated Planning”.