LLM Lab

Memory: threads & checkpoints

An agent has to remember what happened and decide what to do next. Both fit inside a limited context window.

Where this sits in a LangChain agent
Model→Tools→Agent loop (create_agent)→Middleware→Memory (checkpointer)→Integrations (MCP)→Guardrails

Planning

For a long task, an agent first writes a plan: a short list of steps. It carries the plan out one step at a time and revises it when a step fails. Planning turns one large, vague goal into small steps that can each be checked.

Memory

Short-term memory is whatever sits in the context window. It is limited, so something must decide what stays. Common choices: drop the oldest turns, summarise them, or store facts outside the window and retrieve the relevant ones when needed.

user-42checkpoint 1checkpoint 2checkpoint 3latestuser-7checkpoint 1checkpoint 2latest
Two threads in one checkpointer. Each thread_id has its own history of checkpoints; a call with “user-42” resumes from that thread’s latest state and never sees “user-7”.

In the lab, a user mentions a fact early in a conversation and asks about it at the end. Change the window size and the memory strategy, then see whether the agent can still answer.

In LangChain

Short-term memory is the message history of one conversation, saved by a checkpointer and keyed by a thread_id. Reusing a thread continues the conversation; a new thread starts empty. Middleware can trim or summarise long histories before each model call, and facts meant to outlive a conversation belong in a separate long-term store.

from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(model="provider:model-name", tools=[], checkpointer=InMemorySaver())
config = {"configurable": {"thread_id": "user-42"}}

agent.invoke({"messages": [{"role": "user", "content": "My dog is called Biscuit."}]}, config)
agent.invoke({"messages": [{"role": "user", "content": "What is my dog called?"}]}, config)
# same thread_id, so the second call sees the first message
Memory in LangChain is per thread. Swap InMemorySaver for a database-backed checkpointer in production so threads survive restarts.

A worked example

With a 40-word window and the oldest turns dropped, the first turn about the dog no longer fits, so the agent cannot answer “What is my dog’s name?”. Summarising keeps the topics (a trip, a dog-friendly hotel) but loses the name. Retrieval stores every turn and pulls back the one that matches the question, so the name returns even with a small window.

dog is Biscuit trip to Lisbon weather? train, not plane hotels? takes dogs first train?
With a small window, only the last few turns (coloured) fit; the fact about the dog has fallen out (grey). Summaries or retrieval from a memory store are the ways to bring such facts back.

In practice

Agents usually combine all three: recent turns verbatim, a running summary of older ones, and a searchable store of facts and documents. For long tasks they also keep the plan itself in memory and update it as steps succeed or fail, so the model does not lose track of where it is.

Easy to confuse

Check yourself · 1 of 3

An agent keeps only the last 40 words of the conversation. A user told it their dog’s name 200 words ago. What happens?

Check yourself · 2 of 3

What is the weakness of summarising old turns to save space?

Check yourself · 3 of 3

Why plan before acting?

Read more in the books

  • LangChain and LangGraph documentation, LangChain: LangChain: Short-term memory; LangGraph: Persistence and Memory.
  • AI Engineering, Chip Huyen: Ch. 6, “RAG and Agents”: agents (planning) and memory.
  • Artificial Intelligence: A Modern Approach (4th ed.), Russell, Norvig: Ch. 11, “Automated Planning”.
All books
Labmemory inspector
How to read this
  • The numbered list is the conversation so far; the user’s dog is mentioned in the first turn and asked about at the end.
  • Change the context window and the memory strategy. The box shows exactly what the model reads; the answer below shows whether it can still recall the fact.
    Memory strategy