Middleware & guardrails
Middleware is how you customise an agent without rewriting its loop: code that runs before and after every model call or tool call.
Hooks around the loop
LangChain calls middleware “the primitive for customisation”. A middleware can run before the model (trim or summarise messages, inject context), after the model (check or rewrite the response), or around tool calls (retry on failure, ask a person first). Several middlewares can be stacked, and they run in order on every turn of the loop.
Built-in examples
Common built-ins include summarisation (compress history before it overflows), human-in-the-loop (pause before chosen tools), PII handling (redact or block personal data), and model and tool retries. Because the agent is a LangGraph graph, a pause is a real interrupt: the state is checkpointed and the run resumes when the person answers.
A worked example
A user writes “Refund order 7731, my email is lan@example.com”. PII middleware replaces the address with a placeholder before the model sees it. The model proposes refund(order=7731); human-in-the-loop middleware stops the run and shows the call to an operator. On approval the tool runs and the loop continues; on rejection the model is told the call was refused and answers accordingly.
Easy to confuse
- Middleware changes how the loop runs, not what the model knows; it cannot fix a bad prompt or a missing tool.
- Guardrails in middleware are deterministic code you control, unlike instructions in the prompt, which the model may ignore.
Where do before_model and after_model hooks run?
Middleware wraps the steps of the loop, so a before_model hook can edit the state (for example trim messages) on every iteration, and an after_model hook can inspect each model response.
Why does human-in-the-loop middleware need a checkpointer?
Pausing means stopping the graph mid-run. A checkpointer saves the state under the thread ID so the run can continue exactly where it stopped.
A long conversation is about to overflow the context window. Which middleware helps?
Summarisation middleware replaces older messages with a summary before they are sent to the model, keeping the request within the window.
Read more in the books
- LangChain and LangGraph documentation, LangChain: LangChain: Agents, the harness and middleware.
- AI Engineering, Chip Huyen: Ch. 5, “Prompt Engineering”: defensive prompt engineering.