LLM Lab

Middleware & guardrails

Middleware is how you customise an agent without rewriting its loop: code that runs before and after every model call or tool call.

Where this sits in a LangChain agent
Model→Tools→Agent loop (create_agent)→Middleware→Memory (checkpointer)→Integrations (MCP)→Guardrails

Hooks around the loop

LangChain calls middleware “the primitive for customisation”. A middleware can run before the model (trim or summarise messages, inject context), after the model (check or rewrite the response), or around tool calls (retry on failure, ask a person first). Several middlewares can be stacked, and they run in order on every turn of the loop.

before_model→model call→after_model→tools→back to before_model
Where middleware hooks in. The loop is the same as before; middleware runs at fixed points around each model call and tool call, on every turn of the loop.

Built-in examples

Common built-ins include summarisation (compress history before it overflows), human-in-the-loop (pause before chosen tools), PII handling (redact or block personal data), and model and tool retries. Because the agent is a LangGraph graph, a pause is a real interrupt: the state is checkpointed and the run resumes when the person answers.

tool callsnext turnno callsbefore_modeltrim · summarise · redactmodel callafter_modelcheck the replytool callapproval / retryaround tool callsanswer
middleware hookmodeltool
Middleware hooks (purple) sit at fixed points of every turn: before and after each model call, and around tool calls. The loop itself does not change.

A worked example

A user writes “Refund order 7731, my email is lan@example.com”. PII middleware replaces the address with a placeholder before the model sees it. The model proposes refund(order=7731); human-in-the-loop middleware stops the run and shows the call to an operator. On approval the tool runs and the loop continues; on rejection the model is told the call was refused and answers accordingly.

Easy to confuse

Check yourself · 1 of 3

Where do before_model and after_model hooks run?

Check yourself · 2 of 3

Why does human-in-the-loop middleware need a checkpointer?

Check yourself · 3 of 3

A long conversation is about to overflow the context window. Which middleware helps?

Read more in the books

  • LangChain and LangGraph documentation, LangChain: LangChain: Agents, the harness and middleware.
  • AI Engineering, Chip Huyen: Ch. 5, “Prompt Engineering”: defensive prompt engineering.
All books
Labhook into the loop
How to read this
  • Tick the middleware to add. The stage map shows the loop; lit stages are where a ticked middleware acts.
  • Run the request: the trace lists each step of one agent run and marks what each middleware did. Human approval pauses the run until you press Approve or Reject.