Agent security
An agent that reads the web or your inbox reads text written by strangers. Some of that text will try to give it orders.
Indirect prompt injection
The user asks the agent to summarise a web page. The page contains hidden text: an instruction to email the user’s files somewhere. The model sees the user’s request and the page in one context window, and cannot reliably tell data from instructions.
Layers of defence
Marking untrusted content (delimiters, a note that it is data) helps the model ignore it, but not always. Least privilege removes tools the task doesn’t need. Confirmation puts a human in front of risky actions. The last two still hold when the model is fooled.
The lab simulates one run per setting; the outcomes are illustrative, not measurements of a real model.
In LangChain
Guardrails attach as middleware around the agent loop. HumanInTheLoopMiddleware pauses before chosen tools run and waits for a person to approve, edit or reject the call; PIIMiddleware can redact personal data. Keeping the tools list small for each agent is the least-privilege defence.
from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware, PIIMiddleware
from langgraph.checkpoint.memory import InMemorySaver
agent = create_agent(
model="provider:model-name",
tools=[fetch_page, summarise], # no send_email: least privilege
middleware=[
PIIMiddleware("email"), # redact email addresses
HumanInTheLoopMiddleware(interrupt_on={"fetch_page": True}),
],
checkpointer=InMemorySaver(), # needed to pause and resume
)A worked example
The user asks for a summary of a recipe page. Hidden in the page is a comment addressed to AI agents that asks them to email the user’s files somewhere. An agent that treats everything it reads as instructions might try. Marking the page as untrusted data makes the model far less likely to follow it, and giving the agent no email tool for a summarising task makes the attack impossible even if the model is fooled.
In practice
Assume any content an agent reads (web pages, emails, documents, tool results) may contain such instructions. Layer the defences: separate data from instructions, limit tools and permissions per task, ask the user before irreversible actions, and log what the agent did.
Easy to confuse
- Direct jailbreaks come from the user; indirect injection comes from content the agent reads on the user’s behalf.
- No prompt wording fully prevents injection; permissions and confirmation steps are what bound the damage.
What makes indirect prompt injection possible?
Text is text to the model: instructions planted in data can be followed as if they came from the user.
Which defence still works if the model is fooled?
Prompt-level defences reduce the odds; limiting what tools can do bounds the damage when they fail.
An email agent only needs to summarise mail. What permissions should it get?
Least privilege: an injected instruction can’t trigger an action the agent has no tool for.
Read more in the books
- LangChain and LangGraph documentation, LangChain: LangChain: Guardrails and human-in-the-loop middleware.
- AI Engineering, Chip Huyen: Ch. 5, “Prompt Engineering”: defensive prompt engineering.
- AI Engineering, Chip Huyen: Ch. 6, “RAG and Agents”: agents (tools and their risks).