LLM Lab

Parallel work with Send

Some work splits into many independent pieces whose number you only know at run time. Send runs one branch per piece, in parallel, then gathers the results.

Where this sits in a LangGraph application
State→Nodes→Edges & routing→Parallel (Send)→Checkpoints & interrupts→Several agents

Map, then reduce

In map-reduce, a list of items is processed item by item (map) and the results are combined (reduce). LangGraph’s Send does the map: a routing function returns one Send(node, input) per item, and LangGraph runs that node once for each, in the same step, each with its own input. The results land in a state key whose reducer concatenates them; a final node does the reduce.

SendSendSTARTsummarisedoc 1summarisedoc 2summarisedoc Ncombinesummaries (operator.add)END
Map-reduce with Send: one summarise branch per document, all in the same step, their outputs merged by the reducer before combine runs.
import operator
from typing import Annotated, TypedDict
from langgraph.types import Send

class State(TypedDict):
    docs: list[str]
    summaries: Annotated[list[str], operator.add]   # parallel writes are concatenated

def fan_out(state: State):
    return [Send("summarise", {"doc": d}) for d in state["docs"]]

def summarise(item: dict) -> dict:
    return {"summaries": [llm_summary(item["doc"])]}

builder.add_conditional_edges(START, fan_out, ["summarise"])
builder.add_edge("summarise", "combine")
builder.add_edge("combine", END)
Summarise any number of documents in parallel, then combine. The number of branches comes from the state at run time.

A worked example

Eight retrieved documents each take about 2 seconds to summarise. One by one that is 16 seconds; with Send all eight run in one step, so the map takes about as long as the slowest call, around 2 seconds, and the combine node then writes one answer from eight summaries. The same structure serves orchestrator-workers: a planner node emits one Send per subtask it invented.

In practice

Parallel branches multiply load: API rate limits and cost grow with the number of branches, so cap the fan-out. Keep each branch’s input small and self-contained, and make the reduce step robust to a branch that failed or returned nothing.

Easy to confuse

Check yourself · 1 of 3

What does returning [Send("summarise", {"doc": d}) for d in docs] from a conditional edge do?

Check yourself · 2 of 3

Why does the summaries key need a reducer such as operator.add?

Check yourself · 3 of 3

Ten documents take 2 s each to summarise. Roughly how long does the fan-out step take with Send?

Read more in the books

  • LangChain and LangGraph documentation, LangChain: LangGraph: Graph API (Send and map-reduce).
All books
Labfan out and gather
How to read this
  • Choose how many documents arrive. Each column is one parallel branch started by Send; Run fills them in together.
  • The combine node receives every summary through the reducer. The tiles compare the time with parallel branches against doing the same work one by one.