Hand work between LangGraph agents without corrupting shared state
+
- 1State that survives a restart
- 2Hand work between agents
- 3Governed tools an agent can call
- 🏆Engineer the context, not the prompt
You have two agents. One books appointments. One handles billing.
A ticket arrives that needs both: reschedule my install, and my bill looks wrong. So you send it to both.
The first one books Tuesday. The second one sees the account is past due and freezes it. Both finish at almost the same moment, and both save what they decided.
Only one of them is saved. Which one? Whichever finished first — which comes down to how slow an API call was that day. So you book an appointment on a frozen account, or freeze an account you just promised an engineer to. Afterwards it looks like one clean decision was made.
Part 1 built one agent that runs its steps in a fixed order. This post has several: a supervisor that picks who works on what, an agent that passes the job to another one halfway through, and two agents put in each other's way on purpose — to find out what LangGraph does when they disagree.
Verified versions: Python 3.10.12 · langgraph 1.2.11 ·
langgraph-checkpoint 4.2.0 · langgraph-checkpoint-postgres 3.1.2 ·
psycopg 3.3.4 · langchain-openai 1.6.0 · langfuse 4.14.5 against Langfuse
server v3.205.1 OSS · image postgres:17. The lg-checkpoints container and
virtualenv from Part 1 are reused unchanged.
Prerequisites
- A WEC Instance — deploy one if you have not already
- A WEC Inference API key — create one in the portal
- The Postgres container and virtualenv from Part 1
- A Langfuse instance for the last section — see Observe production with Langfuse
Someone has to decide who works on it
Two agents means something has to choose between them. That thing is called a supervisor, and it is just another agent whose only job is picking the next one.
Part 1 wired its steps together in advance with add_edge — step one, then two,
then three. A supervisor cannot do that, because it does not know where the work
is going until it reads it.
Instead the node returns a Command:
from langgraph.types import Command
def supervisor(state: State) -> Command[Literal["scheduler", "billing"]]:
nxt = "billing" if "invoice" in state["ticket"].lower() else "scheduler"
print(f"supervisor routing to {nxt}", flush=True)
return Command(goto=nxt, update={"completed": [f"supervisor->{nxt}"]})
Command does two jobs at once: goto names the next node, and update carries
a state change with it. There is no add_edge from supervisor to anything —
routing happens at runtime, from inside the function.
Save the whole thing as lg_super.py:
import sys
from operator import add
from typing import Annotated, Literal
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.postgres import PostgresSaver
from langgraph.types import Command
THREAD = sys.argv[1] if len(sys.argv) > 1 else "sup-1"
TICKET = sys.argv[2] if len(sys.argv) > 2 else "reschedule my install"
class State(TypedDict):
ticket: str
completed: Annotated[list[str], add]
def supervisor(state: State) -> Command[Literal["scheduler", "billing"]]:
nxt = "billing" if "invoice" in state["ticket"].lower() else "scheduler"
print(f"supervisor routing to {nxt}", flush=True)
return Command(goto=nxt, update={"completed": [f"supervisor->{nxt}"]})
def scheduler(state: State):
print("scheduler running", flush=True)
return {"completed": ["scheduler"]}
def billing(state: State):
print("billing running", flush=True)
return {"completed": ["billing"]}
builder = StateGraph(State)
builder.add_node("supervisor", supervisor)
builder.add_node("scheduler", scheduler)
builder.add_node("billing", billing)
builder.add_edge(START, "supervisor")
builder.add_edge("scheduler", END)
builder.add_edge("billing", END)
if "--draw" in sys.argv:
print(builder.compile().get_graph().draw_mermaid())
raise SystemExit
with PostgresSaver.from_conn_string(DB_URI) as cp:
cp.setup()
graph = builder.compile(checkpointer=cp)
config = {"configurable": {"thread_id": THREAD}}
print("FINAL:", graph.invoke({"ticket": TICKET, "completed": []}, config), flush=True)
The --draw branch prints the graph and exits before touching Postgres, so you can
inspect the topology without a database. It earns its place in the next section.
~/.venv/bin/python ~/lg_super.py sup-1 "reschedule my install"
supervisor routing to scheduler
scheduler running
FINAL: {'ticket': 'reschedule my install', 'completed': ['supervisor->scheduler', 'scheduler']}
And the other branch, to show the routing is data-dependent rather than hardcoded:
~/.venv/bin/python ~/lg_super.py sup-2 "please fix the invoice on my account"
supervisor routing to billing
billing running
FINAL: {'ticket': 'please fix the invoice on my account', 'completed': ['supervisor->billing', 'billing']}
The annotation that is not type hinting
That return annotation looks like ordinary typing you could drop:
def supervisor(state: State) -> Command[Literal["scheduler", "billing"]]:
Drop it and the code runs identically — same routing, same output. So it is optional. Except it is not.
sed 's/ -> Command\[Literal\["scheduler", "billing"\]\]//' ~/lg_super.py > ~/lg_super_noann.py
Ask both versions to draw themselves:
~/.venv/bin/python ~/lg_super.py --draw
~/.venv/bin/python ~/lg_super_noann.py --draw
With the annotation, the two possible routes are there as dotted conditional edges:
__start__ --> supervisor;
supervisor -.-> billing;
supervisor -.-> scheduler;
billing --> __end__;
scheduler --> __end__;
Without it, they vanish, and the graph states something false:
__start__ --> supervisor;
supervisor --> __end__;
scheduler and billing are still declared as nodes, and now nothing reaches
them. The diagram says the supervisor goes straight to the end.
Figure 1. Same code, one annotation apart. The lower graph is wrong.
The reason is mechanical: goto is decided inside the function body, where
nothing can inspect it. The annotation is the only declaration of where a
supervisor is allowed to send work. Omit it and your code still works while every
diagram, and anything else reading the topology, quietly lies.
draw_ascii() needs a package LangGraph does not shipget_graph().draw_ascii() raises
ImportError: Install grandalf to draw graphs: 'pip install grandalf'.
draw_mermaid() has no extra dependency and is what the examples above use.
Handing off without asking the supervisor
A supervisor decides once, at the start. That is a problem when the discovery happens later.
Take a ticket asking to reschedule an install, on an account that turns out to be past due. The supervisor sees a scheduling request and routes accordingly. The scheduler starts work, reads the account, and finds a problem billing owns.
The scheduler can route too — it is a node, and nodes return Command.
Copy lg_super.py to lg_handoff.py, replace scheduler with the version below,
and delete the builder.add_edge("scheduler", END) line — the node decides its own
exit now:
def scheduler(state: State) -> Command[Literal["billing", "__end__"]]:
if "past due" in state["ticket"].lower():
print("scheduler: account is past due, handing off to billing", flush=True)
return Command(goto="billing", update={"completed": ["scheduler(handoff)"]})
print("scheduler: booking it", flush=True)
return Command(goto=END, update={"completed": ["scheduler(booked)"]})
~/.venv/bin/python ~/lg_handoff.py handoff-3 "reschedule my install, account is past due"
~/.venv/bin/python ~/lg_handoff.py handoff-4 "reschedule my install"
supervisor -> scheduler
scheduler: account is past due, handing off to billing
billing running
FINAL: {'ticket': '...past due', 'completed': ['supervisor', 'scheduler(handoff)', 'billing']}
supervisor -> scheduler
scheduler: booking it
FINAL: {'ticket': 'reschedule my install', 'completed': ['supervisor', 'scheduler(booked)']}
Figure 2. The supervisor never knew billing would be involved.
Two details worth keeping. goto=END works from inside a Command, so a node can
finish the run itself. And "__end__" is the string form of END in that
Literal — the annotation needs the literal value, not the constant.
This is the difference between the two published patterns. A supervisor decides from the outside and needs to know every precondition up front. A handoff lets the agent doing the work redirect once it learns something. Most real systems want both, which is what this graph has.
Sending work to two agents at once
So far the supervisor picks one agent. Sometimes you want both — check the calendar
and the account at the same time, rather than waiting for one before starting the
other. goto takes a list for exactly that:
return Command(goto=["scheduler", "billing"], update={"completed": ["supervisor"]})
Both run in the same super-step. Which is where it gets interesting, because now they can disagree.
Copy lg_super.py to lg_conflict.py. Give the state two kinds of field:
class State(TypedDict):
completed: Annotated[list[str], add] # has a reducer
decision: str # no reducer
And have each agent write a different value to the un-reduced one:
def scheduler(state: State):
return {"completed": ["scheduler"], "decision": "book tuesday"}
def billing(state: State):
return {"completed": ["billing"], "decision": "refund first"}
~/.venv/bin/python ~/lg_conflict.py conflict-2
supervisor sending work to BOTH
billing deciding
scheduler deciding
Traceback (most recent call last):
...
File ".../langgraph/pregel/_loop.py", line 692, in after_tick
self.updated_channels = apply_writes(
File ".../langgraph/channels/last_value.py", line 64, in update
raise InvalidUpdateError(msg)
langgraph.errors.InvalidUpdateError: At key 'decision': Can receive only one value
per step. Use an Annotated key to handle multiple values.
Figure 3. Both agents finished. The graph refused to merge their answers.
Read the trace carefully, because three things in it matter.
Both agents ran. billing deciding and scheduler deciding both printed. The
work completed; the failure came afterwards, in after_tick → apply_writes. This
is not two agents colliding mid-flight. It is the graph refusing to reconcile them
at the step boundary.
completed was fine. Both agents appended to it in the same step and nothing
complained, because add says what two values mean. Only decision failed.
LangGraph does not pick a winner. No last-write-wins, no silent choice. Which is the right behaviour — the alternative is a booking agent that sometimes books Tuesday and sometimes issues a refund depending on which task happened to finish first.
Note the order too: billing printed before scheduler, the reverse of the goto
list. Parallel branches finish in whatever order they finish.
Stating the policy
The fix is not to prevent the disagreement. It is to say what a disagreement
means. Copy lg_conflict.py to lg_resolve.py and add the reducer:
PRIORITY = {"refund first": 2, "book tuesday": 1}
def prefer_higher_priority(old: str, new: str) -> str:
"""The policy: a billing hold outranks a scheduling decision."""
winner = max([old, new], key=lambda v: PRIORITY.get(v, 0))
print(f" reducer: '{old}' vs '{new}' -> '{winner}'", flush=True)
return winner
class State(TypedDict):
completed: Annotated[list[str], add]
decision: Annotated[str, prefer_higher_priority]
~/.venv/bin/python ~/lg_resolve.py resolve-2
reducer: '' vs '' -> ''
supervisor sending work to BOTH
billing deciding
scheduler deciding
reducer: '' vs 'refund first' -> 'refund first'
reducer: 'refund first' vs 'book tuesday' -> 'refund first'
FINAL: {'completed': ['supervisor', 'billing', 'scheduler'], 'decision': 'refund first'}
Figure 4. The print inside the reducer makes the merge visible.
Same graph, same disagreement, no error. And the trace shows three things the documentation does not spell out:
The reducer runs pairwise, folded left. Not once with both values — twice, accumulating. First it merges the existing state with billing's answer, then merges that result with scheduler's.
It must therefore be order-independent. Since branches finish in arbitrary
order, a reducer like "take the newest" produces different answers on different
runs. max by priority does not.
It fires before anything runs. That first reducer: '' vs '' -> '' is the
initial state being written, merging the default with the decision: "" passed to
invoke. Your reducer sees empty input and must tolerate it.
Crashing halfway through a parallel step
Part 1 showed a killed run resuming from its last completed node. Parallel work raises a sharper question: if one branch finishes and its sibling dies, is the finished work lost?
Copy lg_conflict.py to lg_partial.py. Drop the decision field, add
RESUME = "--resume" in sys.argv, pass None if RESUME else {...} to invoke as
in Part 1, and replace the two specialists with one instant node and one slow
enough to kill:
def fast(state: State):
print("fast running — finishes immediately", flush=True)
return {"completed": ["fast"]}
def slow(state: State):
print("slow running — 30s window, KILL ME HERE", flush=True); time.sleep(30)
return {"completed": ["slow"]}
~/.venv/bin/python ~/lg_partial.py partial-2
Press Ctrl+C during the window. It takes two presses, and the run ends on
something that looks much worse than a KeyboardInterrupt:
File ".../langgraph/pregel/_runner.py", line 613, in commit
self.put_writes()(task.id, task.writes)
...
RuntimeError: cannot schedule new futures after shutdown
That is fast's checkpoint write being committed to an executor that has already
shut down. It reads like data loss.
It is not:
~/.venv/bin/python ~/lg_partial.py partial-2 --resume
BEFORE values={'completed': ['supervisor', 'fast']} next=('slow',)
slow running — 30s window, KILL ME HERE
FINAL: {'completed': ['supervisor', 'fast', 'slow']}
Figure 5. fast recovered, did not re-run, and next names only the branch that died.
fast is in the recovered state, appears exactly once in the final result, and
next=('slow',) names only the branch that never finished. The write had already
landed in checkpoint_writes — the same table that stopped step_one re-running
in Part 1 now protects a completed sibling.
Worth documenting precisely because the error suggests the opposite. A
RuntimeError in a shutdown path is noise; the state is the thing to check.
A model as the router
Every routing decision so far has been "invoice" in ticket — keyword matching.
That is the approach we measured coming apart on
the gateway's complexity router, where a
plural is enough to miss a keyword entirely.
So let the model decide. Copy lg_super.py to lg_traced.py — same graph, one
node changed:
llm = ChatOpenAI(
model="gemma4",
base_url="https://inference.wiline.com/v1",
api_key=os.environ["WEC_API_KEY"],
temperature=0,
)
def supervisor(state: State) -> Command[Literal["scheduler", "billing"]]:
r = llm.invoke(
"Route this support ticket to exactly one team. "
"Answer with one word, either scheduler or billing, nothing else.\n\n"
f"Ticket: {state['ticket']}"
)
raw = (r.content or "").strip().lower()
nxt = "billing" if "billing" in raw else "scheduler"
print(f"supervisor: model said {raw!r} -> {nxt}", flush=True)
return Command(goto=nxt, update={"completed": [f"supervisor->{nxt}"]})
Credentials come from the environment, which the gateway's .env already holds:
set -a; . ~/llm-gateway/.env; set +a; ~/.venv/bin/python ~/lg_traced.py traced-1
The ticket is deliberately ambiguous — "my bill looks wrong and I need a new install date" mentions both:
supervisor: model said 'billing' -> billing
billing running
FINAL: {'ticket': 'my bill looks wrong and I need a new install date', 'completed': ['supervisor->billing', 'billing']}
One word, no wrapper sentence, and the parse held. It picked billing — defensible, since an unpaid account gates the install, which is the same conclusion the handoff reached earlier by reading the account.
What that one word cost
Asking a model to choose is not free, and the bill is not where you would look for
it. Turning on tracing shows where the time and the tokens went. It is one line, as
in Part 1 — CallbackHandler() reads its credentials from the environment and goes
in the same config dict as the thread id:
from langfuse.langchain import CallbackHandler
handler = CallbackHandler()
config = {"configurable": {"thread_id": THREAD}, "callbacks": [handler]}
Figure 6. Three things boxed: what routing cost, the decision as data, and the payload arriving.
The span tree is the graph:
LangGraph 8.66s
supervisor 8.53s
ChatOpenAI 8.49s 52 → 412 (Σ 464)
billing 0.01s
Three findings sit in that frame.
A one-word answer cost 412 output tokens. Confirmed against the API rather than read off the screen:
set -a; . ~/llm-gateway/.env; set +a
curl -s -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
"$LANGFUSE_HOST/api/public/observations?limit=8" \
| jq '.data[] | select(.name=="ChatOpenAI") | {model, usage}'
{
"model": "gemma4",
"usage": {
"unit": "TOKENS",
"input": 52,
"output": 412,
"total": 464
}
}
The visible answer is seven characters. The bill is 412 output tokens. Whatever the model generated before settling, you paid for — and a routing call looks like the cheapest thing in the graph.
The supervisor is the expensive node, not the specialists. billing took 6ms
and no tokens. The router took 8.49s and 464. In a multi-agent graph, the
orchestration decisions are the cost centre.
The routing decision is recorded as data. The supervisor's output in the trace
is the serialised Command:
{"graph": null, "update": {"completed": ["supervisor->billing"]}, "resume": null, "goto": "billing"}
And billing's input contains ["supervisor->billing"] — the update from that
Command arriving as the receiving agent's state. Handoff, cost and payload in one
frame.
scheduler appears nowhere in this trace — not as a skipped span, not as a
disabled node. The Graph panel draws __start__ → supervisor → billing → __end__.
You cannot tell from a trace which alternatives existed, only which one ran, which
is a second reason the missing Literal annotation matters: the drawn graph is the
only place your alternatives are written down.
What's next
Part 3: the tools an agent is allowed to call. Everything above trusts each node to do only what it should — the next post puts the tools behind a gateway with per-tool permissions, so an agent's capabilities are granted rather than assumed.
Troubleshooting
ImportError: Install grandalf to draw graphs
draw_ascii() needs a package LangGraph does not depend on. Use draw_mermaid().
The graph diagram shows the supervisor going straight to __end__
The routing node is missing its -> Command[Literal[...]] return annotation. The
code still runs; only the topology is wrong.
InvalidUpdateError: At key 'x': Can receive only one value per step
Two nodes wrote the same key in one super-step and that key has no reducer. Either
give it one with Annotated[T, fn], or stop dispatching both nodes in parallel.
RuntimeError: cannot schedule new futures after shutdown after Ctrl+C
A completed branch's checkpoint write raced the executor shutting down. Check
graph.get_state(config).values before assuming anything was lost — in testing the
write had landed every time.
The reducer produces different results on different runs
It is not order-independent. Parallel branches finish in arbitrary order, so a reducer that depends on which value arrives second is non-deterministic by construction.
Further reading
- Part 1 — checkpoint a LangGraph agent — state that survives a crash
- Route by complexity — what keyword routing misses, measured
- Component-level tracing for agent tool calls — finding which step failed