Multi-Agent Systems: When and When Not

Once you have one agent working, the next idea is seductive: split the work across several agents, each an expert, all cooperating. The demos look impressive. The architecture diagrams look serious.

Most of the time, it is a mistake. This article covers the two patterns that show up, the narrow cases where they earn their cost, and the larger set of cases where one well-designed agent wins on every axis you care about.

The two patterns you will actually see

Strip away the marketing and almost every multi-agent design is one of two shapes.

Orchestrator and sub-agents. One agent owns the goal. It breaks the work into pieces and hands each piece to a specialized sub-agent: a research agent, a coding agent, a review agent. Each sub-agent has its own tools and instructions. The orchestrator collects the results and decides what happens next. Sub-agents do not talk to each other; they report up.

flowchart TB
    O[Orchestrator agent]
    O --> R[Research<br/>sub-agent]
    O --> C[Coding<br/>sub-agent]
    O --> V[Review<br/>sub-agent]
    R --> O
    C --> O
    V --> O

Parallel workers. The task splits into independent chunks that do not depend on each other. Fan them out to identical (or similar) agents running at the same time, then merge the results. Think: summarize each of 50 documents, then combine. No agent needs another agent’s output to do its job.

flowchart LR
    S[Split task] --> W1[Worker 1]
    S --> W2[Worker 2]
    S --> W3[Worker 3]
    W1 --> M[Merge results]
    W2 --> M
    W3 --> M

Everything else is a variation. Debate setups, agent “teams”, and role-playing crews are mostly the orchestrator pattern with extra chatter, and the chatter is where reliability goes to die.

When multiple agents genuinely help

Two situations make the extra machinery worth it. Both are specific.

Truly parallel, independent work. If the task splits into chunks that share no state and no ordering, running them concurrently is a real win on wall-clock time. Summarizing 50 documents is 50 independent problems. One agent doing them in a loop is slow; ten agents doing five each finish roughly ten times faster. The test: can worker B start without worker A’s output? If yes, parallelism is genuine.

Clearly separated specialties. Sometimes sub-tasks need genuinely different tools and context. A code-writing agent needs your repo, a linter, and a test runner. A web-research agent needs a browser and a search API. Giving one agent all of it means a bloated tool list and a context window stuffed with instructions for jobs it is not doing right now. Splitting them keeps each agent’s tools focused and its context clean, which reliably improves tool-calling accuracy.

Why does a focused tool list improve reliability?

The model picks tools from their descriptions. Every extra tool is another option it can pick wrong, and another paragraph of description competing for attention in the context.

An agent with 6 relevant tools chooses better than an agent with 30 tools where 24 are irrelevant to the current sub-task. This is the strongest honest argument for splitting agents: not “specialization” as a vibe, but a smaller decision space per call. If you can get the same effect by trimming one agent’s tools, do that first. It is cheaper.

Notice what both cases share: you can name the benefit before you build. “Ten times faster” or “half the tools per agent.” If you cannot name one that concrete, you are adding agents on faith.

When they do not help, which is most of the time

Here is the honest default: most tasks are simpler, cheaper, and more reliable as one well-designed agent. Multi-agent architectures are not free, and the costs are easy to miss until you are debugging one at 2 a.m.

Coordination cost. Every hand-off is a place to lose information. The orchestrator describes the sub-task in text, the sub-agent interprets that text, does work, and summarizes back in text. Each translation drops detail. A single agent keeps the full context in one conversation and never has to compress it into a hand-off.

More failure surface. One agent is one loop that can go wrong. An orchestrator with three sub-agents is four loops, each of which can loop forever, call the wrong tool, or return garbage, plus a new failure mode where the orchestrator mis-routes work or misreads a result. You did not remove the reliability problems of one-agent systems. You multiplied them.

Token spend. This one surprises people. Every sub-agent re-reads its instructions, re-establishes context, and its output gets read again by the orchestrator. The same information passes through the model several times. A multi-agent run can cost several times the tokens of a single agent doing the same task, for the same or worse output.

flowchart LR
    T[One task] --> A["Single agent:<br/>1 context, 1 loop"]
    T --> B["3 sub-agents:<br/>4 contexts, 4 loops,<br/>3 hand-offs"]

None of this means multi-agent never works. It means the burden of proof is on the second agent. It has to pay for the coordination it adds.

A decision guide

Start with one agent. Always. Get it working, get it reliable, measure it. Only then ask whether a second agent buys you something you can name.

Reach for multiple agents when you can answer yes to one of these:

Is the work genuinely parallel? Independent chunks, no shared state, no ordering. If yes, parallel workers save real time.
Do sub-tasks need different tools and context? Not “it feels cleaner”, but a concrete tool list that shrinks and a context window that stops fighting itself.

If neither is a clear yes, the answer is one agent. A messy single-agent prompt is easier to fix than a tangle of agents talking past each other.

Here is the check in code. If this function returns False, you are building one agent.

def should_use_multiple_agents(task):
    # Parallel path: independent chunks are a real win.
    if task.chunks_are_independent and len(task.chunks) > 5:
        return True

    # Specialist path: only if tool lists genuinely diverge.
    tool_sets = [set(s.tools) for s in task.subtasks]
    overlap = set.intersection(*tool_sets) if tool_sets else set()
    mostly_disjoint = all(
        len(overlap) < len(ts) / 2 for ts in tool_sets
    )
    if mostly_disjoint and len(task.subtasks) > 1:
        return True

    # Everything else: one agent.
    return False

The thresholds are judgment calls, not laws. The point is the shape: two narrow yes-paths, and a default of one agent for everything else.

Common beginner mistakes

Reaching for multi-agent first. It feels sophisticated. Start with one agent and only split when you can name the payoff.
Splitting to “specialize” without different tools. If both agents use the same tools, you added hand-offs and gained nothing. Trim the prompt instead.
Ignoring token cost. Sub-agents re-read context every call. Estimate the token bill before you commit, not after.
Chatty agents. Agents debating each other in free text is expensive and hard to debug. Prefer structured hand-offs with a clear result format.
No global step limit. Each agent needs its own cap, and the orchestrator needs one too, or a runaway sub-agent takes the whole system down.
Assuming parallel means faster. If chunks share state or ordering, “parallel” agents will block on each other and you get the cost without the speed.

Questions you will face in production

“The framework makes multi-agent easy. Why not just use it?” Easy to wire up is not the same as easy to run. The framework hides the coordination, not the coordination cost. You still pay in tokens, latency, and debugging when a hand-off drops context. Use the framework if you have already decided you need multiple agents; do not let it make the decision for you.

“How do I debug a multi-agent system that gives a wrong answer?” Trace every hand-off. Log the exact message each agent received and returned. The bug is usually in a hand-off, where the orchestrator described the sub-task poorly or misread the result, not inside a single agent’s loop. That is why hand-offs should be structured and logged, not free-form chat.

“Can sub-agents share memory or state?” They can, through a shared store, but shared mutable state between concurrent agents brings back every race condition you know from distributed systems. Prefer designs where each agent gets its inputs up front and returns its outputs cleanly. If they must share state, treat it with the care you would give any concurrent system, because that is what you have built.

What to remember

The two real patterns are an orchestrator with sub-agents, and parallel independent workers
Multiple agents help in two cases: genuinely parallel work, or sub-tasks with genuinely different tools and context
Most tasks are cheaper and more reliable as one well-designed agent
Multi-agent adds coordination cost, more failure surface, and higher token spend
Start with one agent; add a second only when you can name the specific benefit
Hand-offs are where information and reliability leak; keep them structured and logged

What to study next

That closes out the AI Agents track. The natural next step is agents applied to the tool you already live in: your editor. See how AI coding tools work, which takes the same agent loop from what an agent is and shows what changes when the tools are your file system, your terminal, and your test suite.