Don't Put a Model Where an If-Statement Will Do

A user types a slash command that names exactly which system they want. Your platform takes that message, hands it to a model, and asks: which specialist should handle this?

The model will almost certainly get it right. That's not the problem. The problem is that you had a certainty and you spent a model call converting it into a probability — plus a network round trip, some tokens, and a new line in the trace that can now say something surprising.

I've built agent systems on both sides of this. An open-source deep research platform I own end to end, and an internal enterprise assistant I worked on as one implementer in a cross-functional team. The second one had the shape most of these platforms end up with:

                    user message
                         │
                         ▼
              ┌────────────────────┐
              │  supervisor agent  │   "which specialist handles this?"
              └────────────────────┘
                 │      │      │
      ┌──────────┘      │      └──────────┐
      ▼                 ▼                 ▼
 documentation      ticketing         device info
  (read only)     (read + write)      (read only)

A user says something. A supervisor agent reads it, decides which domain it belongs to, and delegates. The specialist has its own prompt, its own tools, its own permissions.

That middle box is what this article is about.

The specialists were the right call. I'd make that same decision again tomorrow. What I got wrong was reaching for a model to decide between them in cases where nothing was actually undecided.

First, the part that held up

I want to establish this before criticising anything, because "multi-agent was a mistake" is a fashionable take right now and it isn't mine.

Splitting that platform by domain was correct and it stayed correct as the system grew. Documentation retrieval and ticket writes have almost nothing in common — different schemas, different permissions, different write risk, different ways of failing. Loading all of that into one universal prompt would have meant a growing prompt nobody could reason about, and a single agent that could read your knowledge base and modify your tickets in the same breath.

I wrote about how I choose those boundaries in The Model Wasn't the Hard Part. Short version: split where context, authority, ownership, parallelism or failure genuinely need isolating. That test held up under production growth, which is the only kind of evidence I trust for an architectural rule.

So this isn't an argument against specialists. It's an argument about the thing sitting in front of them.

Routing became a reflex

Here's the pattern I fell into, and I think others might have faced similar patterns.

Once you have a supervisor that routes to specialists, routing stops being a decision and becomes the default shape of every request. New capability? Register it with the router. New surface? Point it at the router. The topology is already there and using it costs nothing to type.

What changed my mind was going and looking. I pulled roughly three months of real user queries and worked through how they actually arrived — not just what they asked for, but what each request already carried before any model saw it.

A meaningful share turned up with their destination already settled. Someone had used an explicit command. Someone had clicked a button in an interface that only does one thing. Someone was three messages into a ticket conversation and the next message was obviously about that ticket. The information needed to pick the specialist was sitting right there in the request metadata, in a variable, free.

I'd recommend that exercise to anyone running a router in production, because this is genuinely hard to guess. My assumption before looking was that routing was mostly earning its place, and that the redundant cases were a rounding error. They weren't. The distribution is also what tells you which deterministic paths are worth building — you end up writing rules for traffic you've watched, instead of inventing them for phrasings you imagined.

Running those requests through a model-routed hop buys nothing. You already knew. And you pay for it in three places.

Latency, obviously — a serial model call in front of the model call that does the actual work.

Error surface, which matters more. A router that's right 99% of the time is wrong once in a hundred requests, and it will be wrong in a way you can't reproduce, because the same input can route differently on a different day. You've taken a branch that was previously deterministic and made it a distribution.

Debuggability, which matters most and gets discussed least. When a deterministic branch misbehaves, you read it and you know. When a model-routed branch misbehaves, you're reading a trace and guessing at why a probability landed where it did. Every model in the path is a place where "why did it do that?" stops having a crisp answer.

The ladder

What I use now is an ordering, not a rule. Deterministic signals first, model last, and the model only sees what genuinely survives.

1. An explicit instruction. A command, a flag, an API parameter. The user told you. Parse the string.
2. The entry point. Which surface, which button, which conversation thread. Context you already hold.
3. A cheap deterministic check. Attachment type, pattern match, a lookup against existing state.
4. A model — for what's left.

As code, it's the least glamorous function in the system:

def select_destination(request):
    # 1. The user told us outright.
    command = parse_command(request.text)          # "/tickets", "/docs", ...
    if command:
        return command.destination

    # 2. The entry point already narrows it.
    if request.surface.is_single_purpose:          # a button that does one thing
        return request.surface.destination
    if request.thread and request.thread.destination:
        return request.thread.destination          # already mid-conversation

    # 3. A cheap check on what we can see.
    if request.attachments and all_images(request.attachments):
        return Destination.VISION

    # 4. Genuinely ambiguous. Now a model call is worth what it costs.
    return None

Returning None isn't a failure — it's the handoff. The caller sees it, exposes the routing tool to the agent, and lets the model decide. Everything above it is a decision the application was already in a position to make.

Research-AI, my open-source system, has the same ladder in public code, because it can hand off from ordinary chat to a much heavier multi-step research pipeline and that handoff is expensive enough to be worth getting right:

# 1. Explicit command, parsed straight from the raw string.
is_research_command, topic = parse_research_command(raw_user_input)

# 2. Explicit flag from the UI.
force_research = bool(request_body.force_research) or is_research_command

# 3. Neither? Only then does the model get to choose — by being handed
#    the handoff as a tool it may or may not call.
tool_list = [*base_tools]
if allow_research_handoff:
    tool_list.append(handoff_to_research_graph)

Two things worth pulling out of that. The first two checks are a string parse and a boolean — no model, no network, no tokens. And allow_research_handoff is a separate gate on whether the tool is exposed at all, because in some contexts the answer is already determined and the model shouldn't be offered a choice it isn't permitted to make. Not exposing an option is a stronger control than instructing a model not to pick it.

That's the whole idea. The model isn't doing routing. It's handling the residue after the cheap deterministic signals have taken everything they can — the case where a user typed a sentence that might or might not warrant a twenty-minute research run, which is a real judgement call and exactly where I want a model.

The cost of being wrong isn't uniform

A thing I didn't think about carefully enough early on: routing errors have wildly different consequences depending on what's behind the door.

Route to the wrong read-only specialist and you waste a few seconds and produce an unhelpful answer. Annoying, recoverable, the user rephrases.

Route to a specialist that can write — create a ticket, modify a record, message a person — and a misclassification isn't a bad answer, it's an action taken in the wrong system on the user's behalf. Recoverable only if somebody notices.

So the bar for letting a model make the choice should scale with what's behind the choice. I'm relaxed about a model picking between three retrieval sources. I want a lot more determinism in front of anything that writes. That's the same instinct as enforcing authority in code rather than in a prompt — the model can propose the route, but the further the consequences reach, the more of that decision I want made by something I can read.

Narrowing beats routing

The most useful move wasn't making the router smarter. It was removing decisions from it.

On that platform, ticketing eventually got its own dedicated assistant rather than being one destination among many behind the general path. That sounds like a small organisational change and it isn't. When a domain has enough traffic, enough distinct behaviour and enough write risk, giving it its own front door means the routing decision for that domain stops existing. You haven't improved the classification — you've deleted it.

This is generally the better lever. Improving a router is a tuning problem with diminishing returns and no end state. Removing a routing decision is a structural change you only have to make once, and it's testable.

The rule I'd give someone starting out: if you find yourself repeatedly tuning a router's prompt to stop it misrouting one particular category, that category probably wants its own entry point, not better instructions.

Specialists should be able to lose their jobs

One more thing that platform taught me, and it's slightly uncomfortable.

We added a specialist for a data-warehouse domain during a period of expansion. It was later removed. Nothing dramatic went wrong — it just didn't earn the operational surface it cost, and someone was right to take it out.

That's worth sitting with, because architecture that makes something easy will get more of it. Once the specialist pattern exists, adding another specialist is the path of least resistance for every new capability, and each one brings a prompt, a deployment, a set of tools, a failure mode and a new arm on the router. They accumulate quietly.

A boundary should have to keep justifying itself. If a specialist doesn't have a stable domain contract and a real user need behind it, it's not an architecture, it's a directory.

Where the model calls actually went

I should be honest that removing a routing hop is a small win in isolation. If the rest of the request is a chain of sequential model calls, you've optimised the wrong thing.

Two changes on that platform mattered more than routing did.

Independent tool calls started running concurrently rather than in sequence. When a model asks for three lookups in one turn and none of them depends on another's result, awaiting them one at a time is latency you're choosing to accept for nothing:

# sequential: total time is the sum
for call in tool_calls:
    results.append(await execute(call))

# concurrent: total time is the slowest one
results = await asyncio.gather(*(execute(call) for call in tool_calls))

The catch is in the word independent. This is only safe when the calls have no ordering or side-effect dependency between them — three reads, fine; a read that determines a write, not fine. Working out which calls qualify is the actual work; the gather is the easy part.

And model selection — a fast tier versus a stronger one — got resolved from stored configuration rather than decided at runtime. It's worth saying out loud how easy it is to end up using a model call to choose which model to call. If the task type is already known, that's a lookup.

Both are the same instinct as the routing point: work out what you already know before you ask anything to think about it.

What I can't tell you

Two limits, and the first is the one I care most about getting right.

I don't have independently verified numbers. The query analysis told me where the deterministic paths belonged. It is not a measured before-and-after, and I want to keep those two claims apart. I can tell you the architecture removed model calls from paths where the destination was already determined, and that this necessarily removes latency and a failure mode. I can't give you a defensible figure for what that was worth in production, and I'd rather say so than quote one I can't stand behind. Treat this as an architectural argument, not a benchmark.

Deterministic routing is brittle, and that's the actual trade. A command-based path handles exactly the phrasings you anticipated. Real users don't cooperate, and you will grow a long tail of requests the rules don't cover. If you push determinism too far you end up maintaining an intent-classification system out of regular expressions, which is a worse job than the one you were avoiding. The ladder exists precisely so the tail has somewhere to go: rules take what's unambiguous, the model takes what's left. Deriving those rules from observed traffic rather than from imagination helps a great deal, but it only ever covers what you've already seen — new surfaces and new phrasings keep arriving, and a rule set nobody revisits ages badly. Getting that boundary wrong in the other direction is entirely possible.

The rule

A model in your request path is a decision you've chosen to make probabilistically. Sometimes that's exactly right — natural language is ambiguous, users are vague, and no rule you write will cover the space of ways someone can ask for something. That ambiguity is what models are for.

But you should be spending that on real ambiguity, not on decisions you'd already made.

Before adding a model to a path, the question worth asking is simply: what does this model know that my application doesn't already know? If the honest answer is "nothing, it's just where the architecture puts things" — that's an if-statement. It'll be faster, cheaper, testable, traceable, and right every single time.