· AI
An AI Agent Fleet Is Sized in Minutes, Not Departments
Businesses keep asking which departments to hand to an agent fleet. The better question is which tasks, measured in minutes, not which org chart boxes.
Most companies scope an AI agent fleet by function. Support goes to the bots, sales stays human, and six months later someone is explaining why the support bots keep making promises the contract does not cover. The scoping question is wrong. The dividing line that actually holds up is not which department a task sits in. It is how long the task takes a skilled person to finish, and whether you can cheaply check the output when it is done.
That is not a guess. It is what the best public data on agent reliability says, and what at least one well-known company learned the expensive way.
The steelman for going broad. The case for handing over whole functions, not narrow tasks, is real. Agent capability is not static. METR, the research group that tracks how long a task an AI agent can complete, found the length of task a frontier model can handle at a 50 percent success rate has roughly doubled every seven months over six years of data. If you scope today around today's limits, the argument goes, you are leaving money on the table by the time your rollout finishes, because the ceiling will have moved again.
That argument deserves a fair hearing, because it is correct about the trend. It is wrong about what the trend fixes.
What the data says about the ceiling. METR's own numbers are the place to start. Models hit close to 100 percent success on tasks that take a skilled human under four minutes. Claude 3.7 Sonnet, the model METR used for its headline figure, crosses the 50 percent success mark at roughly one hour of human-equivalent work. Past about four hours, success drops below 10 percent. Those are not small-print caveats. They are the shape of the curve: short, well-defined tasks are close to solved, and the model's reliability falls off a cliff well before a full workday of unsupervised judgment calls.
The doubling trend is real, but it describes the 50 percent line moving outward. It does not describe why agents fail on the tasks past that line, and that matters for a business trying to scope a rollout now rather than in two years.
Why longer tasks fail, and why more model capability alone will not fix it. A 2025 UC Berkeley study, "Why Do Multi-Agent LLM Systems Fail?", annotated 1,642 execution traces across seven multi-agent frameworks and found the failures cluster into three categories: poor system design, agents misaligning with each other, and broken verification of their own work. The single most common failure mode, step repetition, showed up in 15.7 percent of traces. None of that is a raw-intelligence problem a bigger model automatically solves. It is a structural problem with handing an agent a task long enough that nobody is checking the middle of it.
The company that found this out in public. Klarna is the clearest live example. In 2024 its CEO cut customer service headcount and leaned hard on an AI agent to handle support. By May 2025, Bloomberg reported the company was hiring humans back after the CEO said the all-AI approach cost less but left customers worse served than a human would have. Klarna had handed over judgment calls, not just tickets. Compare that to HubSpot's claim that its own Customer Agent resolves 72 percent of tickets without escalation. That is HubSpot's own unverified number, and I would not cite it as proof of anything beyond HubSpot's marketing. But the shape of what it claims to resolve, bounded, templated tickets, is exactly the kind of short, checkable task METR's data says agents are already good at. Klarna's mistake was not using AI. It was sizing the handoff by department instead of by task.
The practical move. Before you hand a function to a fleet, write down the individual tasks inside it and time-box each one the way a skilled employee would do it. Anything under an hour with a clear, cheap way to check the output is a strong candidate today. Anything that runs long because it requires reading the customer's tone, weighing a judgment call, or improvising past a template belongs with a person, or needs to be broken into shorter pieces an agent can actually finish reliably. That is the scoping method we use at MojoAgents when we build out a fleet for a client: task by task against what the evidence says works, not department by department against what sounds efficient on a slide. If you are trying to draw that line for your own team, get in touch.
Sources
References used in this article. Links also appear alongside the relevant claims.
Let's make it happen.
You don't need a polished brief. A couple of lines about where you want your company to go is plenty, and we'll come back with what we'd tackle first.
We scope the work around your goals and timeline before quoting anything, so you know exactly what you're getting.