AI agents in delivery: where they pay off and where they do not

"AI agent" has become an umbrella term for very different things, and that imprecision costs money. Teams build complex orchestration for problems a fixed pipeline would solve better, cheaper, and with less chance of failing at 3am.

A workflow and an agent are not the same thing

The distinction Anthropic proposes in its agent engineering work is the most useful one available today:

• Workflow: the LLM runs inside predefined code paths. You decide the sequence; the model fills in the steps.
• Agent: the LLM directs its own path, picks its tools, and decides when it is done.

Workflows are predictable, cheap to debug and easy to test. Agents are flexible and expensive, in tokens, in latency, and in how hard they are to investigate when they go wrong.

The recommendation that comes with the distinction is blunt: use the simplest thing that works, and add autonomy only when the problem genuinely demands it.

Start with the workflow, almost always

Most software delivery tasks have a known sequence. Generating release notes from commits, classifying and routing a ticket, updating a changelog, translating documentation. None of that needs an agent choosing strategy: it needs a chain with validation between steps.

The rule of thumb: if you can draw the flowchart in advance, it is a workflow. If the path depends on what gets discovered along the way, then it is a candidate for an agent.

The patterns that actually show up

Four recur in practically every serious project:

1. Chaining. One call's output becomes the next call's input, with a programmatic check between them. The check is what stops an error from propagating.
2. Routing. A cheap call classifies, and each class goes to a specialised prompt or model. Saves money and improves quality at the same time.
3. Parallelisation. Running the same analysis from different angles and aggregating, or splitting independent work. Lowers perceived latency.
4. Evaluator and optimiser. One generates, another critiques against explicit criteria, iterating a few times. Works well when there is an objective criterion, and becomes an expensive infinite loop when there is not.

Tools are the part that decides the outcome

An agent is only as good as the tools you give it and how well they are described. A badly named tool, with an ambiguous parameter or an unhelpful error message, produces an agent that tries the same thing five times.

It is worth treating tool definitions with the care of a public API: names that say what they do, parameters with types and descriptions, errors that explain how to recover. The Model Context Protocol standardises exactly that connective layer between application, context and tools, which reduces coupling between your agent and any single vendor.

What benchmarks say about reality

It is worth calibrating expectations against SWE-bench, which evaluates models on real issues from real repositories instead of synthetic exercises. When it was published, it exposed a wide gap between performance on demo tasks and performance on production code with history, dependencies and implicit context.

The numbers have improved a lot since, but the structural lesson holds: well-bounded, verifiable tasks go well; tasks that require understanding the intent behind an entire system go badly. That should guide where you point an agent first.

Where agents pay off in the delivery cycle

The cases with the best risk-to-return ratio share three traits: machine-verifiable results, limited scope, and a low cost of being wrong.

Repetitive mechanical migrations with tests covering the behaviour. Triaging CI failures by correlating logs and commits. Generating tests for legacy code with no coverage, where the test passing is the objective criterion. Investigating a vulnerable dependency, with the proposed upgrade validated by the suite.

What does not pay off today: letting an agent open PRs straight into production without review, or work unsupervised on core code that has no test coverage.

What this means for your team

Pick one repetitive task with a machine-verifiable success criterion and implement it as a workflow first. Measure time saved and hit rate for two or three weeks.

Promote it to an agent only if the fixed pipeline is failing because it cannot anticipate the paths. And before that, invest in the tools and the logging: without a trace of which tool was called, with which arguments and with what result, you will not be able to debug or improve anything.