A prompt is code: version it, review it, test it

One question separates the team that plays with AI from the team that runs it in production: where does the prompt live? If the answer is "in a text field someone edits in the admin panel", the AI feature has no engineering behind it, just well-intentioned improvisation.

The prompt is the configuration that changes behaviour most

In a traditional system, configuration tunes parameters around fixed logic. In an LLM application, the prompt is the logic.

Changing one instruction sentence can alter output format, tone, refusal rate and tool use all at once. It is the most powerful and most fragile thing in the system, and it is still usually the only thing nobody versions.

The config principle from The Twelve-Factor App solves half the problem: separate config from code and never put it somewhere unauditable. The other half is accepting that a prompt is not only configuration, it is behaviour, and therefore it also needs tests.

Why order and structure matter

The Transformer architecture, introduced in the paper that made attention the central mechanism, processes the whole sequence with attention weights between positions. There is no hard separation between instruction and data: everything is context.

That explains three behaviours that look mysterious when you only look at the output:

• An instruction placed after a long block of data often has more effect than the same instruction placed before it.
• Explicit delimiters between instruction and content reduce the chance the model treats content as a command.
• A concrete example beats an adjective. "Answer concisely" is vague; two examples of a good answer define the target.

This is not superstition, it is a consequence of how context is consumed.

Techniques that survive the hype

The prompting literature grew fast and much of it aged badly. Three things still hold.

Explicit structure. Role, task, constraints, output format and success criteria, kept separate. Anthropic's prompt engineering documentation organises the work in that direction, and the gain comes less from magic words than from the clarity the structure forces on you.

Step-by-step reasoning. The chain-of-thought paper showed that asking the model to lay out intermediate steps improves multi-step tasks. The practical detail people forget: reasoning costs tokens and latency, so turn it on where the task is compositional and off where it is simple classification.

In-context examples. A few well-chosen examples, including the hard case and the case that should be refused, move the output more than any instruction rewrite.

Prompts as code, in practice

Treating prompts with code discipline means four concrete things:

1. It lives in the repository, in its own file, with history.
2. It goes through review, and the pull request describes what changed and why.
3. It has tests, meaning a set of cases with expected output that runs in CI.
4. Its version is recorded at runtime, so production logs can say which prompt produced that answer.

Item 4 is what saves the investigation. Without it you are staring at yesterday's bad answer with no idea which text produced it.

The anti-pattern of the prompt that only grows

The typical evolution of a production prompt is by accretion. Every problem found becomes one more instruction line, and six months later there is a two-thousand-word text nobody dares to touch.

The problem is not only cost per call. It is that accumulated instructions conflict, and the model starts honouring whichever is most recent, most emphatic or closest to the end of the context, in ways that are hard to predict.

The antidote is the same as code refactoring: when fix number fifteen arrives, stop adding lines and ask whether it should be in the prompt at all. Often it belongs in code validation, in routing to a different flow, or in a cheaper deterministic second pass.

Not everything should be a prompt

This deserves its own section, because it is the most expensive mistake: using a natural language instruction to guarantee something code guarantees better.

Output format is guaranteed by a schema and validation, not by a polite request. Value limits are guaranteed by checking after the answer. Data access is guaranteed by permissions, never by the sentence "do not show data from other customers".

An instruction is probabilistic guidance. Code is a guarantee.

What this means for your team

If your application's prompt is not in Git today, that is this week's task. It is cheap and it changes everything that comes after.

Next, assemble a small set of cases, ten to twenty, with expected output, and run it in CI on every prompt change. That turns "we think it improved" into evidence.

Finally, review the oldest prompt in the system looking for conflicting instructions. There are almost always two, and the strange answer nobody could explain almost always came from there.