When Software Stopped Being a Recipe
The problem, stated plainly: software engineering trained us to make behaviour repeatable, then handed us a component whose useful output has to be judged across a distribution.
That is not traditional development with a faster autocomplete. It changes what the important artifact is, what verification means, and what an engineer has to be good at.
Stack Overflow's write-up of the 2025 Developer Survey names this directly: ask the same question twice and you can get two plausible answers with different structures and trade-offs. I felt the difference most clearly while reviewing an agent pipeline. Nothing had crashed. The same goal, tools, and context were still there. But one run chose a different route and produced a different implementation from the run before it.
Both answers looked reasonable.
That was the uncomfortable part.
With conventional code, we aim for repeatable behaviour under controlled state. Production systems have never been perfectly deterministic. They have clocks, networks, concurrency, randomness, and dependencies that change underneath them. But the implementation still gives us a concrete thing to inspect. We can trace a branch, reproduce a state, and assert an invariant.
LLM sampling adds a more visible kind of variability. Hosted model updates add a separate versioning risk. The same intent can produce several plausible outputs, and correctness is no longer just "did this line execute as expected?" It is also "does this result satisfy the goal, the constraints, and the standard we agreed on?"
The numbers reflect that tension. Stack Overflow's write-up of its 2025 Developer Survey says 84% of respondents use or plan to use AI tools, up from 76% the year before, while trust in AI accuracy fell to 29%. Their sharpest line is still the simplest:
"It's what makes the field software engineering rather than software hoping-for-the-best."
Adoption is rising. Trust is falling. That is not a contradiction. It is a profession discovering that a useful new component plays by different rules.
Three shifts inside that diagram changed how I build, and one thing carries all three.
The scarce artifact moved
When code generation gets cheap, describing the right system becomes the expensive part.
I used to think of a prompt as the thing that kicked off the work. Now I think of the specification as the thing that makes the work governable. A useful spec has a goal, boundaries, examples, non-goals, acceptance evidence, and a stopping condition. It says what the agent may touch, what it must not touch, and what would prove the task is complete.
That sounds like requirements engineering because it is requirements engineering. The difference is that the spec is no longer a document people read before writing code. It is executable context for a system that can turn intent into implementation.
The tooling is catching up. GitHub Spec Kit and OpenSpec move work from a loose prompt toward durable specifications, plans, tasks, and implementation. Tessl approaches the same pressure differently, through versioned agent skills, evals, governance, and evidence.
I learned this the boring way while building Agentry. The useful part of an agent contract was never the sentence describing the goal. It was everything around it:
- the boundary that prevented a helpful agent from touching the wrong files;
- the evidence requirement that stopped a confident summary from standing in for a review;
- the budget and stopping condition that turned a runaway loop into a bounded loss;
- the identity and approval rules that made every action attributable.
Each field felt like paperwork until a run needed it. Then it became the receipt.
I wrote about the same lesson from the autonomy side in You Don't Pick the Autonomy Level. Your Evidence Does.. The short version is that reach is earned by proof. A larger model, a longer context window, or more agents do not change that.
Tests did not disappear
The lazy version of this argument says tests are obsolete and evals replace them.
That is wrong.
Tests assert specific invariants. They are still the best tool for questions such as:
- Did the parser reject malformed input?
- Did the permission check deny an unauthorized call?
- Did the API preserve the contract?
- Did the retry stop at the configured limit?
Evals answer a different class of question:
- Did the answer satisfy the user's intent?
- Did it stay grounded in the supplied evidence?
- Did it choose the right tool?
- Did it avoid a known failure mode across a representative set of cases?
- How often did quality fall below an acceptable threshold?
A test can tell me the citation field exists. An eval has to tell me whether the citation supports the sentence.
That distinction matters because a green suite can still be looking in the wrong place. I have shipped code where every test passed because the doubles never had the shape of the real primitive. I wrote about that in Green, Reviewed, and Still Wrong. The lesson carries directly into AI systems: verification is only as good as the cases, evidence, and failure shapes it covers.
The hard part is that evals do not settle the question forever. The task distribution changes. Users find new edge cases. Grounding sources move. Providers update hosted models. A threshold that looked sensible in a lab can fail under production pressure.
So the eval suite is not a graduation exam. It is a living description of what failure looks like.
The engineer became the director and the verifier
Microsoft's 2026 Work Trend Index frames the shift as expanded human agency: as agents take on more execution, the premium moves toward judgment, quality control, and redesigning work across people and agents. Anthropic's usage data shows a similar direction from the model side. In its September 2025 Economic Index, directive conversations on Claude.ai rose from 27% to 39% in eight months, while 77% of sampled first-party API business usage showed automation patterns.
The numbers are less interesting than the direction. People are delegating larger units of work.
That can look like the engineer is disappearing from the process. In practice, the job moves to the places where judgment is hardest to automate:
- deciding what deserves delegation;
- writing the contract the system has to satisfy;
- choosing evidence that does not depend on the agent's own confidence;
- designing the rollback before increasing autonomy;
- recognizing when a plausible result is still the wrong result.
The last one is where most of my mistakes have lived.
In Certainty Is the Bug, I described a review finding that sounded precise and urgent. It was also wrong about the failure it claimed. Then two of my own fixes were wrong in different ways. The review was useful, the tests were green, and none of that made the claims true until I reproduced them.
That is the new centre of the craft for me. Not generating more. Building a system in which claims have to arrive with evidence.
The harness is the product
The model gets the attention because it produces the visible result. The reliability lives around it.
The harness decides:
- what context enters the run;
- which tools are available;
- what identity and permissions the run receives;
- which deterministic checks must pass;
- which evals measure quality;
- which actions require approval;
- what gets logged;
- when the system stops;
- how a failed action is reversed.
Prompt engineering matters inside that system, but it is only one control surface. A beautifully phrased prompt cannot compensate for a missing permission boundary, an eval set that rewards the wrong thing, or an audit trail that records only the final summary.
This is why "coding is dead" and "nothing changed, it is just autocomplete" are both unhelpful. The craft did not vanish, and it did not stay put. It moved up a layer.
We still need people who understand code. We also need people who can specify intent, model risk, design evidence, and direct systems that will not produce the same implementation every time.
The field map
- The spec is part of the runtime. Goal, boundaries, examples, non-goals, evidence, and stopping conditions are not planning decoration. They shape what the agent can produce.
- Tests and evals answer different questions. Tests assert invariants. Evals estimate quality across representative cases. AI systems need both.
- A green result is evidence, not a verdict. If the cases miss the real failure shape, the system can be confidently wrong.
- The harness carries reliability. Context, permissions, checks, telemetry, rollback, and audit matter more than one clever prompt.
- The engineer's job moved toward judgment. Delegation increases the value of deciding what correct looks like and proving that the system met it.
Software did not stop needing engineers when it stopped behaving like a recipe.
It started asking us to become better at the parts we used to leave implicit. 🧭