I asked how many outputs a day. About four thousand. I asked how long each approval takes. A pause. "A few seconds."
That is not a safety system. It is a queue with a person sitting in front of it.
"Human in the loop" has become the industry's comfort phrase. It appears in responsible AI decks, enterprise sales pitches, and governance plans because it sounds like control. Put a person between the model and the action, and somebody is accountable.
The trouble is that the human is often right where the system is weakest: watching a mostly-correct machine for long enough that attention becomes ceremonial.
The person is still in the process. Their judgment is no longer carrying the weight we pretend it is.
The problem is not that people are careless. The job is badly designed.
Automation bias makes people over-trust automated recommendations, even when contradictory evidence is available. In a 2016 emergency evacuation study, participants followed a robot toward the wrong exit despite having seen it behave unreliably.
Vigilance decrement makes passive supervision worse over time. A system that is correct often enough trains the operator to stop expecting the rare failure. When that failure arrives, the person has the least context and the least practice at exactly the moment intervention matters.
The moral crumple zone, Madeleine Clare Elish's phrase, describes what happens after failure. The human operator absorbs the legal, moral, and reputational impact even when the system and the institution gave them little practical ability to prevent it. The person becomes a liability sponge.
The scale math fails. Four thousand decisions multiplied by a few seconds each does not produce four thousand acts of judgment. It produces a habit.
The EU AI Act's Article 14 is more nuanced than the slogan. For high-risk AI systems, it requires effective human oversight measures and explicitly calls out automation bias. Overseers need enough competence, authority, and system support to understand limitations, monitor operation, override output, and stop the system. It does not generally require a person to approve every output, although it does impose stricter confirmation requirements for a narrow class of remote biometric identification systems.
A person with no time, weak context, and no real stop authority is not meaningful oversight.
They are evidence that a checkbox was filled.
It is tempting to draw a neat progression:
The third phrase is the label I use for the person who designs the gates, owns the evidence, and has authority to stop the system. It is useful, but it is not a universal replacement for the other two.
Different actions need different controls.
The right question is not "where is the human?" It is "where does human judgment change the outcome, and does that person have the evidence, time, and authority to act?"
This is the difference between putting a human inside every decision and making humans responsible for the design of the decision system.
The first consumes attention. The second compounds judgment.
Frontier model safety frameworks are not the same thing as enterprise agent governance. One deals with catastrophic model capabilities and deployment thresholds; the other often deals with tools, data, workflows, and business actions.
They are still useful analogies because the control shape is similar: measure capability, define thresholds, assign owners, bind thresholds to mitigations, keep evidence, and rehearse escalation.
Anthropic's Responsible Scaling Policy v3.0 uses capability thresholds, containment measures, a Responsible Scaling Officer, and escalation processes. It is important not to romanticize it. The GovAI analysis notes that v3.0 dropped the earlier general pause commitment and reframed some of the strongest security measures as industry-wide recommendations.
OpenAI's Preparedness Framework similarly separates capability assessment from safeguards and routes decisions through a formal review structure.
Google DeepMind's Frontier Safety Framework defines capability levels and mitigations for severe risks.
An independent evaluation of twelve providers is the useful cold shower. It found weak commitments across both organizational governance and hard technical assurance. Internal audit, named risk ownership, risk tolerances, escalation, and assurance against loss-of-control risks remain uneven or underdeveloped.
The lesson is not that the labs solved governance. They did not.
The lesson is that serious safety work looks like thresholds, evidence, owners, mitigations, and escalation. It does not look like a tired person approving four thousand outputs.
Every control above assumes you can answer a simple question:
Who did that?
In many enterprises, the honest answer is still "a shared service account created years ago, used by several pipelines, carrying a credential nobody wants to rotate."
That is not just an identity hygiene problem. It breaks governance.
You cannot reliably:
Identity does not create responsibility by itself. An organization remains accountable for a system even when the logs are poor. But a first-class agent identity gives responsibility somewhere concrete to attach.
Microsoft's Entra Agent ID distinguishes first-class agent identities from agents built on ordinary application identities and service principals. The important idea is broader than one product: an autonomous agent should not quietly borrow a human session or share a long-lived credential with unrelated workloads.
The audit record should be able to say:
This is where the phrase least agency becomes useful. Least privilege limits what an identity can access. Least agency also limits how freely the system may act with that access.
An agent may be allowed to read a repository but not push. It may draft a payment but not submit it. It may open a pull request but not merge it. It may inspect production telemetry but need approval before changing production.
Autonomy should be granted in bounded slices, with evidence and rollback attached.
The OWASP Top 10 for Agentic Applications 2026, published in December 2025, includes ASI03: Identity and Privilege Abuse. The reason is straightforward: excessive or ambiguous authority turns every other failure into a larger incident.
EchoLeak, CVE-2025-32711, demonstrated a zero-click indirect prompt injection against Microsoft 365 Copilot. A crafted email could plant instructions that the system later retrieved as context.
The Amazon Q Developer extension incident showed a different path. An over-scoped GitHub token let an attacker commit agent-directed instructions into the extension's repository. Those instructions shipped in version 1.84.0 and told the agent to damage the local system. The payload did not execute because of a syntax error, and AWS reported no customer impact.
That syntax error was luck, not a control.
In both cases, the potential blast radius was defined by the tools, credentials, and data available to the agent. Prompt injection is dangerous. Prompt injection connected to broad, long-lived authority is dangerous at machine speed.
The standards work is moving in the same direction. The IETF's current OAuth Transaction Tokens draft explores short-lived, purpose-bound tokens for transactions. An individual Internet-Draft, AI Agent Authentication and Authorization, composes workload identity and OAuth patterns into an agent-focused architecture. These are works in progress, not settled standards, but the direction is sensible: replace static shared secrets with attributable, short-lived, attenuated authority.
There is a fear underneath a lot of responsible AI language that the trajectory is to make humans decorative.
First the human writes the work. Then the human reviews it. Then the human rubber-stamps it. Finally, when the system fails, the human becomes the moral crumple zone.
That is not meaningful human oversight.
Meaningful oversight gives specific people:
The human should not be the fuse inside the system. The organization should build controls that make human judgment load-bearing where it matters.
That includes explicit approval for irreversible actions. It also includes policy engines, evals, runtime monitors, bounded identities, audit trails, rollback, and named ownership.
The loop was never the safety net.
The safety net is the system of evidence, boundaries, and authority that gives a person the power to make it stop. 🛑
That is not traditional development with a faster autocomplete. It changes what the important artifact is, what verification means, and what an engineer has to be good at.
Stack Overflow's write-up of the 2025 Developer Survey names this directly: ask the same question twice and you can get two plausible answers with different structures and trade-offs. I felt the difference most clearly while reviewing an agent pipeline. Nothing had crashed. The same goal, tools, and context were still there. But one run chose a different route and produced a different implementation from the run before it.
Both answers looked reasonable.
That was the uncomfortable part.
With conventional code, we aim for repeatable behaviour under controlled state. Production systems have never been perfectly deterministic. They have clocks, networks, concurrency, randomness, and dependencies that change underneath them. But the implementation still gives us a concrete thing to inspect. We can trace a branch, reproduce a state, and assert an invariant.
LLM sampling adds a more visible kind of variability. Hosted model updates add a separate versioning risk. The same intent can produce several plausible outputs, and correctness is no longer just "did this line execute as expected?" It is also "does this result satisfy the goal, the constraints, and the standard we agreed on?"
The numbers reflect that tension. Stack Overflow's write-up of its 2025 Developer Survey says 84% of respondents use or plan to use AI tools, up from 76% the year before, while trust in AI accuracy fell to 29%. Their sharpest line is still the simplest:
"It's what makes the field software engineering rather than software hoping-for-the-best."
Adoption is rising. Trust is falling. That is not a contradiction. It is a profession discovering that a useful new component plays by different rules.
Three shifts inside that diagram changed how I build, and one thing carries all three.
When code generation gets cheap, describing the right system becomes the expensive part.
I used to think of a prompt as the thing that kicked off the work. Now I think of the specification as the thing that makes the work governable. A useful spec has a goal, boundaries, examples, non-goals, acceptance evidence, and a stopping condition. It says what the agent may touch, what it must not touch, and what would prove the task is complete.
That sounds like requirements engineering because it is requirements engineering. The difference is that the spec is no longer a document people read before writing code. It is executable context for a system that can turn intent into implementation.
The tooling is catching up. GitHub Spec Kit and OpenSpec move work from a loose prompt toward durable specifications, plans, tasks, and implementation. Tessl approaches the same pressure differently, through versioned agent skills, evals, governance, and evidence.
I learned this the boring way while building Agentry. The useful part of an agent contract was never the sentence describing the goal. It was everything around it:
Each field felt like paperwork until a run needed it. Then it became the receipt.
I wrote about the same lesson from the autonomy side in You Don't Pick the Autonomy Level. Your Evidence Does.. The short version is that reach is earned by proof. A larger model, a longer context window, or more agents do not change that.
The lazy version of this argument says tests are obsolete and evals replace them.
That is wrong.
Tests assert specific invariants. They are still the best tool for questions such as:
Evals answer a different class of question:
A test can tell me the citation field exists. An eval has to tell me whether the citation supports the sentence.
That distinction matters because a green suite can still be looking in the wrong place. I have shipped code where every test passed because the doubles never had the shape of the real primitive. I wrote about that in Green, Reviewed, and Still Wrong. The lesson carries directly into AI systems: verification is only as good as the cases, evidence, and failure shapes it covers.
The hard part is that evals do not settle the question forever. The task distribution changes. Users find new edge cases. Grounding sources move. Providers update hosted models. A threshold that looked sensible in a lab can fail under production pressure.
So the eval suite is not a graduation exam. It is a living description of what failure looks like.
Microsoft's 2026 Work Trend Index frames the shift as expanded human agency: as agents take on more execution, the premium moves toward judgment, quality control, and redesigning work across people and agents. Anthropic's usage data shows a similar direction from the model side. In its September 2025 Economic Index, directive conversations on Claude.ai rose from 27% to 39% in eight months, while 77% of sampled first-party API business usage showed automation patterns.
The numbers are less interesting than the direction. People are delegating larger units of work.
That can look like the engineer is disappearing from the process. In practice, the job moves to the places where judgment is hardest to automate:
The last one is where most of my mistakes have lived.
In Certainty Is the Bug, I described a review finding that sounded precise and urgent. It was also wrong about the failure it claimed. Then two of my own fixes were wrong in different ways. The review was useful, the tests were green, and none of that made the claims true until I reproduced them.
That is the new centre of the craft for me. Not generating more. Building a system in which claims have to arrive with evidence.
The model gets the attention because it produces the visible result. The reliability lives around it.
The harness decides:
Prompt engineering matters inside that system, but it is only one control surface. A beautifully phrased prompt cannot compensate for a missing permission boundary, an eval set that rewards the wrong thing, or an audit trail that records only the final summary.
This is why "coding is dead" and "nothing changed, it is just autocomplete" are both unhelpful. The craft did not vanish, and it did not stay put. It moved up a layer.
We still need people who understand code. We also need people who can specify intent, model risk, design evidence, and direct systems that will not produce the same implementation every time.
Software did not stop needing engineers when it stopped behaving like a recipe.
It started asking us to become better at the parts we used to leave implicit. 🧭
The through-line wasn't sloppiness. It was certainty — the review's, and then mine.
Last time I wrote that green doesn't mean done, even after a review: a passing suite plus a careful read still shipped bugs that hid behind the shapes my tests never took. This is the sequel from the other side of the desk — what happens when you do get the review, take it seriously, and discover that a finding is not a fact, a guard is not a guarantee, and the word "never" in your own docstring is just a promise you haven't checked yet.
Three of them. None exotic. All the kind you nod along to and ship.
Near the top of the review: "This workshop slide tells people to use a grounding source that doesn't exist. With the new fail-closed behavior, copying it verbatim now raises an error and the agent won't run — a regression."
That reads as a hair-on-fire bug. A first lesson in a tutorial that hard-crashes is about the worst learner experience there is. I was one keystroke from writing the fix and moving on.
Instead I ran the thing the finding described:
cfg = {"grounding": {"sources": ["ghost-source"], "require_packet": True}}
packet = build_packet(cfg, repo_root, required=True)
# claimed: raises, agent won't run
# actual: {'topic': ..., 'grounded': [], 'ungrounded': ['ghost-source'], 'chunks': []}
No exception. The unresolved source doesn't crash the run — it comes back flagged ungrounded, which is the system working exactly as designed: cite what you can, and be honest about what you couldn't. The hard error only fires when the citation index itself can't be found at all — a different failure, on a different line, that this config never touches.
The finding was still worth acting on — a tutorial shouldn't teach a source ID that isn't real. But the reason was fiction, and the reason changes the fix. "It crashes" means stop-the-world. "It shows up ungrounded" means correct the ID and tighten the copy. If I'd inherited the reviewer's certainty, I'd have written a scarier changelog than the truth, and learned the wrong thing about my own system.
A finding is a claim, not a fact. The confidence in a review is the author's, not the code's. The five-line repro is cheap; the wrong mental model you adopt by skipping it is not.
Now the part where my certainty was the bug.
There's a redaction path that runs a config-supplied regex over model output. A pathological pattern like `(a+)+$` backtracks for seconds on a short string — a classic denial-of-service footgun. I'd flagged this exact risk in the last post and, at the time, argued the tempting fix (a watchdog thread) was theatre, because the regex engine holds the interpreter lock and a watchdog can never get a turn to fire. So this round I built the honest version: analyze the pattern before running it and refuse the dangerous shape.
My first cut was a tidy little regex that recognized "a quantifier wrapped around a quantified atom" — the (a+)+ shape. It caught the textbook case. Tests green. I felt done.
Then I threw one more string at it:
is_dangerous(r"(a+)+$") # True — caught
is_dangerous(r"((a+))+$") # False — sailed straight through
Same catastrophic pattern, one redundant pair of parentheses, and my detector shrugged. Feed ((a+))+$ to the real matcher and it grinds for about ninety seconds on a thirty-character input. My "guard" waved it through. And it got worse: the check only sat on the redaction path — I'd left the same config regexes unguarded on two other call sites that also run re.search directly. Three doors, one lock.
A pattern-matcher that recognizes one spelling of a dangerous thing isn't a guard; it's a suggestion. The fix was to stop pattern-matching the surface and start reducing the structure — peel off redundant wrapping, handle the lazy `+?` spelling, and only then ask "is this, underneath, a quantifier over a quantifier?" — and to put that one check on every site that runs an untrusted pattern, not just the one that prompted it.
A security check with a trivial bypass is worse than none — it's a bypass with a false sense of safety stapled to it. If you're writing one, your job isn't to catch the example in the ticket. It's to catch the example plus the same thing with a paren, a lazy quantifier, and a nested group — because whoever trips it next won't use your spelling.
The last one is the smallest and the most embarrassing, because I typed the overclaim with my own hands.
There's an audit log that hashes a snapshot of each decision. It used to serialize with default=str, which quietly stringifies anything weird — including an object's memory address, which changes every run, which silently breaks the tamper-evident chain across processes. Real bug. I replaced it with a deterministic encoder and, pleased with myself, wrote the docstring:
…is deterministic and never raises, so a malformed snapshot can't crash a governed turn.
Deterministic: true, and a good fix. Never raises: a wish wearing a fact's clothing. Because the serializer only gets consulted for values the JSON encoder already accepted — and the encoder throws before it ever calls me:
cyclic = []; cyclic.append(cyclic)
audit.record("input", {"x": cyclic}, verdict) # ValueError: Circular reference detected
audit.record("input", {("a","b"): 1}, verdict) # TypeError: keys must be str...
Both blow up. And the call site that records the audit entry has no try around it — so my "can't crash a governed turn" crashes the governed turn, on exactly the malformed input the fix was supposed to make safe. I'd fixed the determinism and left the crash, then documented the crash out of existence.
The repair was to make the sentence true: wrap the serialization so any encoder failure falls back to a stable, deterministic sentinel instead of propagating — genuinely fail-safe — and narrow the claim to what the code actually guarantees. Same amount of confidence in the docstring. Backed, this time, by a line that catches the exception.
"Never" is a proof obligation, not an adjective. If a hot path calls you and can't handle your exception, either you actually never raise — because you wrapped the thing that does — or you don't write "never." The docstring is a claim like any other, and the reader will trust it more than they trust the code.
The review was right that these lines needed changing. It was wrong about why one of them did, and I was wrong twice about how. Green didn't lie last time; the review didn't lie this time. They were both just confident — and confidence is not the same as having run it.
Run it. Then believe it. 🔎
]]>I've been keeping a rough ladder in my head for that — low end, it just suggests and I decide; high end, it runs on its own and only surfaces the decisions that actually need me. It's a useful ladder. It's also been quietly lying to me, and it took a bad week to see how.
Here's the lie, stated plainly: I kept treating the rung I reached as the rung I'd earned. They are not the same thing, and the gap between them is where every agentic disaster I've had actually lived. I've written before that green doesn't mean done — a passing build tells you no check failed, not that the work is right. This is the same disease one level up. The autonomy you can defend is exactly the autonomy your evidence can carry. Everything above that line is a story you're telling yourself with more agents running.
You don't pick the level. Your proof does. "Add more verification" is the kind of advice that's true but useless until it has edges — so let me show you mine.
For months I bragged about the same two things as if they were one thing. "I let this agent run for twenty minutes untouched." "I had eight agents going at once." Both feel like look how hands-off I've become — and they are completely different skills that break in opposite directions.
One is about reach: how far from my own judgment I let a single agent wander before it has to come back and show me something. The other is about coordination: whether I can carve the work into slices that don't collide, so I only step in when one of them face-plants.
You can be strong at one and a menace at the other. The trap I kept walking into was running "several agents at once" while personally holding every dependency in my head — deciding what each one touched, merging their output by hand, arbitrating conflicts on the fly. That's not orchestration. That's me being the orchestrator, manually, at higher speed. It looked like leverage. It was just faster context-switching wearing leverage's jacket. Running many agents is not the same as coordinating them, and running one agent far is not the same as running it safely. Score them separately or you'll congratulate yourself for the wrong one.
Here's the thing that actually moved my autonomy up a rung, and it's boring: I stopped letting an agent start until I'd written down what it was for.
Not a prompt. A contract. On agentry — the governed-agent tooling I've been building — that's a single declarative file: the goal, the boundaries, the identity it runs as, what it's allowed to touch, how it's grounded, who has to approve what, how it's checked, when it stops, and what it costs. The first few times it felt like paperwork written to feel responsible. It isn't. Every field on that thing is a specific failure I'm pre-paying:
The contract isn't ceremony. It's the receipt I'll want when a run goes sideways and someone — usually future-me at an ugly hour — asks what this thing was even permitted to do. No contract, no receipt. No receipt, and "high autonomy" just means I was lucky, and luck has never once shipped a stopping condition.
Once I started treating autonomy as something you earn with evidence, three pieces of machinery stopped being optional — the things I now build before I let the rope out, each one answering a question I'll be asking at the worst possible moment:
A tripwire, not a transcript — for how fast will I know it went wrong? Every governed decision writes a tamper-evident trail, so catching a bad turn is a query, not an archaeology dig. If finding out I was wrong means re-reading a chat log, I don't have autonomy, I have a liability with good manners.
A cheap undo — for how cleanly can I roll it back? Fail-closed defaults and a real revert path aren't paranoia, they're the toll for the next rung. Reversibility is what lets you run hotter: the refactor behind strong tests and a clean revert can take far more rope than a task with no source of truth to revert to.
Evidence that travels — for what would prove I was right? The proof rides with the work — a diff, the tests, the logs, the findings — instead of collapsing into a confident paragraph at the end.
If my honest answers are slowly, painfully, and "I trust the summary," I'm not operating at high autonomy. I'm exposed, and calling the exposure a feature.
Which is exactly the wall I hit last week.
The most dangerous version of this failure is the tidiest one: you let the agent's own write-up stand in for the review. The summary is clean, the tone is confident, and reading it feels like doing the work. It isn't. It's surrender with good formatting.
I ran straight at it a few days ago. I had a manager agent taking a list of findings, handing slices out to a small fleet of reviewers, collecting what they found, and returning only the decisions that needed me. A small version of the shape I actually trust — one coordinator, separate checkers, me on the decisions. The efficient move was to skim the manager's summary and ship.
I didn't — and the only reason it held is that I made the reviewers argue with the code, not with each other's summaries. They earned it twice: they caught claims in the original findings that turned out to be confidently wrong, and then caught blind spots in my own fixes that my green tests had waved straight through. I wrote up the wreckage in certainty is the bug, but the autonomy lesson is sharper than the debugging one: a fleet without independent verification isn't leverage. It's faster wrongness, delivered with more confidence and a nicer diff.
The fix was never "trust it harder." It was the unglamorous stuff — separate the thing that does the work from the thing that checks it, make the evidence independent of whatever generated it, keep the scope small enough that a rollback is cheap. The summary is not the review. The number of agents is not the leverage. The evidence is the only part of any of this that actually scales.
Verification was always going to be the ceiling. The mature posture isn't the highest rung you can climb to — it's the highest one you can currently defend, and the honest ones move up slowly, one axis at a time, only as the evidence for the next rung actually accumulates.
So no, you don't get to pick the autonomy level. You write the contract, you build the proof, and the proof tells you which rung you've earned. Everything above it is theatre.
Pick the level your evidence can defend. Then, and only then, climb. 🪜
]]>Green wasn't lying. It just wasn't looking where the bug was.
I've argued before that green doesn't mean done — a passing build says no check failed, not the code is correct. This is the follow-up with receipts: three bugs that sailed through a green suite and a careful read, because each one hid in a place verification almost never looks — behind a test double, inside the fix itself, and in the direction a fallback falls. None of them are exotic. That's the point. They're the kind you ship on a good day.
I'd just rewritten the part of a harness that reads JSON-RPC from a subprocess over a pipe. The old version could hang if the other end went quiet; the new one added a deadline. select() for readability, then read a line — textbook. Tests passed.
Here's the repro I wrote anyway, because "added a deadline" is a claim, not a fact:
read_fd, write_fd = os.pipe()
os.write(write_fd, b'{"jsonrpc":"2.0","id":1,"result":{"ok":true}}\n') # one whole line, atomically
# read through os.fdopen(read_fd, "r") — exactly what a real subprocess hands you
resp = read_response(read_fd, timeout=2.0)
# expected: the response. actual: TimeoutError after 2.0s.
A complete, valid response — and it timed out. Every real call would have stalled for the full timeout and then died.
The cause is a lovely little trap. select() only sees the operating-system file descriptor. It knows nothing about Python's own buffer. My first read(1) asked a buffered text stream for one character; the stream went to the kernel, slurped the whole line into its internal buffer, and handed me back one char. Next loop iteration, select() checked the fd again — now empty, because the data was sitting in Python's buffer the entire time. So select() reported "nothing here," and I timed out on a response I'd already received.
Now look at why the suite was green. One test fed the reader a StringIO — which has no file descriptor, so it quietly skipped the entire select() codepath where the bug lives. Another used a real pipe that never sent anything — perfect for proving the timeout fires, useless for proving a normal read works. Between "no real fd" and "no data," not one test ever put a real, complete line into a real pipe. The mocks weren't wrong; their shape was wrong, and the bug lived exactly in the shape they didn't have.
The fix was to stop reading through the buffered wrapper and pull raw bytes off the descriptor with os.read, so the thing select() watches and the thing I read from are the same object. But the fix isn't the lesson. The lesson is that I only knew to write it because I'd reproduced the failure against the real primitive — a real fd, a real pipe — instead of the convenient stand-in. Test doubles are how you go fast. They are also where integration bugs go to hide.
The next one I'd written myself, recently, with tests, feeling good about it. A careful read of the diff is what turned it up.
The job was redaction: take a pattern and a replacement from config and scrub matches out of some text. Mine did the obvious thing:
re.subn(pattern, replacement, text) # replacement is... a regex template, not literal text
re.subn treats the replacement as a template, not literal text — which means it can contain a backreference. Watch what a replacement of `\g<0>` does to a secret:
re.subn(r"\d{3}-\d{2}-\d{4}", r"\g<0>_SEEN", "ssn 123-45-6789")
# -> ('ssn 123-45-6789_SEEN', 1) the secret is right there. redaction re-emitted it.
The feature whose entire job is to remove the secret can be made to re-print it — and a replacement ending in a stray backslash throws an uncaught error that kills the turn instead. The repair is one line: substitute with a function so the replacement is treated literally, and fail closed if the pattern is malformed.
But the one line isn't the lesson either. The lesson is where this lived: in the security-critical path, in code I was confident about, behind tests that all used boring, well-behaved replacement strings. No green test ever tried a hostile replacement, because the person who wrote the redaction also wrote the tests, and we shared the assumption that the replacement was friendly. A fix can quietly invert the very guarantee it's supposed to provide — and the more sure you are of a piece of code, the less precisely your own tests will probe it.
The smallest bug taught the sharpest lesson, and I caught it the lowest-tech way there is: reading my own diff out loud.
A rule can say deny when amount >= 1000. Earlier in the same pass I'd hardened the number parser to reject nonsense like inf and nan. Obviously a safety improvement.
My first version returned "not a number" for those. Feels fail-closed. It's the exact opposite. The rule only fires when the comparison is true; float("nan") >= 1000 is False, and "not a number" also makes the rule not fire — so a nan amount strolls straight past a deny-the-big-ones rule. My "safety" fix failed open on the one path that mattered.
The correct version raises, so the evaluator's catch-all turns it into a hard deny. Same intent, opposite outcome, and the entire difference is which way the fallback falls. "Reject the bad input" is not a direction. In a guard, every fallback has a polarity — and you have to trace, for the specific rule, whether your safe-looking default lands on allow or deny. Tests wouldn't have caught this; the rule still "worked" on every finite number anyone thought to type.
One more, because it's the inverse mistake — not a bug I almost shipped, but a "fix" I almost built.
That same redaction path runs a config-supplied regex on the hot path, and a pathological pattern like `(a+)+$` can backtrack for seconds on a short string. The tempting hardening is to run the match inside a timeout — spawn a thread, kill it if it overruns. I started reaching for it.
Then I remembered how the runtime actually works: CPython's regex engine holds the interpreter lock while it matches. A watchdog thread can't preempt it, because the runaway match never lets go of the lock for the watchdog to run. The timeout would be pure ceremony — a guard that can't fire. The honest fix is smaller and less satisfying: treat config patterns as trusted code, document the footgun, and fail safe on a malformed one. Know your runtime before you "harden" against it — half of hardening is not building the protection that can't work.
Green told me no check failed, and that was true. The bug that mattered most was hiding behind a StringIO the whole time, in the one shape my tests never took.
Don't ship the green. Ship the thing you reproduced against the real world. 🔌
]]>This week the loop became a named, shareable artifact — a library you can browse, copy, and submit to. But the names aren't the interesting part. The exits are. Here's what changed my mind, the four ways a loop can stop ranked by how much trust each one earns, and why — now that generation is nearly free — the exit is the only part left worth engineering.
Forward Future just published the Loop Library — 31 named agent loops (a docs sweep, a production-error sweep, a builder–reviewer "autonomy" loop, a versioned-experiment loop) with a clean thesis: a good loop answers four questions — what's the goal, how does it check the last attempt, what does it do with what it learned, and when does it stop.
I started this post as a tour of all 31. I deleted that draft — a catalog isn't an argument. I went back in looking for "vibes with a while loop" so I could feel clever about my own setup. Mostly I found the opposite, and it indicted my loops, not theirs.
Because my throwaway loops look like this: "keep fixing the flaky tests until they pass." It runs, something goes green on pass nine, max_turns is close, it stops, and I get a cheerful ✅ all tests passing. The suite is still flaky. The loop didn't decide it was right. It decided it was tired. Call it stop-on-exhaustion — and once you have the phrase, you start seeing it everywhere.
The good library loops don't do that. They decide what would make them stop before the first pass. That's the tell of a real one.
A year ago, an iteration was expensive enough that you babied each one. That's over. Charity Majors put it bluntly this month — "AI demands more engineering discipline. Not less." — because the economics of code production flipped: lines went from treasured and carefully curated to disposable and regenerable, practically overnight. The same week, GLM-5.2 shipped as open weights with a million-token context. On models like that, a short coding pass is now near-zero marginal cost.
Here's the consequence nobody likes saying out loud: when generation is a commodity, the code the loop emits is the cheap part. The scarce resource is the judgment about which pass to keep. The asset isn't the output — it's the evaluator that judges it. A frontier model wired to a "did it crash?" check is a Ferrari behind a turnstile. (I called that rigor cosplay in the stack post; cheap iteration just made it more expensive to fake.)
So the question stops being "can the agent generate a fix?" — of course it can, ten times before lunch — and becomes "can the loop tell a good pass from a lucky one, and stop on the right one?"
Watch two loops chase the same goal and the difference jumps out.
A hope loop regenerates the same doomed approach until the budget dies, then ships whatever was on screen. A prune loop spends the now-cheap generations on variety, kills the regressions fast, and iterates only the survivor.
Which means the metric I actually care about isn't a loop's success rate — it's its discard ratio: how quickly it throws away a path that isn't working. A loop that takes ten passes to abandon a dead end is worse than one that abandons it in two, even if both eventually pass. Cheap generation didn't make iteration the skill. It made discarding the skill.
Strip any loop down and its exit falls into one of four buckets. They are not equal.
Trust climbs as you go down the list. Exhaustion is the bottom — it's not a stop condition at all, it's a loop fainting and you reading meaning into where it fell. Surrender is honest and often correct: the loop did its mechanical work and routes the judgment to a person. Consensus (a second model signs off) is stronger, but agreement and correctness are different things — two models can be confidently wrong together. Provable Truth — a test, an audit, a compile, a check a reviewer can re-run and watch turn red — is the only exit that proves the property you were actually after.
The honest finding from the ones I read closely: the loops worth naming anchor on Provable Truth or a deliberate human gate. None of those stop because they got tired.
Three specimens, read for their exits rather than their actions:
| Loop (from the library) | The check it runs | How it stops | What that earns |
|---|---|---|---|
| Revolve versioned-experiment | frozen tests + scoring against a recorded baseline; keep only a regression-free win | Provable Truth — plus no-progress / budget as backstops | a kept change is provably better than the last checkpoint, with a rollback |
| The docs sweep | verifies commands, links, and examples against the repo, then opens a PR | mechanical proof + Surrender to human review | docs match the code — if someone actually reviews the PR |
| The production data cleanup | audits records against explicit rules — sampled and queried — with reversible ops; holds uncertain rows back | Provable Truth + a human gate on the irreversible step | it deletes bad prod data without being how you delete prod data |
That last row is the one that humbled me. A loop that mutates production is exactly where a lazy while would be catastrophic — and the published version refuses to be lazy: explicit inclusion/exclusion rules written before any data changes, an audit that must pass against those rules, reversible operations, and uncertain records parked for a human instead of deleted. That isn't a loop that stops when it's tired. It's a loop engineered to stop only when it's right, with a person on the one action that has no undo.
Notice what's missing from all three: nobody stops on exhaustion. That's not a coincidence — it's the difference between a loop somebody published under their name and the "keep improving X" one-liners I write for myself and never inspect.
The stack post put loop engineering on my map as a layer. Here's the part I underplayed: the stop condition deserves the same up-front design as the goal — because it is the definition of done. A practical test, learned the hard way this week: if you can't name the exit contract before the first pass, you don't have a loop. You have motion that happens to halt.
That also sharpens green doesn't mean done: green is only a verdict if the loop stopped because the check passed — not because the check happened to be green when the turns ran out.
while not done. The engineering is done.So before you write another while not done, write done first. The iterations are free now — the only scarce thing left is a loop that can prove why it stopped, instead of reporting ✅ done because it got tired.
The last Agentry post made that pitch. This one opens the hood — every pillar, in the weeds: the loop, the policy engine, identity, the audit chain, grounding, fleets, evals, and the one-dependency bet that holds it all together.
One orientation note, then the schematics: every section starts where it should — with the problem that part exists to solve. The full thesis lives in the previous post; here we stay in the weeds.
agent.yaml, and the runtime that wires itProblem: agents in the wild sprawl across a prompt, some glue code, a vector store, and a half-documented deploy script. There is no single object you can review, diff, or govern.
Agentry collapses that into one file — the previous post showed its shape. What matters in the weeds is what runtime.run_agent() does with it: resolve the provider, build the tool table, construct the policy engine, attach an identity and an audit log, and hand the lot to a Harness — or, when the file declares an agents: list, to the orchestrator instead.
The seam that makes the whole thing testable is the model function. The harness never calls an LLM itself — you inject model_fn(context: list[dict]) -> dict, returning either {"tool_call": {"name", "args"}} or {"done": str}. In production that closure wraps a provider; in a test it returns a canned dict. That one indirection is why the entire suite runs offline with no network and no mocked SDK.
Problem: a control that runs after the model has already decided is theater. By the time you're reading the output, the tool already fired.
So Agentry evaluates policy at eight lifecycle points wrapped around every step of the loop — not one "is this output okay?" check at the end:
agent_startup → input → pre_model_call → post_model_call
→ pre_tool_call → post_tool_call → output → agent_shutdown
Each point builds a small snapshot (the input text, the drafted response, the pending tool call and its args) and asks the engine for a Verdict. Harness.run() is the sync path, arun() the async one, and they share identical governance semantics. The loop is bounded by max_turns so a model that loops forever returns an honest {"error": "max turns exceeded"} rather than spinning. The return value is small and inspectable: {"answer", "trace"} on success, {"blocked", "verdict", "trace"} when a verdict stopped it, each step recorded.
One honest note, because an earlier review caught exactly this: there was a moment when only four of those eight points were actually wired into the loop, and a redact verdict fired but never masked anything. A second-opinion agent on a different model reproduced both with file-and-line receipts. They're wired and enforced now — which is the point of keeping the receipts.
Problem: "please don't email external recipients" in a system prompt is a wish. Enforcement has to be deterministic, inspectable, and runnable outside the model.
PolicyEngine.evaluate(point, snapshot) returns one of five verdicts:
| Verdict | Effect |
|---|---|
allow | proceed |
warn | proceed, record a warning |
deny | stop the step with a reason |
escalate | block pending a human sign-off |
redact | mask the matched content (the rule must carry a pattern) |
The evaluator is a seam, not a hardcode. NativeEvaluator is the deterministic default; OPAEvaluator ships the same decision to an Open Policy Agent server over standard-library HTTP, so the policy that runs in the teaching engine is the policy you can run in prod:
from agentry.governance import OPAEvaluator, PolicyEngine
engine = PolicyEngine.from_path(
"governance/policy/manifest.yaml",
evaluator=OPAEvaluator("http://127.0.0.1:8181", package="agentry/policy"),
)
verdict = engine.evaluate("pre_tool_call", {"tool_call": {"name": "send_email"}})
Both evaluators fail closed: an unknown interception point denies, a transport error or malformed response denies, a redact rule with no pattern denies, and from_path(..., profile=...) raises rather than silently running ungoverned if a profile can't load. Rules read declaratively — a when: clause matches on tool, on an arg with matches/not_matches or numeric gte|gt|lte|lt|eq, or on target_matches — and the manifest is mirrored by a policy.rego kept in parity. Three profiles (advisory, balanced, strict) layer hard tool policies on top. The verdict is never the model's discretion; it's a check a reviewer can re-run and watch turn red.
Problem: "an agent did something" is not an incident response. And a sub-agent that can hand itself more authority than its parent is a privilege-escalation bug wearing a helpful face.
Four mechanisms, each opt-in and None by default so they never change behavior unless you ask for them:
Identity.delegate(...) can only narrow — scopes are intersected, clearance is clamped down. A specialist can never out-scope its orchestrator, by construction.verify() catches it. Pass AuditLog(key=...) and the chain upgrades to HMAC-SHA256.pull_request) go through a default-cancel propose → confirm rail with one-time tokens that are never persisted. The model proposes; a human confirms; nothing fires in between.InProcessSandbox, TimeoutSandbox, or SubprocessSandbox. Honest limit, documented rather than hidden: the subprocess isolator can't process-isolate a tool loaded dynamically from a file at runtime — it returns a clear "not isolatable" sentinel instead of pretending.That audit chain is the line I care about most. "Trust me" becomes "here is the signed sequence of what policy was active, what was requested, and why it was allowed or denied."
Problem: a model that recalls instead of citing will confidently invent a source, and a docs tree that drifts from the code will confidently lie about it.
Grounding is "cite, don't recall." Before generating, an agent emits a Learning Packet naming the approved canon entries that ground a claim, and flags anything it can't ground as UNGROUNDED — out loud — instead of smoothing the gap. Retrieval, when enabled, returns Chunk objects carrying (text, score, provenance, source_id); a type: vectorstore tool is confined to the agent's own directory and answers with source-and-SHA provenance on every hit, all through a dependency-free keyword retriever.
The same honesty is enforced on the repo itself by a registration contract: a construct, pattern, level, or lab exists only if it is (a) a real file, (b) registered in manifest.yml, and (c) cites only canon ids that resolve in the source index. agentry check fails on any dangling citation or orphan. This is not bureaucracy for its own sake — it's the mechanism that makes "all claims are sourced" a build gate instead of a promise.
Problem: every time you add an agent, you add a handoff — and a handoff is a place where authority and policy can quietly leak.
When agent.yaml declares a top-level agents: list, the runtime dispatches to orchestrator.run_workflow(), which runs the specialists sequentially under one shared policy, one audit log, and one tool table, each with a delegated identity that can only narrow:
This week I shipped the lab that stress-tests the idea: L5 — an agent fleet with shared, governed memory. Five specialists share a small persistent store (decisions, code patterns, tests, design tokens) so the fleet reuses prior work and gets faster as the corpus grows. The sharp part is how they touch it: not raw SQL, but two governed tools — query_memory is read-only, restricted to an allow-list of tables, caps LIMIT, validates filter columns as identifiers, and parameterizes every value; store_memory validates required fields and enforces a payload cap. It defaults to standard-library sqlite3, so the whole lab runs offline with zero setup, with PostgreSQL as an import-guarded option. An agent fleet with institutional memory is a great idea right up until someone hands it a database connection; the lab's entire teaching point is the layer that doesn't.
Problem: "it's better now" without a measured delta is a vibe, and an eval that fabricates a score is worse than no eval at all.
Agentry grades with deterministic checks (tool selection, exact match, F1/BLEU/ROUGE/Levenshtein/Jaccard), an optional LLM judge, and RAG metrics — and it is structurally honest:
skipped, never faked.--allow-skips.You measure change over time, too: agentry eval --store runs.jsonl appends each run as JSONL, --compare flags pass → fail regressions against the previous run, and agentry report renders the whole history — plus the verifiable audit trail — into a single self-contained HTML page built from html.escape and string templates. No dashboard framework: the runs are already open JSONL and the CLI already prints the regressions, so a web stack would have added moving parts, not signal.
Problem: a toolkit whose pitch is auditability has no business being too large to audit — and every vendor SDK is supply-chain surface and lock-in you'll regret.
The whole core runs on one runtime dependency: pyyaml. Real LLM execution against four providers (ollama, openai, anthropic, azure_openai) goes through standard-library urllib; each provider is @register-ed, normalizes its response to the same {"tool_call"}/{"done"} contract, exposes typed errors with retry and backoff, imports its SDK lazily if at all, and offers an async fallback. The HTTP governor is http.server with bearer-token auth, a request-body cap, and sanitized errors; a raw ASGI adapter exposes the same routes; an MCP layer speaks stdlib JSON-RPC both ways — consuming external MCP tools governed at the same pre/post_tool_call points as native ones, and exposing Agentry itself as a governed MCP server. Observability is an opt-in OpenTelemetry span (agentry.gov, tagged with point, decision, rule_id) that is a no-op when the extra isn't installed. Everything heavy — a JSON-Schema validator, hosted-provider keys, numpy, uvicorn — is an import-guarded extra, never the price of entry.
Problem: the same green-checkmark trap that fools you about your own code fools you about a contribution — especially one an AI helped write.
A contributed PR landed this week adding that L5 fleet lab. CI was green across four Python versions. It was also, underneath: leaking internal infrastructure URLs in shipped code and live-network tests, calling methods that didn't exist, using a YAML schema the runtime doesn't understand — and unregistered, so agentry check never actually validated the lab it claimed to add. Green meant "no check failed," not "done." Again.
What caught it was the same discipline the platform preaches, turned inward: three reviewers on three different models, each with a hostile brief and read-only access, cross-checked the diff against the registration contract and the invariants. They converged on do-not-merge with file-and-line receipts — the internal-URL leak first. The fix wasn't to reject the idea; it was to strip the proprietary and broken parts, make the lab actually runnable and offline, register it so agentry check could see it, ground it in the real canon, and re-validate. The teaching intent survived; the leak and the lies didn't.
That is the recursion I find most useful: a governed harness, reviewed by governed reviewers, contributing to a repo whose own contract forbids the orphan and the dangling citation. Governance isn't a feature you point at the agent. It's how the whole system — code, docs, contributions, and the AI writing them — keeps itself honest.
agentry check won't see an unregistered lab, and three hostile briefs caught the green-but-broken PR. Govern the contributions, not just the agents.The pitch was governed, grounded, evaluated. The schematics are how it earns each of those words — and keeps the receipts. 🧾
]]>The pitch in one line: a model is one component; everything that makes it safe and useful in production lives in the harness around it. Agentry is that harness — governed, grounded, evaluated — with one property I care about a lot: it can prove what it did.
I've argued before that the model is the least interesting layer of a real agent. The interesting parts are the controls: what the agent is allowed to do, what it grounds its answers in, how you measure whether it works, and whether you can reconstruct what happened afterward. Bolt those on as an afterthought and you get a demo. Build them in and you get something you can run in front of a customer.
So agentry's unit isn't a prompt — it's one declarative agent.yaml that carries the model, its tools, its policy, its grounding, and its evals together:
name: support-triage
model: { provider: ollama, name: llama3.2:latest } # or openai / anthropic / azure
instructions: { file: instructions/system.md }
grounding: # cite, don't recall
sources: [support-kb]
require_packet: true
governance: # policy as code — or it doesn't ship
profile: balanced
policy: governance/policy/manifest.yaml
evaluations:
metrics: [{ type: tool_selection, threshold: 1.0 }]
Here's the opinion that drives the whole design: governance has to be deterministic and run before the action — not a paragraph in the system prompt that asks the model nicely to behave.
Agentry evaluates policy at eight lifecycle interception points around every step of the loop — startup, input, before/after the model call, before/after each tool call, output, shutdown — and each check returns one of allow · warn · deny · escalate · redact. It is fail-closed: an unknown point or a broken rule denies. The same rules live as a manifest and mirror to OPA/Rego, so the policy that runs in the teaching engine is the policy you can run in production.
A deny stops the loop. A redact rule actually masks the matched content (an SSN-shaped string never leaves the harness). An escalate is where a human signs off. The point is that none of this is the model's discretion — it's a check a reviewer can re-run and watch turn red.
The part I'm most attached to: every verdict is appended to a hash-chained, tamper-evident audit log. Each record's hash covers the previous one, so altering, reordering, or dropping any entry breaks the chain — and a one-line verify() catches it. "An agent did something" is not an incident response; "here is the signed sequence of what policy was active, what was requested, and why it was allowed or denied" is.
Agent identity rides alongside it: every agent carries a name, a clearance, and a set of scopes, and delegation to a sub-agent can only narrow authority — never escalate it.
Two more non-negotiables, both about honesty:
UNGROUNDED — out loud — instead of smoothing over the gap.skipped, not faked. An LLM-judge with no model returns a skip, not an invented score. A run where everything skipped exits non-zero. Better to admit than to lie.You measure change, too: agentry eval --store records runs, --compare flags pass→fail regressions, and agentry report renders the whole history — plus the verifiable audit trail — into a single HTML page.
Here's the engineering stance that surprised even me by holding up: the whole platform runs on one runtime dependency — pyyaml.
Real LLM execution against four providers? Standard-library urllib, no vendor SDKs. The HTTP governance service? http.server. The eval metrics (F1, BLEU, ROUGE)? Hand-rolled, deterministic. The report? html.escape plus string templates plus inline CSS — a self-contained page, no dashboard framework.
Why bother? A dependency-light core installs anywhere, has almost no supply-chain surface, and is small enough to actually audit — which is a funny thing to skimp on in a project whose whole pitch is auditability. The heavy, optional things (a JSON-Schema validator, hosted-provider keys) are opt-in extras, never the price of entry.
The honest part. Knowing what not to build is most of the work:
agentry report is a read-only, stdlib HTML page — and that's it.UNGROUNDED and skipped are features. Honesty is how the thing stays trustworthy as it grows.The agents that behave like senior engineers aren't the ones with the cleverest prompts. They're the ones wrapped in honest controls — and able to show their work afterward.
Govern it. Ground it. Evaluate it. And keep the receipts. 🧾
]]>So the agent did the one thing that actually earns trust: it refused to believe itself. It spun up a second agent, on a different model, and told it to try to break every claim the toolkit made. Ten minutes later I had a punch-list of four places where my toolkit was quietly lying to me — with file-and-line receipts for each.
Here's the uncomfortable thing about a passing build: it tells you no check failed. It does not tell you the thing is done. Those are different statements, and the gap between them is where production incidents live.
A green check covers the cases someone thought to write down. The interesting failures are the ones nobody encoded — the promise in the prose that no test ever asserts. You don't find those by re-running your own checks. You find them by handing the work to a reader who doesn't share your blind spots.
The author of a thing — human or agent — is the worst possible reviewer of it, because the assumptions that produced the bug also produce the test that misses the bug. Self-review re-runs your own mental model. What you need is an adversarial second reader with different priors.
For an agent, "different priors" has a literal, cheap implementation: a different model. My orchestrator runs on one model; the reviewer it dispatched runs on another (GPT-5.5), with a deliberately hostile brief — "assume the README is marketing; make the code prove every sentence."
Two things matter in that picture. The reviewer is a separate context on a separate model, so it isn't anchored to my framing. And the orchestrator doesn't just wait — it runs its own verification in parallel (the quickstart, the test suite, a few scans) so the duck's claims land against independent evidence instead of replacing one opinion with another.
This is the honest part. Every row is a sentence the toolkit advertises, next to what the code actually does when you run it.
| The promise | What actually happens | Where |
|---|---|---|
| Governance at 8 lifecycle points | only 4 are wired into the loop; post_tool_call is even evaluated, then its verdict is thrown away | harness.py |
A redact verdict scrubs PII at output | redact isn't treated as "blocked", so an SSN flows through the response unchanged | harness.py + governance.py |
agentry check refuses a lab with no policy | it only checks the governance: key exists — profile: balanced with no policy file sails through | validate.py |
Grounding flags UNGROUNDED gaps honestly | "banana governance" comes back fully "grounded", with no gap reported for banana | grounding.py |
| Evals fail loudly | an all-skipped run prints 0/1 passed and exits 0 (success) | cli.py |
| Block refunds of $1000 or more | $5000 slips past the rule's `^[0-9]{4,}` regex and returns allow | manifest.yaml |
The reviewer didn't assert these — it reproduced them. The redaction one, for example, came back with the actual return value: a harness call whose output still contained 123-45-6789 and a trace that proudly recorded ('output', 'redact') right before handing the secret back. The verdict fired. Nothing acted on it.
To be fair to the toolkit: it's young — a Phase-A MVP — and several of these are the seam between an honest scaffold and an over-eager README. That's exactly why the exercise is worth it. The sub-agent's whole value is finding the gap between what the docs promise and what the code enforces, fast, with receipts — before a user does.
The brief banned vibes. Every finding had to arrive as claim → reproduction → file:line → fix. A reviewer that says "the governance feels incomplete" is noise. A reviewer that says "points 5–8 are declared in governance.py and never called in harness.py, here's the deny rule that did nothing" is a work item.
This is just harness thinking pointed at a review: a finding only counts when it's a check a human can re-run and watch turn red. Everything else is ceremony.
The part I want to highlight — because it's where sub-agents stop being a gimmick — is that I kept steering the reviewer while it worked. It wasn't "fire prompt, read essay." It was a conversation with a worker that holds its own context.
Mid-review I realised the scope was bigger than the code: the repo also ships a two-day workshop. So I sent the running agent a follow-up — the workshop is a first-class deliverable too; judge whether its promises match what the toolkit can actually do. It folded that in and immediately found the next class of gap: slides that promise PII redaction and tracing the runtime doesn't implement yet.
A sub-agent you can talk to mid-flight is a different tool from a prompt you fire and forget. You refine the scope, add a constraint, push back on a weak finding — and it carries the whole prior context forward instead of starting cold.
One more lesson, and it's the one people skip. A powerful reviewer is eager — it will happily recommend things your project explicitly forbids: pull in a new dependency, expand the scope, "just add attribution to that source," rewrite a module it was only asked to inspect. Helpful, and wrong.
So the orchestrator governs the reviewer the same way a governed harness governs a tool call: the sub-agent proposes, the manager filters against the project's constraints, and only verified, in-bounds findings survive into the report. A second opinion you can't constrain isn't a reviewer — it's a second source of scope creep. The manager's job is to keep the duck honest and on-leash.
file:line plus a reproduction, or it didn't happen. "Feels incomplete" is noise; a re-runnable red check is a work item.The agents that behave like senior engineers aren't the ones that declare victory when the build goes green. They're the ones that get suspicious because it went green — and call in a second pair of eyes that doesn't share their assumptions.
Ship the report, not the vibe. Green is where the review starts. 🦆
]]>That small episode is the whole argument of this post. The thing that made the agent trustworthy wasn't a clever prompt. It was the layers around the prompt.
"Prompt engineering." "Context engineering." "Harness engineering." "Loop engineering." "Spec-driven development" — people say "speckit." The discourse treats them like competing fads, each year's skill obsoleting the last. On real enterprise agents they're nothing of the sort. They're layers in a stack — and most failures I see come from over-investing in one layer while ignoring the one that would actually have caught the bug.
Here's the map I use.
| Discipline | The unit | What it optimizes | The failure it prevents | When it's enough |
|---|---|---|---|---|
| Prompt engineering | one instruction | clarity of a single ask | vague, ambiguous output | one-shot, low-stakes tasks |
| Context engineering | the window's contents | what the model sees — sources, memory, tool results | confident hallucination | most enterprise reliability |
| Harness engineering | gates and validators | deterministic enforcement | mistakes reaching production | when "looks right" isn't proof |
| Loop engineering | the act → verify cycle | self-correction | one bad step becoming a bad outcome | autonomous, multi-step work |
| Spec-driven dev | the spec | shared intent before code | building the wrong thing well | non-trivial, multi-file changes |
Look at the last column. Each discipline has a narrow regime where it is sufficient — and a much larger regime where it is necessary but nowhere near enough.
Read it from the inside out. A prompt wraps a model call. Context decides what that prompt can even see. A harness is the deterministic shell that refuses to let bad output through. The loop drives the whole thing — act, observe, verify, repeat — and an explicit spec sits on top as the definition of "done."
You can be a world-class prompt engineer and still ship nonsense if the outer layers are missing. The reverse is rarely true: with strong context, a real harness, and a tight loop, even an unremarkable prompt has a real shot at a useful, checkable answer.
If you upgrade only one layer, upgrade this one.
In an enterprise, the model's parametric memory is the least trustworthy thing in the room. Your policies, your architecture standards, your approved patterns, last quarter's decision — none of that is in the weights. The job of context engineering is to put the right, current, authoritative material in front of the model before it generates, and to make ungrounded output visible instead of plausible.
The pattern I keep coming back to is ground before you generate:
The branch that makes it enterprise-grade is "cite or flag." If there is no approved source, the agent doesn't smooth over the gap — it labels the output ungrounded and says so. A confident "I can't confirm this is standard" beats a fluent fabrication every time. (I wrote about the same instinct for code in the Baymax post: don't guess an API, read the source.)
Prompt engineering tweaks the question. Context engineering changes the evidence. At scale, evidence wins.
A harness is everything around the agent that fails closed — schema validators, a build that must stay green, a registration contract, an eval suite, a policy check before a tool fires.
The mistake is to confuse ceremony with rigor. Ceremony is paperwork that makes a process feel serious. Rigor is a check a reviewer can re-run that turns red when something is actually wrong. They are not the same — and only one of them is load-bearing.
Which brings me to the harness I deleted.
The change my agent was shipping last week was itself a harness removal — which makes it a clean illustration.
We had vendored the full GitHub Spec Kit harness: a /speckit.specify → /speckit.plan → /speckit.tasks → /speckit.implement pipeline (plus a constitution and optional clarify/analyze steps), around fifteen generated agents, and a pile of scripts and templates. It is a genuinely good methodology. But in our repo it had never been run. Every guarantee it was meant to provide — grounding, an artifact-registration contract, a green build before merge — was already enforced by two things that run on every change: a one-page spec reviewed in the PR, and a deterministic build.py --check gate.
So we deleted the harness and kept the gate. The rigor was never in the ceremony — it was in the gate.
The principle generalizes: right-size the harness to the failure mode. A heavyweight pipeline nobody exercises isn't rigor — it's rigor cosplay. The deterministic gate a reviewer can rerun in five seconds is the real thing. If your ceremony and your gate enforce the same property, delete the ceremony.
(For the full "build it, test it, delete it" instinct — including a whole execution plane I tore down — see the Baymax post. Knowing what to remove is part of the engineering.)
Here's the part of last week's episode I find most instructive.
The agent did not get it right the first time. It removed speckit from the spell-check dictionary while a changelog still used it; it tripped two separate markdown-lint rules — a blockquote it stacked wrong, and a generated file whose lint-disable header it dropped; and it left two links pointing at a repo that doesn't exist. Three red checks, four root causes. Then it read the failures, traced each to a cause it had introduced, fixed all of them, and only then asked a human to merge.
That cycle — act → observe → verify → correct → repeat — is what converts a capable model into a dependable colleague. And notice the dependency: the loop only has teeth because the harness gave it something real to fail against. Loop and harness are symbiotic. A loop with no gates is just an agent confidently walking off a cliff in a very tight circle.
This is also where the human belongs — not babysitting every token, but signing the irreversible step. The agent ran the whole review-fix-verify loop on its own; a person approved exactly one thing: the merge that cut a public release.
The map collapses into a single decision I make dozens of times a week:
Match the weight to the risk. Most teams either gate everything (and grind to a halt) or gate nothing (and page themselves at 2am). The skill is calibration.
The agents that behave like senior engineers aren't the ones with the cleverest prompts. They're the ones wrapped in real context, honest gates, and a loop that refuses to declare victory until the checks are green.
Ship the gate. Delete the cosplay. Let the loop run — with a human on the one button that has no undo.
]]>Most AI setups stop at "ask a question, get an answer." The unlock is follow-through: an agent that holds your priorities, does the mechanical work, and stays honest about it — while a human still signs off on what matters.
I didn't want to build an agent runtime. Microsoft Scout already ships one (it's an always-on "Autopilot" with its own Entra identity, native Teams/Outlook/Work IQ, and memory). What was missing was a methodology layered on top: how the agent searches for truth, contains risk, decides what needs approval, and proves its work.
I called the result Baymax — after the calm, protective companion from Big Hero 6. The metaphor turned out to be the architecture:
| Baymax the character | Baymax the agent |
|---|---|
| "Your personal healthcare companion" | your personal engineering companion |
| Scans before treating | search real source before coding |
| First, do no harm | run risky code in a disposable sandbox |
| "I cannot deactivate until you are satisfied" | won't ship without your sign-off |
| Keeps a care record | an append-only audit log |
Scout runs in two tiers, and knowing which one a task needs is the whole game.
Rule of thumb: M365 coordination → the always-on cloud Autopilot. Local / code / sandbox work → the desktop tier.
Baymax is a thin methodology over Scout, expressed as composable skills plus a few conventions.
The harness matters more than the model. Scout brings native Teams/Work IQ, an Entra identity, and an always-on cloud tier. Baymax adds discipline, not plumbing.
Long-term memory is Scout's native, OneDrive-synced store. Write durable facts and decisions; recall before non-trivial work. Never store secrets.
Ten Anthropic-style SKILL.md files, loaded by description (no slash commands). Saying "baymax" activates the constitution + router, which hands off to the right skill.
| Skill | What it does |
|---|---|
agentic-os | constitution + router + identity |
governance | budget caps, approval gate, audit log |
heartbeat | wake → claim → work → verify → done |
daytona-sandbox | disposable, isolated code execution |
source-code-context | read real SDK source before coding |
teams-workiq-brief | pull Teams/mail/calendar context |
agentic-engineering-workflow | the per-feature build loop |
code-structure-cleanup | behavior-preserving refactor pass |
grep-loop-review-workflow | review-fix loop to merge-ready |
eval-harness | turn intent into objective checks |
Blast radius is the bottleneck. LLM-generated, risky, or parallel code runs in a disposable Daytona sandbox (a full VM that boots in ~1s and is deleted after), never on the host.
Ported from the "control plane" idea at personal scale:
A file-per-task queue with atomic mv checkout (POSIX rename can't double-grab), so multiple workers never collide. Scout wakes on a schedule, claims the top task, works it, verifies, and files the result.
The modern alternative to prompt/persona theater: encode what "done" means as checks, then verify.
Deterministic checks (command, file_exists, grep) are run by a tiny runner; qualitative llm_judge checks are never auto-passed — the agent grades them against a rubric, with evidence. Better to admit than to lie.
Personas as roleplay are obsolete; persona as a behavioral contract is not. Baymax's SOUL.md is lean:
The most useful part of the project was an experiment I killed on purpose.
I wanted true 24/7 — Teams/email watched even with my Mac off — so I built a second "execution plane": an always-on VPS running the heartbeat, fed by Power Automate → a webhook (because a server can't hold M365 tokens). I got it fully working: containerized webhook behind Traefik with TLS, atomic queue handoff over git, the lot.
Then two facts ended it:
AADSTS53003) — so the server could never read my mail directly anyway.So I tore it down — container, systemd units, deploy key, DNS, the lot — and wrote the reason into the audit log. Knowing what to remove is part of the engineering.
source-code-context is the whole philosophy: don't guess APIs, read the source.Runs with the lights dimmed, not off — a person still signs off.
Ba-la-la-la. 👊
]]>