Inside Agentry: How a Governed Agent Harness Actually Works

The problem, stated plainly: you cannot trust an agent you cannot constrain, ground, measure, or reconstruct — and "trust me" is not an architecture. A model emits tokens. Everything that decides whether those tokens are safe to act on in production lives in the code around it.

The last Agentry post made that pitch. This one opens the hood — every pillar, in the weeds: the loop, the policy engine, identity, the audit chain, grounding, fleets, evals, and the one-dependency bet that holds it all together.

One orientation note, then the schematics: every section starts where it should — with the problem that part exists to solve. The full thesis lives in the previous post; here we stay in the weeds.

The unit: agent.yaml, and the runtime that wires it

Problem: agents in the wild sprawl across a prompt, some glue code, a vector store, and a half-documented deploy script. There is no single object you can review, diff, or govern.

Agentry collapses that into one file — the previous post showed its shape. What matters in the weeds is what runtime.run_agent() does with it: resolve the provider, build the tool table, construct the policy engine, attach an identity and an audit log, and hand the lot to a Harness — or, when the file declares an agents: list, to the orchestrator instead.

The seam that makes the whole thing testable is the model function. The harness never calls an LLM itself — you inject model_fn(context: list[dict]) -> dict, returning either {"tool_call": {"name", "args"}} or {"done": str}. In production that closure wraps a provider; in a test it returns a canned dict. That one indirection is why the entire suite runs offline with no network and no mocked SDK.

The governed loop: eight interception points, fail-closed

Problem: a control that runs after the model has already decided is theater. By the time you're reading the output, the tool already fired.

So Agentry evaluates policy at eight lifecycle points wrapped around every step of the loop — not one "is this output okay?" check at the end:

agent_startup → input → pre_model_call → post_model_call
             → pre_tool_call → post_tool_call → output → agent_shutdown

Each point builds a small snapshot (the input text, the drafted response, the pending tool call and its args) and asks the engine for a Verdict. Harness.run() is the sync path, arun() the async one, and they share identical governance semantics. The loop is bounded by max_turns so a model that loops forever returns an honest {"error": "max turns exceeded"} rather than spinning. The return value is small and inspectable: {"answer", "trace"} on success, {"blocked", "verdict", "trace"} when a verdict stopped it, each step recorded.

One honest note, because an earlier review caught exactly this: there was a moment when only four of those eight points were actually wired into the loop, and a redact verdict fired but never masked anything. A second-opinion agent on a different model reproduced both with file-and-line receipts. They're wired and enforced now — which is the point of keeping the receipts.

The policy engine: a swappable seam, not a prompt

Problem: "please don't email external recipients" in a system prompt is a wish. Enforcement has to be deterministic, inspectable, and runnable outside the model.

PolicyEngine.evaluate(point, snapshot) returns one of five verdicts:

Verdict	Effect
allow	proceed
warn	proceed, record a warning
deny	stop the step with a reason
escalate	block pending a human sign-off
redact	mask the matched content (the rule must carry a pattern)

The evaluator is a seam, not a hardcode. NativeEvaluator is the deterministic default; OPAEvaluator ships the same decision to an Open Policy Agent server over standard-library HTTP, so the policy that runs in the teaching engine is the policy you can run in prod:

from agentry.governance import OPAEvaluator, PolicyEngine

engine = PolicyEngine.from_path(
    "governance/policy/manifest.yaml",
    evaluator=OPAEvaluator("http://127.0.0.1:8181", package="agentry/policy"),
)
verdict = engine.evaluate("pre_tool_call", {"tool_call": {"name": "send_email"}})

Both evaluators fail closed: an unknown interception point denies, a transport error or malformed response denies, a redact rule with no pattern denies, and from_path(..., profile=...) raises rather than silently running ungoverned if a profile can't load. Rules read declaratively — a when: clause matches on tool, on an arg with matches/not_matches or numeric gte|gt|lte|lt|eq, or on target_matches — and the manifest is mirrored by a policy.rego kept in parity. Three profiles (advisory, balanced, strict) layer hard tool policies on top. The verdict is never the model's discretion; it's a check a reviewer can re-run and watch turn red.

Prove it, then contain it: identity, audit, approval, sandbox

Problem: "an agent did something" is not an incident response. And a sub-agent that can hand itself more authority than its parent is a privilege-escalation bug wearing a helpful face.

Four mechanisms, each opt-in and None by default so they never change behavior unless you ask for them:

• Identity. Every agent carries a name, a clearance, and a set of scopes. Identity.delegate(...) can only narrow — scopes are intersected, clearance is clamped down. A specialist can never out-scope its orchestrator, by construction.
• Audit. Every verdict is appended to a hash-chained, tamper-evident log: each record's hash covers the previous one, so altering, reordering, or dropping any entry breaks the chain and a one-line verify() catches it. Pass AuditLog(key=...) and the chain upgrades to HMAC-SHA256.
• Approval. Irreversible writes (a refund, an outbound email, a pull_request) go through a default-cancel propose → confirm rail with one-time tokens that are never persisted. The model proposes; a human confirms; nothing fires in between.
• Sandbox. Tool execution can run InProcessSandbox, TimeoutSandbox, or SubprocessSandbox. Honest limit, documented rather than hidden: the subprocess isolator can't process-isolate a tool loaded dynamically from a file at runtime — it returns a clear "not isolatable" sentinel instead of pretending.

That audit chain is the line I care about most. "Trust me" becomes "here is the signed sequence of what policy was active, what was requested, and why it was allowed or denied."

Grounded, with a contract that forbids orphans

Problem: a model that recalls instead of citing will confidently invent a source, and a docs tree that drifts from the code will confidently lie about it.

Grounding is "cite, don't recall." Before generating, an agent emits a Learning Packet naming the approved canon entries that ground a claim, and flags anything it can't ground as UNGROUNDED — out loud — instead of smoothing the gap. Retrieval, when enabled, returns Chunk objects carrying (text, score, provenance, source_id); a type: vectorstore tool is confined to the agent's own directory and answers with source-and-SHA provenance on every hit, all through a dependency-free keyword retriever.

The same honesty is enforced on the repo itself by a registration contract: a construct, pattern, level, or lab exists only if it is (a) a real file, (b) registered in manifest.yml, and (c) cites only canon ids that resolve in the source index. agentry check fails on any dangling citation or orphan. This is not bureaucracy for its own sake — it's the mechanism that makes "all claims are sourced" a build gate instead of a promise.

Fleets: multiplying the work without multiplying the trust boundary

Problem: every time you add an agent, you add a handoff — and a handoff is a place where authority and policy can quietly leak.

When agent.yaml declares a top-level agents: list, the runtime dispatches to orchestrator.run_workflow(), which runs the specialists sequentially under one shared policy, one audit log, and one tool table, each with a delegated identity that can only narrow:

This week I shipped the lab that stress-tests the idea: L5 — an agent fleet with shared, governed memory. Five specialists share a small persistent store (decisions, code patterns, tests, design tokens) so the fleet reuses prior work and gets faster as the corpus grows. The sharp part is how they touch it: not raw SQL, but two governed tools — query_memory is read-only, restricted to an allow-list of tables, caps LIMIT, validates filter columns as identifiers, and parameterizes every value; store_memory validates required fields and enforces a payload cap. It defaults to standard-library sqlite3, so the whole lab runs offline with zero setup, with PostgreSQL as an import-guarded option. An agent fleet with institutional memory is a great idea right up until someone hands it a database connection; the lab's entire teaching point is the layer that doesn't.

Evals that refuse to lie

Problem: "it's better now" without a measured delta is a vibe, and an eval that fabricates a score is worse than no eval at all.

Agentry grades with deterministic checks (tool selection, exact match, F1/BLEU/ROUGE/Levenshtein/Jaccard), an optional LLM judge, and RAG metrics — and it is structurally honest:

• A case with no offline-verifiable expectation is reported skipped, never faked.
• An LLM judge with no model returns a skip, not an invented number.
• A run where everything skipped exits non-zero, unless you explicitly pass --allow-skips.

You measure change over time, too: agentry eval --store runs.jsonl appends each run as JSONL, --compare flags pass → fail regressions against the previous run, and agentry report renders the whole history — plus the verifiable audit trail — into a single self-contained HTML page built from html.escape and string templates. No dashboard framework: the runs are already open JSONL and the CLI already prints the regressions, so a web stack would have added moving parts, not signal.

The one-dependency bet, and the surfaces around it

Problem: a toolkit whose pitch is auditability has no business being too large to audit — and every vendor SDK is supply-chain surface and lock-in you'll regret.

The whole core runs on one runtime dependency: pyyaml. Real LLM execution against four providers (ollama, openai, anthropic, azure_openai) goes through standard-library urllib; each provider is @register-ed, normalizes its response to the same {"tool_call"}/{"done"} contract, exposes typed errors with retry and backoff, imports its SDK lazily if at all, and offers an async fallback. The HTTP governor is http.server with bearer-token auth, a request-body cap, and sanitized errors; a raw ASGI adapter exposes the same routes; an MCP layer speaks stdlib JSON-RPC both ways — consuming external MCP tools governed at the same pre/post_tool_call points as native ones, and exposing Agentry itself as a governed MCP server. Observability is an opt-in OpenTelemetry span (agentry.gov, tagged with point, decision, rule_id) that is a no-op when the extra isn't installed. Everything heavy — a JSON-Schema validator, hosted-provider keys, numpy, uvicorn — is an import-guarded extra, never the price of entry.

The meta-loop: govern the contributions, not just the agents

Problem: the same green-checkmark trap that fools you about your own code fools you about a contribution — especially one an AI helped write.

A contributed PR landed this week adding that L5 fleet lab. CI was green across four Python versions. It was also, underneath: leaking internal infrastructure URLs in shipped code and live-network tests, calling methods that didn't exist, using a YAML schema the runtime doesn't understand — and unregistered, so agentry check never actually validated the lab it claimed to add. Green meant "no check failed," not "done." Again.

What caught it was the same discipline the platform preaches, turned inward: three reviewers on three different models, each with a hostile brief and read-only access, cross-checked the diff against the registration contract and the invariants. They converged on do-not-merge with file-and-line receipts — the internal-URL leak first. The fix wasn't to reject the idea; it was to strip the proprietary and broken parts, make the lab actually runnable and offline, register it so agentry check could see it, ground it in the real canon, and re-validate. The teaching intent survived; the leak and the lies didn't.

That is the recursion I find most useful: a governed harness, reviewed by governed reviewers, contributing to a repo whose own contract forbids the orphan and the dangling citation. Governance isn't a feature you point at the agent. It's how the whole system — code, docs, contributions, and the AI writing them — keeps itself honest.

The field map

• Controls run before the action, at eight points, fail-closed. That's the gap between "the prompt asked the model nicely" and "a verdict a reviewer re-runs and watches turn red."
• Authority only narrows; every verdict rides a hash chain. Delegation can't escalate, and the log can't be quietly edited — containment and proof, not one or the other.
• The fleet shares memory through governed tools, never a raw handle. Read-only, allow-listed, parameterized: institutional memory without the database footgun.
• The contract forbids the orphan; the reviewers run on other models. agentry check won't see an unregistered lab, and three hostile briefs caught the green-but-broken PR. Govern the contributions, not just the agents.

The pitch was governed, grounded, evaluated. The schematics are how it earns each of those words — and keeps the receipts. 🧾