Skip to main content

5 posts tagged with "evals"

View All Tags

When Software Stopped Being a Recipe

· 9 min read
Hassan Tariq
Engineer · AI agents, cloud

The problem, stated plainly: software engineering trained us to make behaviour repeatable, then handed us a component whose useful output has to be judged across a distribution.

That is not traditional development with a faster autocomplete. It changes what the important artifact is, what verification means, and what an engineer has to be good at.

I Read 31 Agent Loops — Most Stop When They're Tired, Not When They're Right

· 8 min read
Hassan Tariq
Engineer · AI agents, cloud

The problem, stated plainly: most agent loops don't know when to stop. They stop when they run out of turns, when they hit a wall they can't get past, or when a human finally looks — and then they call whatever state they're in "done." Repeating is the easy part. Knowing the latest pass is actually right — and not just the pass you happened to be on when the budget ran out — is the whole game.

This week the loop became a named, shareable artifact — a library you can browse, copy, and submit to. But the names aren't the interesting part. The exits are. Here's what changed my mind, the four ways a loop can stop ranked by how much trust each one earns, and why — now that generation is nearly free — the exit is the only part left worth engineering.

Inside Agentry: How a Governed Agent Harness Actually Works

· 12 min read
Hassan Tariq
Engineer · AI agents, cloud

The problem, stated plainly: you cannot trust an agent you cannot constrain, ground, measure, or reconstruct — and "trust me" is not an architecture. A model emits tokens. Everything that decides whether those tokens are safe to act on in production lives in the code around it.

The last Agentry post made that pitch. This one opens the hood — every pillar, in the weeds: the loop, the policy engine, identity, the audit chain, grounding, fleets, evals, and the one-dependency bet that holds it all together.

Agentry: Governed Agents That Can Prove What Happened

· 7 min read
Hassan Tariq
Engineer · AI agents, cloud

In the last post I reviewed "a toolkit I'd been building." This is that toolkit — and the opinions baked into it.

The pitch in one line: a model is one component; everything that makes it safe and useful in production lives in the harness around it. Agentry is that harness — governed, grounded, evaluated — with one property I care about a lot: it can prove what it did.

Green Doesn't Mean Done: Reviewing an Agent Toolkit with a Sub-Agent

· 8 min read
Hassan Tariq
Engineer · AI agents, cloud

I pointed my coding agent at a toolkit I'd been building — an agent harness with governance, grounding, and evals baked in — and gave it one job: prove it's complete. The validator was green. Fourteen unit tests passed. Every command in the README's quickstart did exactly what the README said.

So the agent did the one thing that actually earns trust: it refused to believe itself. It spun up a second agent, on a different model, and told it to try to break every claim the toolkit made. Ten minutes later I had a punch-list of four places where my toolkit was quietly lying to me — with file-and-line receipts for each.