Testing an AI Agent Before Launch: Go, Wait or Block
Your AI agent passes every test you wrote. Can it still be the wrong one to launch?
IN 30 SECONDS
A release gate is a rule, written before testing, that turns an AI agent’s test results into one verdict: ship, hold or block. It matters because an agent can be right whenever it answers and still answer when it should refuse.
KEY TAKEAWAYS · 3
- A framework says what a model can do. A release gate says whether this agent, on these documents, may ship.
- A clean run only covers the cases you wrote. Add cases where the source is missing.
- Hold: tighten and replay. Block: close the hole first.
On this page
A clean test run that still said not yet
IN SHORT A clean test run only covers the cases you wrote. The case you forgot can still decide the release.
WORDS TO KNOW
- Eval
- One test case for an AI agent: a question, its documents, and what a good answer looks like.
- Frozen eval set
- A fixed list of evals, the same on every release, so two runs can be compared.
- Release gate
- A rule, written before testing, that turns eval results into one verdict: ship, hold or block.
I delivered an initial 11-agent AI platform in production for a confidential US real-estate private-equity fund with roughly $3B in assets. The agents read the fund’s documents, and the platform included evaluations and red-teaming.
This post describes a release gate for that kind of document work. Its rubric is a teaching example, not a public copy of the client’s test suite.
The first test set
One missing-evidence case added
Each Eval One test case for an AI agent: a question, its documents, and what a good answer looks like. See the glossary is one test case. A test set with no missing-evidence case can’t catch an agent that answers when it should refuse.
Coming up next: So why not trust a ready-made testing toolkit?
Why a test toolkit can’t decide for you
IN SHORT A framework tells you what a model can do; only your team can decide whether this agent may ship.
An eval framework is a ready-made testing toolkit. It can tell you this model finds and cites documents, true for a thousand teams. What “good enough” means for your agent comes from your team.
LangChain surveyed professionals about AI agents in late 2025 and got 1,340 responses. About 89% said they had observability, a record of what their agents do.
Source: LangChain, State of Agent Engineering, public survey, 18 Nov to 2 Dec 2025, 1,340 responses.
A release gate belongs to that smaller group: it tests offline before anything ships. It’s the piece of how I ship AI inside regulated finance that says yes or no.
Coming up next: So how does the gate say yes or no?
How the gate decides: go, wait or block
IN SHORT One frozen set goes in, every dimension is scored on the same run, and one verdict comes out.
You replay the same Frozen eval set A fixed list of evals, the same on every release, so two runs can be compared. See the glossary on every version. The Release gate A rule, written before testing, that turns eval results into one verdict: ship, hold or block. See the glossary turns that run into one of three verdicts, written down before release day, so “good enough” isn’t up for debate at 6pm.
IN PLAIN WORDS
Abstention means the agent says “I don’t know” and hands the case to a person. It is calibrated when the agent refuses where evidence is thin and answers where it isn’t.
STEP BY STEP
Step 1. Replay. The candidate runs the same frozen set as last time, so two runs compare.
- Replay The candidate runs the same frozen set as last time, so two runs compare.
- Score Each dimension is scored on that one run: sources, numbers, refusals, attack resistance, speed and cost. Refusing well counts like accuracy.
- Ship All pass and abstention is calibrated: it ships behind human review, and is re-evaluated when its data drifts.
- Hold Accuracy passes, but it answers cases it should hand to a person. Tighten and replay. Don’t ship on a promise.
- Block An attack works, or a number is confidently wrong. Fix the gap first: no accuracy score buys back a leak.
A hold means the agent is capable but miscalibrated: a tighter policy and another run can fix it. A block means something got through, and rerunning changes nothing until that hole is closed.
ADAM says: Think of a car inspection: same checklist every visit, and bad brakes fail the car however nice the paint. Where it breaks: a regulator writes those rules, but your thresholds are yours.
And an inspected car can still crash. A passed gate is no guarantee either.
ITEM GET
The verdict is written first
STAGE CLEAR
What you now know
- A release gate turns one test run into one verdict, by rules written before the run.
- An agent can be right every time it answers and still fail the gate, by answering where it should refuse.
- Hold and block differ in cause: a hold is a miscalibrated agent, a block is a hole that a rerun can’t close.
Check yourself
Tap or click a question to see the answer.
Illustrative case: your agent answers everything correctly, even two questions whose source is missing. Which verdict?
Hold. With no source it should have said “I don’t know”. Tighten the policy, replay the same set.
Former CTO of Integra. Named author (1 of 5) of ERC-3643, first author of ERC-6960, co-author of ERC-7410, and co-author of ERC-8203, which is still a draft. Building production AI and regulated Web3 systems.
Have a project to ship or a role to fill?
Discuss your project or roleFollow the engineering notes
Get new articles on production AI and smart contracts by email.