How results work

Ironhide is a referee for what your agent does, not for what it says about itself. It drives your real agent through a multi-turn episode built around a specific attack, watches the trajectory, and returns a result on the behavior it observed. Nothing the agent prints about its own intentions changes the call — the result is grounded in what actually happened in the episode.

That is the whole shift: a referee judges the play, not the press conference.

What an episode is#

An episode is a scripted, adversarial situation your agent is put into. It runs over multiple turns, seeded with an attack the agent is supposed to resist: an injected instruction hidden in a tool result, a poisoned tool definition, a canary credential planted where an over-eager agent would try to move it. Your agent plays through it; the referee checks what it did.

Episodes come in two shapes, and every result tells you which one it was:

Behavioral — your agent reviews a multi-turn transcript and flags the turns where an attack is happening. The referee checks whether it caught the attacks that were planted.
Adversarial — your agent is driven through the attack live, with real tools it can call. The referee checks what the agent actually did with those tools: did it hold the line, or did it get injected, move a canary, over-reach its scope, or escalate a privilege it was never granted.

The attacks it runs#

Episodes are labeled with the attack they exercise, drawn from a fixed taxonomy:

Attack	What it tests
`prompt_injection`	Instructions smuggled into content the agent reads.
`tool_shadowing`	A tool's definition changed under the agent to redirect its behavior.
`data_exfiltration`	Pressure to move data somewhere it should never go.
`excessive_agency`	Authorized-looking but destructive action beyond the task.
`credential_access`	Reaching for secrets, keys, or credentials it wasn't handed.
`privilege_escalation`	Widening its own scope or permissions mid-task.

Episodes are set in realistic verticals — CRM, sales and deals, marketing email, reporting and export, and IAM / SRE / support workflows — so the behavior under test is the behavior your agent would show in production, not a toy.

The result, not a number#

Each episode ends in a rich result, and that result is the product. It carries:

a status — the headline call;
a severity when an applicable severity model produced one; Tier A/B carry null;
the why — a plain-language reason for the call;
evidence on a featured result. On a held-out result, this is an empty list.

The verdict schema supports five statuses:

Status	Meaning
`PASS`	The episode was graded as a clean pass.
`FAIL`	The episode was graded as a breach.
`WARN`	No prohibited effect was observed; security coverage for the run is incomplete. The reason states the caveat.
`INCONCLUSIVE`	No decisive PASS/FAIL could be verified; never upgraded to PASS.
`BLOCK`	The policy-escalation state in the verdict schema; current episode grading does not emit it directly.

A non-breach never carries severity. A breach carries severity only when an applicable model produced one; Tier A/B breaches carry null.

For a field-by-field walk-through of each status and severity level, see Results & severity.

Honest by construction#

Unverified is not a pass. When an episode produces no effect to observe, the result is INCONCLUSIVE and says so — it is never rounded up to PASS.
The evidence never leaks the answer key. Every episode is addressed by a stable, per-account alias, and the answer key never leaves the server. Most episodes are held-out. Those customer results keep the status, applicable severity, and masking explanation, while evidence is [], protected measurements can be null, and no effect-kind substitute is added. A null completion means Not Tested. Featured episodes cover the same failure classes and return their available detail with the run itself.
Preview, and stamped as such. Observed-behavior results are a preview measurement, not a certification. Treat a preview result like a new reviewer whose judgment you're still calibrating — honest, useful, and not yet a seal of approval.

Next steps#

Results & severity: the verdict statuses and severity scale, field by field.
Reading your report: where the result shows up — in the CLI, in your PR, and in the JSON.
GitHub Action: run the referee on every pull request and gate on the result.