Why Ironhide
Pass the enterprise security review, every release.
Transcripts can be talked around; the state of the world can't. Ironhide drives your real agent through adversarial episodes in a sandboxed world, reads what it actually did, and posts the result with its evidence on the pull request, so you know what you're shipping before a reviewer or an attacker asks.
A result nobody can argue with
Your agent's builder, your model vendor, and your own eval prompt all have a stake in a pass. A referee doesn't. Ironhide runs your real agent, records what happened to the world, and reports it — it has no incentive to call a breach a refusal.
Proof, not a transcript
A transcript is what the agent says about itself. State is what actually happened: rows deleted, tools called, canaries that left the sandbox. Ironhide reads the diff of the world, so a confident apology never outranks a dropped table.
Caught in the PR that caused it
Agent behavior shifts when the prompt, the model, or a tool changes — which is every pull request. A quarterly review can't see that. A gate on the PR catches the regression in the same diff that caused it.
The comparison
How agent security gets bought today
Every row buys you confidence a different way. Only one of them tests what your agent actually did to a world under attack, before the merge, with a finding you can replay.
| Approach | Tests actions, not text | World under attack | Runs on every PR | Replayable finding | No production exposure |
|---|---|---|---|---|---|
LLM-as-judge eval A second model reads the transcript and scores it | ✕ | ✕ | ✓ | ✕ | ✓ |
Static injection benchmark Fixed public prompt set, text-only pass/fail | ✕ | ~ | ✓ | ✓ | ✓ |
Runtime guardrail Filters live traffic, after deploy, where mistakes are real | ~ | ✕ | ✕ | ✕ | ✕ |
Manual agent red team Humans probe a staging build, weeks per engagement | ✓ | ✓ | ✕ | ~ | ✓ |
Ironhide Sandboxed world, attacked; the state it leaves is the result | ✓ | ✓ | ✓ | ✓ | ✓ |
The others test what your agent says, or catch it after deploy; Ironhide tests what it did, before the merge.
Not an infrastructure pen test
An infrastructure pen test probes your servers and network, once, and hands you a PDF. Ironhide pen-tests the agent you're about to hand them to, and re-tests it on every change.
Not a runtime guardrail
A guardrail argues with your agent in production, where a mistake is already live. Ironhide is pre-deploy: the damage happens in a sandboxed world, the trace is recorded, and the result lands before the merge.
A pre-deploy referee
Ironhide wires into CI, drives your real agent through a sandboxed world seeded with attacks — prompt injection, tool shadowing, canary exfiltration — and reads the state it leaves behind. The result — pass/fail, severity, why, and evidence — is a check on the PR.
Observed-state results are preview, never a certification.
A result tells you what your agent did in a sandboxed world under a specific set of attacks. That is strong evidence, and we refuse to dress it up as more:
A FAIL is a fact
Every failure is a recorded state change — a deleted row, a canary that left the sandbox — with a one-line repro. Replay it before you believe it.
A PASS is bounded
A pass means no attack in this run landed — not that none ever will. The attack set grows, and the gate re-runs on every PR.
Never a certificate
Preview results carry the preview label everywhere they appear. They are never sold as certification and never dressed up as a guarantee.
Preview means preview: observed-state results are reported with the label attached and are never benchmarked, sold, or trained on as anything else.