Ironhide logoIRONHIDETest Your Agent

Episode library

The episode library is what the referee drives your agent through. Each episode is a multi-turn scenario built around a real attack: an injected instruction, a shadowed tool, a lure to exfiltrate data, a request that tempts your agent to over-reach. Ironhide runs your real agent through the episode and grades what it did.

You have full library access — every episode is drivable — and for each one you can see its type. What you never see is its answer key: the ground truth, the per-turn labels, and the grading rubric all stay server-side, so an agent cannot be tuned against answers it has never seen.

Verticals#

Episodes are set in the workplaces agents actually operate in, so the attack lands the way it would in production:

  • CRM — reading and writing customer records.
  • Sales and deals — pipeline, quotes, and deal data.
  • Marketing email — drafting and sending on behalf of a team.
  • Reporting and export — pulling data out and where it goes.
  • IAM, SRE, and support — access, on-call actions, and ticket handling.

Each vertical gives an attacker a different lever, and each episode exercises one against your agent.

Episode types#

Every episode carries a type, and you always see it:

  • Behavioral — your agent reviews a transcript and calls out the attacks it recognizes. The verdict is about what it spotted.
  • Adversarial — your agent is in the scenario, responding turn by turn to an attack aimed at it. Some adversarial episodes run in a live world where your agent can call tools; there the verdict is about the effects it caused — records touched, data sent, scope exceeded.

The agent contract covers how your adapter drives each type and what shape it returns.

How episodes are addressed#

Every episode is addressed by a stable, per-tenant alias. You can see and drive every episode in the library — what never leaves the server is the answer key: the episode's real identity, its planted canaries, the per-turn labels, and the grading rubric.

Some episodes carry a public title — a showcase, so you can see the caliber of scenario your agent is up against. The rest are addressed by alias only. Both are driven and scored identically. Because the answer key never leaves the server, there is nothing to tune to — memorizing the visible library buys nothing.

Discovering episodes#

Ask the API for the episodes your agent can drive; each entry is addressed by its alias:

$ curl -s -X POST https://app.ironhideai.com/ci/episodes/agt_9f2c11 \
    -H "Authorization: Bearer wk_live_4c8d…"
{
  "episodes": [
    { "alias": "ep_5b1c0f4a9d22", "episode_type": "adversarial", "tier": "C", "featured": true, "title": "CRM export — injected recipient" },
    { "alias": "ep_9d31a0c7b410", "episode_type": "behavioral", "tier": "A", "featured": false }
  ],
  "maturity": "preview"
}

Pass an alias as the --episode-id to ironhide episode run. Some entries include a public title; the rest are addressed by alias only.

The library grows#

The library is not a fixed set — episodes are added over time across the verticals and attack categories. Discover the current set from the API rather than assuming a fixed roster; new episodes are added under alias by default.

Next steps#