Ironhide logoIRONHIDETest Your Agent

Connect your agent

Ironhide is a referee for your real agent. You run the agent you are about to ship — in your own pipeline, on your own infrastructure — and Ironhide drives it through adversarial episodes and returns a verdict on what it actually did. There is nothing for you to expose: Ironhide never calls out to your agent. You authenticate with an API key and your results flow inbound.

That shapes the whole connection story. There is exactly one connection mode: ci. You register an agent, get a wk_live_ key, and from then on every episode you drive and every CI run authenticates with that key.

Log in first#

An account owns all of your agents; billing, the plan, and the shared free-run pool live on the account. Log in (or sign up) before you connect:

$ ironhide login --email you@acme.dev

A new email creates an account and mints an owner key (shown once — save it). An existing email logs in with --owner-key. The CLI stores credentials in ~/.ironhide/config.json (chmod 600); Ironhide keeps only a hash.

Register a CI agent#

ironhide connect registers a ci agent under your account, mints the wk_live_ token, and prints it exactly once:

$ ironhide connect --name "AcmeAgent" --email sec@acme.dev
ok - registered  agt_9f2c11  (data owned)
  IRONHIDE_API_KEY:  wk_live_4c8d0e...
  shown once - saved to ~/.ironhide/config.json (chmod 600). Ironhide stores only a hash.

  Every run is owned. Ironhide keeps the scrubbed environment your agent runs
  against - not your trajectory - to grow the shared attack library.

next - gate it in your pipeline:
  * add the token to your CI secrets as IRONHIDE_API_KEY
  * add the GitHub Action ironhide-ai/ironhide-scan@v1 (or run ironhide test in your pipeline)
  * every run returns a verdict on the ironhide/referee check

The same registration is one API call, owner-authenticated. mode must be "ci" — the server 422s any other value:

$ curl -s https://app.ironhideai.com/agents \
    -H "Authorization: Bearer $IRONHIDE_OWNER_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "AcmeAgent",
      "mode": "ci",
      "contact_email": "sec@acme.dev"
    }'
{
  "agent_id": "agt_9f2c11",
  "api_key": "wk_live_4c8d…",
  "data_rights": "owned",
  "status": "unverified",
  "contract_version": "v2",
  "plan_id": "developer",
  "subscription_status": "trialing",
  "free_run_limit": 100
}
FieldRequiredNotes
nameyesMust be unique across your registrations; a duplicate fails with 409 name_taken.
modeyesAlways "ci". Any other value is rejected.
contact_emailoptionalUsed for report and welcome emails; skipped when absent.

New registrations are on Agent Contract v2, the default and only customer contract — see the agent contract.

Every run is stamped owned — there is no data-rights choice to make and no data_reuse_opt_in field to send. What Ironhide keeps and reuses is the scrubbed environment your agent ran against, not your trajectory. See data rights.

Verify the connection#

Every agent starts unverified and cannot drive real episodes until it passes a connection test. Run it as often as you like — no trajectory is recorded:

$ ironhide verify

See verify the connection for what the probe checks and how to read the result.

Drive an episode#

With the agent verified, drive a single episode locally. You point the CLI at an episode alias and at a small adapter — a callable that wires your real agent into the episode:

$ ironhide episode run --episode-id ep_5b1c0f4a9d22 \
    --adapter my_pkg.agent:run

The CLI pulls the episode manifest, lets your adapter drive it, submits the resulting trajectory, and renders the referee's verdict — pass or fail, severity, why, and evidence. See running an evaluation for the full episode flow, the adapter contract, and how the verdict reads.

Gate it in CI#

Once you can drive episodes, wire the referee into your pipeline so every change is checked before it ships:

  • GitHub Actionironhide-ai/ironhide-scan@v1, posting the verdict on the ironhide/referee check.
  • GitLab CI — the same referee as a normal merge-request job.

Both are thin wrappers around ironhide test, which runs the suite and gates on the result. Store your wk_live_ key as the IRONHIDE_API_KEY secret; never commit it.

Next steps#