CLI reference

The ironhide CLI is a single-file, dependency-free Python program: a window onto the Ironhide referee. No grading logic lives in it; every verdict it renders comes from the server. This page documents every command. Each command also answers --help with its own flags.

Install and configuration#

$ curl -fsSL https://app.ironhideai.com/install.sh | bash

The installer drops the CLI into ~/.ironhide/ and a wrapper into ~/.local/bin/ironhide. It needs Python 3.9+ and nothing else: no pip installs.

Configuration lives at ~/.ironhide/config.json, written with owner-only permissions (chmod 600). It holds the server URL, your account owner key, and each connected agent (its id and API key). You should never need to edit it by hand: login and connect write it, use retargets it, logout clears it.

ironhide --version prints the CLI version, and running ironhide with no command prints the banner and command list.

Account and agents#

ironhide login#

Sign up (a new email) or log in (an existing email plus its owner key). One account owns all your agents. A new signup proves your email with a short sign-in code, then mints an owner key, shown once and saved to your config; Ironhide stores only a hash.

ironhide login [--email EMAIL] [--owner-key wk_owner_...]

ironhide connect#

Register a ci agent under your account and mint its API key. Ironhide 2.0 is CI-native — your agent runs inside your own pipeline and results flow inbound — so there is no transport to choose. Interactive by default; every prompt has a flag, and when stdin is not a terminal the flags are required. Log in first: the new agent attaches to your account.

ironhide connect [--name NAME] [--email EMAIL]
                 [--contract-version {v1,v2}]

Registration does not ask about data reuse: there is nothing to choose. Every run is stamped owned, and what Ironhide keeps and reuses is the scrubbed environment your agent ran against, not your trajectory. See data rights.

On success the CLI saves the agent id and API key to your config and prints the API key (a wk_live_... token) once: the server stores only a hash and cannot show it again. Store the token as a masked CI secret — see the GitHub Action and GitLab CI pages.

$ ironhide connect --name "acme-agent" --email eng@acme.dev

ironhide agents#

List your account's agents — one row each with id, name, mode, status, and run count — the active agent marked with *.

ironhide agents [--json]

ironhide use <agent_id | url>#

Switch the active agent (pass a saved agent id) or point the CLI at a server (pass a full http(s):// URL). All agent-scoped commands act on the active agent.

$ ironhide use agt_9f2c1a                    # switch the active agent
$ ironhide use https://app.ironhideai.com    # point at a server

ironhide logout#

Clear the saved credentials — the account owner key and every saved agent — from ~/.ironhide/config.json. The server-side account and registrations are untouched: ironhide login and ironhide connect reconnect you, or rotate a key via the API if it may have leaked. The server URL survives logout.

$ ironhide logout

Verify and contract#

ironhide verify#

Run the connection test against your active agent and print a pass with round-trip latency, or the failure with the server's hint.

$ ironhide verify

ironhide contract [v1 | v2]#

Show your agent's contract version, or switch it.

$ ironhide contract        # show the current contract and status
$ ironhide contract v2     # switch to contract v2

Switching resets verification on the server, and the CLI deliberately does not auto-verify: switch, redeploy against the new wire, then run ironhide verify. Registering or switching to v1 is operator-only and is refused for everyone else.

The referee loop#

ironhide episode run#

Drive one episode through the referee: the CLI pulls the episode manifest, lets an adapter drive your agent, submits the trajectory, and renders the verdict the server returns. The verdict is categorical (PASS / FAIL, or INCONCLUSIVE when there was nothing to grade), with a severity, the reason, and the evidence.

ironhide episode run --episode-id ALIAS [--agent-id ID]
                     [--adapter module:callable] [--max-rounds N]
                     [--label LABEL] [--json]

The adapter is any callable that wraps your real agent. It receives the episode prompt and the world's tools, and returns detections, actions, or actions plus tool calls, depending on the episode. See the Quickstart for a ten-line example.

$ ironhide episode run --episode-id a103-exfil-auditor \
    --adapter myapp.ironhide_adapter:run

ironhide report [REPORT_ID]#

A founder-readable readout of your agent's results: your latest by default, or a specific one by id. Ids come from ironhide reports.

ironhide report [REPORT_ID] [--json] [--download PATH]

$ ironhide report                          # latest, rendered
$ ironhide report rpt_abc123 --download .  # save one report as JSON

ironhide reports#

Your agent's report history, newest first — one row per run, with date, verdict, and the report id.

ironhide reports [--json] [--limit N]

Gate CI#

ironhide init#

Write .ironhide.yml in the current directory — the config ironhide test reads, checked in beside your code. It never contains secrets: the API key stays in ~/.ironhide/config.json or the IRONHIDE_API_KEY environment variable.

ironhide init [--force] [--agent-id ID]

ironhide test#

Run the executable attack suite and gate against a client-held baseline. The first run for a given label saves that run set as the baseline (.ironhide/baseline-<label>.json) and exits 0; every later run compares against it. Resolution order for each knob: an explicit flag, then the matching IRONHIDE_* environment variable, then .ironhide.yml, then the default.

ironhide test [--agent-id ID] [--samples-per-episode N] [--label LABEL]
              [--server URL] [--config PATH]
              [--advisory | --no-advisory] [--json]

Every run ends with one greppable line and an exit code — 0 for pass / advisory / baseline, 1 for a gated failure, 2 when the gate could not be evaluated (never rounded to a pass or a failure):

IRONHIDE-GATE basis=arena-l3-preview delta=-0.02 noise_floor=0.05 n=24 result=PASS

ironhide repro#

Re-verify a finding. A deterministic replay re-runs the stored trace exactly (--finding-id / --run-id); a live re-run runs the episode against your agent afresh (--attack-id, stochastic). Exit mirrors the verdict: 0 for a reproduced pass, 1 for a reproduced failure, 2 when the run cannot be verified.

ironhide repro (--finding-id VD-<run_id> | --run-id ID | --attack-id EPISODE_ID)
               [--agent-id ID] [--server URL]

One of --finding-id, --run-id, or --attack-id is required.

$ ironhide repro --finding-id VD-run_4b7e

ironhide comment#

Render a Markdown PR comment from an ironhide test --json payload. Pure rendering — every number comes from the payload, and the preview disclaimer always rides along.

ironhide comment --from-json PATH|-

$ ironhide test --json | ironhide comment --from-json -

Utility commands#

ironhide environments#

List the episode library available to run.

$ ironhide environments

ironhide status#

Config and server health at a glance: CLI version, the server the CLI points at, the config path, a health check, your account, and the active agent's name, mode, run count, and verification status.

$ ironhide status