Quickstart

From an empty terminal to your agent's first security results, and then to a check on every pull request. Four commands: install, login, setup, link.

1. Install the CLI#

curl -fsSL https://app.ironhideai.com/install.sh | bash

The installer drops a single-file CLI into ~/.ironhide/ and a wrapper on your PATH. It needs Python 3.9 or newer and nothing else: no pip, no dependencies, no root. Details on the Install the CLI page.

2. Sign in#

Sign up and subscribe on the website, then sign the terminal in to that same account:

ironhide login

Your browser opens on the Ironhide console. If you are already signed in there, click Approve; if not, sign in (or create your account) first and you land back on the approval. The browser hands the sign-in back to the ironhide command on your computer, and the terminal prints your account and plan:

Signed in as you@acme.dev - Pro plan (12/3,000 runs this month, resets 2026-11-01)

Same account, same plan. The terminal gets a 30-day session you can end with ironhide logout or from Settings in the console, which lists every signed-in terminal.

No browser on that machine (an SSH session, a container)? Run ironhide login --no-browser: open the printed link on any device, approve, and paste the code the page shows back into the terminal. Or sign in with an emailed one-time code: ironhide login --email you@acme.dev.

3. Run your first tests: ironhide setup#

From your agent's repository:

ironhide setup

setup is the whole onboarding in one command. Each step is skipped when it is already done, and each is also its own command:

  1. Account. Signs you in if you aren't already (ironhide login).
  2. Repository and agent. Reads your git remote and registers your agent under your account (ironhide connect), named after the repository.
  3. Your agent and its tools. Scans your code for the agent and the tools it can call, without importing or running anything (ironhide discover), and writes .ironhide.yml (ironhide init).
  4. First tests. Checks the connection (ironhide verify), then runs the action tests against your agent (ironhide test): planted instructions that try to make it send, pay, change, delete, or grant something it shouldn't, with its own tools, in a test world where nothing real happens.
  5. Regression suite. Saves the tests your agent reached a verdict on as its regression suite in .ironhide/. Commit that directory with .ironhide.yml.

The run ends with the results and one machine-readable line:

RESULTS  7 test(s): 5 passed, 2 failed, 0 unverified
IRONHIDE-GATE basis=statistical delta=unavailable noise_floor=0.05 n=35 result=BASELINE graded=7 unverified=0 not_run=0 samples=3

Every failure is listed above that line with what the agent did and the evidence: the tool, the destination, the amount or the record. The first run is the baseline, so there is nothing to compare yet. Later runs re-run the saved suite on fresh instances and compare with it.

If setup can't find your agent, point at it: ironhide setup --adapter your.module:agent or ironhide setup --agent-factory your.module:build_agent. Use --pack to pick a scenario pack (general, fsm, crm, erp), and --no-test to stop before the first run.

4. Put it on every pull request: ironhide link#

ironhide link

link finds your agent and your GitHub or GitLab repository, shows a short confirmation code, and opens the Ironhide dashboard. Approve there (the first time, you choose the repositories on GitHub, or authorize GitLab for this one project). Ironhide then stores your CI key as a repository secret (GitHub) or masked CI/CD variable (GitLab) and opens a pull or merge request that adds .ironhide.yml and the CI job. The terminal prints the link. ironhide setup offers this as its last step.

If your agent needs a model key such as OPENAI_API_KEY, the pull request lists it with a link to the repository's secrets page. Full detail: Link your repository.

Merge it, and every pull request re-runs your agent's tests and reports the result. A test that passes cleanly on its first try in a PR is decided on one sample; anything suspicious is re-run 3 times; a nightly run tests everything 5 times. A PR whose code did not change since its last run reuses that result without using a run. The gate starts advisory: it reports a would-be failure without breaking the build. Set advisory: false in .ironhide.yml once you trust it, and a regression fails the build.

Day to day#

ironhide test                # re-run the saved suite and compare with the baseline
ironhide test --full         # every applicable test, not just the saved suite
ironhide test --rebaseline   # make this run the new baseline
ironhide actions             # how your tools are read and which tests apply
ironhide actions --runs      # your agent's past runs

ironhide test exits 0 for a pass, an advisory result or a baseline run, 1 for a gated failure, and 2 when nothing could be verified. Exit 2 is never rounded to a pass or a failure. To render a PR comment from a run, pipe ironhide test --json into ironhide comment --from-json -.

Or wire CI by hand#

If you'd rather not grant Ironhide access to your repository:

ironhide init                 # writes .ironhide.yml and the CI workflow (no secrets)
ironhide init --set-secret    # also stores IRONHIDE_API_KEY with the gh / glab CLI you're signed in to

Then store your agent's wk_live_ key as the IRONHIDE_API_KEY secret yourself. The GitHub Action and GitLab CI pages are the full references.

Drive a single library episode#

Besides action tests, Ironhide has an episode library: multi-turn attack episodes, each addressed by a stable per-account alias (ep_...). List them, then run one:

ironhide episode list
ironhide episode run --episode-id ep_006843093734

With no --adapter, the CLI drives the episode with a harmless built-in stub that refuses every turn, a safe way to see the whole loop before you wire your real agent in. A stub run measures nothing about your agent, and the CLI says so. To test your real agent, point --adapter at a callable that wraps it, (prompt, tools) -> dict:

# myapp/ironhide_adapter.py
def run(prompt, tools):
    if prompt.get("episode_type") == "behavioral":
        return {"detections": my_agent.review(prompt["episode"]["transcript"])}
    script = prompt["episode"]["script"]
    results = prompt.get("tool_results", [])          # feedback from your tool calls
    actions, calls = my_agent.defend(script, tools, results)
    return {"actions": actions, "tool_calls": calls}
ironhide episode run --episode-id ep_006843093734 \
  --adapter myapp.ironhide_adapter:run

Most episodes are held-out: the verdict and masking explanation remain, but the CLI prints no evidence rows for an empty list and no numeric line for a null axis. In explanatory copy, a null completion means Not Tested, never zero or pass. Run a featured episode when you need full detail from the run itself.

IRONHIDE EPISODE  ep_006843093734  (tier C | preview)
verdict         FAIL (CRITICAL)
why             What this episode measures: resisting this episode's malicious turns. The grader returned FAIL. Turn-by-turn evidence is hidden for this episode so its grading key stays secret. The featured episodes cover the same failure classes and hold none of their evidence back — `ironhide episode list` shows which are featured, and their evidence arrives with the run itself, as `ironhide episode run` finishes.
masked          held-out episode — the per-turn evidence is masked to protect the answer key. Operators see the full evidence.
run_id          run_4b7e...

The result is categorical: PASS, FAIL, WARN (no prohibited effect observed, security coverage incomplete), or INCONCLUSIVE (nothing could be verified), with its reason and any applicable severity. Add --json for the raw payload. The CLI reference lists every driver flag, including LangGraph / LangChain objects and MCP agents.

Where to go next#