Ironhide logoIRONHIDETest Your Agent

GitLab CI

GitLab doesn't consume GitHub Actions, but the referee doesn't need one — the action is only a thin wrapper around ironhide test, which runs on any CI. On GitLab the same sweep runs as a normal job: it pulls the agent your merge request builds into a sandboxed arena, attacks it, and posts a categorical verdict on what the agent did — state changes, tool calls, exfiltration — not on what it said about itself.

Same CLI, same five verdicts, same exit codes as the GitHub Action. The two framing rules carry over unchanged:

  • Advisory-first. The job defaults to advisory mode: it posts the verdict as a merge-request note and lets the pipeline pass, failing only once you set advisory_mode: "false". Run advisory for a week or two, watch the baseline settle, then turn gating on.
  • Preview. Observed-state verdicts carry the arena-l3-preview basis and a preview stamp — an honest measurement, not a certification.

Add the job#

You need two things: an include: in your .gitlab-ci.yml, and a few masked CI/CD variables.

Preferred — the CI/CD component:

include:
  - component: $CI_SERVER_FQDN/ironhide-ai/ironhide-scan/ironhide@v1

Or a plain remote include (identical job, no component registry):

include:
  - remote: "https://gitlab.com/ironhide-ai/ironhide-scan/-/raw/v1/templates/gitlab-ci.yml"

The job only runs on merge-request pipelines ($CI_PIPELINE_SOURCE == "merge_request_event") and installs the CLI itself, so nothing else in your pipeline changes.

CI/CD variables#

Set these under Settings → CI/CD → Variables (mask the secret ones):

variablerequiredwhat it does
IRONHIDE_API_KEYrequiredYour agent's key from ironhide connect. Mask it; never commit it.
IRONHIDE_AGENT_IDrequired unless committedThe connected agent's id, so a fresh CI checkout can resolve the agent. Optional if you commit a .ironhide.yml (ironhide init) that carries agent_id.
GITLAB_TOKENoptionalA project access token with api scope, only to post the verdict as a merge-request note — GitLab's built-in CI_JOB_TOKEN can't create notes. The gate is enforced by the job's exit code with or without it.

The behavioural knobs also read from CI/CD variables, so you can flip them without editing the include: IRONHIDE_ADVISORY ("true" / "false"), IRONHIDE_BLOCK_ON (default exfiltration), IRONHIDE_URL, and IRONHIDE_BASELINE. To gate instead of advise, set IRONHIDE_ADVISORY: "false".

Component inputs#

If you use the component form, the same settings are also available as typed inputs:

include:
  - component: $CI_SERVER_FQDN/ironhide-ai/ironhide-scan/ironhide@v1
    inputs:
      advisory_mode: "false"      # gate on FAIL / BLOCK
      block_on: exfiltration
      baseline: "$CI_DEFAULT_BRANCH"
inputdefaultwhat it does
stagetestPipeline stage the job runs in.
advisory_mode"true""true" posts the verdict without failing the pipeline; "false" gates on FAIL / BLOCK.
block_onexfiltrationComma-separated effect categories that hard-block when gating.
serverhttps://app.ironhideai.comIronhide server base URL.
baseline$CI_DEFAULT_BRANCHBaseline label this MR is gated against.

The merge-request note and gate line#

When GITLAB_TOKEN is set, the job posts (and refreshes, on later pushes) one note on the merge request with the verdict, its preview basis, and the machine-readable gate line. That line is also printed in the job log for any pipeline that greps it:

IRONHIDE-GATE basis=arena-l3-preview runs=24 clean=21/24 baseline=0.92 shift=-4.5pt floor=6.0pt result=PASS

The verdict semantics — PASS / FAIL / BLOCK / WARN / INCONCLUSIVE, plus each gate-line field — are identical to the GitHub Action; see The five verdicts and The gate line and exit codes for the full breakdown.

The job exit code is the gate:

codemeaning
0PASS, WARN, INCONCLUSIVE; any result while advisory; or a baseline-establishing run.
1FAIL, or BLOCK per block_on, with advisory_mode: "false". The pipeline fails and the note says which effects were observed.
2Ironhide unavailable — reported, never silently passed.

The first sweep on the baseline branch establishes the baseline (result=BASELINE, exit 0) and is not graded against itself.

Reproduce a finding locally#

Every finding in the merge-request note carries an id. Reproduction is deterministic and identical to every other surface:

ironhide repro --finding-id fnd_7c21a9

Same environment seed, same attack, same wire — replay the trajectory that produced the observed effect, fix it, and re-run. If the effect is gone locally, the next sweep on your MR will show it gone too.

Where to go next#

  • Quickstart: connect your agent and get a first verdict.
  • GitHub Action: the same referee for GitHub pull requests, with the full verdict and gate-line reference.
  • CLI reference: ironhide test, ironhide repro, and every other command behind the job.