Loading...

Operator Checklist

Coding Agents in CI: The Pre-Merge Gating Checklist (2026)

What to gate before letting Claude Code, Codex, or Copilot coding agent open PRs in CI: scoped tokens, branch protection that actually applies to bots, secrets isolation, per-run cost caps, and rollback of agent commits — with GitHub's official requirements and the June 2026 billing episode as evidence.

Outcome: Agent-authored PRs that cannot merge unreviewed, cannot leak secrets, cannot run away on cost, and can be rolled back cleanlyUpdated 2026-07-25

Scope

Coding agents in CI checklist

GitHub's own guardrails for Copilot coding agent are the clearest statement of what 'safe agent CI' means in 2026, and they are stricter than most teams' homegrown setups: by default, Actions workflows on agent-created PRs do not run until a human with write access clicks 'Approve and run workflows' (an org can opt out via a setting GitHub added in March 2026), the developer who asked the agent to open the PR does not count as its required approver, and the agent receives no repository or organization secrets except those explicitly added to a dedicated copilot environment (docs.github.com, Copilot coding agent review and responsible-use pages). Cost needs the same explicitness. Anthropic's announced June 15, 2026 move to meter agentic/CI usage separately from interactive Claude Code — reported by InfoWorld as monthly agentic credit pools of $20 (Pro), $100 (Max 5x), and $200 (Max 20x) with overflow billed at API rates — was paused a day later ('not making this change today'), but the episode is the warning: CI agent spend sits on policy-volatile ground, and every agent run is metered token work that compounds across each PR event on a busy repo. Cap it yourself in the workflow rather than trusting plan pricing. This checklist is the gate list before the first agent-authored PR lands.

Credentials and Token Scope

3 checks

Scope the agent's token to exactly the permissions PR work needs

GitHub's docs for agent workflows specify fine-grained tokens with read access to metadata and read/write only on actions, contents, issues, and pull requests — not a classic PAT with broad repo scope on a maintainer account. The official claude-code-action similarly documents Contents, Issues, and Pull requests read/write as its GitHub App permission set (code.claude.com/docs/en/github-actions). A leaked broad token is an org-wide incident; a leaked scoped token is one repo's PR queue.

high

Isolate secrets: expose only an explicit allowlist to agent jobs

GitHub's design for Copilot coding agent passes no org or repo secrets to the agent except those deliberately placed in the copilot environment. Replicate that for any agent runner: a dedicated environment holding only the model API key and the scoped GitHub token, so a prompt-injected agent cannot read your deploy credentials out of the job env.

high

Run agent jobs with egress and tool restrictions, not just token restrictions

The agent executes arbitrary model-chosen commands inside your runner. Restricting tools via claude_args (--allowedTools / --disallowedTools — the action's docs note Bash is not allowed by default and must be allowlisted per command) and hardening runner egress limits both what a wandering agent does and what an injected one can exfiltrate. The action's own docs/security.md warns that permissive settings like allowing all bots on public repos hand prompt control to outsiders.

medium

Branch Protection and Review Gates

4 checks

Block agent pushes to the default branch; agents get exactly one PR branch

GitHub enforces this for Copilot coding agent — it can only push to the single branch associated with its PR and never directly to main. Enforce the same for self-hosted agents via branch protection: the agent's identity has no push access to protected branches, full stop.

high

Require a human approver who is not the person who invoked the agent

Under GitHub's rules, the requester's own approval of a Copilot-created PR does not count toward required approvals — someone else must review. This closes the self-approval loophole where one person effectively merges unreviewed machine-written code. If you enable 'Require approval of the most recent reviewable push', the agent's last push needs n+1 approvals.

high

Do not auto-trigger CI workflows on agent pushes; require explicit approval to run

On Copilot PRs, Actions workflows do not run automatically after the agent pushes by default — a human must click 'Approve and run workflows', and GitHub only added an opt-out setting in March 2026 (docs.github.com). The reason applies to every agent: workflow runs execute with repository credentials, so an agent that can trigger workflows by pushing can escalate a bad diff into credentialed execution.

high

Route agent PRs through CODEOWNERS for risk-tiered paths

CODEOWNERS on auth, billing, infra, and workflow files means an agent PR touching those paths cannot merge without the owning team — including changes to the agent's own workflow definitions, which is how an agent (or its prompter) would otherwise relax its own guardrails.

medium

Cost Caps Per Run

4 checks

Set a turn cap (--max-turns) and a job timeout on every agent job

The official claude-code-action documents capping agent iterations with claude_args: "--max-turns 5" — note the old max_turns input is deprecated in v1 in favor of claude_args (github.com/anthropics/claude-code-action docs/usage.md) — alongside the workflow-level timeout-minutes job property. These are the only hard ceilings between you and a looping agent burning tokens for six hours on a stuck test.

high

Pin a cheaper model for routine CI tasks via claude_args

Model choice dominates per-run cost, and per-run cost is turns times context times per-token price — the same task on an Opus-class model costs a multiple of a Sonnet-class run for little review-quality gain on routine PR chores. The action documents pinning the model with claude_args: "--model <model-id>" (docs/configuration.md); default to a cheaper model there and reserve top models for explicitly invoked deep work.

medium

Treat CI agent billing as policy-volatile: meter your own spend

Anthropic's announced June 15, 2026 split — separate monthly agentic credits ($20 Pro / $100 Max 5x / $200 Max 20x, then API list prices) for non-interactive use, per InfoWorld — was paused a day later before rollout, but it signals where providers are heading: CI usage metered separately from your subscription. Tag agent API keys per-repo and alert on spend so a pricing change or a runaway workflow shows up in a dashboard, not an invoice.

medium

Restrict trigger events to explicit invocations, not every push

An agent that wakes on every push, comment, and PR event on a busy repo runs hundreds of times a week. Gate triggers on an explicit mention (@claude), a label, or a manual dispatch so each run maps to a human intent — this is simultaneously your cost control and your prompt-injection surface reduction, since arbitrary commenters should never be able to summon the agent.

high

Rollback and Attribution

3 checks

Make agent commits attributable: dedicated bot identity and co-author trailers

When a regression bisects to an agent commit, you need to know instantly that no human wrote it and which run produced it. A dedicated bot committer identity plus the run URL in the commit or PR body turns 'who wrote this?' from an investigation into a link.

medium

Keep agent PRs small and single-purpose so revert is one commit

Squash-merging one bounded agent task per PR means rollback is a single git revert. Letting an agent batch refactors with fixes produces the un-revertable mixed commit that turns a bad merge into an incident. Bound the task in the prompt, and reject agent PRs that exceed the requested scope in review.

medium

Dry-run the full agent pipeline on a sandbox repo before production repos

Verify each gate actually fires: an unapproved workflow does not run, the agent identity cannot push to main, self-approval does not satisfy required review, a deliberately looping prompt hits max_turns and stops. Gates you have never seen trigger are gates you do not have.

low

Common Mistakes

  • Running the agent under a maintainer's classic PAT with full repo scope instead of a fine-grained token limited to contents, issues, pull requests, and actions.
  • Letting the person who invoked the agent approve its PR — the self-approval loop GitHub explicitly disallows for Copilot coding agent.
  • Auto-running CI on agent pushes, giving a compromised or confused agent a path from bad diff to credentialed workflow execution.
  • Shipping agent workflows with no turn cap (--max-turns) or job timeout, then discovering a stuck run has been iterating on a failing test for hours at API prices.
  • Triggering the agent on every push, comment, and PR event on a busy repo — hundreds of metered runs a week that nobody explicitly asked for, invisible until the invoice.
  • Leaving the agent's own workflow files outside CODEOWNERS, so an agent PR can quietly edit the rules that constrain the agent.
  • Planning CI budgets around subscription pricing that providers have already shown they may re-meter — the June 2026 announce-then-pause episode is the precedent.