Coding agents and the public operator
What today's agent runtimes can do, how they are controlled, what benchmarks show and miss, and what a public agency should require before deploying one.
Key points
- The leading coding agents now ship sandboxes and approval modes, but with different defaults; agencies should set them.
- Benchmarks such as OSWorld and ARC-AGI-3 measure general ability, not performance on an agency's own systems.
- Provider safety policies are disclosures; operational safety comes from the buyer's own permissions, logs, rollback and task evaluations.
What has changed
AI agents that read a codebase, run commands and propose changes are a prominent form of agentic AI. Claude Code, OpenAI's Codex CLI and GitHub Copilot CLI are widely used examples. Their usefulness depends less on the underlying model than on the runtime around it: the tools it can call, what it can read and write, and how its work is checked.
How the leading tools are controlled
A comparison published in October 2026 shows that all three now offer sandboxing, but start from different defaults. Codex CLI is sandboxed by default, with read-only, workspace-write and full-access modes, network access off until enabled, and approval prompts before it crosses the sandbox boundary. Claude Code's sandbox is off by default; managed settings can require it and block unsandboxed retries, and its network traffic runs through a proxy with an empty allowlist. GitHub Copilot CLI's local sandbox is off by default, with outbound network access on, and administrators can disable sandbox bypass through managed settings.
What benchmarks show, and miss
OSWorld tests agents on 369 real computer tasks in operating systems such as Ubuntu, Windows and macOS; its authors report that people complete over 72 percent. The ARC Prize Foundation's ARC-AGI-3, released in March 2026, uses interactive environments that human testers solved completely while frontier AI systems scored below 1 percent. Both are useful for comparing models. Neither covers the dispatch, scheduling and vehicle-location systems that public operators run.
Provider policies and buyer controls
Anthropic's Responsible Scaling Policy version 3.0, published in February 2026, adds regular risk reports, a public safety roadmap and external review in certain circumstances, and acknowledges that capability thresholds proved harder to apply than expected. Policies like this are valuable disclosures from the companies that deploy the models. They do not replace the controls an agency applies to its own operations.
ESS view: what an agency should require
Enforce the sandbox, network allowlist and approval mode through managed settings. Run any agent first in shadow mode against a recorded set of the agency's own past cases, in a test copy of its systems, and measure completion and error rates there. Keep complete logs and a tested rollback. Ask vendors to report results on the agency's tasks, not on general leaderboards.