Computer-use benchmarks test a standard desktop, not an agency's software
OSWorld, a leading benchmark for agents that operate computers, uses everyday applications. Agencies should test agents on their own systems before deploying them.
OSWorld evaluates AI agents on 369 real computer tasks in operating systems such as Ubuntu, Windows and macOS. Its authors report that people complete over 72 percent of the tasks. An upgraded OSWorld-Verified release followed in July 2025, and OSWorld 2.0 was announced in June 2026.
Benchmarks like this are useful for comparing models, but they use general-purpose applications. Public operators rely on specialist dispatch, scheduling and vehicle-location software that no public benchmark covers.
ESS view: before deploying any computer-use agent, an agency should run it against a recorded set of its own tasks, in a test copy of its own systems, and measure completion and error rates there.