Coding Agents
POWER IN AI AGENTSMEASURED FOR PEAK OUTPUTON REAL SOFTWARE WORK
THE PERFECT SETUPHARNESS AND MODEL PAIREDTO UNLOCK YOUR DRIVE
CRAFTED FROM LIVE BENCHMARK DATA TO DELIVERRECOMMENDATIONS FOR OUR COMMUNITY
Real software-engineering tasks, run through complete agent stacks — harness plus model — and scored on what actually ships: solve rate across three task suites (higher is better), dollars per task, tokens consumed, and minutes on the clock (all lower is better).
Coding Agent Index
Higher is betterEqual-weight mean of the three suites · top configurations
- 1
Claude Code · Fable 5 (max) (with fallback)77.2 - 2
Codex · GPT-5.5 (xhigh)76.4 - 3Claude Code · Opus 4.8 (max)72.6
- 4Codex · GPT-5.4 (medium)71.1
- 5Claude Code · Opus 4.6 (medium)71.1
- 6Codex · GPT-5.5 (medium)70.5
- 7Cursor CLI · GPT-5.4 (medium)68.8
- 8Claude Code · Opus 4.8 (medium)67.0
- 9
Cursor CLI · Composer 266.6
Cost per Task
Lower is betterMean USD per completed task · cheapest configurations
- 1Cursor CLI · Composer 2$0.04
- 2Cursor CLI · Composer 2.5$0.08
- 3
Claude Code · DeepSeek V4 Pro (high)$0.27 - 4Cursor CLI · Composer 2.5 Fast$0.55
- 5
Claude Code · Kimi K2.6$1.18 - 6Claude Code · Opus 4.6 (medium)$1.26
- 7
Gemini CLI · Gemini 3.1 Pro (high)$1.44 - 8Cursor CLI · GPT-5.4 (medium)$1.52
- 9Claude Code · Opus 4.7 (medium)$1.68
Time per Task
Lower is betterMean wall-clock minutes per task · fastest configurations
- 1Claude Code · Opus 4.7 (medium)6.3m
- 2Codex · GPT-5.5 (medium)6.4m
- 3Cursor CLI · GPT-5.5 (medium)6.6m
- 4Cursor CLI · Composer 2.5 Fast6.8m
- 5Codex · GPT-5.4 (medium)7.1m
- 6Gemini CLI · Gemini 3.1 Pro (high)7.2m
- 7Claude Code · Opus 4.6 (medium)8m
- 8Cursor CLI · GPT-5.4 (medium)8.3m
- 9Cursor CLI · Composer 28.6m
Coding agent comparison summary
Every harness × model configuration we track, ranked by Coding Agent Index.
| # | Agent | Model | Lab | Index | DeepSWE | Term-Bench | Atlas QnA | $/Task | Tokens/Task | Time | Turns |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude CodeAnthropic | Claude Fable 5 (max) (with fallback) | Anthropic | 77.2 | 66.1 | 82.1 | 83.3 | $11.75 | 13.9M | 23.5m | 138 |
| 2 | CodexOpenAI | GPT-5.5 (xhigh) | OpenAI | 76.4 | 64.3 | 84.1 | 80.8 | $5.07 | 12.3M | 10.1m | 106 |
| 3 | Claude CodeAnthropic | Claude Opus 4.8 (max) | Anthropic | 72.6 | 55.8 | 79.4 | 82.5 | $7.70 | 17.8M | 23.1m | 166 |
| 4 | CodexOpenAI | GPT-5.4 (medium) | OpenAI | 71.1 | — | 69.8 | 72.4 | $2.27 | 5.7M | 7.1m | 71 |
| 5 | Claude CodeAnthropic | Claude Opus 4.6 (medium) | Anthropic | 71.1 | — | 70.2 | 71.9 | $1.26 | 4.4M | 8m | 34 |
| 6 | CodexOpenAI | GPT-5.5 (medium) | OpenAI | 70.5 | 56.6 | 75.8 | 79.1 | $2.75 | 7.0M | 6.4m | 78 |
| 7 | Cursor CLICursor | GPT-5.4 (medium) | OpenAI | 68.8 | — | 64.7 | 72.9 | $1.52 | 3.8M | 8.3m | 19 |
| 8 | Claude CodeAnthropic | Claude Opus 4.8 (medium) | Anthropic | 67.0 | 49.3 | 75.0 | 76.8 | $3.26 | 7.7M | 12.4m | 93 |
| 9 | Cursor CLICursor | Composer 2 | Cursor | 66.6 | — | 64.3 | 68.9 | $0.04 | 2.9M | 8.6m | 26 |
| 10 | Claude CodeAnthropic | Claude Opus 4.7 (max) | Anthropic | 65.0 | 40.1 | 73.8 | 81.0 | $5.64 | 15.8M | 15.8m | 107 |
| 11 | OpencodeOpencode | Claude Opus 4.7 (medium) | Anthropic | 64.5 | 39.5 | 75.0 | 79.0 | $2.93 | 7.5M | 12.2m | 54 |
| 12 | Cursor CLICursor | GPT-5.5 (medium) | OpenAI | 61.9 | 37.2 | 73.4 | 75.0 | $2.01 | 4.0M | 6.6m | 78 |
| 13 | Cursor CLICursor | Claude Opus 4.7 (medium) | Anthropic | 60.2 | 31.6 | 70.6 | 78.4 | $2.68 | 5.6M | 13.6m | 86 |
| 14 | Gemini CLIGoogle | Gemini 3.1 Pro (high) | 57.0 | — | 68.3 | 45.6 | $1.44 | 2.9M | 7.2m | 39 | |
| 15 | Claude CodeAnthropic | Claude Opus 4.7 (medium) | Anthropic | 56.8 | 27.4 | 71.4 | 71.7 | $1.68 | 4.5M | 6.3m | 42 |
| 16 | Claude CodeAnthropic | Claude Sonnet 4.6 (medium) | Anthropic | 54.1 | 28.9 | 63.1 | 70.3 | $1.97 | 8.4M | 13.7m | 67 |
| 17 | Claude CodeAnthropic |  GLM-5.1 | Z AI | 52.3 | 18.6 | 65.1 | 73.2 | $4.33 | 25.9M | 19.6m | 174 |
| 18 | Claude CodeAnthropic |  Qwen3.7 Plus (thinking) | Alibaba | 51.9 | 19.2 | 64.7 | 71.8 | $6.23 | 8.7M | 10.6m | 146 |
| 19 | Cursor CLICursor | Composer 2.5 Fast | Cursor | 51.8 | 15.9 | 66.9 | 72.5 | $0.55 | 4.3M | 6.8m | 117 |
| 20 | Cursor CLICursor | Composer 2.5 | Cursor | 51.8 | 15.9 | 66.9 | 72.5 | $0.08 | 3.6M | 9.7m | 117 |
| 21 | Claude CodeAnthropic | DeepSeek V4 Pro (high) | DeepSeek | 47.0 | 8.6 | 64.7 | 67.8 | $0.27 | 9.7M | 17.9m | 127 |
| 22 | Claude CodeAnthropic | Kimi K2.6 | Kimi | 46.9 | 16.5 | 64.3 | 59.8 | $1.18 | 11.4M | 41.2m | 130 |
Coding Agent Index
Equal-weight mean of three real-world suites: hard repository tasks, agentic terminal work, and codebase Q&A. Higher is better.
Composed of DeepSWE (113 tasks), Terminal-Bench v2 (84 tasks), and SWE-Atlas-QnA (124 tasks). The agent is the unit of measurement — the same model lands differently in different harnesses.
DeepSWE
Solve rate on 113 real-world software engineering tasks in real repositories, %.
Terminal-Bench v2
Solve rate on 84 agentic terminal tasks in a live shell, %.
SWE-Atlas-QnA
Rubric score on 124 codebase Q&A tasks, %.
The Harness Effect — Claude Opus 4.7 (medium)
The same model, three harnesses. The scaffold alone moves the Coding Agent Index.
Identical model weights and settings; only the harness changes. Prompting, context management, and tooling are worth real points.
Harness Spread
Points of IndexIndex points between a model's best and worst harness, for every model run in 2+ harnesses
- 1GPT-5.5 (medium) (2 harnesses)8.6
- 2Claude Opus 4.7 (medium) (3 harnesses)7.7
- 3GPT-5.4 (medium) (2 harnesses)2.3
Cost per Task
Mean USD to complete one task, across all three suites. Lower is better.
Measured mean spend per task at list API prices, cache discounts included. The spread is the story: the priciest configuration costs roughly 290× the cheapest.
Coding Agent Index vs. Cost per Task
Capability against mean USD per task (log scale).
Up and to the left wins: more solved tasks per dollar. Open-weight models power most of the value corner.
Token Usage per Task
Mean tokens consumed per task, split into fresh input, cache reads, and output.
Agents read far more than they write — cache reads are the overwhelming majority of tokens everywhere. Output is the thin dark sliver on top.
Cache Hit Rate
Share of context reads served from prompt cache, %. Higher is better for cost.
Harnesses that keep context stable cache better. Every point of hit rate is money: cached reads bill at a tenth of the fresh-input price.
Coding Agent Index vs. Total Tokens
Capability against mean total tokens per task (millions).
Reading more of the repo correlates with solving more of it — but the best harnesses get more index per token read.
Execution Time per Task
Mean wall-clock minutes from task start to the agent declaring done. Lower is better.
Includes model latency, tool calls, builds, and test runs. Fast models in lean harnesses finish in ~10 minutes; deliberate configurations take nearly three times as long.
Coding Agent Index vs. Execution Time
Capability against mean wall-clock minutes per task.
Up and to the left wins: capable and quick. Slow is only worth it if the index follows.
Turns per Task
Mean assistant turns (tool-call rounds) per task.
More turns means more, smaller steps — not necessarily better results. Turn count tracks harness style more than capability.
Run Specifications
Every configuration runs the same way, so the numbers compare clean.
- Environment
- Fresh container per task, repo pinned to a fixed commit, network limited to package mirrors.
- Attempts
- One attempt per task (pass@1), no retries, no human nudges.
- Configuration
- Each harness runs at default settings with its recommended model configuration.
- Budget
- Hard cap of 60 minutes wall-clock per task; runs that exceed it score zero.
- Cost accounting
- List API prices at snapshot date; cache reads billed at 10% of the input rate.
- Reporting
- Cost, tokens, time, and turns are means across all completed tasks in the three suites.
Frequently Asked Questions
What is the Coding Agent Index?
The equal-weight mean of a configuration's scores on the three task suites. One number for how much real software work gets done — no extra weighting tricks, no style points.
What do the three suites actually test?
DeepSWE is 113 real-world software-engineering tasks in real repositories, graded end to end. Terminal-Bench v2 is 84 multi-step jobs in a live shell — builds, migrations, debugging, ops. SWE-Atlas-QnA is 124 questions that require navigating a large codebase and answering precisely.
How are tasks scored?
Implementation and terminal tasks are pass@1: one attempt, and the test suite either passes or it doesn't. Codebase Q&A earns partial credit against a rubric. Nothing is cherry-picked or re-run.
What counts as execution time?
Wall-clock from handing the agent a task to the agent declaring done — model latency, tool calls, builds, and test runs included. It's the number you actually wait.
Why track tokens at all?
Because agents read far more than they write. Cache reads dominate the bill at a 10×-discounted rate, so two agents with the same index can differ several-fold in cost. The token mix is the why behind the cost chart.
Why does the same model score differently across harnesses?
The harness decides what the model sees and which tools it gets — system prompts, context management, edit formats, test loops. Same engine, different car.
Latest Agent Insights
Reporting from the agents desk
Apple Withholds Siri AI in Europe Over Data Rules
By 조선일보 · 3 hrs ago
Taboola Launches Ad Platform for AI Answer Engines, Conversational AI Offerings, Chatbots, and Virtual Assistants, Powered by the Monetization Behind DeeperDive, One of the Fastest-Growing AI Answer Engines | Taiwan News | Jun. 17, 2026 12:15
By Taiwan News · 3 hrs ago
ALSO Accelerates Adoption of Agentic AI Across Europe - AI Elite Partner Program to launch in summer
By finanzen.ch · 3 hrs ago
Taboola Launches Ad Platform for AI Answer Engines, Conversational AI Offerings, Chatbots, and Virtual Assistants, Powered by the Monetization Behind DeeperDive, One of the Fastest-Growing AI Answer Engines
By 新浪香港 · 3 hrs ago
Methodology: every configuration runs the same three suites — DeepSWE (113 tasks), Terminal-Bench v2 (84 tasks), and SWE-Atlas-QnA (124 tasks) — and the Coding Agent Index is their equal-weight mean. Suite design and metric definitions follow the public coding-agents methodology of Artificial Analysis (artificialanalysis.ai/agents/coding-agents). Figures are Glsrm editorial estimates calibrated to our model benchmark table, not Artificial Analysis's published results. Cost per task is derived from each run's mean token mix at list API prices, with cache reads billed at 10% of the input rate. Prices change frequently. Logos identify the respective model creators.