How good is AI getting?
Pick a test. See the best AI score over time, what the test actually measures, and how it compares with people.
Hover or tap a point to see that quarter. The line shows the best score any model had reached by then.
| Quarter | Best score | Record held by |
|---|---|---|
| Q4 2024 | 31% | GPT-4o (Nov 2024) |
| Q1 2025 | 61% | claude-3-7-sonnet-20250219 |
| Q2 2025 | 71% | Claude Opus 4 |
| Q3 2025 | 74% | GPT-5 (high) |
| Q4 2025 | 77% | Claude Opus 4.5 (no thinking) |
| Q1 2026 | 79% | Claude Opus 4.6 (no thinking) |
| Q2 2026 | 84% | Claude Opus 4.7 (max) |
| Q3 2026 | 84% | Claude Opus 4.7 (max) |
| Q4 2026 | 84% | Claude Opus 4.7 (max) |
Room left to improve
Closing in
Best score 84% out of 100%.
Models get more than half right and are improving quickly. Watch this one over the next year.
Humans on this test
There is no published human score. Each task is a real fix that a human developer already made, so in a sense people have solved all of them, but nobody has timed a group of engineers sitting the whole set.
What it tests
Real bug reports from popular open-source Python projects. The model must change the code so the project’s own tests pass. The Verified set is a subset of 500 tasks that human engineers checked were clear and solvable.
Why it matters
It has been the most widely quoted measure of whether AI can do a software engineer’s everyday work.
Caveats
Scores depend heavily on the tools and scaffolding around the model, not just the model itself. Every task comes from a public repository, so some solutions may have been in a model’s training data. In February 2026 OpenAI, which produced the Verified set, stopped reporting scores on it. Its audit found many tasks whose tests reject correct fixes, and signs that leading models had seen the solutions in training. Read the top of this chart with that in mind.
How benchmarks work →Model scores
Top 10 of 33 models by score
| Rank | Model | Developer | Score | Released |
|---|---|---|---|---|
| 1 | Claude Opus 4.7 (max) Anthropic | Anthropic | 84% | Apr 2026 |
| 2 | GPT-5.5 (xhigh) OpenAI | OpenAI | 81% | Apr 2026 |
| 3 | Gemini 3.5 Flash Google DeepMind | Google DeepMind | 79% | May 2026 |
| 4 | Claude Opus 4.6 (no thinking) Anthropic | Anthropic | 79% | Feb 2026 |
| 5 | GLM-5.2 Z.ai (Zhipu AI) | Z.ai (Zhipu AI) | 79% | Jun 2026 |
| 6 | DeepSeek v4 Pro (max) DeepSeek | DeepSeek | 78% | Apr 2026 |
| 7 | Qwen3.7 Max Alibaba | Alibaba | 77% | May 2026 |
| 8 | GPT-5.4 (high) OpenAI | OpenAI | 77% | Mar 2026 |
| 9 | Claude Opus 4.5 (no thinking) Anthropic | Anthropic | 77% | Nov 2025 |
| 10 | Kimi K2.6 Moonshot | Moonshot | 77% | Apr 2026 |
Benchmark data from Epoch AI (CC BY 4.0) and official leaderboards. Updated weekly, last on . Download CSV Methodology