Unblocked ranked the highest performing code review tool
Martian’s open Code Review Bench evaluates AI review tools on the same 50 real pull requests containing 158 known bugs. Unblocked caught more bugs overall, including more critical and high-severity bugs, than any other tool in Martian’s published results.
How to read these numbers. Martian built Code Review Bench and publishes the results of the other tools on its leaderboard. This post compares Unblocked with a selection of them, including the top three. We ran Unblocked through the same published pipeline and judge, on public copies of the same 50 pull requests, and every Unblocked review is public. Unblocked is not yet listed on Martian's own leaderboard. See how it was measured.
All 158 bugs#
Across all 158 known bugs, Unblocked caught 108, or 68.4%. GitHub Copilot came next at 66.5%, with Cubic, Augment and Qodo Deep close behind at about 65%.
What this means for your team#
A benchmark is only useful if it says something about your own pull requests. Here is what these results mean in practice.
Try Unblocked on your next pull request →
Head to head on the most serious bugs#
Pick any tool to compare the two on critical and high-severity bugs.
Test dataset#
Five open-source projects in five languages, ten pull requests from each. Each row shows how many of that project's known bugs Unblocked caught, and links to the public copy of the project where every Unblocked review can be read.
How Unblocked catches them#
Unblocked Code Review has two modes.
- Standard mode - the Unblocked default that tends to post fewer comments.
- Deep mode - which is the basis for this benchmark and built specifically to miss as few bugs as possible. This is typically used with teams where agents write much of the code and a missed bug costs more than an extra comment.
Deep mode reasons about what the pull request is trying to do, reviews it from several perspectives and checks each finding against the code before posting it.
The scoreboard and how it was measured#
The overall score is the ranking Martian's leaderboard uses by default. It combines bugs caught and comment accuracy, counting bugs caught twice as heavily (an F2 score). Comment accuracy is the share of a tool's raised issues that matched one of the 158 known bugs.
| Tool | Overall score | Bugs caught |
|---|---|---|
| Unblocked | 66.2 | 68.4% |
| Qodo Deep | 65.1 | 64.6% |
| Cubic | 64.9 | 65.8% |
| Augment | 64.0 | 65.2% |
| GitHub Copilot | 58.0 | 66.5% |
| Greptile | 51.2 | 51.3% |
| CodeRabbit | 51.1 | 59.5% |
| Cursor Bugbot | 48.6 | 46.8% |
| Claude Code | 47.7 | 48.1% |
| Devin | 44.8 | 41.1% |
Code Review Bench is open source and maintained by Martian. Its 50 pull requests come from Cal.com, Sentry, Grafana, Keycloak and Discourse, ten from each. Each one reintroduces a real change that shipped bugs the project later fixed.
A bug counts as caught when a review comment identifies it. Every tool is scored against the same answer key by the same AI judge. One comment can raise more than one issue, and each issue is graded on its own.
The other tools' results are the ones Martian has published, using each tool's best-scoring version; this post shows a selection of them, including the top three by overall score. Unblocked ran in Deep mode, with every review pass on Claude Opus 5.5, on public copies of the same pull requests in the AI-Code-Review-Eval repositories. Anyone can read each Unblocked comment and check whether it really identifies the bug.
See how the scores are calculated and which tool caught each of the 158 bugs, with a link to the comment that caught it, on our benchmark page.
