# AI code review leaves noisy comments because it lacks context. How do I filter to the good ones? URL: https://getunblocked.com/answers/filter-noisy-ai-code-review-comments/ Published: 2026-11-09T17:00:00Z Updated: 2026-11-08T16:35:14Z Author: Dennis Pilarinos Categories: Context Engine, Engineering Insights **Short answer:** Filter in four layers. Tell the reviewer what counts as a real bug and cap the nits. Skip generated files and anything CI already checks. Give it the context it is missing, such as past PR discussions and migrations. Then track how many comments lead to a fix, and cut whole classes of comment that never do. --- Noisy AI code review is mostly a setup problem, so most of the fix is setup too. Write down what counts as a real bug, cap the minor comments, and skip files and checks that other tools already cover. Give the reviewer the history it is guessing at. Then count which comments lead to a code change and delete the classes that never do. **Accept rate** is the share of AI review comments that end in a fix or a resolved thread because someone agreed. It is the one number that tells you whether filtering worked. The problem is common. An April 2026 study of code review agents (CRAs) found that "12 of 13 CRAs exhibit average signal ratios below 60%" ([Chowdhury et al., arXiv](https://arxiv.org/abs/2604.03196)). In the same study, 60.2% of closed pull requests reviewed only by agents sat in the 0 to 30% signal range ([arXiv](https://arxiv.org/abs/2604.03196)). Its methodology split pull requests by who reviewed them. PRs reviewed only by agents merged 45.20% of the time, against 68.37% for PRs reviewed only by people ([arXiv](https://arxiv.org/abs/2604.03196)). ## Write down what counts as important, and cap the rest Most reviewers ship with a general idea of severity that does not match your repo. Fix that first, in a file the reviewer reads on every pull request. A **severity rule** is a written line that says which kinds of finding block a merge in this repo and which are optional. Claude's Code Review marks each finding as Important ("a bug that should be fixed before merging"), Nit or Pre-existing ([Claude Code docs](https://code.claude.com/docs/en/code-review-pull-requests)). Its `REVIEW.md` file lets you redefine those levels, and the docs say plainly that this is "how you cut nit volume" ([Claude Code docs](https://code.claude.com/docs/en/code-review-customize)). Their example caps the nits ("Report at most five Nits per review") and says style and naming are "Nit at most." GitHub Copilot code review reads `.github/copilot-instructions.md` for repo-wide rules and `*.instructions.md` files for specific paths. It also reads `CLAUDE.md` and `REVIEW.md` if they exist ([GitHub docs](https://docs.github.com/en/copilot/how-tos/use-copilot-agents/request-a-code-review/use-code-review)). So one well-written file can steer more than one reviewer. Keep it short and concrete. For example, a payments service might block a merge only on these: - A query not scoped to the caller's account. - A migration that cannot be rolled back. - Personal data written to logs. - A retry loop with no limit. Everything else is optional, and the file says so. ## Stop comments on code nobody maintains by hand The cheapest noise to remove is a comment on a file no person wrote. Lockfiles, generated clients, vendored code and formatting are already covered by other tools or left alone on purpose. Claude's example `REVIEW.md` lists "Anything CI already enforces: lint, formatting, type errors" under "Do not report," along with generated files and lockfiles ([Claude Code docs](https://code.claude.com/docs/en/code-review-customize)). For paths that need some review but not much, it suggests a higher bar ("only report if near-certain and severe"). Unblocked reads your `.gitattributes` and skips any file marked with GitHub's Linguist generated or vendored markers ([Unblocked docs](https://docs.getunblocked.com/code-review/how-it-works)). A team that already tags those paths gets this without extra setup. Whatever tool you run, make the skip list a file in the repo, not a setting one admin remembers. Then a new engineer can read why the reviewer stays quiet on `src/gen/`. One more rule helps with false alarms. Claude's docs suggest requiring that "behavior claims need a `file:line` citation in the source, not an inference from naming." A comment that has to point at code is much harder to make up. ## Give the reviewer the context it is guessing at Rules trim volume, but they do not make the remaining comments any smarter. A reviewer that sees only the diff will flag a query that looks slow even when a migration last month added the index. It will suggest a pattern your team argued against in a PR two years ago, the same gap behind [coding agents that keep reinventing rejected approaches](https://getunblocked.com/blog/claude-code-institutional-memory/). That is the "lacks context" part of the question, and no severity rule fixes it. The fix is to let the reviewer read what a senior engineer would check. That means past PR threads, the tickets behind the change, the Slack discussion, the migration history. Unblocked Code Review learns from your team's past pull request conversations, and it cites the original conversation whenever a review comment draws on one. Before it reviews, it reads the existing threads on the PR, "so it avoids repeating findings that people have already resolved" ([Unblocked docs](https://docs.getunblocked.com/code-review/how-it-works)). Abhinav Rastogi, Staff Software Engineer at RB Global, put the result this way: "I like that Unblocked's code review is not noisy. It leaves fewer comments, and the model is usually much more aware of the context. For example, when it comments on database queries, it understands migrations and existing optimizations." Context also helps the code before review. The [guide to checking AI code against your conventions](https://getunblocked.com/blog/verify-ai-code-against-conventions/) covers giving the coding agent those rules up front. The answer on [making CLAUDE.md rules stick](https://getunblocked.com/answers/agents-ignore-claude-md-rules-how-to-make-context-stick/) covers why a rules file alone often is not enough. ## Track accept rate and delete what never lands Filtering without a measurement quickly turns into an argument about opinions. A **comment class** is a group of review comments that share a type, such as naming, null checks or query cost. Pick one measure and check it every two weeks: of the comments the reviewer posted, how many led to a code change or a thread resolved because someone agreed. Break it down by comment class. The class that almost never lands is your next rule in `REVIEW.md` or your next skip. Use the feedback buttons on individual comments, but do not count on them alone. GitHub says Copilot uses thumbs up and down "to improve the product." It also says that on a re-review "Copilot may repeat the same comments again," even after a thumbs down ([GitHub docs](https://docs.github.com/en/copilot/how-tos/use-copilot-agents/request-a-code-review/use-code-review)). Unblocked's docs say a short reply explaining why a comment is wrong "reduces similar comments in the future" ([Unblocked docs](https://docs.getunblocked.com/code-review/how-it-works)). Either way, a repeated bad comment means the rule belongs in a repo file. The last line of Claude's check-run details holds the severity counts as JSON, which a CI script can read to fail only on Important findings ([Claude Code docs](https://code.claude.com/docs/en/code-review-pull-requests)). That is a good default gate. Nits stay advice rather than a blocker, and authors stop treating every bot comment as friction. For how the main tools compare on context depth, see [the best AI code review tools in 2026](https://getunblocked.com/blog/best-ai-code-review-tools/). ## Read more - [Best AI Code Review Tools in 2026, Compared](https://getunblocked.com/blog/best-ai-code-review-tools/) - [Tools That Verify AI-Generated Code Against Your Conventions (2026)](https://getunblocked.com/blog/verify-ai-code-against-conventions/) ## Related answers - [Is an LLM-maintained wiki in git a better fix than a context tool for engineering knowledge?](https://getunblocked.com/answers/llm-maintained-wiki-vs-context-tool/) - [How do we drive adoption of an AI coding tool from a third of engineering to everyone?](https://getunblocked.com/answers/drive-ai-coding-tool-adoption-across-engineering/) - [How do I make sure Claude actually calls the context MCP without every engineer editing their CLAUDE.md?](https://getunblocked.com/answers/make-claude-call-context-mcp-without-editing-claude-md/) - [Atlassian Rovo MCP versus a dedicated context tool for Confluence, Jira and code?](https://getunblocked.com/answers/atlassian-rovo-mcp-vs-context-tool/)