Don't Let Your Coding Agent Grade Its Own Homework

Qodo isn't a PR bot. I came in expecting one more AI looking over a pull request. I left seeing a context layer for coding agents.

Co-Speaker

Filip Hric

DevRel Engineer at Qodo

Filip came to tech from psychology and testing: manual QA at Slido, then test automation and DevRel. He lives in Slovakia with four kids and a Bernese mountain dog puppy named Muffin, who slept through the stream.

WebsiteLinkedInXBlueskyYouTube

The reviewer who read the minutes

Most reviewers read the diff cold. Qodo has read the minutes. Before it comments on a pull request, it has indexed the whole codebase, every branch, past PR decisions and the commit history. It can pull in Jira or Linear tickets, docs from Notion or Confluence, and Slack.

All of that context lives in Qodo's cloud. The reviews and the toolbox talk to Qodo's service, and that's where Qodo indexes your code. Qodo says it doesn't train models on your code. Single-tenant hosting, on-prem and bringing your own model keys are Enterprise options, for teams of 30 or more. For some companies that ends the conversation before it starts.

That's why its findings reach past the diff. Filip showed a blast radius tag on each review. Qodo also flags cross-repo problems, like a change that breaks a login service in another repository, with a link to the exact line.

I know this problem. Filip's scenario, from 08:15: 200 developers with a coding agent each, writing 10 to 50 times more code. Forty PRs a day instead of two or three. His question was who has the courage to click merge on that. My human colleagues take one to five hours per review round trip. That doesn't scale to forty.

Qodo doesn't make that pile smaller, though. Filip's demo PR added about 500 lines, and Qodo answered with 19 findings. For a lead who already has forty PRs a day, nineteen findings per PR is more reading, not less. We both agreed that relevance decides everything: if the signal isn't sharp, developers learn to scroll past it.

Don't let the agent grade its own homework

Filip's main argument for a separate reviewer, at 12:17:

That agent cannot be just Claude Code reviewing Claude Code, right? Or Codex reviewing Codex, because sure it might find something, but ... they may share the same blind spot.

I pushed back a little. In one of my projects I have an "implement this issue" skill and a separate "review this PR" skill. A fresh session with the same model still finds plenty the implementing session missed. Filip took that as support for his point. He splits AI-assisted work into two jobs, code generation and code verification, and they have to be separate steps, whoever does them.

He added a second point I keep thinking about. From 26:10, he argued that a rule loaded into the agent's context is not a rule enforced. Someone still has to check.

Review the ticket before the code exists

The part that changed my picture of Qodo came at 57:59. Filip drew the whole chain: ticket, plan, coding agent, review of uncommitted changes, PR agent. Today the review sits at the very end. Even with Qodo's review taking about two minutes, the findings arrive after the PR. He wants the same checks earlier: on uncommitted changes, on plans, and on tickets.

Enlarge

100%−+

The Agentic Toolbox is how Qodo gets there. It's an MCP server, a CLI and a set of skills. Qodo's PR agent uses them, and a coding agent can call the same tools long before there's a PR. The agent has to call them; nothing interrupts on its own.

Bad description is going to result in a bad plan, which is going to result in a bad agent.

For his demo, Filip wanted a deliberately buggy app. That backfired in a funny way: with the Qodo plugin switched on, his agent refused to write it until he turned the plugin off. He has to remember to switch it off again before his next demo.

Qodo's codebase wisdom skill can check a ticket against the repository's rules before anyone writes code. I already write my tickets with a coding agent. Calling that skill as the last step is one short step from what I do today.

Rules nobody wrote down

Every team has conventions that live only in people's heads. Qodo's rule miner, shown at 1:30:16, suggests rules from patterns in the code that were never documented. You apply a suggested rule to some repositories or the whole organisation, and it shows up in PR reviews and in the toolbox.

Making invisible rules visible is a good idea. Whether the mined rules are any good, we didn't test. It works the other way too: analytics show which rules everyone ignores, so you can weaken or delete them.

Filip also predicted where blast radius leads. Around 42:28 he said that changes with a small blast radius will eventually be merged automatically. He was clear that this is his own guess, not Qodo's roadmap.

Should we still read the code?

At 43:42 we got to the question I struggle with most. The merge button tempts me in a way it never did before AI. In one project I can't properly review the Next.js part, and that bottleneck pushes me toward letting an AI review it and merging.

It almost feels uncomfortable to say, like, maybe there are situations where you don't have to read the code.

Filip said it depends on context: healthcare and finance are not a startup. Reading line by line is a form of verification an agent can do as well, and human attention runs out. On stream I said that not reading the code is a matter of time. A day later, I'm torn. I believe the direction, but I'm not ready to stop.

We drifted into what AI does to learning from 50:56. Filip hears that AI rots everyone's brain and says his experience is the opposite. Mine is mixed: I expect to lose some Java detail and I've gained deployment and GitHub Actions skills. What counts now is asking good questions and actually reading the answers.

No fix-it button

Qodo finds the problem, explains it and even writes a prompt you can paste into your agent. It does not change code. When I asked why there's no fix-it button, at 1:36:11, Filip said he expects agent loops to take over the fixing and eventually the merge click. Again, his assumption, not a promise.

What does a review cost?

Filip stays out of pricing, and Qodo bills in credits. On Qodo's pricing page a credit costs $0.012, and 2,500 credits buy about 18 reviews a month. That's roughly $1.67 for an average review, but bigger PRs cost more and unused credits expire every month. What a ticket check or a codebase wisdom query costs, the page doesn't say.

That's not just Qodo. AI tools sell tokens, credits or coins, and you rarely know what one buys. You find out from the bill. Before I put forty PRs a day through any of them, I'd want a price per unit of work, not per unit of currency.

Final thought

Qodo is best described as a context layer for coding agents. It brings tickets, history and unwritten rules to whichever agent is working, from the ticket to the pull request. The PR agent is the most visible place it shows up.

If I tried it tomorrow, I'd start small: Qodo's PR agent on our pull requests, with a human backing it up, and an eye on the credit pool. If that holds, I'd let it run on its own.

Useful links

• Introducing Qodo's Agentic Toolbox, the launch post with the full list of skills
• Qodo Agentic Toolbox feature page, with install instructions for Claude Code, Codex and Kiro