ELO for engineers. With reasoning attached.
Chess solved fair ranking decades ago: rate players by who beats whom, and track how sure you are about every number. DevEval applies the same math — Glicko-2 — to engineering work. Every rating comes with a confidence level and a written justification you can audit.
A match is two developers' last 30 days, read side by side.
AI reads both developers' recent work — pull requests, reviews, discussions — and issues a verdict per dimension and overall: who did better, how confident, and why. That verdict is what feeds the Glicko-2 rating update.
A composite dashboard says “78 vs 65” — and can't tell you why.
A verdict names the winner, the dimension, the confidence, and the evidence. Disagree with one? Open the PRs it cites and argue with the reasoning, not with a number.
Glicko-2, not plain ELO.
Plain ELO gives everyone a number. Glicko-2 also tells you how much to trust that number — which is what makes the ranking usable in an organization, not just a chess club.
Every developer starts with maximum uncertainty, not a fake precise rating. The system knows what it doesn't know yet — and says so.
Each verdict tightens the uncertainty band around a rating. Ratings are always shown with their confidence — a fresh 1500 and a proven 1500 are not the same claim.
Ratings stabilize after roughly 10 comparisons — about two weeks of normal activity. No six-month calibration period before the numbers mean something.
The system schedules competitive, comparable pairs — not random ones. Once a rating is confident it stops comparing, then re-checks periodically, so ratings never go stale.
Relative strength, position, and delivered value.
DevEval separates three questions that a composite score blurs together: how someone compares with peers, where they stand in the organization, and how much value they actually delivered.
ELO / Glicko-2 rating
Relative strength per dimension. It changes after pairwise verdicts, when one developer outperforms another.
Percentile
Where you stand in the organization, for example “better than 87% in code quality”. Answers the review question in one line.
Absolute Points
How much value was actually delivered. Absolute Points are not zero-sum, so they can grow across the entire organization at once. They show the scale of contribution and support quarterly reviews.
One system shows both answers at once: who is strongest (stable) and how much more we ship (growing). A single composite score would blur the two into a number that means neither.
Why pairwise verdicts beat averages.
Composite averages are how engineering analytics ended up ignored. Verdicts are built to be argued with — and to survive the argument.
Run the first comparisons on your own team.
Connect a repo and let the first duels run — ratings stabilize within about two weeks of normal activity.