How We Think About Ranking Models at GLSRM

The question we get more than any other, by a wide margin, is some version of "just tell me the best model." We understand the appeal. It's a clean question and it deserves a clean answer. We're also convinced, more with every release cycle we cover, that it's the wrong question, and that answering it honestly does readers a disservice even when we could get away with answering it lazily.

Here's the thing a single ranking number can never quite admit about itself: it was built by someone deciding, in advance and usually silently, how much reasoning matters relative to writing quality, how much speed matters relative to depth, how much cost matters relative to raw capability, and then folding all of those judgment calls into one figure that looks objective. It isn't. It's an opinion wearing a scoreboard's clothes. This is how we think about the problem instead, and why we build our own model coverage the way we do.

A Single Number Was Always a Compression Artifact

Every ranking is a compression of many dimensions into one, and compression always throws something away. The question is never whether a ranking makes tradeoffs — it always does — it's whether those tradeoffs are visible or hidden. A leaderboard that blends coding skill, conversational tone, factual recall, and price into a single composite score has made a dozen quiet decisions about relative importance on your behalf, and you have no way of knowing whether those decisions match what you actually care about.

We'd rather be upfront about the compression than pretend it isn't happening. When we say a model is strong, we try to say strong at what, relative to what alternative, and for what kind of reader — because "best," stripped of that context, isn't a finding. It's a marketing claim wearing an analyst's coat, and we don't think our job is to launder one into the other.

This is also why so much of our coverage is spent on how benchmarks actually work rather than just what they currently say. A reader who understands that a pairwise preference score measures something different from a code-correctness score, and that either can be nudged by contamination or clever formatting, is a reader who can hold our own verdicts to the same standard we're asking labs to meet. We'd rather build that literacy than ask for blind trust in a number we produced ourselves.

What 'Multi-Dimensional' Actually Means in Practice

In practice, this means we track models across separate lanes rather than blending everything into one figure: reasoning and multi-step problem solving, code generation and debugging, writing and communication quality, tool use and agentic task completion, plus the practical dimensions that pure capability scores tend to ignore entirely — cost per task, response latency, and consistency across repeated attempts at the same kind of request. A model can lead in one lane and trail in another, and we think that's worth showing plainly rather than smoothing over.

This isn't a failure to commit to an opinion. We have opinions, plenty of them, about which models are genuinely ahead in which lane at any given moment — that's a large part of what this publication exists to do. What we've decided not to do is manufacture false precision by averaging lanes that don't actually average into anything meaningful. A model that writes beautifully and reasons unevenly, and a model that reasons rigorously and writes flatly, are not the same distance from some abstract ideal just because a spreadsheet says their blended scores match.

Picture two hypothetical models sitting at the same overall composite score. One gets there by being outstanding at structured reasoning and merely adequate at conversational writing. The other gets there the opposite way — excellent prose, average reasoning. A blended score tells you they're tied. A reader trying to pick a model for technical documentation and a reader trying to pick one for customer-facing chat need to know they are looking at two very different tools that simply happen to average to the same number, and that's precisely the distinction a single score is built to erase.

We Weight Real Work Over Clean Puzzles

We spend a lot of our own attention on exactly the gap the rest of the industry talks around: the distance between a clean benchmark question and the messy shape of an actual task. Public benchmarks are indispensable as a shared reference point, and we cite them constantly, but we don't treat them as the final word, because they were built to be gradeable at scale, not to resemble the ambiguous, multi-part, context-heavy requests that make up most real usage.

So alongside the public numbers, we weight our own coverage toward tasks that look more like actual work: a multi-turn support conversation that shifts topic halfway through, a coding request that has to respect an existing codebase's conventions rather than start from a blank file, a writing task with a genuinely ambiguous brief instead of a clean spec. None of this is a rejection of benchmarks. It's an acknowledgment that the benchmark and the workload are two different distributions, and a publication that only reports the first one is only telling half the story its readers actually need.

We also try to stay honest about where our own coverage has the same limitation. We can't simulate every reader's exact workload any more than a public benchmark can, and we don't pretend otherwise. What we can do is favor tasks with enough mess and ambiguity in them that a model's performance there tells you something closer to how it will hold up on your desk, rather than how it performs on a test written specifically to be easy to grade.

Why We Refuse to Crown One 'Best' Model

We've made a deliberate editorial choice not to run a single "best model" headline, and it's not because we're avoiding an opinion — it's because the framing itself is usually a category error. Picture a team choosing a model for a high-volume, latency-sensitive classification task, and another team choosing a model for occasional, complex, high-stakes analysis where cost barely matters and depth matters enormously. There is no honest sense in which the same model is "best" for both of those jobs, and a ranking that pretends otherwise is optimizing for a clean headline over a useful answer.

What we publish instead looks more like a set of tradeoffs than a trophy. This model is stronger on complex reasoning but costs meaningfully more per task. That one is close behind on quality but faster and cheaper at scale, which matters enormously if your usage is high-volume. Another is behind both on raw capability but meaningfully more consistent run to run, which can matter more than peak skill for a workflow that can't tolerate surprises. We'd rather hand a reader that map and let them place themselves on it than hand them a crown that quietly assumes their priorities match some average reader's priorities, which they usually don't.

A single crowned winner also ages badly in a way a tradeoff map doesn't. Name one model "the best" and the claim starts decaying the moment a competitor ships an update, because the entire piece was built around a title rather than a set of reasons. A tradeoff-based comparison ages more gracefully, because the underlying reasoning — this one favors depth, that one favors throughput, this other one favors consistency — tends to stay useful even after the specific names sitting in each slot have changed.

Showing Our Work Instead of Asking for Trust

The last piece of this is methodology, and we think it matters as much as any individual verdict. We try to be specific about what we actually tested, when we tested it, and against what alternatives, because a ranking without that context is just an assertion. Models update, sometimes quietly, and a verdict from months ago can quietly go stale without anyone announcing that it has. We'd rather revise a ranking in public than let an outdated one sit there accumulating false authority simply because nobody got around to changing it.

We also try to be honest about disagreement — including when our own read of a model's real-world performance doesn't match what a lab's self-reported benchmark numbers would suggest. That gap isn't embarrassing to point out; it's often the most useful thing we can tell a reader, because it's exactly the gap a released number can't see on its own. Showing our work means a reader can disagree with our conclusion and still find the piece useful, because they can see precisely which tradeoff we weighted differently than they would have.

None of this is a hedge dressed up as rigor. We have real opinions about real models, and we publish them plainly. What we won't do is pretend a single blended number can stand in for a question that's actually about your workload, your budget, and your tolerance for risk, none of which we get to define for you. The honest answer to "what's the best model" was never a name. It's a short, specific list of what you're optimizing for — and once you have that list, the right model tends to be a lot easier to find than the leaderboard made it look.