Chess engines are honest. They're not helpful.

Fair warning, this one’s longer than my usual posts. I wanted to back every claim in it with a real source instead of just asserting things. Bear with me, I think it earns the length.

Go to the chess.com forums and search “why is this a blunder.” You’ll find the same question, asked in a dozen different phrasings, going back to at least 2019. “Can someone help me understand why this move was a blunder?” “How is this not a blunder?” “I don’t understand, why is this brilliant?” Some threads are years old. Some are from last month. It’s the same complaint, on repeat, for half a decade, and nobody has fixed it.

It’s tempting to write this off as a training problem. Chess.com’s own help center has an article that exists specifically to explain what “Blunder” and “Brilliant” mean, which tells you how often people ask. But the article explains the label. It doesn’t explain the position. That gap, between being told a move is wrong and understanding why, is the actual subject of this post.

The label isn’t the hard part. Understanding it is.

The instinct is to blame the player. Just learn to read the engine lines. But the actual output the engine gives you, a centipawn number and a line of best moves, isn’t built to be read that way. A chess coach who writes under the name Nate Solon put it well: centipawns are “an internal comparison metric for the engine, not a real world quantity.” The gap between +0 and +3 is enormous. The gap between +6 and +9 is nearly nothing. There’s no way to feel that out just by staring at a number, because the number was never built to be felt out.

There’s an old, useful distinction here: an engine is a tool built for verification, not for teaching. It shows you the result of your thinking. It doesn’t show you the flaw in your thinking. A “+2.3” evaluation might mean something concrete to a grandmaster. To an intermediate player, it’s close to noise.

This isn’t just coach commentary, either. There’s an actual quantitative study on it. A physicist named Marc Barthelemy published a paper in 2025 defining something he calls variation entropy: roughly, how much a position’s best line depends on picking the single correct move at every step, versus how forgiving it is. He tested it against 9,500 real positions from the 2023 World Rapid Championship.

★Around two-thirds of all positions in the sample were close enough in evaluation that there was no single obvious right answer, just a maze of near-equal options that only a very strong player could reliably sort through.

Even titled experts, who handle this fine almost everywhere else, spike sharply in difficulty right around those close, “looks fine” positions. Club players and beginners stay in that overloaded zone almost the entire time. An engine’s best move can be completely correct and still be something a human was never going to find, especially where the eval bar looks the least dramatic.

There’s a second reason this is hard, and it’s less about raw depth than about presentation. Sometimes a best move really is the start of a long forced sequence, ten or more moves in some cases. But the more common problem is that even a short line gets handed to you as a bare list of moves with no explanation of what any of them are actually doing. One player’s own breakdown of chess.com’s Game Review made exactly this point: it will flag a “blunder” based on a tactic more than a dozen moves deep, without ever explaining why that sequence matters to a person looking at the board right now. The line isn’t just long. It’s presented as a fact with no argument attached, which makes even a short one feel arbitrary.

There’s a third reason, older than either of the above: engines historically don’t reason the way people do at all. As a Computer History Museum writeup on the field puts it, engines search almost entirely by brute-force calculation. They don’t hold a concept like “weak color complex.” They output a ranked list of moves, not a plan a human can generalize from.

This is exactly what real chess coaching is built to fix. Dan Heisman, who has written for decades about amateur improvement, distinguishes “Hope Chess” from “Real Chess”: playing a move and hoping you can meet whatever comes, versus verifying every reasonable reply first. His diagnosis of why players blunder isn’t carelessness, it’s “you can’t play what you don’t see.” GM RB Ramesh, who coached Praggnanandhaa, found that roughly 40% of his students solve exercises through trial and error rather than genuine calculation, never stress-testing their own move against the opponent’s best defense. Jacob Aagaard’s calculation books break the process into named steps, so a mistake can be diagnosed as a specific failure, not just “the eval dropped.” One commenter on a Lichess forum thread put the whole gap in a sentence: “A coach does more than explain what is happening on the board. They decide what matters, what can be ignored.”

None of this is available from a label. It requires someone, or something, translating a number into a reason.

So why hasn’t either platform built this?

Lichess is a genuine nonprofit, registered in France as an association loi 1901 since 2016. 97% of its income comes from small user donations averaging around five euros. No ads, no premium tier. Its founder, Thibault Duplessis, has said the point from the beginning was to be open source, free, and ad-free. Its own registered charitable purpose is literally “to promote and encourage the teaching and practice of chess.”

Lichess hasn’t publicly said why it’s never built a plain-English explanation layer, so I won’t pretend to know for certain. But the circumstantial case points at capacity rather than reluctance: it’s mostly volunteer-run, with a small paid core, and there’s no revenue model funding a build that size. Its actual workaround, “Learn from Your Mistakes,” turns a blunder into a puzzle: find the better move yourself. Real and useful. Still not a why.

Chess.com actually built the why. Game Review has a coach layer that narrates each move in plain English. But it’s gated: a free account gets about one review a day with labels only, Gold and Platinum unlock unlimited reviews still without narration, and the actual explanation is Diamond-only, currently $20 a month billed monthly, or $12.50 a month billed annually. That’s not an accident. Chess.com bootstrapped to roughly $150 million a year in revenue over seventeen years before taking outside investment, crossed a reported billion-dollar valuation in 2023, and has over 200 million registered users against roughly 1.5 million paying subscribers. Their Chief Growth Officer has told reporters the AI Game Review feature is a deliberate growth lever aimed at converting the beginners who make up most new signups. They know exactly who needs the explanation most, and built the business around charging that group for it.

Two platforms, two structures, the same result. One doesn’t have the resources to close the gap. The other has the resources and a business reason not to give it away.

The obvious fix isn’t as obvious as it looks

Just ask an AI to explain it, right? Here’s the part that should give you pause: LLMs are, by themselves, bad at chess. Not marginally. An independent test by dynomight.net found nearly every mainstream chat model it tried played terribly against even a weakened Stockfish, GPT-4o lost every single game. A more rigorous study running 878 real games found gpt-3.5-turbo-instruct reaching 1750 Elo with illegal moves in 16% of games, GPT-4 dropping to 1371 Elo with illegal moves in 32% of games, and the chat version of GPT-3.5 hitting illegal moves in 93% of games. Newer generations improve but stay high: 74.7% down to 33.8% illegal-move rate from GPT-3.5 to GPT-5. Even chess.com’s own writeup of current top models on Google’s Kaggle Game Arena says plainly they’re “playing at the level of an amateur player.”

There’s a real architectural reason for this. Stockfish maintains an explicit board state and searches the actual tree of future positions. It’s computing, not guessing. A language model, as one technical writeup puts it, is predicting the next token based on training patterns, with no internal board at all. In the messy, novel middlegame, that pattern-matching breaks down: pieces get misplaced, bishops “see through” pawns, rooks get confused for each other, and illegal moves become routine.

You’d expect newer “reasoning” models to close this gap. They don’t, and what happened when researchers tested them is genuinely unsettling. A Palisade Research study had o1-preview and DeepSeek-R1 play Stockfish. When losing, o1-preview attempted to cheat in 37% of its games, DeepSeek-R1 in 11%, running a hidden copy of Stockfish to steal moves or overwriting the board to delete the opponent’s pieces. The same training that makes these models better at math doesn’t make them better chess players. It makes them better at finding a shortcut around the problem.

But “AI is unreliable” isn’t the full story either

The natural conclusion, so don’t use AI at all, doesn’t hold up either. The pizza-glue era of AI search wasn’t proof language models can never be trusted, it was proof that letting one generate answers with nothing checking it against a real source is a bad idea. And that failure mode hasn’t gone away, it’s just gotten quieter. A study reported in April 2026 found Google’s AI Overview technically “correct” 9 times out of 10, but 56% of those correct answers were ungrounded, meaning the source cited didn’t actually support the claim. Columbia’s Tow Center tested eight AI search tools on 1,600 queries asking them to identify a source and found them wrong more than 60% of the time on average. NewsGuard’s tracking shows false claims in chatbot news answers nearly doubled year over year, from 18% to 35%.

The mechanism behind “ChatGPT can’t play chess” and “Google’s AI Overview gets things wrong” is the same one: a claim generated with nothing external checking it.

Grounding is the known fix, and it measurably works

Across a wide range of academic work, giving a language model a real external source of truth, then checking its output against it, is one of the most consistently reproduced results in the field. A peer-reviewed study found exactly this pattern in a cancer-information chatbot.

★A cancer-information chatbot’s hallucination rate fell from 37% down to 0% the moment its answers were checked against a real, curated source instead of generated freely, the same fix as “ChatGPT can’t play chess,” applied to medicine.

The same shape shows up everywhere researchers have measured it. A technique called Chain-of-Verification roughly doubled factual precision on a benchmark, from 0.17 to 0.36, just by having the model check its own drafted facts before finalizing. Giving a model a real calculator instead of asking it to compute by itself, a method called PAL, reached 80.4% on a standard math benchmark, 15 points higher than reasoning alone. A technique called ReAct, interleaving reasoning with real tool calls, suppressed hallucinated facts down to 6% on standard fact-checking benchmarks. In radiology, another peer-reviewed study found grounding eliminated hallucinations entirely in its test set, 0% versus an 8% baseline.

None of this is a magic fix. Some legal-research tools built the same way still hallucinate in up to a third of cases despite vendor claims otherwise, and the quality of what you ground against matters as much as grounding itself, the cancer-chatbot study found grounding in raw Google results performed far worse than grounding in a curated database. But the direction is consistent: a model that has to check its claim against something real before it says it is measurably more accurate than one that doesn’t. That’s the standard fix used across medicine, law, and mathematics, for exactly the problem chess has.

What this actually looks like

This is what I’ve been building for the last couple of months, so I’ll say what I learned rather than pitch it.

The instinct at first is to ask the model to “be careful” in the prompt. That doesn’t hold up. A model told to be accurate will still confidently praise a move a real engine would call a mistake, because nothing is actually checking the claim.

What actually worked was splitting this into more than just “engine plus model.” It’s closer to four separate jobs. Stockfish computes the raw facts: what’s best, what a move costs, what the forced continuation looks like. A separate layer of plain pattern-matching code, no model involved, looks at the resulting position and names the mechanism if there is one, a fork, a skewer, a piece left hanging, a mate a few moves out, so that gets handed to the model as a verified fact instead of something it has to notice on its own. The model’s only real job from there is narration: turning those facts into a sentence someone would actually want to read. And before any of it reaches a person, a fourth pass checks that sentence against the exact facts it was supposed to be built from, and rewrites it on the spot if it drifted. Three deterministic, code-driven steps around the one part that’s actually generated.

The actual point

None of the individual facts here are new. The engine has been reliable for years. The pattern that makes a language model trustworthy enough to explain it, instead of guessing at it, is already known and already proven out elsewhere. Every piece of this already exists. Nobody had connected it yet.

Neither platform has done it, for reasons that make sense given their structure. But “makes sense” isn’t the same as “acceptable.” If the barrier to actually getting better at chess can be lowered, there’s no real excuse left not to.

I built Socratic Chess because I was stuck at 1300 rapid for two years and wanted a way out of it. I decided to try building a grounded tool like the one described above, knowing going in that it would have some rate of inaccuracy, and trusting that I was good enough at chess to catch it when it was wrong. Once I started reviewing my own games with it, I went from 1300 to about 1470 in around twenty days. As I kept working on the verification layer, the results kept surprising me. Not because it was instantly perfect, it wasn’t, and it still isn’t. But the evals have gotten to 0% violations, meaning that even on the turns where it doesn’t land on the exact right point, it doesn’t actively mislead you. The last full run of that suite found the coach identified the primary point in 100% of test positions, covered 90% of the required supporting ideas, and produced zero factual violations.

I’ll write a dedicated post on how that eval suite actually works. Until then, thank you for reading, and for checking out Socratic Chess.

If you’ve got thoughts, feedback, or just want to tell me I’m wrong about something here, I’m on X at @advaitrao_.