Grigory Potemkin, so the story goes, built villages that were nothing but a facade. Painted fronts, no houses behind them. They were erected along the Dnieper so that Tsarina Catherine, sailing past, would see prosperity where there was none. The trick worked because of where the observer stood. From the boat, on the water, at the right distance, a facade and a village are truly indistinguishable. So the deception is not in the wood and paint. It is in the fact that the test was never capable of telling the two apart in the first place.
A recent machine-learning paper has resurrected this image. It describes something uncomfortably close to home, and it is worth sitting with past the point where it stops being about machines. The authors call it potemkin understanding. A large language model can define a haiku flawlessly. It can cite the 5-7-5 structure and explain why it works. It doesn’t end here. Ask the same model to write one, and it may produce something that does not scan. Worse, it does not even notice. The explanation was real. The competence behind it was not. Ask it to grade its own output and, often, it disagrees with itself. Yes, it follows the pattern. Also, no, it does not. This is not a wrong answer repeated consistently. It is an incoherent one.
The paper’s real contribution, though, is not the anecdote. It is the mechanism it names: the reason why anyone ever thought an exam could measure understanding at all. They call it the keystone. A small set of questions that can only be answered correctly by someone who has genuinely grasped the concept. We trust the YKS, AYT and TYT here, the SAT in the United States, the bar exam, the comps and the quals, precisely because we believe no other route to a passing score exists. But the paper is careful about why that trust is earned in humans, and the reason is thinner than academia ever had to notice. It is not because correct answers on a finite set of questions logically entail correct understanding. To be frank, they do not, and never could. It is because the ways a human tends to misunderstand a concept happen to be few and structured. Someone who thinks a haiku is 5-8-5 will write bad haiku, but in a predictable, catchable way. The keystone works because human error is, empirically, well-behaved. It was never a proof. It was a wager on a regularity nobody bothered to state out loud. For a very long time, nothing came along to call the bet.
Obviously, this is not a new problem in philosophy. Only the venue is new. Wittgenstein raised it as the rule-following paradox, and Kripke sharpened it decades later with “quus”: a deviant function that gives the same result as ordinary addition on every sum you have ever actually performed, and diverges only past the point where you happened to stop checking. The lesson is simple and unsettling. No finite run of correct answers can, strictly speaking, settle which rule a mind is following. Philosophy treated this as a seminar puzzle. Faintly embarrassing, safely irrelevant. After all, no one had ever met an actual quus-adder. The Potemkin paper reports meeting several. It gave a name and a spreadsheet to a thought experiment everyone had agreed to stop worrying about.
Now let us take the argument one step further than the authors do, because this is exactly where it stops being a problem about benchmarks. The paper concludes that our tests are invalid for machines. Fair enough. But look at what the machines actually passed. Definitions. Classifications. The right answer to the right question, in the right format. In other words, the whole visible body of what we call “knowing”. If a system with no coherent grasp of a concept can reproduce all of this, then all of this was never the knowledge itself. It was its facade. Yet this facade is precisely what our schools teach, what our exams reward, what our journals publish and cite. So the honest conclusion is not “the machine has Potemkin understanding.” It is harsher. The knowledge on which such success rests is, at best, a form of Potemkin knowledge. Painted fronts that someone who has never lived in the village can put up.
One might object that this is unfair. Humans who pass the exam usually do understand. Or at least we like to believe so. Besides, the paper itself says so too. There are certain reasons to believe it is true. But only thanks to the wager. The human passes because human error happens to be shaped a certain way, not because the exam ever reached the understanding itself. What the exam measures is the facade. We simply assumed that behind a human facade there must be a house. And here is the uncomfortable part: modern epistemology has been perfectly content with that assumption. It defined knowing as what can be stated, reproduced, and checked against an answer key. It rarely asked what it means to actually know a thing. It only asked whether the knower produces the expected output. Naturally, a regime like this was bound to be imitated. It was built to be imitable. What we lack is not a better benchmark. What we lack is an approach to knowledge that is truthful and essential rather than merely procedural.
Here is the part academia will not enjoy. Every exam, every dissertation defense, every professional license, every peer review operates on exactly the assumption the paper found to be false for language models: that a curated, finite, keystone-shaped test certifies something coherent underneath it. That certainty was never a property of the test. It was a favor borrowed from the ordinary regularity of human error. Quietly cashed for a century. Never audited, never even named, until a system came along whose errors did not do us the courtesy of being human-shaped. The scandal is not that the model turned out to have a facade instead of a foundation. The scandal is that we never once checked whether the same is true of the diploma on the wall. We never had a machine willing to grade its own answer twice and tell us, without flinching, that it disagreed with itself.
Nobody has run that experiment on the tenured. Nobody wants to. Yet the paper’s method is not, in principle, restricted to language models. Ask an expert to produce an instance of the very concept they have just defined flawlessly in a seminar. Then show it back to them in a different room, under a different name, and ask if it qualifies. Somewhere between definition and deployment, competence has a way of turning into a performance that only has to survive one pass. We built an entire civilization of credentials on a wager about human error that we never once wrote down. The first time something failed to honor that wager, we called it the machine’s problem. It is not. The machine did not fake our knowledge. It showed us what our knowledge was made of. So it is worth asking, quietly, off the record, in the room where the grades are actually assigned: whose facade gets discovered first, once someone finally points the boat the other way?
Cemil Caca Arslan holds a PhD in Public Law. His research focuses particularly on the fundamental implications of techno-politics and techno-feudalism for the state.
