Where Self-Improving AI is the Beginning of Infinity (And Where It Hits a Wall)

Yesterday, Anthropic shipped Claude Fable 5, its most capable public model yet — with one quiet exception: point it at frontier AI research, and it degrades its own output. No notice, no fallback. Intelligence that can improve intelligence is now rationed. The premise had landed days earlier: Claude already writes over 80% of Anthropic’s code, runs its experiments, and is edging into the judgment calls research depends on.

The caution may be warranted. But the precedent is stark: one company has begun drawing the borders of where the most consequential technology of our time may compound—unilaterally, and partly invisibly. The question is no longer whether self-improving AI is coming, but where the compounding is allowed to run, and who gets to verify it. The answer begins twelve centuries ago.

The Freedom to Recombine Ideas

We rarely think of algorithm as a name, or algebra as a book. But centuries before they became the foundation of modern AI, they were exactly that. Ninth-century mathematician al-Khwārizmī had a brilliant mind, but also the luck of living in the exact place where the world’s frontier knowledge had just arrived: Greek geometry, Persian astronomy, and Indian arithmetic with its weird new zero, all translated for the first time into the same language. Ideas that incubated for centuries on separate continents were suddenly brought together as algebra.

Recombination of ideas is the core engine of human progress. It is also a compounding one: every invention multiplies the combinations available to the next. Economist Martin Weitzman called this “recombinant growth.” As the knowledge of one generation becomes the foundation for the next, human civilization has been executing its own, slow algorithm of recursive self-improvement.

Scaling the Human Mind

Because of our biological limits, we learned to build technologies and fine-tune institutions—from writing and R&D teams to computation and the web—to augment our ability to process what is known.

Yet the final step of recombination is still bound to us. For two ideas to actually fuse, they must still intersect within the same mind. Furthermore, as the sheer volume of human knowledge expands, innovators must climb longer just to reach the frontier of a single discipline, let alone multiple ones. Economist Ben Jones called this the “death of the Renaissance Man”—an era where invention arrives later in life, driven by narrower and narrower specialists.

Artificial intelligence removes this constraint. By absorbing the vast majority of codified human knowledge, it can map and traverse the rugged landscape of everything known on our behalf. It is al-Khwārizmī’s lucky accident—the collision of separate worlds into a single place—made permanent and available to everyone.

AI’s Comparative Advantage

When it comes to recombining ideas, humans and machines face completely different economics. For a person, cognitive costs increase steeply with distance: mastering far-apart fields is difficult and eventually impossible. Our best attempts to work around this limit require heavy scaffolding. In academia and corporate R&D, overcoming this friction means funding dedicated centers, aligning complex incentives, and spending years just to build a shared language. Merging distant fields often requires the blank slate of a startup as a forcing function—like Pixar pulling computational physicists into the same room as Disney-trained storytellers—just to force truly novel recombinations.

This is why radical, cross-disciplinary leaps remain so impossibly rare. But for a machine that has read everything, the cost barely climbs at all.

The asymmetry carries a sharp implication: AI’s advantage over us is not uniform but widens with distance. Within a specialized domain, human experts stay competitive as top verifiers and arbiters of agentic output—some call it judgment, others taste or curation, but it really boils down to what the human has seen before and how they measure it relative to the machine. But out at the seams between disciplines, where our own cognitive costs are incredibly high, the machine is already superhuman. There is just one catch…

The Verification Divide

A flood of recombinations is not the same as progress. As the distance between domains increases, the nature of the bet fundamentally changes. Distant recombinations are highly skewed: they yield a mountain of useless dead-ends, offset by the rare, world-altering breakthrough.

A distant recombination is therefore a lottery ticket. Without verification, the rare jackpot is indistinguishable from the pile of losing tickets, and its value collapses to the average of the pile: almost nothing.

Generation determines what you hold. Verification determines what it’s worth.

For a recombination loop to actually compound, two steps must happen in sequence: you generate a new idea, and then you verify it. Whatever comes next will be built on the assumption that the first idea holds.

That confirmation is not a formality. It is the difference between knowledge, which accumulates because each checked layer can bear the weight of the next, and unverified output, which merely accumulates idea debt. It is verification that makes the loop truly recursive, rather than just impressively fast.

When you commoditize a scarce input, you do not remove the bottleneck. You simply relocate it. For all of human history, intelligence—the generation of the next idea—was the scarce resource that progress waited on. Make it abundant, and the constraint immediately moves downstream to verification: the cost of establishing that any given output is actually correct.

Crucially, that cost is not a property of the idea itself, but of the domain. In mathematics and software, verification is often nearly free and instant—the proof holds, the tests pass, the code compiles—and the loop is closed at machine speed.

But in domains burdened by long, unforgiving feedback loops—from entrepreneurship and business strategy to frontier science dealing with unknown unknowns—verification cannot simply be automated. Here, the loop of recursive self-improvement grinds to a halt.

Within these domains, AI may well have driven the cost of generation to zero, but verification still costs precisely what the world has always charged for it. And that newly scarce step is exactly where value will accrue.

The Recombination-Verification Matrix

Map these two dimensions against each other—the vast knowledge distance AI can bridge versus the last-mile verification costs the real world imposes—and the future of progress fractures into four quadrants.

This is the domain of the known knowns. When the cognitive distance is short and the answers are easily checked, AI acts as a bulldozer for friction—effortlessly automating incremental discoveries, routine improvements, and daily grunt work. It won’t yield world-changing breakthroughs, but driving the cost of execution to zero fundamentally raises the baseline of human productivity, allowing experts to shift their time higher up the intelligence value chain.

The Land of Expert Verifiers (Within Field, Hard Verification)

AI generates relentlessly, but progress is entirely bottlenecked by the human capacity to check the work. Because verification remains stubbornly difficult, top experts must transition from being primary creators into ultimate arbiters. By using the machine to clear away the legwork, these top verifiers can scale their true scarce resources—their taste, experience, and intuition—across an unprecedented volume of output.

The Rapid Takeoff (Across Fields, Easy Verification)

Where distant recombination meets cheap verification, the AI gold rush unfolds first. It is the unexplored frontier where the moment an AI finds a novel combination, it can independently prove it is correct and close its own loop. Gains stack rapidly without a human-in-the-loop, making this the quadrant where AI will deliver massive, world-altering benefits the fastest.

Fool’s Gold (Across Fields, Hard Verification)

This is the same frontier, but nothing found here can be cheaply confirmed. A real discovery and a confident hallucination are indistinguishable. This is no accident: the most distant recombinations often land precisely where no discipline owns the tests yet—no established benchmark, no standard eval, no single expert qualified to referee across the gap. The structure of the problem puts the biggest prizes behind the worst measurement: AI’s advantage is largest exactly where our ability to verify is weakest.

A New Golden Age of Discovery

Last month, an internal OpenAI reasoning model disproved a conjecture Paul Erdős posed in 1946. It involved the planar unit distance problem—the deceptively simple question of how many pairs among n points in a plane can sit exactly one unit apart. For eighty years, it stood as one of the most stubborn open questions in combinatorial geometry, and for eighty years, the field assumed a particular grid arrangement was the absolute limit. The model proved otherwise, autonomously discovering an infinite family of configurations that beats the grid outright.

The way the model got there is the core thesis of this piece: it cracked an elementary geometry problem by importing heavy machinery from algebraic number theory. These were two fields with no immediately obvious reason to meet, joined across exactly the distance no human was positioned to cross. This is al-Khwārizmī’s move, made by a machine.

And it counts.

It is an insight the field can build its next result on—not merely a plausible claim—for a reason that has nothing to do with the solution, and everything to do with the domain. In mathematics, the second step is nearly free. A proof either holds from its first line to its last, or it does not. Verification is fast and absolute, a stark contrast to the years a clinical trial or a macroeconomic forecast might require. The distant recombination delivered the breakthrough, but it was the low cost of verification that let the loop close behind it.

Mathematics is anchored by a single, universal standard of proof. But the further a field drifts from that absolute certainty—the longer and fuzzier its verification gets—the more that same flood of confident, far-flung ideas simply piles up unconfirmed, leaving real discoveries indistinguishable from mirages.

Progress runs at the speed of automation where checking is cheap, and at the speed of verification everywhere else. The widening gap between those two speeds is set to become the defining fault line of discovery.

There is only one looming exception to this rule: the simulation problem. If world models become capable of running infinite, highly realistic counterfactuals—effectively turning real-world friction into the mere cost of compute—the verification bottleneck vanishes. We are seeing early glimpses of this in physics and robotics. Should this extend to modeling complex markets and social systems, the machine delivers verification-in-a-box. At that point, the 2x2 matrix collapses, the path to superintelligence is cleared, and the only things left unautomated will be the things humanity has literally never measured.

The Endless Frontier

For founders, inventors, and scientists asking themselves where to deploy their time, the clash between AI’s infinite capacity for recombination and the annoying friction of real-world verification acts as a compass. It points directly toward the new frontier.

The most obvious opportunities sit in the Rapid Takeoff quadrant—distant problems paired with frictionless verification. But this space is crowding quickly, and the frontier AI labs hold an inherent structural advantage here. In these domains, cheap verification doesn’t just check the work; it generates the exact data needed to train the next model. Modern reasoning models scale through automated reward signals, and this quadrant supplies an infinite synthetic loop of them for free.

Fable’s launch made the advantage explicit: AI research itself — instantly benchmarked, self-grading — is the purest Rapid Takeoff territory on the map, and it is the first domain a lab has drawn a border around.

Ironically, the more defensible opportunities sit in Fool’s Gold. The true prize in this quadrant isn’t generating answers, but building scalable forms of verification. Whoever engineers the measurement infrastructure that turns a long-lag question into a short one—the simulation that replaces the physical lab, or the formal eval a soft discipline has never had—does not make a single discovery. Instead, they convert an entire region of the map from Fool’s Gold into a Rapid Takeoff Zone. They supply the missing half of the loop, and in doing so, they define exactly where the golden age is permitted to spread.

As in the Land of Expert Verifiers, building this infrastructure demands harvesting the insights of the world’s top talent. It requires progressively extracting the intuitive “weights” locked inside their minds, converting their tacit knowledge into replicable, measurable digital traces that finally close the loop for full automation.

Notice how this shifts the economic rents. For most of history, the scarce resource was the bridge—the rare person or team who could hold distant fields in a single mind commanded a massive premium. Today, spanning that distance is practically free. The new rents sit one step downstream, captured entirely by the two jobs the machine cannot do for itself: choosing which recombinations are worth advancing, and verifying which ones actually paid off.

For scientists and professionals, the lesson is clear: the method of verification is no longer the boring half of the work—it is the work. For entrepreneurs, builders, and creatives, the mandate is exactly the same. In an age of infinite output, a durable business isn’t built by generating more of it. It is built by being the arbiter whose verification the world trusts.

Take Nobody’s Word For It

Science built an institution for this exact problem three and a half centuries ago. When the Royal Society was founded, it took as its motto nullius in verba—take nobody’s word for it. The point was never that ideas were scarce. The point was that an idea counts for nothing until it has been checked, and that the act of checking is the institution’s reason to exist.

We are about to inherit more ideas than any civilization has ever held: recombinations conjured at near-zero marginal cost, across distances no single mind could bridge. Whether that becomes a golden age of discovery for your startup, your science, or your art, or simply a planetary waste of tokens, turns on the oldest discipline we possess: the refusal to take the machine’s word for it.

And that refusal cannot stop at the model. The labs building self-improving systems are also drawing the borders of self-improvement: which domains may compound at frontier speed, which must be routed elsewhere, and which restrictions users are even allowed to see. Their caution may be justified; a loop that improves intelligence is not an ordinary product feature. But nullius in verba was never a judgment about motives. It was a rule for claims. A safeguard is a claim too. A civilization mature enough not to take the machine’s word for its answers cannot take any lab’s word for where the loop may close.

Recombination is now free. The future belongs to whoever is allowed to verify it.

A version of this article appeared on Substack.