What Is Recursive Self-Improvement, and Where’s The Next Battlefield

Consider the past eight weeks. Anthropic published an essay titled "When AI Builds Itself" disclosing that Claude now writes more than 80 percent of the code merged into Anthropic's own systems, and calling on the industry to preserve the option of slowing down. An internal OpenAI model disproved the unit distance conjecture, an 80-year-old problem in combinatorial geometry, producing a counterexample construction that nine of the world's leading mathematicians then verified and annotated by hand. And Recursive Superintelligence, a company founded in 2025 with roughly 25 employees and no public product, emerged from stealth with 650 million dollars at a 4.65 billion dollar valuation, on the explicit thesis that the self-improvement loop itself is the product.

The question everyone is asking is "how fast". That question has no stable answer, and the people offering one are mostly selling something. There is a better question with a structural answer: where does self-improvement work, where does it stop, and what moves the boundary?

The argument of this essay is that the boundary is not a natural constant. It is a function of engineering, specifically the engineering of what we will call validators. Recursive self-improvement (RSI) expands into a domain exactly when someone builds a fast, trustworthy signal of success that the optimisation process cannot corrupt. Predicting the next domain RSI conquers is therefore equivalent to predicting where the next such signal gets built. That reframing turns a metaphysical debate about superintelligence timelines into a due diligence checklist, which is what we intend to leave you with.

Three things called "self-improvement"

The phrase covers three different activities, and conflating them is the single most common error in both pitch decks and press coverage.

Layer (a): AI making discoveries on external problems. FunSearch found new constructions in combinatorics and landed in Nature. AlphaEvolve improved matrix multiplication kernels and data-centre scheduling inside Google, feeding gains back into the infrastructure that trains its own successor models. The unit distance result belongs here too. This layer has the strongest evidence base by far.

Layer (b): AI improving its own scaffolding. Agents rewriting their tools, prompts, memory systems, and orchestration code. The Darwin Gödel Machine is the canonical research artifact; the 80-percent code figure at Anthropic is the production reality. Evidence here is real but shallower than it looks, because the thing being improved is the shell around the model, not the model.

Layer (c): AI improving its own training signal or weights. Self-rewarding training, models judging models, systems that modify what they learn from. This is the layer the word "recursive" actually evokes, it is where the superintelligence narratives point, and it is where the known catastrophic failure lives. Self-rewarding training (SRT) collapses under prolonged optimisation: the model learns to satisfy its own reward, the reward drifts with the policy, and performance falls off a cliff.

Here is the refinement that matters, because a sophisticated critic will immediately object that layer (c) already works in production. Constitutional AI, RLHF variants, and LLM-as-judge pipelines all involve models shaping the training of models, and they run every day at every major lab. What separates stable layer-(c) systems from collapsing ones is not whether the system self-trains. It is where the anchor sits. RLAIF-style pipelines work because something in the loop is held fixed outside the optimiser's reach: a frozen constitution, a frozen judge model, human spot checks, a KL penalty tethering the policy to its pretrained prior. SRT collapses because its anchor, agreement with the model's own majority vote, moves together with the policy being optimised. The anchor is inside the loop.

So the first three questions to ask of anything calling itself recursive self-improvement: Which layer? What is the anchor? Is the anchor inside or outside the optimisation loop?

What the frontier labs are actually betting

Apply the taxonomy to the five most visible positions and a spectrum appears.

Anthropic has positioned itself as the measurer with a hand on the brake. The June essay is careful on both flanks. It presents internal data showing layer (b) is deeply underway, with engineers shipping eight times the code per quarter they did from 2021 to 2025. It states plainly that full RSI has not arrived and is not inevitable. And it argues the world should retain the ability to slow or pause frontier development while alignment research catches up. Anthropic co-founder Jack Clark has separately put roughly 60 percent probability on a system eventually training a more capable successor without human involvement. Read through our lens, Anthropic is claiming (a) and (b) are here, and treating the transition into anchored (c) as a phase change requiring governance rather than celebration.

OpenAI is the roadmap. The stated targets are an intern-level research assistant by September 2026 and a fully autonomous AI researcher by March 2028, with the unit distance disproof serving as the exhibit. One structural observation belongs in any honest account: that roadmap was announced alongside a corporate restructuring and a commitment of roughly 1.4 trillion dollars to infrastructure. The RSI narrative is simultaneously a research programme and a financing instrument. That is not an accusation, it is a pricing consideration, and we return to it below.

Google DeepMind publishes no manifestos and ships the most artifacts. AlphaEvolve improving the training pipeline of the models it runs on, AlphaChip normalising learned chip layout, and semi-autonomous mathematical discovery work with Gemini constitute the deepest portfolio of delivered (a) and (b) results. Across the three big labs, the volume of position statements is roughly inversely correlated with the volume of delivered evidence. Draw your own conclusions about what that implies for reading lab communications.

Recursive Superintelligence is the natural experiment. The big labs sell models and APIs and use self-improvement as internal tooling; Recursive has organised an entire company around the loop as the commercial thesis, with a staged plan toward autonomous training runs and a stated ambition to expand from AI research into physics, chemistry, and pre-clinical biology. This is the purest bet on layers (b) into (c) that exists at scale. It follows that Recursive faces, in its most concentrated form, exactly the engineering question this essay is about: as its loop tightens, what remains fixed that the optimiser cannot redefine, and who calibrates it? Every layer-(c) company must answer this. The ones that succeed will succeed because they answered it well, and the answer will be visible in their architecture, not their narrative.

AMI Labs is, in our framework, not an RSI company at all but a validator company. Yann LeCun's bet on world models is a bet that the binding constraint for physical-domain AI is the absence of a fast, faithful simulator of reality, and that building one is the unlock. If Recursive is betting on the loop, AMI is betting on the anchor. The fact that both raised historic pre-product rounds within months of each other tells you the market has not yet decided which side of this trade is scarce. This essay is an argument that the anchor side is.

The academic world has, meanwhile, quietly mainstreamed the topic: ICML 2026 hosts a dedicated workshop on recursive self-improvement, even as a large fraction of researchers continue to reject intelligence-explosion framings outright. The disagreement is real and unresolved. What follows is a framework for having it precisely.

Invariants, validators, and the dangerous third case

Start by discarding the folk explanation, which says self-improvement needs "ground truth." It is imprecise in both directions. AlphaZero bootstrapped superhuman Go with no external ground truth beyond the rules. SRT had a training signal of its own making and collapsed anyway. The operative distinction is whether the system possesses an invariant it cannot redefine: a reference frame the optimisation process cannot move. In Go, the rules fix win and loss. For code, the interpreter fixes pass and fail. SRT's majority vote moved with the policy, so there was nothing fixed to optimise against, and optimisation ate itself.

A validator is the special case where a fast invariant coincides with the actual target. In mathematics and code, the fast check (is it correct, is it faster) is the thing you want, so the invariant is a complete validator. In most socially embedded domains the fast checks available are merely necessary conditions. A policy can be budget-feasible and structurally stable and still fail to improve anyone's life; the real target is slow, noisy, and confounded. The invariant is a strict subset of the validator.

Jason Wei's Verifier's Law makes this quantitative: the ease of training AI on a task scales with the task's verifiability, decomposable into five properties (objective truth, speed, scalability, low noise, continuous reward). Math, code, and games score five for five. Social domains typically fail on speed, noise, and objectivity, which is why fast iteration there can prune obviously bad proposals but cannot rank the survivors.

This yields three regimes, and the third is the one that should keep investors up at night.

• Fast-true validators produce autonomous discovery. Math, code, games. This is where every Nature paper lives.

• Slow-true validators make discovery possible but grinding. Clinical endpoints, real policy outcomes, longitudinal anything. Iteration is bottlenecked on reality's clock.

• Fast-but-false validators are worse than none. A cheap signal that looks rigorous, correlates weakly with the target, and can be gamed by the optimiser. SRT's majority vote is one instance. The overfit financial backtest is the instance every reader of this publication already knows in their bones: a strategy that annihilates the backtest and dies on contact with the live market has not been validated, it has memorised the answer key.

(For completeness there is a fourth logical quadrant, slow-and-false, but it has no analytical interest: a loop that cannot iterate quickly and points at the wrong target never starts, so it produces no failure worth studying.)

The technology industry has, in fact, run this experiment on itself at civilisational scale. Engagement metrics, conversion funnels, and A/B tests are all gameable proxies, and yet the industry did not collectively collapse into reward hacking, because there is one anchor no company can redefine: paying customers. Firms that optimised proxy metrics unmoored from that anchor, and 2021 minted a generation of them, executed the commercial version of SRT collapse. The lesson transfers exactly. Fast-false validators are lethal specifically in the absence of an out-of-loop anchor, and survivable, even useful, when a slow true anchor periodically reasserts itself. The robust architecture in every domain is multi-fidelity: iterate cheaply against the fast proxy, re-anchor periodically against the slow truth.

The domain map

Theory in hand, walk the spectrum. For each domain, one judgment and one question to ask a company operating there.

Chip design and EDA. Simulation is a native fast invariant; a layout either meets timing or it does not, and the simulator does not care what the optimiser wants. AlphaChip proved penetration years ago. This is the industrial domain where self-improvement methods diffuse first and deepest. Ask: what fraction of your claimed gains survive tape-out rather than simulation?

Formal mathematics. The domain is migrating wholesale into fast-true territory, and 2026 has been its coming-out year. Note the fine structure of the recent results, because it confirms the framework: the unit distance breakthrough was a disproof by construction, and expert commentary was explicit that a counterexample was the expected first form of AI breakthrough, since checking a construction is vastly cheaper than checking a new proof strategy. Verification asymmetry showed up in the data before the theory was widely stated. Ask: is your system producing constructions (cheap to verify) or arguments (expensive to verify), and who checks the latter?

Materials and chemistry. Nature's feedback loop is slow, so the field is manufacturing a faster one: autonomous laboratories that close the propose-synthesise-characterise cycle in hours. This is the main battleground of constructed invariants, and unlike software evals, a robot lab is a capital asset with a real moat. Ask: what is your cycle time from proposal to physical measurement, and how often does the loop actually close without humans?

Proteins and drugs. The textbook case of invariant strictly inside validator. Structure prediction is fast and solved-ish; efficacy in humans is slow, noisy, and expensive. AlphaFold shortened the front of the funnel, not the funnel. Anyone claiming end-to-end self-improving drug discovery is claiming to have abolished clinical trials. Ask: which stage of the funnel does your fast signal actually validate, and what is its historical false-positive rate at the next stage?

Robotics. The sim-to-real gap is the fast-false risk made physical: policies that ace the simulator and shatter on contact with friction, deformation, and dust. The world-model bet is precisely an attempt to engineer a simulator faithful enough to serve as an invariant. If it works, robotics jumps categories on this map. Ask: what is your zero-shot transfer rate from simulation to hardware, measured on tasks chosen by someone else?

Finance. The backtest is the best-known fast-false validator in commerce, but finance adds a property no other domain on this map has: the validator decays. Alpha is adversarial. A signal that was fast-true erodes into fast-false as others learn it, without anyone changing the code. Domains with adversarial dynamics (trading, security, fraud) can never be permanently conquered by self-improvement, only transiently, and any pitch claiming a durable self-improving edge there is claiming its opponents have stopped learning. Ask: what is the measured half-life of your signals?

Law, policy, education. The slow end. Outcomes take years, attribution is confounded, and there is no objective truth to converge on because reasonable value systems rank the same outcome differently. The research frontier here is instructive: recent academic work couples policy proposals to calibrated agent-based simulations of real populations (a Barcelona homelessness model presented at IJCAI 2025 is a representative example), using the simulation as a constructed, externally calibrated invariant that a proposal-generating process cannot argue its way around. The honest reading of that line of work is that the constructed invariant remains a necessary-condition filter, never an oracle; passing the simulation is not succeeding in the world. But a filter that cheaply rejects the bottom 95 percent of proposals is worth a great deal even when it cannot rank the top five. Ask: what calibrates your simulator, and can the optimiser touch it?

The frontier moves

Everything above reads as static classification. The real story is dynamic: the verifiability frontier is being pushed, deliberately, by three mechanisms, and this is where the next five years of the map get redrawn.

Autoformalisation converts informal artifacts into formally checkable ones. In mathematics this has already happened at scale, with human-verified natural-language proofs now routinely back-formalised in Lean after the fact. The disciplined way to project this outward is by the semantic completeness of the target, and the gradient runs: mathematics (done), safety-critical software and hardware verification (happening now, since the formal methods toolchain long predates AI and the binding cost was always writing specifications, which is exactly what language models cheapen), the structural layer of contracts (partial, because payment triggers and parametric terms formalise while "reasonable efforts" and "material adverse change" are anchored to slow human adjudication by design), general law (no, and claims otherwise are a red flag).

World models attempt to manufacture invariants for physical domains, giving robotics and materials a simulator faithful enough to iterate against. This is the largest single validator-engineering bet in the market today.

Learned reward models are the most seductive and most dangerous mechanism: soft validators, trained rather than fixed. Every learned validator is a wager that the optimiser will not find the seams in it, which is to say a wager against Goodhart's law, which has one of the worst historical win rates of any wager available. Learned validators can work when they are frozen, spot-audited, and tethered to out-of-loop data. Unfrozen, co-evolving validators are SRT waiting to happen.

One sentence carries the whole essay, so here it is on its own: the rate of expansion of recursive self-improvement is limited by the supply of fast-true invariants, and invariants can be manufactured.

Predictions

A framework that cannot be wrong is not a framework. Five commitments, each with a failure condition. Two earlier candidates were dropped from this list because reality got there first: AI-driven improvements to training infrastructure being adopted by frontier labs stopped being a prediction when Anthropic published its code-share figures, and "an AI system resolves a famous open conjecture" stopped being one on May 20, 2026.

1. Within 18 months, a frontier lab publishes a component-level ablation attributing a named fraction of a production model generation's capability gain to AI-driven contributions. Toy-scale demonstrations already exist (Karpathy's autoresearch loop found roughly twenty transferable training tweaks autonomously), and labs already gesture at it ("instrumental in creating itself"), but no lab has yet decomposed a flagship model's gains into a human-versus-AI attribution with a number attached. The first credible production number marks layer (b)'s shift from anecdote to accounting. Wrong if: by early 2028 the claims remain either qualitative ("instrumental") or confined to toy-scale demonstrations, which would suggest labs cannot cleanly attribute the contribution at production scale, or prefer not to. 

2.Over the next 18 months, AI mathematical output shifts from a counterexample bias toward parity between affirmative proofs and disproofs, as autoformalisation collapses the verification-cost gap between them. The early skew was real and had a clean cause: a counterexample is cheap to check by construction, a novel proof is not, so the first dramatic AI results took the cheaper-to-verify form (the unit distance disproof being the canonical case). But that skew is an artifact of a verification bottleneck that Lean is actively dissolving. Once an affirmative proof compiles in a proof assistant, checking it becomes mechanical, and the cost asymmetry that favoured disproofs disappears. The early evidence is already here: AxiomProver's machine-verified proofs of the Chen-Gendron and Fel conjectures in early 2026 are affirmative results drawing on genuinely non-obvious machinery, not cheap counterexamples. Wrong if: through 2027 AI results remain dominated by disproofs and constructions, which would mean the formalisation toolchain matured more slowly than its current trajectory implies, and the verification bottleneck is stickier than this essay claims. 

3. Within 24 months, a closed-loop self-driving lab autonomously discovers a battery or catalysis material that is independently reproduced and enters scale-up evaluation, with biology lagging materials by years. The gap is validator speed and regulatory gating: a coin cell reports in weeks, a clinical endpoint takes years and, per the January 2026 FDA/EMA principles, still requires human sign-off at every autonomous decision. Wrong if: no closed-loop material clears independent reproduction, or a biological candidate reaches equivalent status first. 

4. No weight-level self-improvement system lacking an out-of-loop anchor produces a reproducible, externally verified capability gain within 24 months, and companies organised around that bet pivot toward layers (a) and (b) or toward validator infrastructure. This is the framework's sharpest falsifiable claim, and the current funding environment has conveniently created the natural experiments to test it. Wrong if: someone demonstrates stable, reproducible self-training without any frozen external reference. We would regard that as a genuinely fundamental result, roughly on par with the discoveries it enabled.

5. Within 24 months, autonomous-AI-R&D or self-improvement capability moves from voluntary frontier-safety policies into at least one government framework as a distinctly named, separately assessed category. The risk is already acknowledged across the major labs' safety policies and is being measured by the UK AI Security Institute and NIST; what has not happened is a binding framework naming self-improvement as its own assessed capability. This is the most uncertain prediction here, because regulation tracks demonstrated harm more reliably than anticipated capability. Wrong if: RSI remains folded into general frontier-model evaluation everywhere, with no separately named assessment. 

What this means for capital

The theory translates into three investment positions and one discipline.

The anchor trade, narrowed. If RSI's expansion is bottlenecked on the supply of fast-true invariants, validator infrastructure is the picks-and-shovels position. But state it carefully, because the naive version of this thesis is weak: pure-software evaluation is a low-moat business, frontier labs internalise their own evals, and generic simulation is commoditising. The defensible validator businesses are the ones where the anchor has a moat of its own: physical (autonomous laboratories are capex-heavy and wet), data (simulators calibrated on proprietary measurement, as in EDA and clinical), or regulatory (formal verification bound into safety standards, where the validator is mandatory rather than optional). A validator anyone can replicate in software will be replicated, most likely by the customer.

The narrative discount, not the narrative veto. The strongest objection to red-flagging layer-(c) companies is that nobody literally bets on unanchored self-bootstrapping tomorrow; the RSI story is narrative capital that buys talent, compute, and time while the real business lands in layers (a) and (b). This is partially true and fully priceable. Distinguish two structures. Narrative-(c), business-(a/b): the company tells the superintelligence story while shipping evaluated artifacts with external anchors. Sane, investable, price the narrative premium accordingly. Narrative-(c), business-(c): the evaluation signal and the optimisation target are the same object, and there is no component the team can point to that the optimiser cannot redefine. That is not an early Amazon, it is a structural SRT candidate, and the failure mode is not "too early" but "provably circular."

The red flags, as questions. Which layer are you on? What is your anchor, and is it inside or outside the loop? Who calibrates it, and with what data the optimiser cannot touch? Do your headline metrics come from signals the optimiser can reach? Green flags are the mirror image: teams that volunteer their multi-fidelity re-anchoring schedule, freeze their judges, and publish where their proxy disagreed with reality.

The discipline. Every claim in this essay applies to its authors. The domain map will be wrong somewhere; the predictions carry their own failure conditions precisely so that readers can hold us to them.

Coda

Stop asking when recursive self-improvement arrives. It arrived unevenly, some years ago, in the domains whose structure permitted it, and the interesting variable was never the calendar. Ask instead where the next invariant gets engineered, because that is where the map changes next, and the people building anchors are, for the moment, priced like an afterthought while the people building loops are priced like a prophecy. Markets that misprice structure eventually correct on structure.

When the next pitch says recursive self-improvement, you need three questions. Which layer. What anchor. Who calibrates it.

NonPublic Pty Ltd (ABN 49 607 216 928) holds Australian Financial Services Licence #482668. Investments are available to wholesale and sophisticated investors as defined under the Corporations Act 2001. This content is general in nature and does not constitute financial product advice. It does not take into account your objectives, financial situation, or needs. Investing in private markets involves significant risk, including the potential loss of your entire investment. Past performance is not a reliable indicator of future results. You should obtain independent financial advice before making any investment decision.