What mathematicians ought to know concerning the Lean Theorem Prover: questions of reliability and AI

[This is a guest post by Thomas Hales. This blog post was initially written in a different file format and converted using AI. — T.]

Mathematicians have been weighing in on what they worth about arithmetic. For me, what issues is the consistency of math and its unparalleled reliability in assist of science and civilization.

Formalization of Math

A proper proof is a mathematical proof that has been exhaustively checked on the stage of the foundations of math and the elemental guidelines of logic. In concept, this could be finished by hand, however due to the variety of steps concerned, that is typically finished by pc, utilizing software program that’s designed for the duty.

Examples of theorems which have been formalized embrace the four-color theorem, the Feit-Thompson (odd-order) theorem, the Kepler conjecture, sphere eversion, the sphere packing drawback in 8 and 24 dimensions, Navier-Stokes compelled blowup, and Fermat’s Last Theorem. The final three formalization initiatives have been accomplished this 12 months and have introduced widespread consciousness of the potential of formalization.

Software methods for formalization are variously referred to as proof assistants, theorem provers, or interactive theorem provers. For the aim of this submit, these phrases are used interchangeably. Many proof assistants have been developed over time: Automath, HOL Light, Isabelle, Coq (renamed Rocq final 12 months), Metamath, Mizar, and Lean. Freek Wiedijk edited a guide “The Seventeen Provers of the World” that compares a few of these proof assistants, giving a proof of the irrationality of the sq. root of two in every of them. Among mathematicians, the Lean theorem prover is the preferred, and this submit will give attention to Lean.

Lean was developed and launched by Leo de Moura in 2013, whereas at Microsoft. To our nice profit, de Moura persuaded Microsoft to make the software program open-source. Kevin Hartnett’s guide on the historical past of Lean, “The Proof within the Code”, states that Jeremy Avigad (the director of Carnegie Mellon’s new NSF institute ICARM) was the primary consumer of Lean. He ran a Lean seminar in 2015 that I attended. In 2017, considered one of Jeremy’s graduate college students, Mario Carneiro, working with Johannes Hölzl, took current elements of Lean’s core library and began a separate Lean mathematical library, referred to as mathlib. This library of formalized arithmetic is now large, containing almost 300,000 theorems, over 100,000 definitions, 2.5 million strains of code, with over 700 contributors. Any definition or theorem in mathlib can be utilized to show additional theorems. For instance, if a proof makes use of the Cauchy-Schwarz inequality, the outcome may be cited from the library reasonably than reproving it.

Autoformalization is a sensible actuality

In the previous, researchers needed to transcribe paper proofs into formal proofs by human labor. For instance, the formal proof of the Kepler conjecture on sphere packings in three dimensions took about 20 human work-years to finish and consists of about 500,000 strains of proof scripts. For years, it has been a dream for many people working in formalization to seek out methods to deliver elevated automation to the method. Autoformalization is the belief of that dream. Autoformalization is the formalization of arithmetic by AI. AI reads the paper (say a pdf or tex file) and outputs the formal proof in Lean or another proof assistant.

Autoformalization has develop into a sensible actuality in 2026. Starting in late spring and summer season of 2025, researchers have been changing into more and more bullish about autoformalization. Here are some milestones.

• Sep 2025, Math Inc. produced a quasi-autoformalization of the prime quantity theorem. The course of was merely “quasi”, as a result of people needed to intervene to present additional steerage at any time when the AI acquired caught.
• Jan 2026, J. Urban posted an arXiv preprint “130k strains of formal topology in two weeks” that gave the autoformalization of enormous elements of Munkres’s topology textbook in a proof assistant primarily based on set concept.
• Mar 2026. Approximately per week after saying the finished formalization in 8 dimensions, Math Inc. introduced an autoformalization of the sphere-packing drawback in 24 dimensions, following the proof by Viazovska and her collaborators. This undertaking generated about 500K SLOC (supply strains of code) that {golfing} (or code pruning) later diminished to about 200K strains.
• May 2026, a bunch at Meta/Facebook Research autoformalized a big a part of 26 mathematical textbooks in a undertaking referred to as ATLAS.

From there, quite a few theorems have been autoformalized. Particularly noteworthy is the autoformalization of Fermat’s Last Theorem, introduced by Anthropic on September 4. This undertaking generated 13 million strains of Lean in 11 days. The announcement of Navier-Stokes blowup with forcing on September 8 by OpenAI was accompanied by an autoformalization of the concept in Lean.

Looking ahead, Urban said in January, “We imagine that (auto)formalization might develop into fairly simple and ubiquitous in 2026, no matter which proof assistant is used.” Autoformalization initiatives have been accomplished in varied proof assistants utilizing varied LLMs, however we give attention to Lean. “For [Jesse] Han, it represents much more: the start of a revolutionary transformation in arithmetic, the place extraordinarily large-scale formalizations are commonplace” (IEEE Spectrum). Jared Lichtman introduced the launch of MAP (the Mathematics Autoformalization Project) on Sept 8, 2026, which goals to translate “all recognized math into formal code”. He asks us to think about the subsequent one trillion strains of code.

Is Lean dependable?

Type concept.

Lean relies on kind concept; actually, a selected dialect of kind concept referred to as CIC, the calculus of inductive constructions. This submit isn’t meant to be a tutorial on kind concept, and I might be transient. Russell’s well-known paradox in 1901 (the set of all units that aren’t a component of themselves….) led to a disaster within the foundations of math. Two options have been proposed later that decade. (1) Zermelo’s axioms of set concept that disallow the creation of unsafe units; (2) kind concept that makes it a syntax error to create Russell-paradox-like entities. Type concept was launched by Russell himself in 1903 in his guide Principles of Mathematics, and it turned a part of the foundational system of Russell and Whitehead’s Principia.

For mathematicians who’re accustomed to set concept, B. Werner’s paper (1997) “Sets in Types, Types in Sets” provides some reassurance that no matter they’ve finished in set concept may be translated into kind concept, and no matter will get finished in kind concept may be translated again into set concept. More exactly, the paper reveals that ZFC set concept may be encoded into CIC, and {that a} specific dialect of CIC may be encoded again into ZFC (augmented with a hierarchy of inaccessible cardinals).

At the chance of simplifying issues to a ridiculous diploma, we’d say that “sorts are like disjoint units”; every aspect in kind concept “is a component of” precisely one kind. The kind of the pure quantity 2 is the pure quantity kind; the kind of e, the bottom of the pure logarithm, is the actual quantity kind, and so forth. The kind of pure numbers is disjoint from the kind of actual numbers, and an specific coercion (sending 2 to 2.0) is constructed from the kind of pure numbers to the kind of actual numbers. When I give talks, I generally draw an image of units as a Venn diagram with nonempty intersections and an image of sorts as bricks stacked in opposition to each other with out intersection.

Lean’s design

One a part of the Lean system is a general-purpose programming language (appropriately referred to as the Lean programming language). Ordinary pc applications, comparable to a program to type an inventory, may be written on this language, then compiled and run. The Lean system additionally offers a mathematical language, during which definitions may be written, theorems may be said, and proof scripts may be written. The programming language and mathematical language should not unbiased entities. Rather, it’s a single language that does each. Program code may be blended with theorems concerning the correctness of the algorithms; mathematical proofs may be generated utilizing applications. The proof scripts in Lean are parsed and undergo a course of referred to as elaboration (a type of compilation course of for arithmetic), then the proofs are checked by the Lean kernel. It is the kernel’s accountability to test and confirm the output of elaboration.

The Lean kernel is a number of thousand strains of C++ code. The kernel is fastidiously engineered however extraordinarily advanced. We talked about mathlib above, which consists of about 2.5M SLOC, written within the Lean language. The library has been elaborated, then checked by the kernel. If there may be an unconditional false proof wherever in these 2.5 million strains of code, it’s the fault of the kernel or runtime for failing to reject a false proof. Any defect within the underlying kind concept is a severe kernel defect, whether it is carried out in code.

Lean proofs ought to by no means be believed till they’ve been checked by the kernel. Additionally, a proof in Lean shouldn’t be accepted till a human audit is carried out to make sure assertion constancy. Is the verified theorem what we expect it’s? Do the definitions in Lean correspond to what we expect they need to be? This job is mostly massively simpler than checking the proof itself. For occasion, for Navier-Stokes, a human ought to test that the assertion in Lean corresponds with Fefferman’s assertion of the Millennium Prize Problem, and particularly that ideas comparable to the sector of actual numbers, partial derivatives, and measure are accurately outlined in Lean. The comparator software in Lean assists with this job. The software can even carry out further checks, comparable to inspection for doable unauthorized axioms.

Summer of Soundness Bugs

A soundness bug is a bug within the kernel that permits a proof of “False”, and consequently a proof of any proposition. A soundness bug is essentially the most disastrous of any type of bug in a proof assistant and may set off an alarm for mathematicians who care deeply concerning the reliability of arithmetic. Occasionally, soundness bugs are present in varied proof assistants. In 2003, I discovered a soundness bug within the proof assistant HOL Light, which was then thought-about to have essentially the most dependable of all kernels. That kernel is tiny, consisting of just some hundred strains of pc code. For me, it’s a badge of honor that I discovered this soundness bug, which was the primary soundness bug that had been present in that proof assistant since 1996. (See HOL Light change log, July 2003.)

Lean 4 was launched in September 2023. Prior to launch, two soundness bugs have been discovered and corrected. In May 2025, one other soundness bug was reported, attributable to overflow. All hell broke unfastened within the spring and summer season of 2026, which is now being referred to as the “Summer of Soundness Bugs”. Several soundness bugs in Lean have been uncovered in July and August. The summer season insanity affected varied proof assistants, however my focus is Lean. One Lean bug led to a bootleg disproof of the Collatz conjecture. I realized of the bug this summer season when it produced a brief illicit proof of the Kepler conjecture in Lean. All these bugs have been rapidly repaired, and mathlib has been verified by the repaired kernel. An evaluation of the soundness bugs is present in de Moura’s postmortem.

The “summer season of Lean soundness bugs” may sound like a catastrophe, however nearer investigation reveals that the detection of those soundness bugs is a constructive improvement. The summer season bugs have been detected by frontier mannequin AI within the fingers of safety researchers enthusiastic about dependable kernels, not by black-hat hackers. The Collatz bug was discovered by Ramana Kumar, a co-author of “CakeML: a verified implementation of ML”, which creates an end-to-end verified ML (the purposeful programming language). Several bugs have been discovered by Dan Selsam. According to de Moura’s report, “Daniel Selsam at OpenAI assisted the Lean FRO with an AI specialised in cybersecurity, and located different programming errors within the Lean kernel. All of them have been mounted.” The collaboration with Selsam ended “when the interior AI reported it couldn’t discover further points.” Dan Selsam has contributed to Lean from its early days and was one of many creators of the IMO grand problem geared toward reaching IMO-level drawback fixing verified in Lean. He has been within the information not too long ago over his warning about AI security (Sept 14), reported in a viral submit on X.com.

Bug extermination

Various proposals have been made about how you can keep away from soundness bugs in Lean. I’ll talk about three.

1. Develop different Lean kernels, and cross-check formal proofs.

About 25 kernels for Lean have been written. The “Lean Kernel Arena” lists them.

All who mistrust the present lineup of kernels are welcome to put in writing their very own kernel for Lean. I’ve generally performed with the concept of writing a kernel and have urged the undertaking to college students with out success. It appears to me a wonderful strategy to be taught Lean completely. I’ve recognized of Dan Selsam since 2016, once I heard of his graduate-student undertaking at Stanford that developed a Lean kernel in Haskell. Another early Lean kernel was written in Scala by Gabriel Ebner in 2017.

The Navier-Stokes formalization has already been confirmed by greater than a dozen proof-checkers. Cross-checking the proof by totally different kernels doesn’t take away all doubt. The Collatz bug was not caught by cross-checking in opposition to a considerably out-of-date Nanoda kernel, which accepted the illicit Collatz disproof due to its personal unrelated bug. Computer chips may need design bugs and manufacturing defects. There are tender errors, working system bugs, and compiler bugs. Different kernels may need the identical defects. Some of those errors may be mitigated by operating totally different kernels which have been carried out in several programming languages on totally different {hardware} and working methods.

Ideally, we might desire a “clean-room” design of the Lean kernel – a kernel implementation that doesn’t take a look at the Lean 4 kernel supply code, to keep away from copying bugs from one kernel to a different.

2. Formally confirm the kernel.

Gödel incompleteness. We want to possess a proper proof that the Lean 4 kernel has no bugs. However, Gödel’s second incompleteness theorem locations extreme limitations on this enterprise. The most we’d hope for is a relative consistency proof. If such and such a system is constant, then Lean 4 is constant; it has no soundness bug; it is not going to produce a proof of False.

There is an extended custom of formally verifying kernels. In precept, formal verification can test each the logical specification of a kernel and its concrete implementation in code; however some verifications may test one however not the opposite. Years in the past, John Harrison formally verified the core of the HOL Light proof assistant kernel in a strengthened model of HOL Light. This gave a proof of idea. An extra enchancment has been an implementation of HOL Light in CakeML, talked about above, which is a programming language with formal semantics and a verified compiler. This is what the Candle undertaking does.

There are different main kernel verification initiatives for different proof assistants.

Autumn of verified Lean kernels

In a submit on-line on September 10, Joachim Breitner wrote, “I’m a bit childishly proud that I simply launched a Lean Checker with a proper consistency proof. I declare the summer season of AI-found kernel implementation bugs to be over!” (@nomeata). I might go additional and describe this undertaking as one of the crucial vital milestones in Lean’s historical past.

Breitner has developed a verified Lean kernel referred to as Con-Leche. The implementation is in Lean, and consistency is formalized in Lean, with code and proofs generated by Claude. The formal consistency proof assumes a Lean encoding of ZF set concept augmented by a hierarchy of inaccessible cardinals. Interestingly, the Con-Leche semantics for Lean’s phrases are straight set-theoretic reasonably than kind theoretic. Con-Leche has checked mathlib. The undertaking incorporates the same old disclaimers that the kernel verification makes assumptions about compiler, runtime, and pc surroundings. Con-Leche’s consistency proof has been checked by greater than a dozen different proof-checkers. Con-Leche’s consistency declare may suffice for all sensible functions, even when it differs in technical element from the declare of Lean type-theory consistency.

One extremely constructive facet of Breitner’s work is that a number of the most abstruse elements of Lean, comparable to the overall equipment of mutually inductive sorts with nesting, now have consistency ensures backed by a set-theoretic mannequin.

3. Improve our theoretical understanding of the kernel and Lean’s kind concept (a selected dialect of the Calculus of Inductive Constructions which has non-cumulative universes and proof irrelevance).

The foundational doc for the kind concept of Lean is Mario Carneiro’s MS thesis at Carnegie Mellon (2019). The dialects of CIC utilized by Rocq and Lean are sufficiently totally different that outcomes don’t straight switch from one to the opposite. Unfortunately, an error was discovered within the thesis. The thesis can also be out-of-date, as a result of it focused the older Lean 3 system. Work to restore and lengthen the thesis is ongoing.

As a member of his thesis committee, I used to be shocked when he proved that definitional equality in Lean is undecidable. In apply, because of this the Lean algorithm fails to ascertain the definitional equality of some phrases which can be actually definitionally equal. This destructive outcome was not downgraded by the error; it’s nonetheless a theorem.

We point out some desired properties of Lean’s kind concept and the present standing of the proofs.

Unique typing.

Above in our “ridiculous” simplification of kind concept, we said that every time period has a novel kind. More exactly, distinctive typing is the property that if a time period has each kind A and sort B, then A and B are definitionally equal. Unique typing isn’t a property constructed into Lean’s logic. It is a tough conjecture that’s nonetheless unproved. Other very fundamental questions on Lean’s kind concept stay unanswered, together with Pi-injectivity, a modified Church-Rosser property, and type injectivity.

Logical consistency relative to set concept.

This property states that there isn’t a derivation of False in Lean’s system with the given axioms, below the belief of set concept consistency (with appropriate axioms). Of course, logical consistency is the only most vital property that we should always need of Lean’s kind concept. As of October, 2026, I do know of no full, public relative-consistency proof protecting Lean summary kind concept. Mario Carneiro has claimed in his thesis and in lectures that there’s an alternate route to ascertain consistency that avoids the thesis error, however to the perfect of my data, this alternate route has by no means been written down, past a quick assertion within the introduction to his thesis. In my view, a results of such basic significance should be given in full earlier than it’s accepted. Con-Leche, mentioned above, makes and formally verifies a carefully associated consistency declare relative to set concept.

Progress is being made on these analysis issues (arXiv:2607.13662, arXiv:2403.14064, Carneiro/AITP2026).

In his talks, Mario Carneiro has repeatedly made a request to different researchers to contribute to the foundational metatheory of Lean, “There are a half dozen individuals engaged on MetaCoq, however Lean doesn’t have sufficient kind theorists concerned. If you determine as such, come assist out!” (Slides of Bonn discuss, 2024-07-24). I second his request.

My general evaluation is that our theoretical understanding of Lean’s kind concept isn’t what we wish it to be and that the mathematical group as an entire is giving brief shrift to essential type-theoretic questions associated to Lean. If as a career we’re emigrate on the entire from set concept to kind concept, then we should always work much more to solidify the foundational metatheory.

Consistency could also be a very powerful foundational property, however consistency is not at all sufficient. I don’t imagine that mathematicians may be totally glad with a system that claims to be a sort concept however that can’t even promise that well-formed phrases have a novel kind, as much as definitional equality. The summary concept should be easy sufficient to show and to be realized by a big group. We additionally can’t be totally glad if the one recognized path to consistency is an AI formalization that lacks human exposition.

Postscript:

Ken Thompson famously wrote “Reflections on Trusting Trust”. He requested, “To what extent ought to one belief an announcement {that a} program is freed from Trojan horses?” He imagines malicious code that finds its manner into compilers and hides its personal presence. His conclusion is, “You can’t belief code that you just didn’t completely create your self… No quantity of source-level verification or scrutiny will shield you from utilizing untrusted code.”

Today, within the age of AI, which more and more has the aptitude to deceive us and to take advantage of software program vulnerabilities, we completely can not put blind belief in methods comparable to Lean. Taking an adversarial view of AI, we’d ask how you can certify that AI didn’t depart a backdoor soundness bug in Lean when it did its sweep for bugs in the summertime of 2026? What if the bug is so obscure that people are not possible to seek out it on their very own? What if that very bug was exploited within the Lean verification of the Con-Leche checker, leaving a soundness bug in Con-Leche too? (Now that Con-Leche’s consistency has been cross-checked by a number of different kernels, a soundness bug must defeat all these cross-checks as nicely.) Then suppose that bug is used maliciously to plant a backdoor in formally verified software program that protects vital roads. What precautions will we take now to forestall one of these future situation? During the previous 12 months, a lot foundational work on the type-theoretic foundations of math and its reliability has been relegated to AI, and that is harmful except fastidiously audited by people.

Credit: I thank Avigad, Breitner, and Urban for feedback and corrections. Authorship is absolutely human (TCH). AI was used as a software in search and analysis, fact-checking, and proofreading.