180 Million Lines of Proof: How AI Just Changed Software Safety

One hundred and eighty million lines of mathematical proof code were generated in just a few weeks. This staggering output comes from a new collaboration between Anthropic and partner Theorem, which has demonstrated that large language models can now perform rigorous formal verification of existing software. The system did not write new applications from scratch but instead focused on proving that established code functions correctly under defined conditions.

Historically, formal verification has been a luxury reserved for high-stakes environments like avionics or nuclear systems. The process is incredibly labor-intensive, often requiring teams of specialists to spend years on a single component. For instance, verifying the seL4 operating system kernel took twenty person-years of expert work. By automating this process with Claude, Anthropic aims to make such rigorous checks as routine as standard unit testing, potentially revolutionizing how developers ensure software reliability.

The methodology involves Claude translating crash tests from 123 open-source libraries into mathematical problems. The model then analyzes the compiled machine code to produce extensive proofs. These proofs are subsequently checked by an independent program called Rocq. According to the researchers, the workflow is nearly autonomous, requiring minimal human intervention. The team describes the process as fully autonomous proof generation, although the underlying formal foundations and tools were designed by human experts at Theorem.

During the verification process, Claude identified hundreds of bugs that conventional testing methods had failed to detect. These errors often lurked in obscure corners of the code, only manifesting under very specific conditions such as unusually large input files or particular access sequences. In one notable case, the team found as many security bugs in a single week within a widely used PDF reader as are typically reported for that software in an entire year. Another logical error had remained undetected for over a decade before the AI flagged it.

Despite these successes, significant caveats remain. Anthropic notes that formal verification does not guarantee that software is entirely error-free. The proofs are valid only under strict constraints, including specific processor architectures, single-threaded execution, and input sizes under 64 MB. Since real-world applications often run on diverse hardware and configurations, this limitation is a critical factor for developers to consider.

The computational cost of this approach is also substantial, with individual proofs consuming hundreds of millions of tokens. However, efficiency is improving rapidly. A smaller test run indicated that proofs are reusable, and newer models are becoming more token-efficient. For example, Opus 5.5 solved half the proofs with approximately 40 percent fewer tokens than its predecessor, Opus 5. This trend suggests that the economic barrier to entry may lower over time.

If AI can both write code and prove its safety, the industry may see a major leap in automation. Anthropic expects formal verification to become as practical as standard testing soon. Theorem has already outlined a roadmap targeting critical infrastructure libraries like glibc, OpenSSL, and cURL. This shift moves the focus from simply asking if AI can program to determining if it can guarantee that its output is safe.