180 Million Lines: Claude's Massive Push to Mathematically Verify Open-Source Code

180 million lines of mathematical proof code were generated by Anthropic’s Claude in a matter of weeks, marking a significant shift in how software correctness is established. This massive output represents an automated approach to formal verification, a process that has historically been too resource-intensive for broad application in the tech industry.

The initiative, conducted by Anthropic’s Frontier Red Team in collaboration with partner Theorem, focused on 123 open-source libraries within Google’s OSS-Fuzz security project. Instead of creating new software, the model translated existing crash tests into mathematical problems and analyzed compiled machine code to produce extensive proofs. These proofs are subsequently checked by an independent verification program named Rocq. According to Anthropic, the process requires minimal human intervention, allowing researchers to describe the proof generation as fully autonomous, even though the underlying formal foundations and tools were designed by human experts at Theorem.

The scale of this effort contrasts sharply with previous benchmarks in the field. Formal verification has long been considered prohibitively expensive; for instance, a team of specialists required 20 person-years to verify the seL4 operating system kernel. Consequently, such rigorous methods were typically reserved for high-stakes environments, such as avionics systems in aircraft. By leveraging AI to handle the computational load, Anthropic and Theorem have demonstrated that this level of scrutiny can be applied to a much wider range of software.

A critical byproduct of the verification process was the discovery of hundreds of bugs that conventional testing methods had missed. The model was permitted to fix these errors autonomously. Many of the defects were subtle, manifesting only under specific conditions such as unusually large input files or particular sequences of data access. In one notable instance, while verifying a widely used PDF reader, the team identified as many security vulnerabilities in a single week as are typically reported for that software over an entire year. Additionally, Claude detected a logical error that had remained undetected for more than a decade.

Despite these successes, the project has defined limitations. Anthropic clarifies that the label "formally verified" does not equate to being entirely error-free. The proofs are valid only under strict conditions, including a specific processor architecture, single-threaded execution, and inputs smaller than 64 megabytes. Real-world applications often operate across diverse hardware and configurations, meaning the verification is context-dependent. Furthermore, the computational cost is significant, with individual proofs consuming hundreds of millions of tokens. However, efficiency is improving; newer models like Opus 5.5 are solving half of the proofs with approximately 40% fewer tokens than its predecessor.

If AI systems can both write and verify code, the implications for software development could be profound. Developers may place greater trust in AI-generated software, reducing the time spent on manual review and lowering overall costs. Theorem’s roadmap now targets critical infrastructure libraries, including glibc, OpenSSL, and cURL. Anthropic anticipates that AI-driven formal verification will soon become as routine as standard testing, potentially transforming the fundamental question in software engineering from whether AI can program to whether it can guarantee the safety of its own code.