How Anthropic Used Claude To Turn Fuzz Testing Into Proofs

🔍 Read the full analysis: How Anthropic Used Claude To Turn Fuzz Testing Into Proofs on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Anthropic’s Frontier Red Team and its collaborator Theorem say they used Claude to generate machine-checked proofs that 239 fuzz-test entry points in 123 open-source libraries do not crash on inputs smaller than 64 megabytes when compiled into specified RISC-V programs. The effort also found bugs that the teams say existing fuzzing had missed, but the result applies only to the modeled binaries and conditions—not to every build or use of the libraries.

The team used existing fuzz tests as the properties to prove. A fuzz test feeds inputs into a small library entry point and treats crashes or failed checks as bugs. Anthropic and Theorem compiled the library and test into an instrumented RISC-V executable; Claude then wrote a proof in Rocq, a formal proof system, and Rocq’s checker verified it against the specification. The stated property was that the program would not crash for any input under 64 MB, including crashes from AddressSanitizer memory errors or failed assertions, under the modeled assumptions.

Anthropic says the project ran over a few weeks and produced about 180 million lines of Rocq proof. A typical proof was around 600,000 lines, for a binary with a median size of 18,000 RISC-V instructions. The company says about 180 million lines did not need human review because the proofs were checked mechanically. Trust instead rests on the proof checker and the formal models and theorem statements, along with supporting components such as the hardware and operating system.

The process reportedly found hundreds of bugs, including cases that required unusually large inputs, rare data layouts or sequences of calls to expose. Examples in the source account include a decompressor issue requiring an input of roughly 20 MB and an image-decoder read requiring a 64 MB input and more than 2 GB of memory. The account also describes a file-restriction logic bug that did not necessarily cause a crash. These are reported examples from the project, not evidence that every bug in the libraries was found.

When Fuzz Tests Become Proof Targets

Fuzzing tests many inputs but cannot generally establish that every possible input or code path has been covered. A proof, if valid for its specification and model, can establish the stated property for all inputs within its defined limits. This gives the reported work a different security value from simply running more tests: it attempts to rule out a category of crashes at particular entry points rather than estimate how often one might occur.

The approach also changes what can be checked. A fuzz test can encode a higher-level requirement that fails through an assertion, not just memory safety. Anthropic gives a compression example: if a test asserts that decompressing compressed data returns the original, proving the test cannot fail can establish that round-trip property for the modeled inputs. The team also reports that proof attempts exposed requirements not stated in the original tests, including a need to rule out stack overflow from deep recursion in a destructor.

The result is not a general certificate that software is safe. Its practical reach depends on the exact binary, the entry point, the specification and the assumptions used to model execution. Still, if the reported method can be applied reliably at lower cost, it could let developers use formal verification on more code than traditionally feasible. Anthropic frames the work as a response to the possibility that AI systems can find and combine software flaws; that broader security argument is the company’s rationale, not a finding established by these proofs.

From Existing Tests to Rocq Proofs

Formal verification aims to prove that software meets a specified property, but writing a useful specification and completing the proof can demand substantial specialist effort. Anthropic cites the seL4 microkernel as an example of the cost: two person-years to build and 20 person-years to verify. That comparison is background supplied by Anthropic, rather than a cost measurement from the new project.

The collaboration avoided writing a new behavioral specification from scratch for each library. Instead, it used an existing fuzz test as the starting point, compiled the code and test to RISC-V, and modeled the resulting program and its Linux process environment. Claude generated the Rocq proof; when a proof attempt failed, the failure could point to a potential defect. The team could then patch the code, regenerate the specification and try to prove the revised program.

Anthropic describes the work as, to its knowledge, the first large-scale application of AI to formal verification of production software at the assembly-code level. It says it is publishing proofs that did not require patches, some previously unavailable trusted components, and specifications for tests it could not prove. That qualification matters: the reported set includes proof attempts that did not succeed, as well as completed results.

“discovered hundreds of bugs”

— Anthropic

Limits of the Verified Binaries

The proofs cover single-threaded, static RISC-V binaries linked against musl, according to the source account. They do not establish the same properties for the x86 or ARM builds linked against glibc that are more commonly used. It is also not clear from the supplied material how many of the reported bugs have been independently confirmed, patched or assigned vulnerability identifiers.

A no-crash proof is only as broad as its specification and model. It does not prove that a library has no security or correctness defects, and it cannot establish properties the test does not express. Some logic defects may not crash at all. The stated input limit is under 64 MB, and the result depends on assumptions about the executable and its environment. Anthropic also notes that some proof attempts could not be completed; the source does not give a full account of their status or the total computing cost.

The report says the proofs can consume hundreds of millions of tokens, though newer models and reuse of shared proofs can reduce that requirement. The supplied information does not provide a complete per-library cost breakdown, independent replication results or a detailed account of how the method performs across other architectures and build configurations.

Publication and Wider Validation

Anthropic says it is releasing the proofs that required no code patch, trusted components that were not already public, and specifications for fuzz tests it could not prove. Those materials should let outside researchers inspect what was formalized and how the checker validates the claims. The source account does not specify a release schedule for every artifact.

The next useful test will be whether researchers and maintainers can reproduce the results, confirm the reported defects and adapt the method to the platforms and build configurations used in deployed software. Until then, the reported proofs should be read as guarantees about particular modeled RISC-V programs—not as assurance that the corresponding libraries are free of bugs in general.

Key Questions

What did Claude prove?

Anthropic and Theorem say Claude generated Rocq proofs for a no-crash property at 239 fuzz-test entry points in 123 open-source libraries. The property covers inputs under 64 MB for the specified compiled programs and assumptions.

Does this mean the libraries are bug-free?

No. The proofs address specified properties for particular binaries and entry points. They do not rule out every type of defect, every untested behavior or problems in other builds.

How did the team check Claude’s work?

Claude wrote proofs in Rocq, and the Rocq proof checker checked them against the formal specifications. The claim relies on the checker and the models used for the program and its environment, not on people reading every line of proof.

Why did the method find bugs fuzzing missed?

Fuzzers test selected inputs and paths, while a successful proof must account for every input covered by its specification. The project account describes defects that required large inputs, rare layouts or particular call sequences, which ordinary fuzzing may not reach.

Do the results apply to common x86 and ARM builds?

Not as reported. The proofs cover single-threaded static RISC-V binaries linked against musl. The source says they do not establish the same result for commonly used x86 or ARM builds linked against glibc.

Source: ThorstenMeyerAI.com