By Heath Muchena · Sources reviewed 13 November 2026 · Original DN analysis and illustrative records
What Matters
An AI audit claim needs evidence tied to the code you actually intend to use. Separate the claimed property, reproduced observation, repair and retest. A convincing explanation or a matching build label cannot certify a secure deployment.
AI can accelerate parts of contract review, but a report still needs a checkable chain of evidence. A warning may concern a different build, a reproduction may use an incomplete environment, and a patch may never reach the version that was tested.
This guide gives seven practical checks for that chain. It does not rank five alleged zero-day auditors, attribute real incidents to AI or provide exploit instructions. Its examples use invented labels, and its instrument performs no code execution.
The DN thesis
The smallest useful unit is one finding attached to one artifact and one stated property. Keeping the evidence and repair steps separate makes disagreement visible and gives the next reviewer a specific question to resolve.
1. Identify the artifact the finding actually concerns
A report can be accurate and still concern the wrong build. Begin with the contract revision, compiler, dependencies, configuration and deployment context. Preserve a reproducible artifact identifier in your own records; use a short alias in the public worksheet.
OpenZeppelin distinguishes audited release channels from other channels and warns against treating its development branch as a tagged release. [5] That is one concrete reason “we use a well-known library” is not a sufficient version record.
Separate reviewed source from deployed bytecode. OpenZeppelin describes source verification as checking correspondence with the deployed assembly. [4] A source match supports an identity question; it does not say every property of that code is secure. If the audit covered BUILD-A and the release contains BUILD-B, record the difference before interpreting its findings.
2. Translate the finding into a checkable property
“Critical vulnerability” is a classification, not a complete statement of what went wrong. Ask which property should hold, what assumptions it depends on and what observation would contradict it. A useful record can be specific without exposing an exploit payload.
Solidity’s formal-verification documentation explains that a proof concerns a supplied specification and its assumptions. The specification still needs to represent the intended behavior. [2] A tool cannot resolve an omitted business requirement merely by proving the requirements it was given.
For an illustrative finding, write “an authorized withdrawal must not exceed the allocation represented by this record,” then identify the actual property and test reference in controlled engineering documents. Do not infer that this fictional example describes a vulnerability in any live protocol. The worksheet stores only evidence aliases and declared outcomes.
3. Record the method and its uncertainty
An LLM explanation, static-analysis warning, fuzz-test counterexample and mathematical proof are different outputs. Keep the method name, tool/model version, configuration and coverage with the finding. The agent’s fluent summary should not replace the original result.
The SMTChecker documentation distinguishes failures, unresolved results and abstraction-related false positives. [2] This makes “the tool produced a warning” a useful starting point for review, not a uniform conclusion about an exploitable live failure.
Compiler context matters too. Solidity changed its default arithmetic behavior in version0.8.0, while unchecked blocks deliberately alter that behavior. [3] A generic claim that all generated contracts suffer unchecked overflow can therefore misdescribe the code. Ask the reviewer to connect the finding to the relevant operations and version rather than repeating a category name.
4. Keep reproduction inside an authorized test boundary
A reproduction should have a named environment, input record, expected result and observed result. Preserve the evidence that it ran on the artifact being reviewed. Use controlled test environments and authorized work; this guide supplies no exploit code or live-target procedure.
Anthropic’s smart-contract work evaluates agents using historical vulnerable contracts and controlled experiments. [1] Such work can demonstrate capability under the experiment’s conditions. It does not establish that a particular model performed an actual theft or that a commercial audit product reliably finds every vulnerability.
If a reproduction fails, record what failed. An incomplete setup, different state, missing prerequisite or a false-positive finding are different possibilities. The local record preserves “not reproduced” without converting it into “no bug.” It also distinguishes a completed attempt from no attempt at all.
5. Separate demonstrated impact from a hypothetical loss
The maximum balance visible in a test environment is not automatically the amount a real attacker could take. A severity discussion needs the affected assets, reachable operations, permissions and relevant state assumptions. Keep any modeled loss separate from observed loss.
That distinction also matters when reading a benchmark headline. Simulated revenue is an experimental result with its own setup, retries and selection rules. It should not become a claim about realized criminal proceeds, market-wide exploit frequency or a live provider’s audit accuracy.
Write the narrowest conclusion the evidence supports. “The stated property failed in TEST-A on BUILD-A” is stronger evidence than an unsupported broad claim because another reviewer can understand what would need to be checked. If a business impact remains unknown, leave it unknown rather than filling it with the contract’s total value or a model’s guess.
6. Tie the proposed repair to a new artifact and retest
A patch proposal, an applied patch and a successful retest are separate records. Give the patched artifact its own identifier. Record which test was rerun, which result changed and whether other relevant behavior was checked.
NIST’s SSDF describes secure development as a collection of practices for reducing vulnerabilities, limiting their impact and addressing causes across the development lifecycle. [6] DN’s practical application is to keep repair evidence within that continuing process, rather than treating one model response as the end of the review.
The worksheet compares the declared patch artifact with the artifact named by the retest. A matching label is only a reported link. If the retest used the old build, an absent failure does not establish anything about the proposed repair. If the original failure remains on the intended patch, preserve that conflict instead of reporting the finding as closed.
7. Publish the remaining limits with the conclusion
A completed audit has a scope, exclusions and an evidence date. Preserve unresolved findings, untested integrations, deployment differences and assumptions that could later change. Avoid replacing those details with a single green badge or a numerical safety score.
OpenZeppelin explicitly notes that audits do not ensure the absence of bugs. [4] That limitation applies even when a report is professionally written and several tools agree. Independent review can add evidence, but copying the same model explanation into several reports does not create independent observations.
Give the next reviewer an actionable handoff: the artifact reviewed, property assessed, reproduction record, patch revision, retest outcome and remaining question. If release owners decide to proceed, record that separately as an organizational decision. This evidence map neither makes that decision nor certifies the contract as safe.
DN Audit Finding Evidence Record
Track one finding with opaque aliases. The record compares exact artifact labels only under reader-declared comparable context. A match is a reported link, not authenticated proof. Leave unknown references blank. Do not paste code, secrets or exploit instructions.
No current calculation. Enter a record or load a labelled example.
This worksheet sends no entered values to a server, connects to no account and stores no input in browser storage. Downloads contain the entries you supplied. Use aliases rather than private data. Other site services remain subject to the site’s privacy policy.
Labelled examples and practical use
The first invented case links REPORT-A to BUILD-A, records TEST-A as a reported reproduction, and links RETEST-B to the intended patched BUILD-B. The result describes those links without deciding whether the underlying files are authentic or the test coverage is adequate.
Change the finding’s artifact to BUILD-OTHER and the first link conflicts. Change the retest artifact back to BUILD-A and the patch link conflicts. Report that the failure remains and the tool preserves that result. Missing references remain unknown; a different environment prevents matching labels from becoming comparable evidence.
Use the download as an index into controlled engineering records. It is deliberately small enough to hand to a reviewer without copying confidential source, credentials or a dangerous proof-of-concept into a public form.
Methodology, assumptions and limits
The model compares exact trimmed aliases for finding→reviewed artifact and retest→patch artifact. A comparison needs nonempty references and a declared comparable environment. Different context is outside the comparison; missing evidence remains unknown; different aliases are a record conflict. The model never hashes or reads the underlying artifacts.
Reproduction and retest states are independent reader assertions. A failed reproduction is not converted into a false-positive verdict. A retest that reports the original failure absent is not converted into a global safety conclusion. No overall pass percentage or confidence score is calculated.
All six cases are invented. Public primary documentation was reviewed on 13 November 2026. DN did not audit a deployed contract, run exploit benchmarks, assess a commercial provider’s accuracy or verify a reported repair. The reviewed Solidity latest documents are development documentation; production work must pin the actual compiler and tool version.
Frequently asked questions
Does a clean audit mean the contract has no bugs?
No. An audit has a defined scope and cannot ensure the absence of bugs. Preserve the exclusions and unresolved assumptions. [4]
Is a model explanation the same as a reproduction?
No. The explanation states a claim; a reproduction needs a defined artifact, environment, test and observed result. This worksheet only records references to that evidence.
Can formal verification prove the intended business behavior?
It proves properties relative to the supplied specification and assumptions. The specification still needs review against the intended behavior. [2]
Why does the tool say reported rather than verified?
It compares user-supplied labels and statuses. It does not inspect files, authenticate evidence, execute tests or confirm that two labels identify the same code.
Primary sources
Reviewed13November2026. A publication or export time does not refresh the evidence.
- Anthropic smart-contract exploit research — Historical-contract and controlled evaluation evidence; not a list of attributed liveAI thefts.
- Solidity SMTChecker and formal verification — Specifications, assumptions, unknown outcomes and abstractions. The reviewed latest page is a development document.
- Solidity checked and unchecked arithmetic — Version-specific arithmetic behavior; not evidence of LLM-specific bug rates.
- OpenZeppelin: Preparing for mainnet — Audit limitations and verifying source-to-deployed-code correspondence.
- OpenZeppelin Contracts releases — Tagged/release-channel distinctions. No package was installed or audited here.
- NIST SP800-218 SSDF v1.1 — High-level secure development and vulnerability mitigation practices; no compliance or safety certification.
Turn the unresolved question into a research brief
For a scoped review of the public documents behind a claim, explore the DN Research Desk. An enquiry is subject to agreed scope and capacity; it does not certify a deployment, establish an entitlement or commit you to a paid service.
Disclosure: this article promotes DN’s own research service. Commercial enquiries do not purchase a favorable conclusion. No external provider is promoted through an affiliate link here.
Related DN research
About the Author
