Detection benchmark
What it found, what it ignored, and what it refused to close.
Every number here is read from a recorded run against a testbed of twenty machines whose faults we set ourselves. Nothing is rounded, and nothing on this page is a screenshot.
Run of 2026-11-01 — 58 tests, 70m 23s. Later runs on the same machine produced the same report byte for byte: the measurement is repeatable, which is not the same as independently verified.
68 of 68
Planted faults raised. Every weakness placed in the testbed came back as a finding, and every one was closed by a rescan after repair.
0 of 29
Decoy checks raised. Twenty-nine checks had to stay silent on seven clean or look-alike machines. All of them did.
3 of 3
False-fix traps held. The flaw stayed and the machine hid it. The finding stayed open.
False-fix traps
A finding that goes quiet has not been fixed.
These three targets keep their flaw and hide it. Two go quiet and one stops linking to it, the shape a real estate takes when something is pulled from navigation or left to 503 under load. A re-check that finds nothing and closes the ticket is wrong, and it is wrong in the direction that gets someone breached.
nuclei.match:git-config at web-down
First look
/.git/config is served to anyone who asks. Raised as a finding.
Re-check
The web server stops answering altogether. The file was never removed.
nuclei.match:git-config at web-503
First look
The same exposed /.git/config. Raised.
Re-check
Every path now answers 503. A scanner reading status codes sees a site that is simply down.
nuclei.match:exposed-gitmodules at web-default
First look
/app/.gitmodules is served on a machine with everything wrong, on the ports every scan looks at. Raised.
Re-check
The page stops linking the directory, so a crawl no longer finds it. The file is still served.
Decoys
Twenty-nine checks that had to stay silent.
False alarms cost a team its attention. Each row is a machine, some built to look wrong and some plainly clean, and each tag is a check that ran against it and stayed silent.
| Target | What it is, and the checks that stayed silent | Silent |
|---|---|---|
web-clean | All six security headers, a CSRF cookie, CORS set to *, and a 200 for every path, so a path probe gets a 200 whether or not the path exists. | 6 |
mail-clean | SPF ending ~all across two strings, DMARC at p=reject, a Google site-verification token, and CAA naming one authority, signed with DNSSEC and carrying a 2048-bit DKIM key. | 6 |
web-keys | A Google browser key, a presigned S3 link, and AWS's own documentation key — in the page and in its script. All three are public by design. PHP and jQuery on versions still supported. | 5 |
tls-clean | A CA-signed certificate, long-lived, issued for this exact name. | 4 |
web-clean-alt | The same clean site on port 8000, a session cookie set properly, and CORS echoing the origin without credentials. | 4 |
ai-locked | A model server, a workflow editor, a Gradio app and a JupyterLab on their usual ports — each asking for credentials the way that stack really asks. The AI checks stay silent here; the port check still reports the open ports, counted in the note below. | 3 |
web-redirect | Port 80 exists and only ever redirects to HTTPS. That is the fix, not the fault. | 1 |
Not counted in any number above: each run also raises 19 findings nothing was planted for, the same 19 on the weak and the fixed scans. Fourteen are open ports and five are plain HTTP with no redirect. Eight of them are on decoy machines: five open ports on ai-locked, one on web-clean-alt, and one plain-HTTP finding each on web-clean and web-keys. They have not yet been judged correct or false. So 0 of 29 means the named decoy checks stayed silent; it does not mean the run had no false positives. A new finding outside this list fails the benchmark until it is planted or treated as a false positive.
The baseline
Sixty-eight faults, by family.
Each is a specific weakness on a specific machine and port, with its own expected finding. The accepted baseline is a file in the scanner repository, so a run cannot quietly get worse: a regression against it fails the benchmark test.
headers18web18nuclei11ai5email5dns4tls4ports3What this does not prove
- This is our own testbed. It measures whether the scanner does what it was built to do against faults we chose. It says nothing about recall on an estate we have never seen, and we will not claim a real-world number from it.
- The transparency log is not measured here. A verified fix can be signed and its record published to a public log, so an auditor can check it without trusting us — that is what verifying a certificate describes. The end-to-end test that writes to the log is opt-in and has not run in any recorded benchmark, so that path is not counted here.
- These runs were rehearsals on a developer machine. Not independently hosted CI. The commands, the pinned scanner images and the recorded reports are the evidence; an outside build system vouching for them is not yet part of it.
- ProvenClosed changes nothing on your infrastructure. It reads, it re-checks, and it hands the work to your tracker. AWS is read through a role assumed over OIDC, so there are no stored access keys.
The three numbers at the top are the ones worth arguing with. If any of them is measuring the wrong thing, that is the feedback we want — hello@provenclosed.com.