The Accuracy Gate: How Pentestas Filters 90% of False Positives Before You See Them
The Accuracy Gate: How Pentestas Filters 90% of False Positives Before You See Them
Every security team has spent a Monday triaging Sunday's scan. The CSV has 200 rows. By coffee, the count of "this is a real bug" is single digits. The rest are stack-trace pattern matches, false reflections, or findings that the scanner flagged without ever verifying exploitability.
The Accuracy Gate is how Pentestas refuses to ship that CSV. Every finding goes through three filters before it's persisted — and only findings with demonstrated impact make the final report.
What the Accuracy Gate is
Think of the Accuracy Gate as an airlock between the raw tool output and the customer-visible findings list. Every signal a scanner module produces has to pass through three stages:
Stage 1 — Junk filter. Pattern-based rejection of signals that clearly aren't exploitable. An SQL injection payload that "extracted" the string style.css is extracting a CSS filename from your HTML — drop. An XSS payload reflected inside a JSON error response with full backslash-escaping is not executable — drop. An SSRF probe that "saw" a redirect on the outbound request but the target was the same origin as the caller — drop.
Stage 2 — Orthogonal verifier. Every vulnerability class has a second-opinion verifier that uses a different signal than the original detector. Examples:
- SQLi detector sees a SQL error text. SQLi verifier fires a time-based blind payload (
' AND SLEEP(5)--) and measures response-time delta. If the delta isn't present, the error was a verbose debug page, not an injection. - XSS detector sees payload reflected in the response. XSS verifier renders the page in a headless Chromium with a JavaScript-execution hook. If no script fires, the payload was inserted but never executed.
- SSRF detector sees the target make an outbound request. SSRF verifier uses an out-of-band DNS callback oracle; if the DNS hit arrives at the attacker-controlled name, confirmed. If not, dropped.
- IDOR detector sees a 200 on a different user's ID. IDOR verifier compares response bodies; identical response = cache artifact, not IDOR.
- Auth bypass detector sees a 200 after a supposedly-authenticated request missing a token. Verifier additionally checks that the response body is role-appropriate content, not a generic "welcome" page.
The verifier runs in a fresh HTTP session — no cookies from the scanner's crawl, no lingering state, no CSRF token auto-inclusion. The call is hermetic; if it reproduces under those conditions, the signal is robust.
Stage 3 — AI false-positive filter. On Pro+ plans, Claude reads each verified finding + full evidence and judges exploitability in context. Can mark as false-positive (drop), keep as-is (standard), or bump/lower severity. Typical outputs the AI filter catches:
- CORS wildcard on a marketing site with no authenticated surface — "yes,
Access-Control-Allow-Origin: *, but the site has no auth cookies. No attacker value. Drop." - XSS reflection in a 404 handler — "yes, reflected; but the response is text/plain; the browser renders it as plain text; not executable. Drop."
- Stored XSS in a dev-only endpoint — "exploitable, but only internal users with admin role can hit the endpoint. Keep, but lower severity from CRITICAL to MEDIUM."
This filter does NOT drop CRITICAL findings without human review. CRITICAL findings may have severity adjusted, but they're never auto-dropped. Only the scan detail page shows filtered findings (strikethrough with AI rationale) — they're available for review if your team disagrees with the filter.
What "verified" means
Every finding Pentestas persists carries a boolean verified flag:
verified: true— the second-pass verifier re-confirmed the finding independently. Highest confidence.verified: false— the detector fired, but the verifier wasn't applicable or didn't confirm. The signal is real enough to investigate, but lacks the higher-confidence badge.
Filter ?verified=true in the findings list for the highest-signal subset. Every finding in the subset has, by construction, reproduced under two different oracles.
Post-scan re-verification
Findings don't just get verified once. Every verified:true HIGH or CRITICAL finding is additionally re-fired against the live target on a cadence:
- Daily on Enterprise
- Weekly on Pro
If the previously-triggering payload no longer produces the oracle signal, the finding is marked resolved with a timestamp. Your audit trail becomes "found on date X, resolved on date Y" automatically — no manual closure, no drift.
If the payload still triggers, the finding is bumped back to active and a webhook fires. Regression after a fix is caught within 24 hours instead of at the next manual pentest.
Why this matters more than "we have AI analysis"
Many vendors advertise "AI-powered analysis". The differentiation is in what the AI does. There are broadly two patterns:
Pattern A — AI dresses up the report. Tool fires every payload, collects signals, Claude writes a narrative for each one, team reviews narrative and decides which are real. This is how most "AI-enhanced" scanners work. The triage burden hasn't moved; you now have AI-written narratives for findings that still include a lot of false positives.
Pattern B — AI is the filter. Tool fires every payload, collects signals, signals go through the Accuracy Gate before being persisted. By the time a finding is in the list, it has been triple-filtered. The AI narrative on the finding matters less than the AI filter that prevented the noise from reaching you.
Pentestas is Pattern B. The observable difference for a customer: a 30-endpoint SaaS scan that a legacy scanner reports as "78 findings, 60% false-positive" is a Pentestas scan reporting "14 findings, 100% reproducible, 8 of them inside attack chains". The Monday morning triage is over by 9:30.
The economics
Engineer time spent triaging false positives is the hidden cost of the typical security-scanner purchase. At a $200K/year loaded cost of an AppSec engineer, every 30 minutes a week spent disproving false positives is ~$2,500 a year. At a typical noise ratio of 70% false positives against 40 scan findings/week, that's closer to 5 hours a week = $26K/year in engineer time for a single scanner's noise.
Pentestas's Accuracy Gate removes 85–95% of that noise. The freed-up time becomes — in theory — time spent on proactive security work. In practice it often becomes time spent enabling more scans (more targets, higher cadence) which improves coverage without raising triage burden. The org ends up with more coverage and less friction simultaneously.
Industry application
Fintech
Payment apps generate the highest false-positive rates on typical scanners — tokenised card numbers pattern-match as PII, idempotency keys pattern-match as SQL error fragments, every transaction-ID UUID looks like a credential to a heuristic matcher. The Accuracy Gate reduces this torrent to the findings that genuinely matter: auth bypasses on payment endpoints, IDORs on transaction-lookup, mass-assignment on account-update. The Monday-morning triage is unblocked.
Medtech
HIPAA programmes often mandate zero-tolerance on false positives — every flagged finding gets investigated, which makes high-FP tools prohibitively expensive to operate. Pentestas ships findings with proof-of-exploit traces that stand up under a compliance officer's review. "Here is the request, here is the response, here is the exploitable behaviour" beats "the scanner flagged it" every time.
Legaltech
Legal platforms often have document-heavy attack surfaces — thousands of custom fields, free-text PDFs, client-specific templates. The FP rate from string-matching scanners against legal text is catastrophic. Pentestas's Accuracy Gate drops every finding that didn't reproduce, which brings the legal-platform scan from "unusable" to "weekly evidence" in practice.
Banks + financial services
Banks run many different scanners against many different environments. The noise compounds. Consolidating onto a single ai pentest that filters pre-persist reduces the noise fan-out at source. A bank running Pentestas against its public-app + internal agents typically produces a weekly findings delta of 5–15 items, most of them ticketable.
Insurance
Insurance platforms have the same pattern as fintech — high PII volume, heavy regulatory scrutiny, low appetite for "maybe bugs". The verified-only filter (?verified=true) lets the insurance AppSec team operate as a "red light only" programme: nothing gets investigated unless the Accuracy Gate confirmed it reproduces end-to-end.
What the Accuracy Gate does NOT do
Honesty matters for a filter this aggressive. Three caveats:
-
It won't catch a bug whose oracle is complex. If a vulnerability requires, say, a specific cookie set over a specific subdomain with a specific timestamp relationship, and the combination isn't in our verifier catalogue, the finding may not get a
verified:trueflag even when real. These cases appear asverified:falsefindings the scanner persists without the high-confidence badge. -
It doesn't catch human-pentester-only findings. Business-logic bugs, bespoke protocol attacks, chained social-engineering exploits — these need a human. Pentestas's pipeline is tuned for OWASP-Top-10-adjacent work; deeper logic bugs remain in the human-pentest budget.
-
It can occasionally over-filter. Every filter has a small rate of false negatives (real bugs dropped). The AI FP filter is the most aggressive layer; if you disagree with the rationale, click Disagree and the finding returns to the active list. Pentestas audit-logs the override.
See it
Run any Pentestas scan and click Show AI-filtered in the findings list. The strikethrough rows are the ones the Accuracy Gate (specifically the AI FP layer) dropped — with each one's rationale alongside. Your team becomes confident that drops are correct dropouts, which is the trust that unlocks the whole continuous-pentest programme.
Further reading
- Validation docs — full filter architecture
- AI false-positive filter — the third stage
- AI penetration testing explained — the umbrella methodology