Skip to content
BACK TO BLOG
FILE // BUILD_LOG 10
Security By Ryan Sebastian 2026-07-04 12 min readUpdated 2026-07-04

Penetration Testing vs Vulnerability Scanning: We Ran Both on the Same App

Every definitional post tells you scans and pentests are different. We can show you. We ran 75 scanner modules against our own application and published what they caught, and what they could not.

THE 40-WORD ANSWER

A vulnerability scan is an automated check that finds potential weaknesses and ranks them by severity. A penetration test is a human-led engagement that exploits those weaknesses to prove impact. A scan finds, a pentest proves, mature teams run both.

The Difference in One Table

A scan and a pentest answer two different questions: “what might be wrong here” versus “what can an attacker actually do here.” Everything else, the cost gap, the timeline gap, the compliance treatment, follows from that one distinction.

DimensionVulnerability scanPenetration test
MethodAutomated tools match known patterns, CVE signatures, and misconfigurationsA human engineer, backed by tooling, actively exploits weaknesses
DepthSurface and configuration level; known vulnerability classesBusiness logic, chained attack paths, authorization flaws
Typical costFree to a few hundred dollars per run; our deep scan is $499Roughly $5,000 to $35,000+ per 2026 guides from DeepStrike, Blaze Infosec, Intruder, and Astra; our fixed price is $2,499
TimeMinutes to hoursMarket norm is 4 to 6 weeks from scoping to report; our engagement delivers in 5 to 7 days
OutputSeverity-ranked finding list, false positives includedReport with reproduction steps, business impact, remediation, attestation letter
Exploitability evidenceNone. A scan flags possible weaknessesProven. Each high-severity finding is reproduced step by step

The Demonstration: 75 Scanners Against Our Own App

We did not have to theorize about what scanning finds, because we ran the experiment on ourselves. Our engine, PhantomDragon, orchestrates 75+ scanner modules through a 17-phase pipeline, and we pointed the whole thing at one of our own applications. The full write-up is public: I ran 75 automated scanners against my own app.

That run surfaced findings in six categories (specific vulnerabilities withheld for responsible disclosure):

Information leakage

Medium-High

Verbose error messages, server version headers, exposed .git directories, backup files, debug endpoints left on in production.

Security header gaps

Low-Medium

Missing or misconfigured Content-Security-Policy, X-Frame-Options, HSTS, Permissions-Policy, Referrer-Policy.

SSL/TLS weaknesses

Medium

Deprecated cipher suites, weak DH parameters, incomplete certificate chains.

Hidden attack surface

High-Critical

Forgotten admin panels, accessible staging environments, undocumented API endpoints that still respond.

Injection vectors

High-Critical

SQL injection, XSS, SSRF, and template injection payloads tested across every discovered endpoint and parameter.

Authentication and session issues

High

Weak session token entropy, missing rate limiting on login, insecure cookie flags.

One number from that experiment matters more than any definition: in our testing, the overlap between any two scanners averaged 30 to 40 percent. That means 60 to 70 percent of what each tool finds is unique to that tool, which is the entire argument for scanning breadth over a single-scanner checkbox.

The Honest Limits of That Run

The same write-up is explicit about what 75 scanners could not do. Every finding above was a flag ranked by pattern severity, not a demonstrated breach. Nothing in the output proved that an attacker could chain those flags into stolen data. The run also could not touch:

Business logic flaws: can a user apply a discount code twice, or transfer a negative amount? Scanners do not understand business rules.

Complex authentication bypasses: multi-step auth flows, OAuth misconfigurations, race conditions in session handling.

Chained vulnerabilities: combining a low-severity information leak with a medium SSRF into critical-severity data exfiltration.

False positive filtering at the end of the pipeline still needed human review before any finding could be trusted.

That gap, between flagged and proven, is exactly the space a penetration test exists to fill.

What Only a Human Engineer Catches

Three vulnerability classes are effectively invisible to automation, and they are consistently where the worst breaches live.

Business logic abuse

Price tampering, coupon stacking, negative quantities, skipping a payment step by replaying a request. Your business rules are not in any signature database, so no scanner can test them. An engineer reads your flows and asks: what happens if I do this out of order?

Chained attack paths

Scanners score findings one at a time. An attacker combines them: an information leak reveals an internal hostname, an SSRF reaches it, a weak internal service gives up credentials. Three low-to-medium flags become one critical breach, and only a human connects them.

Authorization nuance

A scanner can compare a 200 to a 403. It cannot know that user A must never see tenant B's invoices. IDOR and broken object-level authorization stay human-found classes because "should this user see this object" is a business rule, not a pattern.

What the Artifact Looks Like

The deliverable is where the difference becomes concrete. A scan hands you a list; a pentest hands you evidence a third party will accept.

SCAN OUTPUT

  • Severity-ranked finding list
  • Header and TLS grades
  • CVE and version matches
  • Machine exports (CSV, JSON)
  • False positives included; triage is on you

Our $499 deep scan packages this as an executive PDF in about 48 hours, plus a CSV export.

PENTEST REPORT

  • Scope statement and OWASP / NIST methodology
  • Step-by-step reproduction for each high finding
  • Severity with business impact, not just pattern scores
  • Remediation guidance and a free re-test after fixes
  • Signed attestation letter for auditors and customers

See the format yourself: the anonymized sample report.

When a Free Scan Genuinely Is Enough

If you have never scanned, hold no sensitive data yet, and have no compliance driver, a free scan is the right first step, not a downsell. Paying for a pentest before you have fixed the basics wastes most of the engineer's time on findings a machine would have caught.

You have not yet fixed the basics: missing security headers, weak TLS, exposed files. Scan, fix, rescan, all free.

You are pre-launch or pre-revenue with no personal data in the system yet.

You want a fast outside-in read on what an agency or contractor actually delivered.

You already pentest annually and just need drift checks between engagements.

Our free Ghost Scan checks your site across 9 security categories in minutes. No card, no signup.

When You Need the Pentest

The moment a third party needs proof (an auditor, an enterprise security team, an investor) a scan stops being enough, because a scan cannot prove exploitability. The common triggers:

SOC 2 audits

SOC 2 does not explicitly require a penetration test, but auditors expect one as monitoring evidence, commonly mapped to CC4.1, and Type I versus Type II timing matters. SOC 2 pentest guide.

ISO 27001:2022

Control A.8.8 (technical vulnerability management) lists periodic penetration testing among vulnerability-identification methods, and A.8.29 covers security testing in development and change. ISO 27001 pentest guide.

Enterprise security reviews

Vendor questionnaires ask for a recent independent test and an attestation letter. A scanner PDF is usually rejected because it carries no exploitability evidence. the fixed-price pentest.

Pre-launch with sensitive data

If you are about to put health, financial, or personal data behind a login, test before real records are at stake, not after. your first pentest, explained.

One honesty note on PCI DSS: our engagement covers only the external application-layer slice of requirement 11.4, meaning 11.4.3 and the 11.4.4 retest loop. It does not cover internal or segmentation testing; if you need those, you need an additional provider.

Why Our Engagement Runs Both

This is not a scan-or-human choice, and our $2,499 fixed-price pentestis built on that premise: PhantomDragon's 75+ scanner modules run first for breadth, then one engineer verifies every finding, kills the false positives, chains the paths, and attacks the business logic. Automated coverage plus human proof, in one engagement, with a free re-test after you patch.

The wider 2026 market has converged on the same shape. XBOW, Horizon3.ai NodeZero, Synack, and PentestGPT all pair automation with human oversight; hybrid human plus AI is the published consensus, not our invention. We compare the approaches in detail in AI penetration testing vs manual. Scope honesty applies here too: we test web applications and APIs only, not mobile apps, internal networks, physical security, or red teaming.

The Cost Ladder

Start free, escalate only when a real driver appears. Our ladder has three rungs:

Free Ghost Scan

$0

Surface scan across 9 categories. No card, no signup. The right first step for almost everyone.

Deep Security Scan

$499

75+ scanner modules, OWASP Top 10 coverage, executive PDF in about 48 hours, CSV export.

Penetration Test

$2,499

One engineer plus the full scanner sweep, executive PDF in 5 to 7 days, free re-test, signed attestation letter.

For market context: 2026 cost guides from DeepStrike, Blaze Infosec, Intruder, and Astra put typical web application pentests at roughly $5,000 to $35,000+, Intruder cites day rates of $1,000 to $3,000, Fractional CISO prices compliance-grade tests around $5,000 with quality engagements at $15,000 to $30,000, and Astra publishes subscription pentest pricing at $1,999/yr (Basic) and $5,999/yr (Plus) per target. Full breakdown with sources in our penetration testing cost guide; ongoing coverage lives on the pricing page. Buying from Sri Lanka? See the VAPT Sri Lanka guide.

Frequently Asked Questions

Is a vulnerability scan the same as a penetration test?

No. A vulnerability scan is automated pattern matching that lists possible weaknesses. A penetration test adds a human engineer who verifies and exploits findings to prove impact, tests business logic, and documents reproduction steps. Compliance frameworks treat the two as different controls.

Will a vulnerability scan pass a SOC 2 audit?

Usually not on its own. SOC 2 does not explicitly require a penetration test, but auditors expect one as monitoring evidence, commonly mapped to CC4.1. Most B2B SaaS companies submit an annual pentest report plus continuous scanning.

How often should I scan versus pentest?

Scan continuously or at least monthly, because dependencies and attack surface change constantly. Pentest at least annually and after major changes. Sri Lanka CERT|CC, for example, recommends periodic security assessments at least once a year.

How much does each cost?

Scans run from free to a few hundred dollars; our Ghost Scan is free and our deep scan is $499. For pentests, 2026 cost guides from DeepStrike, Blaze Infosec, Intruder, and Astra put typical web application engagements at roughly $5,000 to $35,000 or more. Our fixed-price web and API pentest is $2,499.

Can AI scanners replace a human pentester?

Not in 2026. Platforms like XBOW, Horizon3.ai NodeZero, Synack, and PentestGPT all pair automation with human oversight, and hybrid human plus AI is the published consensus. Automation wins on breadth and speed; humans win on business logic, chained attacks, and authorization flaws.

Find out what your app is leaking, free

The Ghost Scan checks your site across 9 security categories in minutes. No card, no signup, no sales call. If it turns up something worth proving, the fixed-price pentest is the next step.

METHODOLOGY NOTE: The scanner findings referenced above come from our published 75-scanner run against our own application. Market prices are attributed inline to their 2026 published sources (DeepStrike, Blaze Infosec, Intruder, Astra, Fractional CISO). We sell the $2,499 pentest described here; read with that in mind.