Choosing a Vulnerability Scanner: What to Look For

The right vulnerability scanner sees all of your assets, reports accurately enough that your teams trust it, helps you decide what to fix first and fits into the tools people already use, at a price model that doesn't penalize growth. On paper, most products tick the same boxes. The real differences show up when you point them at your own environment. This guide covers the criteria we'd weigh when choosing a scanner and how to run a proof of concept that tells you something useful.

Start with your environment, not the product

Before you read a single datasheet, write down what you need to scan and why. That list becomes your requirements and, later, your scoring sheet.

Work through these questions:

• What do you have? Servers, network devices, laptops, cloud accounts, containers, web applications and APIs, and roughly how many of each.
• Where is it? On-premises, remote, in one or more cloud providers, or in segmented networks that a central scanner can't reach.
• How fast does it change? Auto-scaling groups and short-lived containers need a different approach from a stable server estate.
• What drives the program? If you need quarterly external scans for PCI DSS, they must be performed by an Approved Scanning Vendor (ASV). Check whether that's included or will be a separate service.
• Who will run it? A tool that needs a full-time administrator won't suit a two-person security team.

Sort the results into must-have, should-have and nice-to-have. That keeps a polished demo of a feature you'll never use from swaying the decision.

Asset coverage: what can it scan?

Coverage is the first filter. A scanner that misses a category of assets leaves a blind spot you'll have to fill with another tool.

Discovery matters as much as assessment. Ask how the scanner finds assets you didn't know about, and whether it can compare what it discovers with your asset inventory.

Credentialed scanning and agent options

Accurate results depend on the scanner seeing what is actually installed. There are three main ways to get that view:

• Credentialed network scans log in to each host remotely and read the package database, registry and configuration.
• Agents run on the host and report back, which suits laptops and systems that are often off-network.
• Agentless cloud scanning reads workload data through the cloud provider's APIs, often by analyzing disk snapshots, without logging in to each instance.

Most organizations end up using a mix. When comparing options, ask:

• How are scan credentials stored, and can they come from your existing secrets vault?
• What is the minimum privilege the scanner needs on each platform?
• Does the scanner clearly report hosts where authentication failed?
• What resources does the agent consume, how is it updated and what happens when it's offline?
• How are agent and network results for the same host merged, so you don't get duplicates?

Prioritization data: KEV, EPSS and context

Every scanner finds more vulnerabilities than you can fix at once. The useful ones help you decide what to fix first. CVSS severity alone doesn't do this well, because a large share of published vulnerabilities score high or critical.

Look for built-in support for these public data sources:

• CISA's Known Exploited Vulnerabilities (KEV) catalog. A list of vulnerabilities with evidence of exploitation in the wild, maintained since 2021. US federal civilian agencies must fix listed entries by set due dates under BOD 22-01, and the list is a strong signal for everyone.
• EPSS. The Exploit Prediction Scoring System, maintained by FIRST, estimates the probability that a published CVE will be exploited in the wild in the next 30 days. Scores are published daily.
• CVSS v4.0. FIRST published CVSS v4.0 in November 2023. Check whether the scanner shows v4.0 scores where they exist, alongside v3.1.

Public data is only half of prioritization. The other half is your context: asset criticality, internet exposure, data sensitivity and ownership. A good scanner lets you tag assets and factor those tags into ranking.

Many products also add a proprietary risk score. That can be useful, but ask how it's calculated. If the vendor can't explain why one finding outranks another, your engineers won't trust it either. Make sure you can filter and sort by KEV status and EPSS directly.

Finally, ask where the detection content comes from and how quickly new checks ship after a vulnerability is disclosed. Public enrichment data can lag, so a vendor that relies on it alone will be slower to detect new issues.

Accuracy: false positives and false negatives

Accuracy decides whether your remediation teams take the scanner seriously, and you can't judge it from a datasheet.

On false positives, check how the scanner handles patches that Linux distributions backport into older package versions. This is a common source of bogus findings. Look at whether each finding shows its evidence, such as the version string, file path or package it relied on, and whether it distinguishes confirmed findings from inferred ones.

False negatives are harder to measure because you're looking for what's missing. The best approach is to scan systems whose state you already know, such as an isolated test machine left deliberately unpatched. Compare what each scanner reports against what you know is there.

Integrations: ticketing, CMDB and SIEM

A scanner that doesn't connect to your other tools becomes a report generator that someone has to copy from by hand.

• Ticketing. Look for two-way sync, grouping by remediation (one ticket per patch or fix, not one per CVE per host), routing to the right owner and automatic closure once a rescan verifies the fix.
• CMDB and asset inventory. The scanner should pull in ownership and criticality, and push back assets it discovers that your inventory doesn't know about.
• SIEM. Sending findings and asset data to your SIEM lets analysts see whether a host involved in an alert has known vulnerabilities.
• API. Check that the API is documented, covers everything the interface does and allows bulk export of raw data.
• Access control. Look for single sign-on, role-based access and an audit log of the scanner's own activity. The scanner holds credentials for much of your environment, which makes it a high-value system to protect.
• CI/CD. If you build software or containers, check whether scans can run in your pipelines and fail builds on policy.

Reporting and dashboards

Different audiences need different views. Engineers need fix lists grouped by remediation action. Managers need SLA compliance, finding age and trends. Auditors need evidence of scan coverage and frequency.

Check whether reports can be scheduled, customized and exported, and whether the scanner reports on its own coverage. Two numbers are worth tracking from day one: the percentage of known assets scanned in the last 30 days, and the percentage of scans where authentication succeeded.

How is vulnerability scanning priced?

Pricing models vary widely. Make sure you understand how your environment will be counted before comparing quotes.

Compare the total cost over three years rather than the first-year price. Include infrastructure for scan engines, administration time and the triage effort that inaccurate results create. Ask about renewal price caps before you sign.

How do you run a proof of concept?

A proof of concept (POC) against your own environment is the most reliable way to choose. Keep it structured so the results are comparable.

1. Shortlist two or three products. More than that stretches your team too thin to test any of them properly.
2. Set success criteria and weights first. Build the scoring sheet before you see any results, so the most impressive demo doesn't set the rules.
3. Pick a representative sample. Include a subnet with mixed operating systems, a cloud account, a container registry, one web application and a few remote laptops. Add some systems whose vulnerabilities you already know, to serve as ground truth.
4. Keep conditions equal. Give each scanner the same scope, the same credentials and similar scan windows. Get change approval, and tell your operations and monitoring teams when scans will run.
5. Measure the results. Compare discovered assets with your inventory, record the authentication success rate, check findings against your ground-truth systems and manually validate a sample of 20 to 30 findings from each product. Note scan duration and any impact on systems or the network.
6. Test the workflow, not just the scan. Push findings into your ticketing system, fix something, rescan and check that the ticket closes. Build the report your manager will actually ask for.
7. Involve the people who'll fix things. IT operations should review the findings and the ticket format. Their view on noise and usability is often decisive.
8. Test support. Raise at least one real support request during the POC and note how quickly and how well it's answered.

Vendors often offer to run the POC for you. Accept their help with setup, but have your own team drive it, and don't rely on a vendor-prepared demo environment.

Frequently asked questions

Do we need more than one scanner?

Sometimes. Many organizations use one platform for infrastructure and a separate tool for web applications or containers. Keep the number small and bring all findings into a single place for tracking.

Should we choose agent-based or network-based scanning?

Usually both. Agents suit laptops and cloud workloads, while network scanning covers devices that can't run an agent and shows what's exposed from the outside.

Is an open-source scanner good enough?

For a small, stable environment, it can be. Budget for the staff time to maintain it, keep content current and build the reporting and ticketing integration yourself.

How long should a proof of concept last?

Two to four weeks is usually enough to run several scan cycles, test the workflow and validate a sample of findings.

Key takeaways

• Define your assets, constraints and must-haves before looking at products.
• Prioritize coverage, credentialed or agent-based visibility and accuracy over feature counts.
• Expect built-in KEV and EPSS data, plus a way to add your own asset context.
• Check integrations with ticketing, CMDB and SIEM, and test the full fix-and-verify workflow.
• Understand the pricing model and three-year cost for your environment, including triage time.
• Run a structured proof of concept against your own systems, with ground truth and a scoring sheet agreed in advance.