Building a Python Vulnerability Scanner with NVD API Integration

Manual vulnerability tracking doesn’t scale. I built a Python scanner that monitors 47 homelab services using the National Vulnerability Database API. It detects vulnerabilities in installed packages, sends alerts for critical CVEs, and integrates with existing Prometheus monitoring.

Here’s how to automate vulnerability management for your homelab.

found you

One thing to know before building this. In February 2024 the NVD slowed and then largely stopped enriching new CVEs with CPE data — for months, the large majority of new entries went unanalyzed, and only a fraction of CVEs published since have a CPE at all. This scanner matches installed package versions against CPEs. Without them it doesn’t throw an error; it quietly returns nothing, which is the worst way for a security tool to fail.

If you build something like this now, treat OSV and your distro’s security tracker as primary sources rather than the “future improvement” I list at the end, and alert on scan coverage as well as on findings.

The Manual Vulnerability Problem

Security advisories arrive faster than humans can track them. CVE-2024-XXXXX published. Check your systems. Repeat 50-100 times per month. By October 2024, the NVD recorded over 240,000 CVEs, growing about 39% year over year in 2024, largely because the Linux kernel became a CNA that February.

Problems with manual scanning:

• Advisory fatigue: 50+ notifications per week across projects
• Version mismatch: “Does Docker 24.0.5 include CVE-2024-1234 fix?” requires GitHub archaeology
• Delayed response: Critical CVE published Monday, discovered Friday
• Coverage gaps: Missed Python package updates, forgotten services

What I needed: Automated scanner that checks installed versions against NVD daily, filters noise (Low/Medium severity), and integrates with alerting infrastructure.

NVD API 2.0: Structured Vulnerability Data

The National Vulnerability Database provides free API access to CVE details. API 2.0 (launched 2022) replaced rate-limited 1.0 with better search and filtering.

Key API capabilities:

• CVE search: Query by CPE (Common Platform Enumeration), keyword, or date range
• Scoring data: CVSS v2/v3 metrics, severity ratings (Low/Medium/High/Critical)
• Version ranges: Affected version start/end for precise matching
• Update frequency: CVEs added within 24 hours of publication

API access:

import requests

# Public API (rate limited: 5 requests / 30 seconds)
url = "https://services.nvd.nist.gov/rest/json/cves/2.0"
headers = {"Accept": "application/json"}
params = {"keywordSearch": "docker", "resultsPerPage": 20}

response = requests.get(url, headers=headers, params=params)
cves = response.json()

API key benefits: Request rate increases to 50 requests / 30 seconds with a free API key from NVD.

Python Scanner Architecture

My scanner uses three components: package inventory, NVD query engine, and alert dispatcher.

System design:

How it works:

1. Inventory collection: Scan homelab hosts via SSH, collect dpkg -l (Debian) or rpm -qa (Red Hat) output
2. NVD query: For each package, query NVD for CVEs matching package name
3. Version matching: Parse affected version ranges, check if installed version vulnerable
4. Severity filtering: Drop Low/Medium unless specific packages (OpenSSH, sudo, kernel)
5. Alert dispatch: Critical/High vulnerabilities → Slack notification + Prometheus metric

Scanner implementation: https://gist.github.com/williamzujkowski/9ea76a7c2d5e0b40d45f65a81774992e

Homelab Deployment: Scanning 47 Services

I deployed the scanner as a cron job on my homelab management server (Ubuntu 24.04, 8GB RAM). It runs daily at 06:00 UTC, scans all hosts, and reports findings.

Infrastructure scanned:

• Docker containers: 23 services (Nginx, PostgreSQL, Redis, Traefik, etc.)
• Virtual machines: 12 Ubuntu/Debian VMs
• Physical hosts: 8 servers (Proxmox, K3s nodes, NAS)
• Network devices: 4 items (router firmware, Pi-hole, Unifi controller)

Deployment script:

# Install scanner dependencies
pip install requests nvdlib

# Create inventory file
cat > /etc/vuln-scanner/hosts.txt <<EOF
homelab-server-01
homelab-server-02
docker-host
proxmox-node-01
EOF

# Setup cron job (daily at 06:00 UTC)
echo "0 6 * * * /usr/local/bin/vuln-scanner --config /etc/vuln-scanner/config.yaml" | crontab -

Configuration: https://gist.github.com/williamzujkowski/d56a2e449cdadd843f86c9c5af8fed56

Version Matching: The Hard Part

NVD stores affected version ranges as CPE 2.3 strings. Matching installed versions against these ranges requires parsing CPE format and semantic version comparison.

CPE 2.3 example:

cpe:2.3:a:docker:docker:*:*:*:*:*:*:*:*
Vendor: docker
Product: docker
Version: * (wildcard - all versions)

cpe:2.3:a:docker:docker:24.0.5:*:*:*:*:*:*:*
Version: 24.0.5 (specific version)

Version range challenge: CVE affects Docker 24.0.0 through 24.0.5. Installed version: 24.0.4. Match algorithm:

def version_in_range(installed, vuln_start, vuln_end):
    """Check if installed version falls in vulnerable range."""
    from packaging import version

    installed_ver = version.parse(installed)
    start_ver = version.parse(vuln_start)
    end_ver = version.parse(vuln_end) if vuln_end else None

    if end_ver:
        return start_ver <= installed_ver <= end_ver
    else:
        # No end version = open-ended vulnerability
        return start_ver <= installed_ver

Edge cases handled:

• Wildcard versions: CPE * means all versions vulnerable
• Pre-release tags: Docker 24.0.5-rc1 vs 24.0.5
• Epoch versions: Debian package epochs (1:24.0.5-1)
• Missing version data: Some CVEs lack precise version ranges

Accuracy: of the 77 findings in the run below, 3 were backport false positives — versions where the distro had patched without bumping the upstream version string. That’s a count I can defend. I don’t have a methodology clean enough to quote a rate, and CPE format variations mean the miss rate is unknown rather than small.

Integration: Prometheus Metrics

I exposed vulnerability counts as Prometheus metrics for dashboard visualization and alerting.

Metrics exported:

# HELP vulns_total Total vulnerabilities detected
# TYPE vulns_total gauge
vulns_total{severity="critical"} 2
vulns_total{severity="high"} 7
vulns_total{severity="medium"} 23
vulns_total{severity="low"} 45

# HELP vulns_by_package Vulnerabilities grouped by package
# TYPE vulns_by_package gauge
vulns_by_package{package="docker",severity="high"} 1
vulns_by_package{package="nginx",severity="medium"} 3

Grafana dashboard: https://gist.github.com/williamzujkowski/d56a2e449cdadd843f86c9c5af8fed56

Alerting rule (Prometheus):

groups:
  - name: vulnerability_alerts
    rules:
      - alert: CriticalVulnerabilityDetected
        expr: vulns_total{severity="critical"} > 0
        for: 1h
        annotations:
          summary: "{{ $value }} critical vulnerabilities detected"
          description: "Run vuln-scanner --details for CVE list"

A caveat on the alerting that took me too long to notice. for: 1h means the condition must hold continuously for an hour before the alert fires — so paging happens after an hour, not within it. Worse, a cron scanner exits when it finishes, so unless you push to a Pushgateway the series goes stale within minutes and > 0 has no data to evaluate. The for window can then never be satisfied and the alert never fires at all.

Use for: 0m — a once-daily batch signal has no flapping to debounce — and add a companion rule on absent(vulns_total) so a scanner that silently stops running pages you instead of going quiet. A monitoring rule that cannot fire is worse than no rule, because you believe you are covered.

Scan Results: 77 Vulnerabilities Found

First scan of my homelab (2024-01-30) detected 77 vulnerabilities across 47 services.

Breakdown by severity:

Critical vulnerabilities:

High severity examples:

• PostgreSQL 14.7 → 14.10 (3 CVEs fixed)
• Nginx 1.24.0 → 1.25.3 (2 CVEs fixed)
• Redis 7.0.11 → 7.0.15 (2 CVEs fixed)

False positives: 3 of the 77 findings were backport artifacts — packages the distro had patched without bumping the upstream version string. That is a count, not a rate; as noted above I don’t have a methodology clean enough to quote a rate.

Handling Backported Patches

Debian stable backports security fixes without version bumps. Package shows as vulnerable in NVD but is actually patched.

Example: OpenSSL 3.0.2 on Ubuntu 22.04 carries fixes for CVEs affecting later 3.0.x releases while the version string stays 3.0.2-0ubuntu1.12. (Debian 12 ships a 3.0.x of its own with the same backporting behaviour — the mechanism is the same, the version strings differ per distro.)

Detection workaround:

def check_debian_backports(package, version, cve_id):
    """Check if CVE fixed via Debian Security Tracker."""
    url = f"https://security-tracker.debian.org/tracker/{cve_id}"
    response = requests.get(url)

    if response.status_code == 200:
        # Parse HTML for "fixed in version X"
        if f"{package}/{version}" in response.text:
            return True  # Backport fix applied
    return False

A warning about the obvious implementation. Substring-matching f"{package}/{version}" against the tracker’s HTML page does not work — that form does not appear on those pages, so the check returns False every time and suppresses nothing. And the logic is inverted on its own terms: the tracker lists the fixed version, so a match on the installed version would indicate the vulnerable state, meaning any lucky collision hides a real finding rather than clearing a false one.

Use the machine-readable feed at https://security-tracker.debian.org/tracker/data/json and compare against the fixed version explicitly. And set a timeout= — an unresponsive tracker otherwise hangs the whole scan.

Automated Remediation: Patch Suggestions

Scanner generates patch suggestions based on CVE fix versions.

Example output:

[CRITICAL] CVE-2024-XXXX detected in openssh-server
  Installed version: 9.2p1
  Fixed in version: 9.5p1
  CVSS score: 9.8 (Critical)
  Suggested action: apt-get install --only-upgrade openssh-server

[HIGH] CVE-2024-YYYY detected in docker-ce
  Installed version: 24.0.5
  Fixed in version: 24.0.7
  CVSS score: 7.5 (High)
  Suggested action: apt-get install --only-upgrade docker-ce

Automation safety: Suggestions generated, but updates NOT auto-applied. Homelab stability > speed. Critical CVEs get manual review before patching.

Performance and Resource Usage

Scanner completes full homelab scan in 8.4 minutes.

Performance breakdown:

• Inventory collection: 2.1 minutes (SSH to 47 hosts, run package list commands)
• NVD queries: 5.3 minutes (394 API calls, rate limited to 50/30sec with API key)
• Version matching: 0.7 minutes (local computation)
• Alert dispatch: 0.3 minutes (Slack webhook, Prometheus push)

Resource consumption:

• Memory: 180MB peak (loading full CVE JSON responses)
• Network: 43MB download (NVD API responses)
• Disk: 15MB (cached CVE database, 7-day retention)

On caching. An in-process dict is not a cache for this workload — a cron job starts cold every morning, so it never gets a hit. If you want the speedup you have to persist to SQLite or similar with an explicit TTL, and that is a piece of work rather than a one-line addition. Worth knowing before you budget for it.

Comparison: Commercial vs Open Source Scanners

No false-positive column, deliberately. I have not run Nessus and Qualys across the same hosts under the same conditions, and a comparison table is exactly where an unmeasured number does the most damage — it ranks other people’s products against mine on evidence I do not have.

Why custom Python scanner: Full control over filtering logic, easy integration with existing homelab tools (Prometheus, Slack), no vendor lock-in, learning experience.

When to use Trivy/Grype: Container-focused environments. Both tools excel at scanning Docker images before deployment. I use Trivy for CI/CD pipeline, custom scanner for deployed infrastructure.

Limitations and Future Improvements

Challenge 1: CPE matching accuracy

• Problem: NVD CPE data incomplete. Some packages missing CPE entries entirely.
• Impact: unknown. CPE format variations mean some vulnerabilities are missed and the miss rate is not something this setup can measure
• Future fix: Add OSV (Open Source Vulnerability) database as secondary source

Challenge 2: Network device scanning

• Problem: Router firmware, managed switches don’t report package lists
• Workaround: Manual version tracking in config file, NVD query by firmware version
• Impact: Increased manual maintenance for 4 network devices

Challenge 3: Transitive dependencies

• Problem: Python package depends on vulnerable library, but scanner only checks top-level packages
• Example: Django 4.2.0 depends on vulnerable Pillow 9.5.0, scanner misses Pillow CVE
• Future fix: Parse dependency trees (pip freeze, requirements.txt analysis)

Improvement roadmap:

1. Add EPSS (Exploit Prediction Scoring System) integration for prioritization
2. Implement KEV (Known Exploited Vulnerabilities) catalog checks
3. Expand to container image scanning (layers, base images)
4. Add remediation automation (dry-run mode, approval workflow)

Further Reading

Research and standards:

• National Vulnerability Database (NVD) - NIST’s CVE database
• Common Platform Enumeration (CPE) Specification - CPE 2.3 format documentation

Python libraries and tools:

• nvdlib - NVD API 2.0 wrapper
• CVE Binary Tool - CLI vulnerability scanner
• Trivy - Container and OS vulnerability scanner
• Grype - Vulnerability scanner for containers and filesystems

Related projects:

• EPSS Project - Exploit prediction scoring
• CISA KEV Catalog - Known exploited vulnerabilities
• OSV (Open Source Vulnerability) - Open source vulnerability database

Implementation references:

• Scanner source code: https://gist.github.com/williamzujkowski/9ea76a7c2d5e0b40d45f65a81774992e
• Configuration examples: https://gist.github.com/williamzujkowski/d56a2e449cdadd843f86c9c5af8fed56
• Grafana dashboard: https://gist.github.com/williamzujkowski/d56a2e449cdadd843f86c9c5af8fed56

---

Start tracking vulnerabilities automatically. Deploy the scanner, configure NVD API access, integrate with monitoring. Most homelab security gaps come from delayed patching, not zero-day exploits.

Automated scanning won’t fix vulnerabilities for you. But it will tell you what’s broken before attackers find it. In my homelab, both critical findings were patched within 24 hours of the scan surfacing them, against the weeks they would previously have sat unnoticed.