@consenti/scanner

Every cookie profile declares a set of categorized trackers. What a site actually fires in a real browser is a different question — third-party scripts, tag managers, and forgotten pixels have a way of drifting out of sync with whatever's written down. @consenti/scanner is a local, offline CLI that closes that gap: it crawls a site under three consent states — no consent given, reject-all, accept-all — and reports every cookie, request, and script it actually saw fire, cross-referenced against what state it fired in.

Install

@consenti/scannerisn't published to npm yet — it lives in the Consenti monorepo as a workspace package. Clone the repo and build it locally:

git clone https://github.com/santoshe61/consenti.git
cd consenti
npm install
npm run build --workspace=@consenti/scanner
npm run install-browsers --workspace=@consenti/scanner   # one-time: installs Playwright's Chromium

Usage

node apps/scanner/dist/cli.js scan https://example.com --depth 2 --output-dir ./scan-results

Exits non-zero when the report has unclassified trackers or anything firing before consent — usable directly as a CI/CD gate.

Configuration flags

Output layout

Every scan gets a short random id (e.g. 116gltxj) and its own directory, so concurrent or historical scans never collide or overwrite each other's screenshots:

<output-dir>/<scan-id>/
  scan-results.json
  scan-results.html
  screenshots/
    <page>/no-consent.png
    <page>/reject-all.png
    <page>/accept-all.png

The id is embedded in the report itself (ScanReport.id) and shown at the top of both the console output and the HTML report.

Progress output

A scan can take a while — several page loads × three consent states × settle time each. The CLI prints a live spinner with the current activity (crawling, navigating a page, detecting/clicking a CMP banner, capturing signals, …) on an interactive terminal, or plain one-per-line progress otherwise (piped output, CI logs). Pass --verbose to always print every step as its own line — useful when debugging a slow or stuck scan.

What it does

• Crawls same-origin links from the start URL, respecting robots.txt (unless --skip-robots-txt), bounded by --depth/--page-limit.
• Scans each page under three consent states, clicking through any detected CMP banner (Consenti's own, or a generic enterprise/open-source-style banner) via multi-locale accept/reject button-label matching. If no banner is detected at all, the report says so explicitly (cmpDetected: false) rather than guessing.
• Captures cookies, localStorage, sessionStorage, IndexedDB, network requests, script tags, and iframe origins per state.
• Classifies every third-party cookie/request/script against a bundled offline dataset. Unmatched trackers are never guessed into a category — they land in the report's manual-review list instead.
• Behavioral detection — any third-party domain making requests post-load is flagged regardless of what its cookies are named.
• CNAME/first-party-cloaking detection — subdomains that CNAME to a different registrable domain are flagged, catching server-side-tagging setups that make a third-party tracker look same-site.
• Tag manager detection — recognizes GTM/Tealium container script tags.
• Fingerprinting heuristics — flags canvas, audio, and font-enumeration API usage as a separate non-cookie tracking category.

HTML report

Alongside the machine-readable scan-results.json, the CLI writes a self-contained scan-results.html — no fetch, no CDN, no external assets, so the file works standalone once written. Every section is independently scrollable and also exposes its underlying data as raw JSON for programmatic use:

• Summary — totals plus the site-wide list of trackers flagged for manual review.
• Pages — one collapsible card per scanned page: findings, cloaking, fingerprinting, tag managers, screenshots.
• Crawl metadata — pages scanned, pages skipped by robots.txt.
• Suggested setup — a consent template, UI template, and profile generated from the site's actual discovered trackers (see below).
• Frontend-only profile — the same suggestion as a self-contained ConsentiProfile({...}) snippet, ready to paste into a site running @consenti/ui with no backend at all.
• Full report (raw JSON) — the complete ScanReport, identical to the sibling .json file.

Suggested setup: from raw findings to a Consenti profile

A scan's findings are grouped by purpose (necessary/functional/preferences/analytics/ marketing) into one of Consenti's 8 built-in compliance groups, chosen heuristically from what was found — opt-in (GDPR/ePrivacy-style) whenever any non-essential tracker turns up, since it's the safe universal default: it satisfies opt-in jurisdictions outright and is a strict superset of what opt-out jurisdictions require. Anything that couldn't be confidently classified is excluded from the suggestion and called out for manual review — consistent with the scanner never auto-assigning necessaryor writing to a live profile on its own. Treat the output as a starting point for the dashboard's consent-template/UI-template/profile authoring flow, not something to import blindly.

Bypassing bot-averse CMPs, and checking GPC behavior

Some CMPs alter what they show based on trivial automation signals — navigator.webdriver, or a User-Agentcontaining "HeadlessChrome". --skip-user-agent-checksneutralizes both, so a scan reflects what an ordinary visitor sees rather than what a detected bot sees. It's off by default — a deliberate, named action to bypass a site's own check, not something applied silently on every scan — and only affects requests made after the page has already loaded (the very first navigation request necessarily goes out before this tool has a chance to correct anything), which covers the common case since most CMPs run their bot-check from an init script after load, not before.

--enable-gpc simulates a visitor with Global Privacy Control turned on — sets navigator.globalPrivacyControl = true and sends Sec-GPC: 1on every request. Run a scan with and without it to see whether a site's tracking behavior actually changes in response: several US state privacy laws require honoring GPC as an opt-out-of-sale/sharing signal, and a site that doesn't is worth flagging regardless of what its cookie banner claims.

Known limitations

Extending the classification database

Every domain/cookie-name → vendor → category mapping the scanner matches against lives in apps/scanner/source-data/ in the repo — a single, open, community-maintained dataset (vendor name, category, confidence, purpose description, retention, TCF vendor id, source citation) designed to grow toward the depth of what large commercial CMPs ship. It stays local and free for anyone self-hosting Consenti; see the README.md in that directory for the schema and how to add a vendor.

Programmatic use

import { runScan, writeReport, writeHtmlReport, hasBlockingFindings, DEFAULT_SCAN_OPTIONS } from '@consenti/scanner'

const report = await runScan(
  'https://example.com',
  { ...DEFAULT_SCAN_OPTIONS, depth: 2 },
  message => console.log(message) // optional — progress updates as the scan runs
)

console.log(report.id) // e.g. '116gltxj' — also the name of its output directory
await writeReport(report, './scan-results')      // → ./scan-results/<report.id>/scan-results.json
await writeHtmlReport(report, './scan-results')  // → ./scan-results/<report.id>/scan-results.html

if (hasBlockingFindings(report)) process.exitCode = 1