Sampling & extrapolation

Scanning every object in a petabyte-scale bucket on every run would be slow and expensive, and it is not necessary. Argus scans a carefully chosen sample of each datastore and extrapolates to the full population using standard statistics - so you get an exposure estimate you can defend to an auditor without paying to read every file.

Two things make this trustworthy rather than hand-wavy: Argus scans the files most likely to matter first, and it reports its estimates with an explicit confidence interval, not a bare number.

Choosing what to scan

Argus does not sample blindly. Before it reads anything, it ranks the objects in a datastore by how likely each is to contain sensitive data, using signals like:

• File type - structured data (CSV, JSON, spreadsheets, Parquet) and documents (PDF, DOCX) rank above source code or config.
• Name and path - names suggesting exports, backups, customers, or employees rank up; names suggesting tests, samples, or temp files rank down.
• Size and age - files in a realistic size range for real datasets, and recently modified files, get a nudge up.
• Learned likelihood - a machine-learning ranker, trained on what previous scans actually found, refines the ordering over time; when it is confident about a file, its judgment leads.

The highest-ranked files are the ones that get scanned, so the sample is biased toward finding real exposure rather than being a blind random draw.

Fair coverage

Ranking alone could leave whole corners of a datastore unscanned, so Argus balances it with fair coverage. Sample sizes are set adaptively per datastore: small datastores are scanned in full, while large ones are sampled at a fraction that scales sensibly with size, with per-datastore caps that keep any single scan bounded in time and cost. A tenant-wide scan intensity setting (low, medium, or high) shifts that balance toward speed or thoroughness.

Reading structured vs unstructured data

How Argus samples within a file depends on its shape:

• Structured data (CSV, JSON, XML) is sampled record-aware: Argus keeps the header and draws records from across the file - beginning, middle, and end - rather than just the first rows, so a sensitive column buried deep in the file is still caught.
• Unstructured data (PDF, DOCX, TXT) is sampled by pulling content from across the document rather than only the top.

Extrapolating to the whole

Once the sample is classified, Argus estimates what the full datastore contains using a Wilson score confidence interval with a finite-population correction - a well-established statistical method for estimating a proportion from a sample. Instead of a single guess, every estimate comes as a range: a point estimate plus a lower and upper bound at a stated confidence level (95% by default), and a margin of error.

That is what lets Argus say something honest like “an estimated 12,000 records contain PII, and we are 95% confident the true number is between 10,400 and 13,600,” rather than a false- precision single figure. How that confidence is computed and when a sample is too thin to trust is covered in Confidence scoring.

Incremental scans

A full scan re-ranks and re-samples everything. An incremental scan (available on a manual run) is cheaper: it focuses on objects that are new or have changed since the last scan, re-verifies a portion of what was previously found sensitive, and spends the remaining budget catching up on things it has not looked at before. This keeps routine re-scans fast while still surfacing drift.