A small sample can tell you where to look next without pretending to be the whole story

A team opens a large file, reviews 25 records, and immediately finds oddities: inconsistent dates, unclear categories, duplicate-looking entities, or values that do not match the source document.

That review may be extremely useful. It also may not support a claim such as, “12 percent of the file is wrong.”

The difference comes down to the question the sample was asked to answer.

A discovery question asks what kinds of patterns, exceptions, or failure modes may be present. An estimation question asks how common something is across a defined population. Small, deliberately chosen reviews can be excellent for discovery. Estimation needs a suitable sample design, a defined population, and an honest account of uncertainty.

Confusing those jobs creates two problems. Teams sometimes dismiss a small review because it is not statistically representative. Or they give it too much authority because the examples are memorable. A better approach is to treat the review as a triage instrument: a fast way to decide what deserves the next, more targeted piece of work.

Start with the decision, not the row count

There is no universal rule that 20, 30, or 50 records is the right exploratory sample. The useful size depends on the decision, the variation in the file, and the consequence of missing a problem.

Before selecting records, give the review a job. That job might be to learn whether a field has more than one meaning, see whether a proposed matching rule creates false matches, identify segments requiring different handling, or determine which records need source verification.

Consider an invoice file with a field called quantity. A review across several business units may show that quantity means items in one system, cases in another, and hours in a third. The review has not measured the frequency of the issue. It has exposed a definition problem that could make later analysis meaningless.

The next work is clearer: locate unit-of-measure fields, map the source-system conventions, and check where quantity appears without enough context to interpret it.

Choose for coverage when exploring

A convenience sample of the first rows in a spreadsheet often tells you more about file order than about the data. For exploratory work, choose records that expose meaningful variation.

That may mean including large and small accounts, different regions, source systems, product lines, and time periods. Include records with and without key fields. If a value looks unusual, include some of those cases too.

This is diagnostic selection, not a census in miniature. Its purpose is to find different ways the data can behave.

A revenue-operations team, for example, might inspect 24 company records before setting deduplication rules. It finds that the website field contains corporate domains, location-specific pages, marketplace profiles, and a handful of social-media URLs. The right conclusion is not that some measured share of websites is flawed.

The right conclusion is that a single website-matching rule may confuse unlike things. A sensible follow-up could profile the full field by URL pattern, separate domains from page URLs, and trace unusual cases back to their sources before deciding what should count as a match.

Keep a review log that turns examples into tests

A compact review log helps prevent anecdotes from becoming conclusions. Four columns are often enough:

The “possible explanation” column matters. It makes clear that an observation is not proof of cause. The “next test” column gives the review practical value.

Many basic reliability checks can begin this way. Missing values, repeated identifiers, duplicates, and invalid formats are all worth noticing. But a full-file rule check and a source-accuracy check are different tasks. A field can satisfy every formatting rule and still be wrong, as when a high-dollar value is entered with an extra zero.

That kind of finding may justify tracing material records to source documents. It does not, by itself, establish an overall error rate.

Match the follow-up to what you found

The best next step should be proportionate to the risk and the question.

A formatting inconsistency may call for a full-file profile. A suspected difference between source systems may call for stratified checks, so each system gets examined separately. A potentially material value error may require source verification. A decision that depends on prevalence requires a probability-based sampling and estimation exercise designed for the population at issue.

That last distinction matters because a percentage from a small, deliberately varied review is not a population estimate. Even a properly drawn sample estimate carries sampling error. It also does not eliminate other problems, such as incomplete coverage, ambiguous measurement, coding mistakes, or processing errors.

Use language that preserves what the review actually showed. Say, “In this review, we found three date conventions.” Say, “This suggests testing all records from Source B.” Avoid turning a discovery into a claim that Source B is unreliable or that a fixed percentage of the file is defective.

Know when the first pass has done its job

Exploratory review can stop when newly examined records are mostly repeating known patterns and the team has a concrete, prioritized next test. That is a practical stopping point, not proof that no unseen issue remains.

Continue or redesign the review when important segments have not been seen, core definitions remain unclear, or the cost of an error is high.

A small sample earns its place by reducing uncertainty about what to do next. It need not settle the whole story to make the next investigation sharper, cheaper, and more defensible.