How to Choose a Sample Size for a Business Data Review

“Sample 385 records” is common advice, and often wrong. That figure fits one narrow planning problem: estimating an unknown percentage in a large population at 95% confidence with a margin of error of plus or minus 5 percentage points.

A business review may instead need to find a rare error, compare two workflows, or reach defensible conclusions for several regions. Those jobs require different sample plans. The right question is not, “What is the standard sample size?” It is, “What decision must this review support?”

Start with the decision

Before opening a calculator, write a one-sentence objective. It should identify the population, the unit being sampled, and the decision the result will inform.

For example: “Estimate the share of this quarter’s invoices with a coding error within plus or minus 5 percentage points.” The unit here is an invoice, not a customer or supplier. Sampling invoices can support an invoice-level conclusion; it does not automatically describe the share of customers affected.

Most reviews fall into one of three categories:

• Estimate a rate or average. How many orders ship late? What is the average time to resolve a case?
• Detect an error or exception. Is there a good chance of finding at least one defect if a meaningful problem exists?
• Compare alternatives. Did a revised workflow improve on-time completion enough to justify a wider rollout?

Choose the category first. A formula that suits one can be misleading for another.

When you need an estimate

For a percentage under simple random sampling, a common planning formula is:

n₀ = z² × p(1 − p) / e²

Here, n₀ is the initial number of completed, usable records; z is set by the confidence level; p is the expected percentage; and e is the desired absolute margin of error.

At 95% confidence, z is approximately 1.96. If no credible prior estimate exists, use p = 0.50. This is conservative because it produces the largest required sample for a proportion.

With p = 0.50 and a margin of error of plus or minus 5 points, the result is about 385 completed records. Tighten the target to plus or minus 3 points and the requirement rises to about 1,068. Precision gets expensive quickly because the sample requirement grows with the inverse square of the margin of error.

Those are planning targets, not guarantees. The final uncertainty depends on the achieved sample, the actual rate, the sampling design, and the quality of the underlying records.

Account for the size of the population

The large-population formula can overstate the need when the review covers a small, known list. If the sample is a meaningful share of the population, apply a finite-population correction or consider reviewing every record.

Suppose there are 2,000 active suppliers and the goal is a 95% confidence estimate within plus or minus 5 points. Using the conservative 50% planning assumption, the target falls from about 385 to about 323 completed simple-random records after the finite-population adjustment.

That does not mean 323 is always enough. It assumes every selected supplier is eligible, usable, and selected at random. If only 60% of drawn records are expected to survive eligibility and data-quality checks, draw about 539 records to obtain 323 usable ones. Document the expected usable-record rate rather than applying a generic uplift.

Detection is not estimation

An error-detection review asks a different question: how many records are needed to have a specified chance of finding at least one error?

If errors occur in 1% of records and draws are independent, random, and taken from a large population, the chance of seeing at least one error in n records is:

1 − (1 − q)ⁿ

where q is the error rate. To have at least a 95% chance of finding one error at a true 1% rate requires 299 records.

That result does not estimate the error rate to a stated precision. It only supports a detection goal. For a finite population, related records, clustered operations, or targeted high-risk sampling, use a design-specific calculation.

Comparisons need power, not a generic margin of error

If the purpose is to compare two periods, vendors, processes, or treatments, define the smallest difference that would change the decision. Then plan for enough statistical power to detect that difference.

A comparison plan needs the baseline rate or variability, the minimum meaningful difference, the significance level, desired power, and expected allocation across groups. A one-group margin of error cannot answer whether the review can distinguish a worthwhile improvement from normal variation.

For example, a workflow trial should begin with: “What percentage-point improvement in on-time completion would justify rollout?” That threshold, not a familiar sample-size number, drives the calculation.

Plan segments before drawing records

A total sample can look substantial while being too thin for a decision-critical subgroup. If leadership expects separate results for enterprise, small-business, and public-sector accounts, each group needs adequate precision or power of its own.

Small but important groups may need oversampling. If selection rates differ across groups, retain the selection information and use appropriate weighting for overall estimates. Do not assume a proportionally allocated total sample will meet every segment’s needs.

Put representativeness ahead of arithmetic

A large convenience sample is not made representative by its size. Define the sampling frame: the complete practical list from which records can be selected. Check whether it excludes old accounts, cancelled orders, difficult-to-reach locations, or records with missing fields.

Use a reproducible random-selection rule. Keep track of exclusions, duplicate records, ineligible cases, and missing values. Nonresponse and coverage gaps can bias a result, while a larger sample only reduces random sampling uncertainty.

Clustering also matters. Records drawn from the same store, branch, account, or weekly batch often resemble one another. They provide less independent information than the same number of dispersed records, so a complex design may require a larger sample and a suitable variance method.

Make the plan fit on one page

Before selecting records, document: the decision; population and frame; record-level unit and time period; primary measure; precision, detection probability, or minimum meaningful difference; confidence and power settings; segment requirements; sampling method; expected usable-record rate; adjustments for population size or design; final draw count; and the person approving the plan.

Seek statistical review when the decision involves compliance, financial reporting, safety, legal exposure, experimentation, or weighted, clustered, repeated, or otherwise complex data. For routine reviews, a short written plan will still prevent the most common mistake: choosing a number before deciding what the number must accomplish.