How to Test Whether a Public Data Query Has Quietly Truncated Its Results

A public-data query can succeed, return plausible rows, and still leave out part of the answer.

That failure is easy to miss. An analyst searches a municipal permits portal, receives exactly 1,000 records, exports a clean spreadsheet, and starts ranking contractors. No error appears. But the same filters, run as a count query, show 1,347 matching permits.

The arithmetic was fine. The retrieval was incomplete.

Quiet truncation means a source returned only part of the matching result set without making that fact obvious to the person using it. Sometimes the limit is documented. Sometimes a next-page link, transfer-limit flag, or page token is present but overlooked. Sometimes a dashboard, export button, and underlying API each have different rules.

A successful response proves that a request succeeded. It does not prove that every matching record arrived.

Start with an exact-scope count

The most useful test is a reconciliation: compare the number of unique records retrieved with an expected count for the *same* query.

“Same” matters. The count and the extraction should use the identical filters, date range, geography, endpoint, record type, and exclusions. A portal’s headline count for an entire dataset cannot validate a filtered extract.

Where the source supports it, obtain either an exact count or an official inventory of matching record IDs before downloading the full records. Then retrieve all available pages and compare the result.

A sound workpaper captures four things:

• the full query specification;
• the retrieval timestamp;
• the expected count or ID inventory;
• downloaded rows, plus the number of unique record IDs.

Also retain source metadata: page tokens, next-page links, transfer-limit indicators, stated page sizes, and any warning that a total is estimated.

Count unique records, not just rows

Downloaded-row counts can be misleading. Paging can repeat records. A changing source can shift records between pages. A later join can duplicate otherwise valid records.

For completeness testing, the main comparison should usually be:

> expected matching records versus unique retrieved record IDs

Consider this reconciliation:

The raw download has more rows than the reported total, but the unique-ID count reconciles. The duplicates need explanation, yet there is no remaining numerical evidence that records are missing.

That depends on the identifier being fit for purpose. It should identify the queried entity, be nonblank, and remain stable during the extraction period. If there is no reliable ID, a documented fallback comparison may still be possible, but the completeness evidence is weaker.

Read the source’s paging contract

Each platform has its own signals. Learn the contract for the specific endpoint rather than assuming one familiar rule applies everywhere.

For example, Socrata requests default to 1,000 returned results when no limit is specified. Its documentation describes paging with limit and offset, and warns that results are not implicitly ordered. When paging is available, use a deterministic order on a durable unique field. Otherwise, the sequence can be unstable.

ArcGIS feature-service queries can be constrained by a service’s maxRecordCount. Where supported, returnCountOnly and returnIdsOnly offer useful validation paths. An exceededTransferLimit=true response is a clear signal that more results remain; continue until the source indicates the limit has not been exceeded.

Other APIs provide a continuation URL or token. A next-page link means the current response is not the whole result. Follow the chain until no continuation signal remains.

CKAN DataStore deployments can have configurable defaults and maximums. They may also return estimated totals under some configurations. A number labeled “total” is useful evidence, but first establish whether it is exact, estimated, and tied to your query scope.

A missing error message is not a completeness signal.

Treat round numbers as a prompt, not a verdict

A result count that lands exactly on 10, 100, 1,000, 2,000, 10,000, or 50,000 deserves a closer look. Those values often resemble default page sizes or maximum-result settings.

They are not proof of truncation. A query can genuinely match exactly 1,000 records. But a round threshold is a cheap diagnostic: inspect the documentation, response metadata, and pagination behavior before relying on the result.

The same caution applies to a completed CSV export. A file can download successfully while containing only the rows the interface was willing to export.

When the numbers do not reconcile

Suppose the source reports 12,486 matching records, but the extraction contains 10,000 unique IDs. Do not call the result exhaustive.

First, confirm that the count and retrieval used identical scope. Then check page size, offsets or tokens, ordering, and duplicate IDs. Look for a documented request cap or an ignored continuation signal.

If those checks do not explain the gap, consider whether the source changed during collection. New, deleted, or reordered records can create gaps and repeats in offset-based paging. An official snapshot, a source-provided cursor, a bounded historical period, or a bulk download may provide firmer evidence.

For dashboards, test the display, export, downloadable file, and API separately. They can represent different layers of the same public source and apply different limits.

When a reconciliation remains unresolved, describe it plainly: the analysis covers the records retrieved at a stated time, but it should not be read as an exhaustive count.

Before publishing “all,” “every,” “complete,” or any supposedly exhaustive total, require a successful reconciliation or state the limitation. Completeness is not something a polished result table silently guarantees. It is a claim that needs evidence.