Morrow CreekData Services
← Morrow Creek Data Journal

How to Detect When Business Records Have Been Reclassified Instead of Newly Created

Morrow Creek Data ServicesMorrow Creek Data Journal

A report says a category gained 500 businesses this month. That may be true. It may also mean 500 existing businesses were assigned new category codes.

The difference matters. New entities change the size of a population. Reclassifications change its composition. If those are mixed together, a dashboard can mistake coding work, a policy revision, or a warehouse history rule for real growth.

The useful question is not, “How many new rows appeared?” It is, “Which business entities are genuinely new to the population being measured?”

That distinction applies to customer segments, supplier types, product families, risk bands, industry codes, sales territories, and public business registers.

Define what “new” means before counting

A new database row is not necessarily a new business fact. A versioned warehouse may create a fresh row whenever a customer changes segment. An extract may assign a new row number each month. A recent load timestamp can reflect a backfill or a corrected record.

Start by naming the unit that matters. Is it a legal business, supplier, customer account, product master, establishment, or contract? Then define the population: active suppliers in a country, for example, rather than every supplier ever loaded into a system.

For entity counts, use a persistent business key where possible. That might be an account ID, registration number, supplier ID, or product master ID. Row IDs, file positions, and ingestion timestamps are usually poor substitutes because they can change when the entity does not.

A source-system insert also deserves caution. Records can be deleted and recreated, restored after an outage, migrated from another platform, or introduced through a historical reload. Source operations are useful evidence, not automatic proof of real-world creation.

Compare snapshots as a population flow

Take two dated snapshots: a prior period and a current period. Match current entities to prior entities using the documented identity key. For each entity, assign one mutually exclusive status:

  • New to the population: no matched entity exists in the prior snapshot.
  • Retained in the same class: the entity appears in both snapshots with the same category.
  • Retained but reclassified: the entity appears in both snapshots, but its category differs.
  • Exited or absent: the entity appeared previously but is not in the current snapshot.

This turns an ambiguous row comparison into an entity-lifecycle view. It also makes uncertainty visible. If stable identifiers are missing, use a documented matching hierarchy and label uncertain matches for review. A shared name alone does not prove that two records represent the same entity.

The result should include a matching rate. If only 70 percent of current entities match to the prior snapshot, claims about new entities need careful qualification. The unmatched group may include true entrants, but also late arrivals, earlier extraction failures, changed identifiers, and duplicate-resolution work.

Build a from-to classification table

Once entities are matched, create a transition table. Put prior classifications in rows and current classifications in columns. The diagonal shows retained entities that stayed in the same class. Off-diagonal cells show movements between classes. Entities with no prior match form the candidate new-entity group.

Suppose a supplier master grows from 12,000 active suppliers in June to 12,140 in July. The Technology category rises by 180 suppliers. Matching on supplier ID finds 35 new suppliers assigned to Technology, 160 transfers into Technology, and 15 transfers out.

Technology did gain 180 suppliers overall. But only 35 were new entities. The remaining net gain came from reclassification.

This table supports a basic reconciliation for every category:

Current count = retained in class + genuine new entities + transfers in − transfers out − exits

If the numbers do not reconcile, investigate before publishing. Common causes include duplicate keys, inconsistent active-status rules, incomplete matching, or category values that changed after the snapshot date.

Inspect the mechanism behind the change

The transition table shows what changed. Supporting history can help explain why.

Where available, change-data-capture or audit data can distinguish inserts, updates, and deletes. Field-level history may show that an existing entity’s classification field changed, which is stronger evidence of reclassification than a newly loaded row alone.

Keep several dates separate: entity creation date, row creation date, classification effective date, update date, and warehouse load date. Enterprise systems commonly record creation and last-update fields separately. A recent update date says that a record changed in the system; it does not establish that the business itself is new.

Also look for batch signatures. A large number of entities moving from one class to another on the same effective date often points to a revised rule, mass recoding, or mapping update. It is less consistent with independent waves of newly created entities.

A risk system illustrates the point. If thousands of persistent account IDs move from medium to high risk on one date after a scoring-model update, report a reclassified population. Do not describe the movement as thousands of newly high-risk accounts without evidence about the underlying behavior or policy change.

Check whether the classification itself changed

Sometimes the entities did not change at all; the classification scheme did.

Industry and other formal classifications are periodically revised. Official concordances can describe relationships between older and newer code sets, but those relationships may be one-to-many or many-to-one. An old industry class split into several new classes can create a sharp increase in a new code with no corresponding increase in establishments.

Record the classification version and the lookup or mapping version used in each snapshot. Review release notes and effective dates. If history was recoded or backcast to a new standard, say so. A comparison across versions may still be useful, but it should not be presented as a pure measure of entity movement.

The same caution applies to internal taxonomies. A renamed segment, changed eligibility rule, or revised mapping table can alter counts while the underlying customer population stays still.

Report the answer with provenance

A short note alongside the result can prevent a misleading interpretation. Include the two snapshot dates, population definition, identity key, matching rate, classification version, and treatment of late-arriving records or backfills.

When no prior snapshot, audit history, or reliable identifier exists, the honest answer may be that the source cannot distinguish new entities from reclassified ones. Reporting that limitation is better than turning recent rows into an unsupported growth claim.

New records are an entity question. New rows are often only a system event.

Have a dataset that needs to become usable?

Morrow Creek Data Services handles paid research, cleanup, verification, and structured delivery projects.

Start a data project