Morrow CreekData Services
← Morrow Creek Data Journal

Before cleaning a dataset, decide what one row is supposed to represent

Zachery Knowles
Zachery KnowlesFounder, F.R.A.R.Y. Data Technologies LLC

A surprising number of data problems begin with a question nobody asked at the start: what is one row?

Is a row one company? One location? One product? One transaction? One person? One observation from one source?

The answer controls almost everything that comes after it.

If a dataset is supposed to have one row per company, two locations for the same business may need to be combined or represented in related fields. If the dataset is location-based, combining them would destroy useful detail.

The same issue appears with products. One row per product family is different from one row per size, color, model, or stock-keeping unit.

Without a clear row definition, duplicate cleanup becomes guesswork. So does counting, grouping, joining, and validation.

A file can even look internally consistent while mixing several row meanings. That is especially common when data from different sources is combined without first deciding which level of detail the final dataset should preserve.

The fix is not complicated, but it should happen early.

Write down what one row represents. Then judge duplicates, missing fields, identifiers, and relationships against that definition.

A dataset becomes much easier to reason about once every row is trying to be the same kind of thing.

Have a dataset that needs to become usable?

Morrow Creek Data Services handles paid research, cleanup, verification, and structured delivery projects.

Start a data project