Scope the collection before the first record.

Start with the decision the dataset must support.

Public-web collection is not simply “getting information from websites.” The project needs a business purpose that determines which records matter, which fields are necessary, how current the information must be, and what evidence a reviewer needs. A list intended for market mapping can require different coverage and provenance than a file intended to update an internal catalog.

State the intended decision or workflow in plain language. Then describe who will use the file, how it will be reviewed, and what the user must be able to do with each row. This prevents the scope from accumulating fields that are expensive to collect but do not improve the intended outcome.

Define exactly what one record represents.

Many collection failures begin with an undefined record. “Products,” “companies,” and “locations” sound specific until one company has several offices, one product has multiple variants, or the same organization appears under related brands. Decide whether one row represents a legal entity, operating location, website, product family, individual item, document, event, or another unit.

Give the record a stable identity rule whenever possible. A company might be identified by its legal name and jurisdiction; a location by its street address and organization; a product by a manufacturer identifier and variant. If no stable identifier exists, define the combination of fields used to detect duplicates and describe how uncertain matches should be handled.

Record-boundary questions

• Can one source page produce more than one record?
• Can one record appear on more than one approved source?
• Are subsidiaries, franchises, departments, and branches separate records?
• Are discontinued, archived, out-of-stock, or inactive records included?
• What evidence is sufficient to treat two similar entries as the same entity?

Approve the source and access boundaries.

Name the websites, public directories, documents, or source categories that may be used. A scope can allow a defined list of domains, a customer-provided URL inventory, or an approved class of public sources. It should also identify excluded sources and which source controls when two permitted sources conflict.

Morrow Creek does not bypass authentication, paywalls, access controls, rate limits, technical restrictions, or source terms. A source can be public in the ordinary sense and still be unsuitable for the requested collection method or use. Source feasibility is reviewed before the work is confirmed, and a change in site structure or access conditions can require a scope review.

Build a field specification, not a wish list.

For every requested field, define its name, meaning, data type, allowed format, source, and treatment when unavailable. Separate required fields from optional fields. If a field needs interpretation, describe the rule that turns the source wording into the delivered value.

A good field specification also prevents hidden transformations. For example, decide whether prices remain as displayed strings or become numeric values with separate currency fields; whether category labels remain source-native or map to a controlled customer taxonomy; and whether dates use the source timezone or an agreed normalized timezone.

Each field should answer

• What fact does this column represent?
• Which approved source can support it?
• What data type and format should be delivered?
• Is the field required for every included record?
• How are blanks, conflicting values, ranges, and multiple values represented?
• Does the record need the source URL, observation date, or document reference for this field?

Set inclusion, exclusion, coverage, and stopping rules.

“Find all” is rarely measurable without a defined population. Establish the geography, category, date range, source set, product family, organization type, or other boundary that controls inclusion. If the source does not expose a complete population, use a defensible stopping rule such as a supplied URL list, a named directory, a fixed source set, a confirmed record target, or exhaustion of an approved navigation path.

Write exclusions with the same care as inclusion rules. Common decisions include whether to exclude records without a required identifier, records outside the date window, duplicate locations, discontinued products, noncommercial entities, or entries that lack qualifying public evidence.

Decide how provenance, freshness, and conflicts are recorded.

Provenance is part of the schema, not an afterthought. Decide whether each record needs a source URL, page title, document name, observation date, source category, or field-level reference. If a public page can change, an observation date helps explain what was visible during production.

Specify the maximum acceptable age for time-sensitive facts. Also define source precedence. When two approved sources disagree, the scope can use a controlling source, retain both values in separate fields, flag the conflict, or leave the disputed field unresolved. The correct choice depends on the intended use; unsupported certainty is not created simply to fill the record.

Translate quality expectations into acceptance tests.

Acceptance rules should be testable against the delivered file. They can cover exact columns, file type, data types, required-field completeness, allowed values, uniqueness, source presence, observation dates, row expectations, and the absence of records outside the approved boundary.

A requirement such as “accurate data” is too broad by itself. Define which fields are verified, which sources control verification, how unavailable values appear, and how exceptions are counted. If the project allows a limited exception rate, name the denominator and the fields included in that calculation.

Example of a confirmable request.

This example does not determine price or feasibility by itself. It gives the scope review enough structure to confirm the source boundaries, clarify edge cases, estimate the work, and define the exact acceptance contract before commitment.

What happens after submission.

1. Start a private paid-job request and describe the business outcome, record, sources, fields, and delivery needs.
2. Morrow Creek reviews lawful use, technical feasibility, source restrictions, coverage, exceptions, and measurable acceptance.
3. Material questions are resolved inside the private workspace.
4. You receive the exact scope and price before committing to production.
5. Completed customer files are checked against the confirmed requirements before payment and release.

For related planning, review the complete custom data project guide, company list research guide, public-web collection service, and product data collection service.