How to Use Value-of-Information Analysis to Decide Which Data to Collect Next

Most data wish lists begin with a reasonable question: “What do we wish we knew?” A better question is narrower: Which unknown could change a decision before the decision window closes, and improve the expected result by more than the cost of finding out?

That is the purpose of value-of-information analysis. It treats data collection as an investment decision, not as a general effort to reduce uncertainty. More information can be useful, but only when it enables a better action.

Start with the decision, not the dataset

Name the pending decision, its deadline, and the actions actually available. For example, a company may need to decide whether to launch a regional service this quarter, delay the launch, or decline it altogether.

Then describe the consequence of each action under plausible conditions. The metric might be expected profit, expected cost, expected loss, or a multi-criteria utility score that includes financial, operational, and reputational effects. Revenue alone is often too narrow, especially where service levels, legal constraints, or safety thresholds matter.

With that model in place, calculate the best action using what is already known. This is the baseline. Any new data source must beat it after accounting for its full cost.

Ask the switch question

For each uncertain input, ask: if the value turns out lower or higher than expected, would our preferred action change?

If the answer is no, further data about that input may improve a forecast but has little immediate value for the decision. A variable can have a wide confidence interval and still be decision-irrelevant.

Consider a campaign expected to convert between 3.0% and 4.5% of prospects. If both outcomes clear the launch hurdle, more precise conversion data may not be the next priority. Fulfillment capacity may be less uncertain, yet far more valuable to investigate if one plausible capacity level creates late-delivery penalties that reverse the launch decision.

This switch question prevents teams from mistaking visible data gaps for important ones.

Use EVPI to set an upper limit

Expected value of perfect information, or EVPI, answers a hypothetical question: what would it be worth to know the uncertain state of the world with certainty before deciding?

Suppose a regional launch has a 40% chance of high demand and a 60% chance of low demand. Launching would produce a $200,000 gain under high demand and a $120,000 loss under low demand. Not launching produces $0.

With current information, the expected value of launching is $8,000:

(0.40 × $200,000) + (0.60 × -$120,000) = $8,000

So launch is the current choice. With perfect demand information, the company would launch only when demand is high. That has an expected value of $80,000. The EVPI is therefore $72,000.

That figure is a ceiling, not a budget recommendation. Even flawless demand information is worth no more than $72,000 for this decision. A real study, which will be imperfect and may arrive late, should be worth less.

Partial perfect information applies the same logic to one uncertainty at a time. It is useful for ranking a long list of unknowns before commissioning research. Investigate the factor most likely to alter the choice and improve the outcome, not necessarily the factor with the largest variance.

Estimate the value of a real study

Expected value of sample information, or EVSI, evaluates an actual proposal: a pilot, survey, audit, test, external dataset, or measurement improvement. Unlike EVPI, it reflects imperfect accuracy, finite samples, noisy measurements, and biased or ambiguous results.

Continue the launch example. Imagine a pilot that classifies demand correctly 75% of the time. A favorable result would lead to launch, while an unfavorable result would lead to no launch. Under these illustrative assumptions, the expected value after using the pilot is about $42,000, compared with $8,000 without it. Its EVSI is about $34,000.

If the pilot costs $20,000 all-in, its expected net benefit is about $14,000. If it costs $40,000, it should not go ahead for this decision.

“All-in” matters. Include acquisition or collection cost, data preparation, quality checks, analyst time, governance work, decision delay, and opportunity cost. Also consider whether the organization can act on the result. Evidence delivered after a launch, pricing, purchasing, or budget deadline may have little value, however accurate it is.

Turn a data wish list into a decision agenda

A compact working table can make the analysis practical. For each uncertainty, record its plausible range, the action favored at the low and high ends, the likelihood of a switch, the upside of switching correctly, the proposed evidence source, its expected accuracy and timing, and its all-in cost.

The output need not imply false precision. For a small decision, a decision table with a few credible scenarios may be enough. For larger stakes or linked uncertainties, simulation and more formal decision analysis may be appropriate.

Challenge the model before trusting its output. Are there missing actions? Does the payoff model omit a binding operational constraint? Are the assumptions about study accuracy credible? Does a financial metric overlook a safety, legal, compliance, or reputation threshold?

A data project can be justified in expectation and still produce an unhelpful result. That does not mean the original decision was wrong. It means expected value describes the quality of the choice before the outcome is known.

Before collecting the next dataset, ask: What decision will this evidence change? What result would reverse today’s choice? How likely is that result? What is the benefit of switching correctly? Will the evidence arrive in time, and what will it truly cost?

If those questions have no clear answers, the most valuable next step may not be more data. It may be a clearer decision model.