The Counterfactual Question: What Difference Did We Actually Make?

By Dr Tristana Perez, Head of Impact

What is a counterfactual?

When we talk about impact, we are ultimately trying to describe the change that happened as a consequence of our involvement. A counterfactual helps us do this by defining the scenario in which we don’t intervene to compare against one in which we do provide capital or design an intervention.

To construct a counterfactual we ask ourselves: What would have happened anyway? At MedAccess, key elements include mapping out what would have happened to the health product we’re exploring regardless of our support: Would the business have scaled and entered new markets regardless? Would countries have purchased an innovative product, and in what quantities? What alternative standard-of-care would people have relied on in the absence of our involvement?

For example, when we examined the added value of dual HIV/syphilis rapid diagnostic tests for pregnant women, we found that HIV testing rates were already very high, reflecting decades of sustained investment in HIV programmes. As a result, our guarantee was unlikely to shift existing HIV testing trends.

In contrast, syphilis testing rates were much lower, despite the severe and well‑documented risks to mothers and babies. Without targeted intervention, this gap was unlikely to close on its own. This is where our involvement had the potential to generate meaningful impact through expanded access.

This “path not taken” becomes the benchmark against which we assess the change our capital, expertise, or partnership helped unlock.

Why do we use counterfactuals?

Counterfactuals allow us to identify the differential, and therefore further define our contribution. It helps us consider not only whether impact occurred, but what portion of that impact can reasonably be attributed to our engagement.

In doing so, counterfactuals help articulate the type and scale of impact we are contributing to – whether that is catalytic market shaping, improved access, or system‑level change.

However, in many real‑world situations clear attribution is not possible. Markets are dynamic, multiple partners are involved, and change often emerges through complex, interdependent pathways. In these cases, counterfactuals still play an important role: they help us describe the direction and magnitude of change we are enabling, even when we cannot isolate our exact contribution with precision.

What are the key issues with using counterfactuals?

Counterfactuals are powerful, but they come with inherent challenges. The level of optimism, or conservatism, we apply directly shapes the degree of change we claim. Too optimistic, and we risk under‑estimating our potential contribution; too pessimistic, and we risk overstating it.

Because counterfactuals describe a scenario that never materialised, they cannot be validated once an investment is made. This makes them an area ripe for impact washing, where the comparator scenario can be designed to allow for exaggeration of positive impact through assuming no change or declining access without intervention. Without clear standards, counterfactuals can be stretched to justify almost any claim.

In our session with impact management practitioners, we surfaced these tensions clearly: counterfactuals are useful, but they must be constructed responsibly.

What did we discuss together?

Our group explored several core questions:

• Are counterfactuals always necessary? At what level of depth and rigour should we define them when we already feel confident that the investment is generating positive change?
• What role should counterfactuals play in capital and resource allocation? How should they inform decisions about where concessional, catalytic, or strategic capital goes?
• How far downstream should counterfactuals extend? Should we consider only immediate effects, or also the knock‑on outcomes for households, systems, or markets?

While the discussion brought out differing perspectives, there was a shared recognition that the purpose of counterfactuals is not perfection, but clarity of intent and honesty about contribution.

Where we landed

Despite varied views, a common thread emerged. Counterfactuals should ultimately reflect the organisation’s mission and what it is directly aiming to achieve with its involvement. In MedAccess’ case, our north star is asking ourselves whether our contribution will meaningfully increase access to effective medical products, ensuring they’re available where and when they’re needed.

Whether the goal is catalytic capital, increased access to medicines, or problem‑driven market transformation, the counterfactual should help us judge whether we are addressing the problem we set out to solve.

Fair, reasonable demonstration of change, and not maximalist claims, should guide our approach.

What we took away

Participants agreed that clearer guidance on minimum expectations for impact investors would be valuable. This includes establishing baseline standards for constructing counterfactuals, articulating assumptions transparently, and ensuring consistency across teams and investments.

Ultimately, counterfactuals are not an academic exercise. They are a practical tool for strengthening accountability, sharpening decision‑making and ensuring we direct capital where it can make the most meaningful difference. In a resource-constrained reality, counterfactuals help us make the most impactful choices.