TL;DR

Attribution can never tell you whether your ads caused a sale, only which ad touched the customer last. Geo holdout testing can. You split your markets into two groups: one sees your ads, one doesn't. The difference in conversion rates between them is the lift your ads actually caused, stripped of every attribution argument. Until recently this required a measurement partner and a six-figure budget. Google's open-source Meridian GeoX methodology and modern automation have brought it within reach of accounts spending a few thousand a month. If you run paid ads and have never run a holdout, you don't actually know what your ads do. You know what your dashboards say.

The question attribution can't answer

Every attribution model, from last-click to the fanciest data-driven setup, answers the same question: which touchpoint gets credit for this conversion?

It's a useful question. It is not the important one.

The important question is: would this conversion have happened without the ad?

A branded search click is the classic case. Someone searches your company name, clicks your ad, buys. Attribution gives the ad full credit. But that person was already looking for you. Most of those sales happen anyway, ad or no ad. The ad didn't cause the sale. It taxed it.

No attribution model can catch this, because attribution only sees people who converted. It never sees the counterfactual: the people who would have converted anyway. To measure causation, you need a group that didn't get the treatment. You need a holdout.

How a geo holdout test works

The design is simple enough to explain in a paragraph.

Take your geographic markets, cities, states, or DMAs, and split them into two matched groups with similar historical sales patterns. In the treatment group, your ads run as normal. In the holdout group, you turn them off (or reduce spend sharply) for the test period. Then you compare conversion rates between the two groups.

The difference is your incremental lift. If treatment markets convert at 2.4% and holdout markets at 2.0%, your ads are causing a 20% relative lift. Everything above the holdout baseline is real, caused demand. Everything at or below it would have happened anyway.

Because the groups are geographic, the test is immune to the things that break user-level measurement: iOS tracking restrictions, ad blockers, cookie deletion, cross-device journeys. You're comparing aggregate sales in places, not tracking individuals. Privacy-proof by construction.

Why geography, and not audiences

Platform-native lift tests usually split by user: some users see ads, some don't. That works inside one platform's walled garden, but it can't answer cross-channel questions, and it depends on the platform's own identity graph, the same platform that profits from the result.

Geographic splits have three advantages:

Platform-neutral. You can hold out Meta in one set of cities and Google in another, or hold out everything, and compare on equal footing.
No identity tracking required. Aggregate regional sales data is enough.
Harder to game. The analysis runs on your actual revenue by region, not on a platform's modeled conversions.

The tradeoff is statistical power. Regions are noisier than individuals, so geo tests need careful market matching and enough time to separate signal from noise. That's where the methodology matters.

The Meridian GeoX methodology

Google's current open-source solution is Meridian GeoX, a framework for designing and analyzing geographic incrementality experiments. It can run independently or help calibrate Meridian marketing mix models. Two important parts of designing a geo experiment are:

Market matching. Not all cities are interchangeable. Meridian GeoX uses historical sales data to find treatment and holdout groups whose pre-test behavior tracks each other closely, so differences during the test can be attributed to the ads rather than to the markets just being different.

Power analysis. Before the test starts, you estimate how long it needs to run and how large the spend difference must be to detect the lift you care about. This is the step that saves you from the most common failure mode: running a test for two weeks, getting an inconclusive result, and concluding that "testing doesn't work." The test didn't fail. It was underpowered from the start.

A well-designed geo test typically runs two to four weeks, depending on your conversion volume and the size of the effect you're trying to detect.

What small accounts get wrong

Three mistakes show up constantly:

1. Holding out too little spend. If treatment and holdout differ by only 10% of budget, the expected difference in sales is tiny and the test needs months to detect it. A holdout has to be a real holdout.

2. Testing everything at once. Turning off all ads in half the country measures your total marketing lift, which is interesting once, but doesn't tell you which channel earns the next dollar. Test one channel at a time.

3. Reading results daily. Regional sales are noisy. A test that shows +30% on day three and -10% on day nine is normal. You decide at the end, at the pre-committed significance threshold, or you don't have a test. You have a anxiety dashboard.

From test result to budget decision

A holdout test that ends in a slide deck is trivia. The point is to change where money goes.

The output you want is a per-channel incremental ROAS: revenue caused per dollar spent, not revenue claimed. That number slots directly into budget allocation. Channels with high attributed ROAS but low incremental lift get cut. Channels that look mediocre in the dashboard but show real lift get funded.

This is also how you catch the branded search trap at scale: brand campaigns routinely show enormous attributed ROAS and near-zero incrementality, because the demand existed before the ad.

Where GhostEngine fits

GhostEngine runs geo and spend holdouts built on Google's Meridian GeoX methodology as part of the platform, not as a one-off consulting project. Tests are designed with proper market matching and power analysis, run on a schedule, and the resulting incremental lift feeds directly into budget allocation, so the channel that actually caused the sales gets the next dollar. Details are on the Why GhostEngine page.