Causal inference is the discipline of estimating what an action actually changed, by comparing what happened with what would have happened if you had acted differently. For marketers that means separating the sales your advertising caused from the sales that would have arrived anyway, which is a different question from which channel the customer touched last or which line on a chart moved at the same time as your spend.
In my former life running a 50-person growth marketing agency that spent over $10 million a year on Facebook ads, the hardest conversations were never about creative or targeting. They were about credit: the platform said one thing, Google Analytics said another, and the CFO wanted to know what would happen to revenue if we cut the budget in half. That last question is a causal one, and none of the dashboards could answer it, because every dashboard reports what happened, not what would have happened otherwise. I’ve since built marketing mix models for companies like Monzo, Booking.com and The AA, and the same question sits underneath every one of them. I’ll show you the handful of ideas from causal inference that let you reason about it, without the notation getting in the way.
Counterfactuals: the missing half of every causal claim
The core idea in causal inference is the “potential outcome” (the result you would see under each possible decision). For any given week there’s the sales number you’d get if the campaign runs and the sales number you’d get if it doesn’t. The causal effect of the campaign is the gap between the two. The trouble, which statisticians call the fundamental problem of causal inference, is that you only ever observe one of them. Once the campaign has run, the week without it is gone forever, and it’s this unobserved outcome that we call the counterfactual.
In the course’s invented Northstar case, a campaign week records £135,000 of sales. If the counterfactual (the same week without the campaign) would have been £100,000, the campaign added £35,000. If it would have been £120,000, the campaign added £15,000. The observed sales are identical in both stories; only the counterfactual moved, and with it the answer. This is why I push teams to rewrite “the campaign drove £135,000” as “the campaign ran in a week with £135,000 of sales, of which we estimate £15,000 to £35,000 was incremental, compared with no campaign”. It’s clumsier, but it makes the comparison explicit, and the comparison is where all the arguments are hiding. The incrementality guide goes further into how platforms, experiments and models each define that baseline.
It’s worth keeping in mind that “compared with what?” has more than one valid answer. An experiment might compare a campaign with no campaign, or 20 percent more spend with current spend. A marketing mix model usually compares actual media with a baseline of zero media, which is a counterfactual nobody has ever run. These aren’t estimating the same quantity, so a marginal lift from a small budget increase shouldn’t be quoted as if it were the average return on all historical spend.
Why correlation and causation come apart in marketing
The correlation vs causation problem in marketing is baked into how budgets are set. Spend is a decision made by people who are trying to be smart, and smart people spend more when they expect sales to be good. That single habit creates most of the bias you’ll meet in attribution and in MMM, and it shows up in three slightly different forms.
Confounding: a shared cause moves both spend and sales
A confounder is a variable that causes both the treatment (your spend) and the outcome (your sales). Seasonality is the classic example. A garden furniture brand spends more in spring because that’s when people buy garden furniture, so a chart of spend against sales slopes steeply upward even if the ads did nothing at all. Promotions, product launches, pay day and competitor stock-outs work the same way. Leave the shared cause out of the analysis and the model hands its effect to whichever variable happens to move with it, which is usually your media. The confounding variables guide goes through which controls fix this and which make it worse.
Selection: the ad reaches people who were already going to buy
Selection bias is confounding at the level of the individual. Targeting algorithms are built to find the people most likely to convert, and retargeting shows ads to people who have already put something in their basket, so the people who see your ads were always going to buy at a higher rate than the people who don’t. Comparing exposed and unexposed customers then measures the targeting, not the advertising. Paid search is the extreme case: the ad only appears when someone types your brand name, which is about as strong a signal of intent as you’ll ever get. Google’s own researchers list ad targeting and seasonality under selection bias in Chan and Perry’s 2017 paper on the challenges of media mix modeling, and a 2018 follow-up by Aiyou Chen and colleagues at Google proposed a correction for search specifically. The same problem is why multi-touch attribution tends to flatter lower-funnel channels.
Reverse causality: spend follows demand
Reverse causality is when the arrow runs the other way, with sales causing spend. Plenty of budgets are set as a percentage of last month’s revenue, bid strategies automatically spend more when conversion rates are high, and ecommerce teams pour money into the channels that look like they’re working this week. In each case a good sales week produces a big spend week, so spend and sales correlate even if the spend contributed nothing. Weekly data makes this hard to untangle, because both the decision and the response can happen within the same seven days.
Watch a confounder inflate your return on ad spend
The simulator below builds a world where you know the answer. A hidden seasonal demand signal pushes sales up and down, and the marketing team spends more in peak weeks. You choose how closely spend follows demand and what one dollar of advertising truly earns, and then compare three estimates: the naive slope of sales on spend, the estimate from a regression that also includes a season index, and the truth.
At the default settings the naive regression more than doubles the true return, and more than half of the credit it hands to advertising is really seasonal demand. Drag “how much spend follows demand” to zero and the naive slope falls back close to the truth (any remaining gap is sampling noise), because when the budget ignores the season there’s nothing for the regression to confuse. Then put the follow slider back up and add noise to the season index. This is the realistic case, since you never observe “demand” directly, only proxies like last year’s sales, search trends or a holiday calendar. A noisy control removes some of the bias but not all of it, which is the honest version of “we controlled for seasonality”.
If you want to run the same experiment with your own settings, or see how much the estimates vary from one simulated world to the next, the Python version repeats the simulation 500 times and adds a fourth estimate where spend was randomized.
With the default settings and a moderately noisy season index, the script reports a naive estimate of about $3.39 against a true $1.50, a season-adjusted estimate of about $2.12, and a randomized estimate of $1.50 with the widest spread of the four. That last point matters: randomization removes the bias, but a single small experiment can still be noisy, which is why statistical power deserves as much attention as design.
Drawing the causal question before you model it
The most useful habit I’ve picked up from the causal inference literature is drawing the problem as a directed acyclic graph, or DAG (a diagram where each arrow means “directly causes” and no arrow loops back on itself). It sounds academic, but in practice it’s a whiteboard sketch that forces everyone to say out loud how they think spend gets set and how customers respond. The simplest useful version for marketing has three arrows: demand causes spend, demand causes sales, and spend causes sales.
The diagram tells you what to do. Any path from spend to sales that starts with an arrow pointing into spend is a “back door”, and Judea Pearl’s back-door criterion says you can estimate the causal effect if you adjust for a set of variables that blocks every back door without blocking the front door or opening new paths. Here that means controlling for demand. In a real business the diagram has more boxes (promotions, price, distribution, competitor launches, branded search, the weather), and the same rule decides which of them belong in the model. Cinelli, Forney and Pearl’s crash course in good and bad controls is the clearest short treatment I know of, and it’s why I’d rather spend an hour arguing over a diagram than a week tuning hyperparameters.
The diagram can’t be discovered from the spreadsheet, though. It comes from knowing how the business actually sets budgets (ask the media team, not the data), how customers respond and when each variable is measured. Two analysts with different diagrams will build different models from the same data, and that disagreement is useful, because it’s a disagreement about the business rather than about the code. A good exercise is to list three reasons your team changes spend, then ask of each one whether the same reason can change sales without any advertising. Every “yes” is a confounder you need in the model or a reason to distrust it.
Causal inference methods marketers can actually use
There are only a few ways to fill in a counterfactual, and they differ mainly in how much you have to assume.
Randomisation: make spend independent of everything else
Randomisation is the gold standard because it breaks every back door at once. If a coin flip decides which users, regions or weeks get the ads, then demand, seasonality and intent are balanced between the groups on average, and the difference in outcomes can be read as the effect of the ads. That’s the logic behind platform conversion lift studies, user-level holdouts and geo-lift tests where whole regions go dark. The catch is cost and noise: you have to give up some sales in the holdout, and advertising effects are small relative to the natural variation in sales, so tests need to be bigger and longer than most teams expect. The incrementality testing guide covers how to plan one around a specific decision.
How much better are experiments than clever observational methods? The best evidence comes from Gordon, Zettelmeyer, Bhargava and Chapsky, who compared 15 large randomized advertising experiments at Facebook with the estimates that standard observational methods would have produced from the same data. Even with rich demographic and behavioral data to condition on, the observational methods often failed to recover the experimental lift, and were frequently far off. I summarize the paper and its 2023 follow-up in the guide to Facebook experiments vs observational methods. It’s the best antidote I know to the belief that more data automatically means less bias.
Natural experiments: let the world randomize for you
Sometimes you get randomization for free, when something outside your control changes exposure for reasons unrelated to demand. My favorite marketing example is the study by Seth Stephens-Davidowitz, Hal Varian and Michael Smith on Super Bowl ads for films. Studios buy the slots before anyone knows which teams will play, and the home cities of the two teams watch the game in much larger numbers, so the extra exposure in those cities is close to random with respect to film demand. Comparing box office across cities gave them a causal estimate that a simple spend-vs-sales regression couldn’t. In everyday marketing, natural experiments look like a platform outage, a tracking change that cuts one channel in some regions, a TV strike or a supply problem that pauses a campaign. They’re worth writing down when they happen, because they’re often the only unconfounded variation you’ll get in a year.
Quasi-experiments: strong assumptions you can check
When you choose which regions get the campaign but can’t randomize properly, methods like difference-in-differences and the synthetic control method estimate the counterfactual from untreated comparison units. They swap the “we measured every confounder” assumption for a different one, such as “the treated and comparison regions would have followed parallel trends”, which you can at least partly inspect in the pre-period. In the course’s geo dataset, a treatment-only before/after comparison reports a lift of 17, difference-in-differences reports 9, and the constructed truth is 8.
Where marketing mix modeling sits in the causal picture
A marketing mix model is an observational model: it learns from the spend you actually chose, not from spend you randomized. That makes it a causal inference method only to the extent its assumptions hold. When someone reads a channel’s ROI off an MMM, they’re implicitly assuming the model includes every shared cause of spend and sales (seasonality, promotions, price, distribution and so on), that the functional form for adstock and saturation is roughly right, that there’s enough independent variation between channels to separate them (see multicollinearity in MMM), and that the past is a fair guide to the decision being made. None of those come for free, and the model’s R² tells you nothing about any of them, which is why I treat fit statistics as a minimum bar rather than evidence of truth.
None of this is an argument against MMM, which is the only method that covers every channel, including offline ones, over a long enough period to see carryover and diminishing returns, and it’s what you need for budget allocation. But it means the model should be built and read with the causal assumptions on the table. Google’s Meridian documentation on control variables makes the same point: include confounders, include strong predictors, exclude mediators, and ask the marketers how budgets are actually set. And the strongest MMMs are anchored by experiments, either as priors or as calibration targets, so that at least some channels have a randomized estimate pulling the observational one towards the truth. Calibrating MMM with experiments walks through how that works in Robyn and Meridian.
A checklist for reading any causal claim about marketing
When a vendor, a platform or your own analyst tells you a channel “drove” a number, these are the questions I’d ask before believing it.
Causal inference doesn’t hand you a formula that turns a spreadsheet into the truth. What it gives you is a vocabulary for the assumptions every ROI number already makes, and the discipline to test the ones that matter most for the decision in front of you. If you want to practise this on a full case, the Vexpower MMM course works through it on the invented Northstar brand, from drawing the causal question to defending a budget recommendation.
