Geo-level Bayesian hierarchical media mix modeling (Sun et al., 2017), explained

Geo-level Bayesian hierarchical media mix modeling is an approach, set out by Google’s Yunting Sun, Yueqing Wang, Yuxue Jin, David Chan and Jim Koehler in 2017, that fits a marketing mix model on regional data instead of national totals, letting each region have its own media effect while pulling those effects towards a shared average. The paper showed that, when regional data exists, this “partial pooling” gives tighter credible intervals and far less biased ROAS than a national model, because there are more observations and more useful variation in spend.

This is the paper I’d point to if someone asked why Google Meridian wants your data by region. It’s the third of Google’s 2017 MMM papers and the most practical one, because it takes the problem the first two diagnosed (too little information in two years of national data) and offers a fix most advertisers can actually use. Partial pooling is also the idea in it that’s hardest to picture from equations alone, so this page includes a demo where you can push it to both extremes and watch the error.

Why regional data helps: more observations, more variation

The paper starts where Jin et al. (2017) left off. A Bayesian MMM on national weekly data suffers from a small sample and too little variation in spend, unless you give it strong priors, and its colleagues’ answers were either to pool brands in a category or, here, to use regions. A country can usually be split into geos (cities, DMAs, states) where data can be gathered, and as long as the regional series aren’t perfectly correlated with the national one, they carry a larger effective sample size.

The second benefit is range, and I think it’s the more important one. An advertiser rarely cuts national spend to near zero, or might keep a channel too small to see, but spend in individual regions usually varies much more widely than the national total. That matters because a narrow range of spend forces the model to extrapolate the response curve, which is one of the data limitations in Chan and Perry’s challenges paper. The catch is an assumption: you can only take advantage of the extra variation if the mechanism by which media works is similar across regions.

The authors are practical about what a geo has to be. You need to be able to serve ads by that geography with reasonable accuracy and track both spend and the response metric there; they suggest aggregating cities into larger units, because small cities are sparse and noisy and because people see an ad in one city and buy in another. They also note it’s easier to get geo-level data for some media (spot TV, and digital, where publishers can usually break spend down by region) than others. Geo lift tests help here too, since randomly changing spend in some regions creates exactly the independent variation the model needs.

How geo-level Bayesian hierarchical media mix modeling works

The model, which the authors call GBHMMM, keeps the transforms from Jin et al. (adstock for carryover, then a Hill curve for saturation) and adds a hierarchy. Sales per capita in geo $g$ and week $t$ are a geo-specific intercept, plus each channel’s adstocked and saturated spend times a geo-specific coefficient, plus controls with geo-specific coefficients, plus noise:

The hierarchy is in how those geo coefficients are drawn: $\beta_{m,g} \sim \text{normal}(\beta_m, \eta_m^2)$, and likewise for the intercepts and control coefficients. Each region is treated as a sample from an overall population, $\beta_m$ is the national mechanism, and the standard deviation $\eta_m$ controls how far regions can deviate from it. The carryover and shape parameters ($\alpha$, $K$, $S$) are shared across regions but differ by channel, because letting them vary by region would bring back the identifiability problems Jin et al. found with the Hill curve. The maximum carryover $L$ is fixed by the modeler.

Two preparation steps matter a lot. Spend and sales have to be standardized per capita by each geo’s target population, otherwise big regions look like big effects and a given spend could saturate a small city but not a large one. The authors add that census population isn’t always the right denominator; for an auto brand, sales may be a better proxy, since a wealthy small town might be a bigger market for BMWs than a poor big one. On priors, they recommend bounded priors, or ones with most of their mass near zero such as a half normal, for the standard deviations, having found that improper flat priors slowed convergence. They fit the model with a Gibbs sampler that alternates between slice sampling the transform parameters and sampling the hierarchical linear model, which is just a standard Bayesian regression once the transforms are fixed.

The paper also gives two useful diagnostics. If the geo model produces wider intervals than the national model, the regions probably don’t share a mechanism, and separate models make more sense than a joint one. And not every coefficient needs a hierarchy; some can be fixed across regions while intercepts vary, with WAIC or cross-validation deciding which. National ROAS and marginal ROAS come out as spend-weighted averages of the regional ones, computed for each posterior draw, using the same counterfactual definitions as Jin et al.

Partial pooling, and why it beats both extremes

The heart of the method is partial pooling. With no pooling, each region’s estimate uses only its own data, which is unbiased but very noisy for small regions. With complete pooling, every region gets the national average, which is stable but wrong wherever regions really differ. Partial pooling pulls each region’s estimate towards the average by an amount that depends on how noisy that region’s own data is relative to how much regions genuinely vary, which is what the between-region standard deviation ($\eta$ in the paper, $\tau$ in the demo) controls.

With the defaults (a true spread of 0.5 between regions and about six weeks of noisy data each), every region’s own estimate has a standard error bigger than the real differences between regions, so partial pooling cuts the error of the no-pooling estimates by about a quarter on the first draw (and by a third or more on most draws, which you can see by pressing “New regions”) while also beating a single national number. Try setting the weekly noise low and the weeks high: regions can then stand on their own and pooling helps much less. Set the true spread to zero and complete pooling wins, since any spread you allow is just noise. The “Set τ from the data” button estimates the spread from how much the regional means vary beyond what their noise would explain, which is a rough version of what the Bayesian model does when it puts a prior on $\eta$ and learns it.

In practice, the useful lesson is that you rarely know in advance how different your regions are, and a hierarchical model lets the data decide, within the limits of the prior you set on that spread. That’s also why the prior on the between-region standard deviation is worth thinking about rather than leaving at a default: too tight and every region gets the national answer, too loose and you’re back to ten noisy separate models.

What the simulations found

The simulations compare the geo model with the national model of Jin et al. on data generated from the same model class: one channel, one control, five geos and 104 weeks, with an unobserved seasonal demand and spend correlated with it at 0.5, repeated over 100 datasets. The true parameters are $\alpha = 0.5$, $K = 2$, $S = 3$, $L = 4$ weeks and $\beta = 2$, and the authors say in a footnote that these settings are more favorable to the modeler than is typical. Both models get the same priors, and the national data is the average of the regional data, so the only difference is the level of aggregation.

In the first scenario the national model overestimated the response curve, and therefore average ROAS, because low variation in spend let its parameters compensate for each other (underestimating the slope and baseline while overestimating the coefficient). Its marginal ROAS was much less biased, for the same reason Chan and Perry give: the model sees plenty of data around current spend and very little near zero. In the second scenario, where some regions were near saturation and others weren’t, the geo data covered much more of the curve, and the national model’s bias was more than 300 times the geo model’s.

Ad targeting bias: geo data helps, but doesn’t cure it

The third scenario tests the hardest problem in MMM, selection bias from ad targeting, by making the control variable only partly capture the underlying demand that spend follows (a correlation of 0.8 instead of a perfect proxy). Both models were biased, as you’d expect when demand leaks into the media coefficient. However, the geo model’s ROAS error (its mean squared error) was about one fifth of the national model’s.

The reason is neat. Averaging regional spend into a national total strengthens its correlation with the shared base demand, so national spend is more confounded with demand than spend in any single region; the regional variation that isn’t driven by demand helps the model separate the two. It’s worth being precise about the claim, though: geo data reduced targeting bias in this setup, it didn’t remove it, and the authors point to Chen et al.’s search query controls for correcting it properly. The confounding variables guide covers that side of the problem.

When geo data is missing for some channels

The fourth simulation tackles a situation every practitioner hits: digital spend is available by region, but national TV or magazines aren’t. The naive fix is to impute regional spend from the national figure in proportion to population, which adds no useful variation over time. With three channels, the authors imputed one, two and then all three, and the geo model got steadily worse as more channels were imputed, with bigger biases and wider intervals, for the same reason the national model struggles: too little variation.

The more encouraging finding is that imputing one channel didn’t ruin the others. When only one of three channels was imputed, the estimates for the other two were slightly worse than with full geo data but still reasonable, although the authors expect that to break down if channels are more correlated. Their conclusion is pragmatic, and it’s the line I’d quote to anyone deciding whether regional data is worth the effort: “it is better to build a geo-level model when the geo-level data is available” (or partially available), as long as regions share a similar media mechanism.

The auto category case study

The real example uses monthly data for 12 auto brands across 18 US DMAs from January 2013 to December 2014, with new car registrations from R.L. Polk, TV spend from comScore, Google search spend and query volume from Google, and other variables from Neustar MarketShare. The 18 DMAs covered more than half of new car registrations, and TV made up more than 80 percent of the brands’ national marketing budgets. Sales were standardized per capita using each DMA’s average monthly sales as the target population, and the model used delayed adstock with a maximum of three months, since car purchases take a while.

Because a single brand nationally has only 24 monthly observations, the authors compared category-level models instead: 288 observations for the national category model against 5,184 for the category-geo model. The geo model gave considerably tighter intervals, and both agreed on the shape of the effects, with TV peaking around a month after exposure and decaying over the next two, and search peaking much sooner. National TV spend turned out to be extremely skewed with many months near zero, while search spend was roughly normally distributed, and TV looked closer to saturation at its maximum historical spend than search did.

From this paper to Meridian

If you read Meridian’s model specification after this paper, the structure is recognizable: geo intercepts, geo-level media coefficients drawn from population-level distributions with their own standard deviations, the same normalized adstock and Hill transforms shared across regions, and Bayesian priors throughout. Meridian adds a shared time-varying intercept for trend and seasonality, the option of a separate noise level for each region, priors set directly on ROI so experiments can feed in, and reach and frequency inputs, the last of which comes from a 2023 Google paper by Zhang and colleagues that describes itself as an extension of this model.

So the practical question this paper leaves you with is whether you have, or can get, regional data. The Meridian page covers how to prepare it, the data requirements guide covers how much you need, and the older brief on geo models has a gentler introduction. If you’re reading the Google series in order, the research papers bibliography lists what came next.

In my experience the biggest obstacle to geo-level modeling is getting honest regional spend for offline channels, which is exactly where this paper shows imputation hurts, and the statistics are the easier part. Thankfully the digital channels that make up most budgets today can almost always be broken down by region, and even a partial geo model, with the offline channels at national level, gives you more to work with than two years of national totals. If you’re new to the underlying ideas, the pillar guide to marketing mix modeling and the course start from the beginning.