Residuals show what the model has left unexplained
A residual is the observed outcome minus the fitted value. Its pattern can reveal systematic prediction errors, changing variance or unusual observations. It cannot by itself prove that a channel coefficient is causal. A model can predict sales well while allocating credit incorrectly between correlated channels.
- Fit on training periods
- Plot residuals through time
- Evaluate future predictions
- Challenge channel interpretation
Use the R MMM tutorial for the full teaching model. The standalone residual diagnostic script uses a simpler linear benchmark on the same synthetic CSV so the diagnostic workflow is easy to inspect.
A reproducible workflow makes data checks, modelling assumptions and decision sensitivity visible. The mmm tools, code and platforms learning path connects this idea to the assumptions, practical checks and next steps.
To work through the connected decisions, read MMM data leakage: audit the path from raw data to holdout and Robyn model selection: review the Pareto candidates.
Run a reproducible benchmark
Place the R script and CSV in one directory, then run Rscript residual-diagnostics.R mmm-practice-synthetic.csv. It fits on the first 80% of weeks, predicts the remaining weeks and writes a PDF with residual-versus-time, residual-versus-fitted, a normal Q-Q plot and a holdout prediction chart. It prints train and holdout RMSE.
The practice data was constructed with known linear effects and no random noise. Very small errors are expected and are not a benchmark for a real marketing dataset. The script includes the known seasonal construction to demonstrate a correctly specified teaching model, not a recommended production control set.
Read patterns before reaching for a test
A run of positive residuals means the model repeatedly underpredicted during that period. Investigate missing events, changed business conditions or lag structure. A fan-shaped residual-versus-fitted plot suggests that error scale changes with the outcome level. An extreme point deserves a source-data and business review before removal.
A Q-Q plot concerns the residual distribution under the fitted specification. It does not test every assumption behind a budget recommendation. With nearly deterministic synthetic residuals, even its shape should not be overinterpreted. Use real-data diagnostics in combination with domain knowledge and sensitivity analysis.
Keep the future period genuinely separate
Do not repeatedly alter the model to improve the same holdout and then describe that period as untouched. Use a validation workflow for selection and preserve a later evaluation period where feasible. Fit preprocessing on training data only, and carry past media state into the holdout without using future observations.
The validation guide puts these checks alongside decomposition and experiment agreement. If prediction looks strong but plausible models imply very different channel returns, residual diagnostics have not resolved the identification problem.
Keep the plots, configuration and data hash with the decision record. Explain which pattern triggered a model change and whether that change improved a fresh evaluation, rather than presenting a clean plot without its selection history.
Sources and further practice
This article’s worked example is illustrative, not a measured client result. Use the linked methodology to check the assumptions before applying it.