Economic Consulting · The Causal Confounder
Causal Confounder Guide - DiD Damages
Estimate damages with difference-in-differences, then work out which of three facts from discovery biases the estimate.
What is a natural experiment?
Establishing that one thing causes another is harder than showing that they moved together. A confounder is a variable that is correlated with both the supposed cause and the outcome, and its presence can make a naive before-and-after or treated-versus-untreated comparison look like evidence of a causal effect when the true story is that some third factor - the economy, the season, an unrelated event - was driving both sides of the comparison at once.
The cleanest way to rule out confounding is a randomized controlled trial: assign who gets the treatment purely by chance, and on average nothing else differs systematically between the treated and untreated groups, so any difference in outcomes can be attributed to the treatment itself. Much of the time, though, randomizing is impossible or unethical - you cannot randomly assign which regions get a new policy or which firms face a price change - so researchers instead look for a natural experiment: a real-world situation where circumstance created something close to random or as-if-random assignment, without anyone designing it that way.
Difference-in-differences is one of the standard tools built for exactly this setting. It compares the change over time in a treated group to the change over the same period in a comparable untreated group, using the untreated group's movement as a stand-in for what the treated group would have done anyway. The method rests on a testable assumption, parallel trends: before treatment, the two groups need to have been moving together, which is what justifies using one as the counterfactual for the other.
The design is only as good as what does and does not cancel in that comparison. A factor that affects both groups equally - a broad economic shift, a seasonal pattern - shows up in both differences and cancels out, which is the whole point of having a control group. A fixed, constant difference between the groups also cancels, because it sits in both the before and after figures for that group equally. What survives, and biases the estimate, is a shock that hits the treated group alone and falls inside the post-treatment window - nothing in the untreated group's data offsets it, so it gets absorbed into the estimate as if the treatment itself caused it.
This logic - work out what a comparison assumes away, then check whether anything violates that assumption - is the core skill of applied causal inference, and it shows up anywhere someone wants to attribute an outcome to a cause from observational data: policy evaluation, litigation damages, and empirical research across economics and the social sciences generally.
How it works
You are the testifying economist in a damages case. Northgate Retail (the treated region, where the defendant's pricing policy took effect) sold 520k units a month before the policy and 400k after. Southvale (the untouched control) went from 500k to 460k over the same window. Pre-period months are shown for both regions so you can check parallel trends: both were rising at the same rate before the policy, so the design is admissible.
Phase 1 asks for the naive difference-in-differences estimate: the treated group's change minus the control group's change, in thousands of units. Phase 2 presents three facts from discovery - a national fuel surcharge that hit every region, a warehouse fire that hit only Northgate during the post period, and the fact that Northgate has always sold about 20k more than Southvale - and asks which single one biases the DiD estimate.
Phase 3 gives you the fire's documented impact (30k units) and asks for the adjusted causal effect: the naive DiD with the treated-only shock stripped out. Numeric answers are accepted within 3 (thousand units) and each question locks after one check; you can restart the case at the end.
How scoring works
Phase 1: +2 points for the naive DiD within a tolerance of 3 (thousands of units). Phase 2: +3 points for picking the one confounder that biases the estimate. Phase 3: +3 points for the adjusted effect within the same tolerance of 3. Maximum 8 per run.
The DiD questions record skill attempts under regression; the confounder pick records under selection-bias.
There is exactly one correct pick in phase 2: the Northgate warehouse fire, because it is the only shock that is both treated-group-only and inside the post window.
DiD is a subtraction with a built-in counterfactual
The estimator is DiD = (T_post - T_pre) - (C_post - C_pre). The control's movement is the counterfactual: what the treated group would have done anyway. Anything that moved both regions - seasonality, a recession, an industry-wide cost shock - lands in both differences and cancels.
Sign discipline matters here because everything is negative. Northgate fell 120; Southvale fell 40. The estimate is -120 - (-40) = -80, not -160. Write both group changes down first, then subtract, rather than trying to net four numbers in your head.
Check the parallel-trends panel: DiD is only credible if the two groups were moving together before treatment. The game shows both regions rising 6 per month pre-policy with a trend gap of zero, which is what licenses using Southvale as Northgate's counterfactual at all.
Only treated-only, post-period shocks bias the estimate
Two things cancel automatically in the subtraction, and both are offered as decoys. A fixed level gap between the groups (Northgate always sold 20k more) sits in that group's pre and post figures equally, so it vanishes in the first difference - DiD never required the groups to be identical, only to move in parallel. A shock hitting both groups (the national fuel surcharge) lands in the control difference too, so it vanishes in the second difference.
What cannot cancel is a shock that lands on the treated group alone inside the post window. The warehouse fire hit only Northgate, after the policy started. Nothing in Southvale's numbers offsets it, so the full 30k gets absorbed into the estimate as if the defendant's policy caused it.
This gives you a two-question test you can apply to any proposed confounder: did it hit only the treated group, and did it hit inside the post period? Both yes means bias; either no means it cancels.
Adjust, then own the smaller number
Once a treated-only post shock is documented, the fix is arithmetic: adjusted effect = DiD - shock. Here the fire's impact is -30, so the causal effect of the policy is -80 - (-30) = -50. The defensible damages figure is 50k units, not 80k - the naive number overstated the claim by 60%.
The professional instinct being trained: the plaintiff's expert is happy with -80, and your value is finding the alternative explanation before the other side's expert does. A damages number survives cross-examination only if the treated-only shocks have already been carved out of it.
Note which direction the adjustment goes. The fire made Northgate's fall look worse, so removing it shrinks the estimated damage. If a treated-only post shock had helped the treated group, stripping it out would enlarge the estimate instead.
A worked example
Step 1, the naive estimate. Northgate: 400 - 520 = -120. Southvale: 460 - 500 = -40. DiD = -120 - (-40) = -80. Enter -80 (tolerance 3). Northgate fell 120, but 40 of that was happening everywhere, so the plaintiff's number attributes an 80k monthly decline to the policy.
Step 2, the confounder. Fuel surcharge: hit both regions, cancels in the control difference - not it. Persistent 20k baseline gap: sits in both Northgate's pre and post, cancels in the first difference - not it. Warehouse fire: Northgate only, during the post period - nothing offsets it. Pick the fire.
Step 3, the adjustment. Logistics filings put the fire at -30k units. Adjusted effect = -80 - (-30) = -50. Enter -50. The defensible damages figure is 50k units a month, and the naive estimate overstated the claim by 60%.
Common mistakes
• Adding the two declines instead of differencing them. -120 - (-40) is -80; getting -160 means you subtracted in the wrong direction.
• Picking the baseline gap as the biasing confounder. "The regions were never identical" feels like a flaw, but a fixed level difference cancels in the first difference. DiD needs parallel trends, not equal levels.
• Picking the common shock. Anything that hits both groups is exactly what the control difference exists to remove.
• Adjusting in the wrong direction in phase 3. The shock is -30; subtracting it means -80 - (-30) = -50, a smaller loss. Landing on -110 means you added the fire's damage to the policy's instead of removing it.
• Ignoring the pre-trend evidence entirely. The game hands you the parallel-trends check; in an interview, failing to mention it before quoting a DiD number is the flag graders look for.
Why interviews test this
Difference-in-differences is the workhorse of economic consulting - damages estimation, antitrust impact studies, labor and policy litigation - and interviews test exactly the sequence this game runs: set up the 2x2, state the parallel-trends assumption, compute the estimate, then hunt for what else could explain it. The confounder step is where candidates separate: anyone can subtract two numbers, but naming which facts bias the design and which cancel shows you understand the identification behind the formula.
Expect follow-ups of the form "opposing counsel says the regions were always different - does that kill your estimate?" The trained answer: no, level differences cancel; what would kill it is a treated-only shock inside the post window, and here is how I would test for one.
Play The Causal Confounder on Outcry
The Causal Confounder is free to play on Outcry, in the Economic Consulting section, and every run is timed and scored. Read the guide once, then play The Causal Confounder and check your score against the strategy above. The guide book has a written guide for every other game, and the practice hub and mock assessments put the same skills under a real screen's clock.