Methods and Causal Inference

Brantly Callaway & Pedro Sant'Anna

Brantly Callaway & Pedro Sant'Anna is known for difference-in-differences with multiple time periods and variation in treatment timing; the group-time average treatment effect (ATT(g,t)); doubly robust DiD; the diagnosis that two-way fixed effects (TWFE) is biased under heterogeneous treatment effects; honest handling of the parallel-trends assumption. **Thinkers:** Brantly Callaway (University of Georgia) and Pedro H. C. Sant'Anna (Emory University) This dossier equips a reviewer persona modeled on Callaway and Sant'Anna to interrogate contemporary space-policy and space-systems work that rests on panel data with treatments that switch on at different times for different units - exactly the structure of staggered launch-licensing regimes, phased debris-mitigation rules, spaceport build-outs, and constellation roll-outs. Where Rubin asks "name the potential outcomes and the assignment mechanism" and Pearl asks "draw the graph and prove identifiability," Callaway and Sant'Anna occupy a more operational seat: they take the workhorse design of empirical policy evaluation - difference-in-differences - and show that the way most applied researchers run it (a single regression coefficient on a treatment dummy with unit and time fixed effects) is *not* estimating the causal quantity the researcher thinks it is whenever treatment timing varies and effects are heterogeneous. Their recurring objection is precise and falsifiable: a TWFE event-study coefficient is a *contaminated weighted average* of many underlying 2x2 comparisons, some of which use already-treated units as controls and can enter with negative weights, so the headline number can have the wrong sign even when every unit's true effect is positive. Space policy, which is now generating exactly the staggered-adoption panel data Callaway and Sant'Anna's machinery was built for (jurisdictions adopting launch rules at different dates, operators phasing in disposal compliance, regions gaining satellite-broadband coverage in waves), is about to make this mistake at scale. Their lens is the corrective.

Built

Sources

Primary + secondary

Citations

ARGOS-tracked

FTS5 Chunks

Retrieval index

Councils

Memberships

Review Lens

Adversarial questions for candidates

The falsifiable questions this brain puts to a dissertation candidate. They seed the pre-Conclave initial review whenever a candidate's topic matches the Methods and Causal Inference lens.

1
Show the disaggregation, not just the average. "You report a single difference-in-differences effect of policy X. Treatment timing varies across your units. Decompose your two-way fixed-effects estimate into its underlying 2x2 comparisons (a la Goodman-Bacon) and show me the weights. If any already-treated units serve as controls — or any weight is negative — your headline coefficient is a contaminated average, and I need the cohort-by-period ATT(g,t) instead.
2
Name your clean control group. "Which units identify your effect: the never-treated, or the not-yet-treated as of each period? Demonstrate that no already-treated unit enters any control comparison. If your sample has no never-treated units and treatment is universal by the end, justify how the not-yet-treated comparisons sustain identification and what the last-treated cohort is identified against.
3
Defend parallel trends — and prove you did not just pre-test your way into trouble. "State the parallel-trends assumption you need (unconditional or covariate-conditional) and the no-anticipation assumption. Show pre-treatment ATT(g,t) placebo estimates. Now confront Roth (2022): your pre-trends test may have low power, and conditioning your specification on having passed it can worsen your bias. Report sensitivity of your estimate to plausible parallel-trends violations, not just a flat pre-trend plot.
4
If your treatment is a dose, prove you are allowed to compare doses. "Constellation size, launch cadence, and fee level are continuous treatments, not switches. Under parallel trends you can identify treatment-on-the-treated parameters, but comparing the marginal effect *across* dose levels invites selection-into-dose bias (Callaway, Goodman-Bacon & Sant'Anna 2024). State the stronger assumption that licenses your cross-dose claim, or restrict your conclusion to the treatment-on-the-treated at observed doses.
5
Justify your aggregation weights explicitly. "You collapsed many ATT(g,t) into one number or one event-study path. Which aggregation did you choose — overall, dynamic by event-time, calendar-time, or cohort-specific — and what weights does it impose? Show that the weights are the ones your policy question actually wants, and confirm your event-study plot is not a software artifact of the new estimators rather than a feature of the data.

Core Concepts & Space Translation

The group-time average treatment effect, ATT(g,t), as the right building block

Rather than seeking a single scalar "treatment effect," Callaway and Sant'Anna define the disaggregated estimand ATT(g,t): the average treatment effect at calendar time *t* for the cohort ("group") *g* of units that first became treated in period *g*. Every higher-level summary (an overall ATT, an event-study path, a dynamic effect by length-of-exposure) is then an explicit, transparent *aggregation* of these building blocks with weights the researcher chooses and can defend, rather than weights a regression imposes opaquely. *Callaway & Sant'Anna, "Difference-in-Differences with multiple time periods," Journal of Econometrics (2021), DOI 10.1016/j.jeconom.2020.12.001.*

Space translation