--- marp: true theme: default paginate: true --- # Cost-Overrun Hazards in Earth-Observing Missions ### A Competing-Risks Model Separating Instrument-Driven from Launch-Driven Slip **Candidate:** JPL_ASTRO_EARTH_08 **COLLEGIUM 1st Battalion** **JPL / NORTH STAR category:** Earth Science Missions **Hall-of-Shoulders anchors:** Fine and Gray; Fogel; Callaway and Sant'Anna; Flyvbjerg (controlled rival) **Date:** 2026-06-15 A question of stewardship: how a program office holds reserve against the delays it can foresee. Defense brief. Design-stage. Every number is illustrative and not yet executed. --- ## Answer first: the contribution (H0 / H1) **H1 (contribution).** For Earth-science missions, instrument-driven slip and launch-driven slip are statistically distinct competing risks, and instrument-driven slip is the dominant subdistribution hazard for missions carrying first-of-kind active sensors but not for passive-radiometer heritage missions. **H0 (null).** The two slip sources are not separable competing risks; either their subdistribution hazards are indistinguishable across the cohort, or the dominance does not differ by sensor archetype. The deliverable is the pre-registered design and the falsification conditions, not estimated coefficients. --- ## Why it matters, stated up front - Reserve is finite and is allocated once, at confirmation, under standing policy. - If slip is one hazard: hold reserve centrally against an undifferentiated pool. - If slip is two hazards with archetype-dependent dominance: steer reserve. - First-of-kind active-sensor missions (JPL's specialty): reserve and gate against instrument maturation, with entry TRL at KDP-B as the leading indicator. - Heritage passive-radiometer continuity missions: reserve against launch-manifest and provider dynamics. - The decision-relevant quantity is the cumulative incidence of each slip cause by archetype, which is exactly what the subdistribution model estimates. --- ## The problem frame - **Current state:** schedule slip, the proximate driver of cost growth, is modeled as a single delay variable; reserve is held against an undifferentiated pool. - **Desired state:** an archetype-specific basis for steering reserve to the dominant first-slip hazard. - **Gap:** three literatures touch the problem; none integrates them. - **Consequence of inaction:** a mission is under-reserved against the wrong hazard, overruns or descopes, and the program office cannot tell which lever it should have pulled. --- ## Slip has two structural origins - **Instrument development:** a science instrument fails environmental test, cannot close its calibration budget, or carries a detector or laser below its assumed maturity. Owner: instrument provider. Lever: TRL gating, instrument reserve. - **Launch-vehicle availability:** manifest congestion, a shared-vehicle anomaly, or the provider's own development slip, exogenous to the spacecraft's readiness. Owner: launch-services program. Lever: vehicle diversity, manifest margin. - Different owners, different physics, different levers. One variable hides all of it. --- ## The gap: three literatures, never joined - **Spacecraft cost-and-schedule:** establishes that low TRL drives slip, but models slip as one continuous outcome (Dubos et al.; Bearden; Kipp et al.). - **Competing-risks survival:** supplies the estimator that separates two cause-specific hazards, but is mature in biostatistics and never applied to mission slip (Fine and Gray; Austin et al.). - **Project-overrun economics:** supplies the rival explanation, estimating optimism, that any separation must survive (Flyvbjerg et al.). - The contribution is the integration, not any single element. --- ## Theoretical framework: three anchors, one logic - **Fine and Gray (the estimator):** the subdistribution hazard and its cumulative incidence function are the decision-relevant, predictive quantity. - **Fogel (structural decomposition):** do not leave slip as an aggregate; decompose it into channels measured against a bounded counterfactual. The cumulative incidence of a removed risk is that counterfactual. - **Callaway and Sant'Anna (heterogeneity):** refuse to pool a heterogeneous hazard; sensor archetype enters as an explicit effect modifier, not a pooled coefficient. --- ## Why the cumulative incidence function, not Kaplan-Meier - Treating a competing event as ordinary right-censoring biases the event probability **upward**, regardless of independence (Austin et al.; Andersen et al.). - For an archetype contrast the bias is doubly dangerous: the two archetypes face different mixtures of the two causes, so the bias differs between them and could manufacture or mask a difference. - The cumulative incidence function, modeled through the subdistribution hazard, is the only defensible basis for the comparison. --- ## The named causal mechanism - **Driver:** a mission carries a first-of-kind active sensor below its assumed maturity at KDP-B. - **Mechanism:** the least-mature sensor technology fails test, cannot close calibration, or runs long, so instrument development becomes the binding path. - **Observable effect:** an instrument-driven first slip raises the instrument-slip cumulative incidence faster than the launch-slip cumulative incidence in the active stratum. - **Consequence:** the committed launch date moves, accruing standing-army and rework cost that overruns the baseline. - For heritage passive-radiometer missions the chain inverts: a direct-rebuild instrument carries little maturation risk, so the first slip is disproportionately launch-side. --- ## Data: four named sources - **CADRe (via ONCE, data-use agreement):** confirmation-baseline and as-flown schedule, milestone dates, mass and power, and the Part A narrative cause of slip. - **NICM / NICM-E:** instrument parameters and the taxonomy that defines the archetype. - **GAO Assessments of Major NASA Projects (public):** the independent, auditor-side cause narrative. - **TechPort (public API):** the entry TRL of the least-mature sensor technology at KDP-B. - The design rests on the join of all four on the mission key. --- ## Unit of analysis and the competing events - **Unit:** the mission-development spell, from KDP-B to the earlier of first slip or launch readiness. One spell per mission. Time in months from KDP-B. - **Three terminal states:** instrument-driven first slip, launch-driven first slip, or administrative censoring at launch. - **Effect modifier:** sensor archetype (first-of-kind active vs passive-radiometer heritage; mixed/ambiguous as a third robustness stratum). - Left truncation is accommodated where record entry post-dates KDP-B. --- ## Cause-coding: two-source reconciliation - The one genuinely hard measurement act: coding each first slip as instrument-driven or launch-driven. - Read the CADRe Part A narrative (project-authored) and the GAO narrative (auditor-authored) for the same project-year. - Code high-confidence only when both name the same dominant cause. - The two sources carry **opposite** directional biases, so a doubly-named cause is robust to either bias alone. - Un-codable events are flagged, held out of the primary contrast, and recoded both ways in sensitivity analysis. --- ## The estimator For event type k (k = instrument, launch): subhazard_k(t | X) = subhazard_k0(t) * exp(X * beta_k) - The subdistribution hazard governs the cumulative incidence function CIF_k(t | X). - The coefficient on the archetype-by-instrument-side interaction is the parameter of interest for H1. - The cause-specific Cox model runs in parallel for the etiologic rate interpretation; Gray's test is the nonparametric check across archetype strata. - Estimation: ridge-penalized partial likelihood, penalty by cross-validation, events-per-variable cap, given the small cohort. --- ## Identification, not an instrument - No randomized assignment of archetype; the contrast compares naturally occurring groups. - Three argued identification claims: 1. The events genuinely compete (one first slip alters the at-risk set for the other). 2. The archetype is a pre-outcome characteristic fixed at KDP-B. 3. The covariates respect temporal ordering, measured at or before KDP-B. - Disciplined by the Callaway-Sant'Anna refusal to pool and the Fogel bounded-counterfactual reading of the CIF. --- ## Threats to validity - **Internal:** cause-coding error (bounded by two-source reconciliation and recoding); endogenous TRL gating (entry TRL measured at KDP-B; descope history carried as a covariate). - **External:** bounded to NASA Earth-observing missions of the era; era confounding of the launch market absorbed by calendar-period effects. - **Construct:** narrative-based slip-cause categories; binary archetype on a continuous novelty spectrum (third stratum guards the dichotomy). - **Statistical-conclusion:** small cohort and an interaction term; power is the central concern. --- ## Analysis plan: a fixed six-step sequence 1. Assemble the cohort and **freeze the cause-coding** before any modeling. 2. Construct the archetype variable and the entry-TRL covariate. 3. Estimate nonparametric cumulative incidence functions; run Gray's test across strata. 4. Estimate cause-specific Cox and Fine-Gray models, without then with the archetype interaction. 5. Test H1 by the interaction sign/significance and the within-stratum dominance contrast. 6. Run the pre-registered robustness battery last. Ordering is itself a method claim: it removes analyst latitude. --- ## Expected results (illustrative, design-stage, not executed) - **Active stratum:** the instrument-slip CIF rises faster to a higher plateau than the launch-slip CIF; the subdistribution hazard ratio on the entry-TRL deficit is above one (illustrative direction, not a number). - **Heritage stratum:** the two CIFs converge; launch-slip dominance reverses or vanishes. - **Formal signal:** Gray's test rejects equality across strata; the archetype-by-TRL interaction is distinguishable from zero with the predicted sign. - Result tables (T6.1 through T6.3) are specified now and left empty by design. --- ## What each outcome would mean - **H1 holds:** archetype-steered reserve and KDP-B TRL gating; a sharper rule than pooled reserve. - **H0 holds (tight intervals):** slip is one hazard after all; central pooled reserve is vindicated, not merely assumed. A result worth having. - **Partial support (separable, no archetype dependence):** still model slip as two hazards, which changes the reserve-modeling architecture even without archetype steering. - Every outcome carries a usable lesson. --- ## Confidence and uncertainty - **High confidence:** the estimator choice, the cumulative-incidence-not-Kaplan-Meier commitment, the external-validity boundary, the value of pre-registration. - **Moderate confidence:** the instrument-side mechanism (well-evidenced); the archetype contrast beyond complexity (tested by the complexity control). - **Lower confidence, flagged:** the launch-side heritage-arm dominance, the thinner half of the evidence base, most likely to return inconclusive on a small cohort. - A non-rejection with wide intervals is reported as **inconclusive**, not as support for H0 (pre-committed). --- ## Pre-registered robustness battery 1. Slip threshold varied one to four months; the pattern must be stable. 2. Un-codable events recoded both ways; a flip is reported as fragile. 3. Ridge penalty and events-per-variable cap varied. 4. Optimism control removed and reintroduced (adjudicates the Flyvbjerg rival). 5. Mixed-sensor third stratum brought in to confirm the binary is not forcing a dichotomy. Power and the minimum detectable effect are computed **before** estimation. --- ## Rival explanations, each with a pre-committed adjudication - **Estimating optimism:** controlled by the reference-class proxy; removed and reintroduced. - **Archetype as a complexity proxy:** the Bearden complexity index is held constant; an effect that vanishes under it is reported as a complexity effect. - **Cause-coding artifact:** two-source reconciliation plus both-way recoding. - **Anticipatory descoping (toward a false null):** entry TRL at KDP-B; descope history as a covariate. - **Era confounding of the launch market:** calendar-period fixed effects; within-era estimation where the sample permits. Three of the five adjudications can take the contribution away. --- ## How the argument is built **The goal:** a defensible, archetype-specific basis for steering scarce reserve at confirmation, delivered as a pre-registered, falsifiable design. | # | The argument turns on | Developed in | |---|----------|-------------| | 1 | The problem is real | Ch 1, 2.1, 3.1 | | 2 | The problem is material | Ch 1.4, 3.2-3.5 | | 3 | The design addresses the mechanism | Ch 2.2-2.6, 5 | | 4 | The design improves on the alternatives | Ch 2.2.3, 5.1, 6.1.3 | | 5 | The residual risk is acceptable | Ch 4.5-4.6, 5.4, 6.3-6.5 | --- ## Residual risk that survives the design - **R1 Cause-coding error:** bounded by two-source reconciliation and both-way recoding; a flip is reported as fragile. - **R2 Small-sample power:** bounded by the pre-estimation minimum-detectable-effect analysis and the wide-interval-inconclusive rule. - **R3 Anticipatory descoping:** bounded by KDP-B-dated TRL and a descope-history covariate; the rival most likely to bias toward H0. - **R4 Thin launch-side arm:** the heritage-arm dominance is the more evidence-thin half, to be confirmed on the cohort. Residual risk is acceptable in the specific sense that it is bounded, stated, and tested, not zero. --- ## Scope, delimitations, and design-stage honesty - Bounded to NASA Earth-observing missions reaching KDP-B, roughly 1990 to present (about thirty to sixty missions). - Not commercial constellations, not planetary or astrophysics, not non-US programs. - The active-versus-passive axis only; not novelty in the bus, avionics, or ground system. - **Architecture traceability is out of scope:** this is an econometric survival study, not an enterprise-architecture deliverable. Decision relevance is carried in prose, as reserve policy. - No coefficient, hazard ratio, or CIF plateau is reported as a finding. --- ## Future-research program 1. Secure the CADRe data-use agreement and ONCE access. 2. Assemble the cohort and freeze the cause-coding. 3. Execute the pre-registered specification exactly as written. 4. Report the falsification decision exactly as the rule defines it, whichever way it breaks. 5. Extend later to other novelty axes and to other mission classes, each as a new pre-registered test. --- ## Contribution restated - The first application of the Fine-Gray competing-risks apparatus to NASA Earth-mission schedule slip. - Slip reframed as two separable hazards whose dominance is conditioned by sensor archetype. - A Fogel-style bounded counterfactual and a Callaway-Sant'Anna heterogeneity discipline, with the Flyvbjerg optimism rival explicitly controlled. - The deliverable is the pre-registered design and its falsification conditions. The design is finished; the execution awaits the data; the rule for reading the data is fixed in advance. --- ## Selected references - Fine and Gray (1999), *J. Am. Stat. Assoc.* doi:10.1080/01621459.1999.10474144 - Austin, Lee and Fine (2016), *Circulation.* doi:10.1161/circulationaha.115.017719 - Andersen, Geskus, de Witte and Putter (2012), *Int. J. Epidemiol.* doi:10.1093/ije/dyr213 - Dubos, Saleh and Braun (2008), *J. Spacecraft and Rockets.* doi:10.2514/1.34947 - Kipp et al. (2012), *IEEE Aerospace.* doi:10.1109/aero.2012.6187407 - Bearden (2004), *IEEE Aerospace.* doi:10.1109/aero.2004.1368229 - Irons, Dwyer and Barsi (2012), *Remote Sens. Environ.* doi:10.1016/j.rse.2011.08.026 - Callaway and Sant'Anna (2021), *J. Econometrics.* doi:10.1016/j.jeconom.2020.12.001 - Leunig (2006), *J. Econ. Hist.* doi:10.1017/s0022050706000283 - Flyvbjerg (2018), *Transp. Res. A.* doi:10.1016/j.tra.2018.07.013 Full 149-entry bibliography in the dissertation back matter. --- # Thank you **Cost-Overrun Hazards in Earth-Observing Missions:** **A Competing-Risks Model Separating Instrument-Driven from Launch-Driven Slip** Candidate JPL_ASTRO_EARTH_08 | COLLEGIUM 1st Battalion | Earth Science Missions The design is finished; the execution awaits the data; the rule for reading the data is fixed in advance. If it helps those who steward these missions hold the right reserve against the right hazard, it will have served its purpose.