--- marp: true theme: default paginate: true --- # Fault-Management Maturity and Mission-Anomaly Survival A hazard model of safe-mode entries and recovery outcomes Doctoral defense brief Candidate JPL_AUTONOMY_EDL_04 COLLEGIUM 1st Battalion Category: Autonomous Systems and Robotics Hall-of-Shoulders anchors: Fogel, Taleb 2026-06-15 --- ## The answer first: the contribution - The duty of stewardship over a distant spacecraft does not end at launch, and the question of whether the vehicle can help itself in a fault deserves a measured answer rather than an intuition. - This dissertation delivers the first population-level, conditional hazard estimate of whether onboard fault-management autonomy lowers a spacecraft's risk of mission-ending loss once it has entered a fault state. - The deliverable is one number: the hazard ratio exp(beta_1) on the autonomy variable, with its confidence interval and an explicit accept-or-reject decision on the null. - H0 (null): autonomy has no effect on the hazard of mission-ending loss conditional on fault entry. The coefficient is zero; the hazard ratio is one. - H1 (alternative): higher autonomy lowers that hazard, controlling for complexity, distance, and age. The coefficient is negative; the hazard ratio is below one. - It converts the NASA and JPL "autonomy buys survivability" intuition into an architecture-trade parameter. --- ## Why it is falsifiable - The claim is a single estimable quantity, not a narrative. - H1 is refuted by a non-negative coefficient, by a confidence interval that includes one, or by a sign reversal once confounders are added. - A hazard ratio above one with an interval excluding one refutes H1 in the strongest way: higher autonomy associated with worse conditional survival, a substantively important negative finding. - The decision rule is fixed in advance and the refutation conditions are pre-registered. --- ## The problem: safe mode is routine, and sometimes terminal - Robotic spacecraft routinely enter safe mode when onboard monitors detect a fault. Safe mode protects the vehicle and interrupts the mission. - Fault entry is not exotic. Galileo safed itself four times; CloudSat's battery anomaly threatened its constellation; Dawn's reaction-wheel faults recurred across its tour. - In some fraction of cases the entry into a fault state precedes permanent loss. - The decision-relevant question is not whether faults occur (they do) but what happens next, and whether the spacecraft's own capability changes the outcome. --- ## The gap: two literatures that do not meet - Reliability statistics (Castet and Saleh and others) model the unconditional time from launch to hardware failure across hundreds of spacecraft. They do not condition on fault entry and do not use autonomy as a covariate. - Fault-management engineering builds and flies autonomy (the Deep Space One Remote Agent closed the detection-isolation-recovery loop onboard) and argues from architecture and case narrative. - No study estimates the effect of autonomy on the hazard of mission-ending loss conditional on fault entry, with confounder control. - That conditional, population-level estimate sits in the seam between the two literatures and is the object of this work. --- ## Why conditional, and why a hazard model - Conditioning on fault entry isolates the manager's real question: given that my spacecraft will get into trouble, does onboard autonomy help it get out. - An unconditional comparison is dominated by who encounters faults at all, not by post-fault survival. - A hazard model, not a logistic regression on outcome, is the natural estimator: it uses the dwell time in the fault state, handles right-censoring of still-operating missions, and accommodates recurrent episodes clustered within a spacecraft. - Mission-ending loss is rare; the hazard framework tolerates a small event count better than any mean-based approach. --- ## Theoretical framework: the Fogelian anchor - Fogel's cliometric discipline: a claim that "outcome could not have occurred without factor" is an unmeasured counterfactual until stated quantitatively, embedded in an explicit counterfactual, and exposed to falsification. - The hazard ratio on autonomy is the social-saving analogue: a single number that supports or refutes the autonomy claim. - Conditioning operationalizes the counterfactual: the within-stratum comparison of comparable fault episodes that differ only in autonomy level. - The most likely residual confounder, program quality, is signed, not assumed away, which bounds the estimate as a ceiling on autonomy's benefit. --- ## Theoretical framework: the Talebian anchor - Taleb's tail-risk discipline governs the dependent variable. Mission-ending loss is a rare, heavy-tailed event; the historical record undersamples the tail and sample means understate exposure. - Consequence one: prefer a hazard formulation that handles censoring and small event counts over any mean-based estimator. - Consequence two: precaution over point optimization. Autonomy's value may concentrate in the worst episodes, so a pre-specified tail subgroup is built in. - The non-naive precautionary principle forbids treating any single point estimate as license to optimize against the most-likely case while ignoring the catastrophic case. --- ## The named causal mechanism - Driver: higher onboard fault-management autonomy. - Mechanism: onboard detection, isolation, and recovery execute without waiting for a ground command cycle. - Observable effect: faster, light-time-independent resolution of the fault episode before it becomes terminal. - Operational consequence: a hazard ratio exp(beta_1) below one on the autonomy variable. - Strategic implication: an evidence-based architecture-trade parameter for deep-space autonomy investment. - The chain is a claim about a process, not a bare correlation, and it is most legible at large distance where the ground loop is slowest. --- ## Data: four named sources - NTRS anomaly and lessons-learned reports: fault narratives and pre-flight autonomy coding. - GAO project assessments: independent corroboration of losses and program complexity and cost context. - JPL mission anomaly and incident-surprise-anomaly records: fault-entry and recovery timing. - TechPort technology-readiness classifications: the exogenous anchor for the autonomy score. - The sources are data, not bibliography; an episode record is assembled by linkage across all four. --- ## Unit of analysis and variables - Unit: the fault episode, a discrete entry into a safe mode or comparable fault state. A mission contributes multiple episodes, clustered on the spacecraft. - Time-to-event: from fault entry to confirmed recovery (censored) or mission-ending loss (event); still-operating missions are right-censored. - Treatment, autonomy: an ordinal three-level score (ground-loop-dependent; onboard detection with limited response; onboard detection-isolation-recovery), read per level. - Controls: complexity index (subsystem and instrument count, cost class), distance regime (time-dependent), spacecraft age. --- ## The autonomy score: the delicate construct - It is the treatment, so its measurement quality decides whether the study can answer its question. - Built in three documented passes: a TechPort technology-readiness anchor; a detection-isolation-recovery placement read from pre-flight design documentation; an independent second-reader re-coding with rubric adjudication. - Scored from pre-flight material so a mission-ending loss cannot retroactively contaminate it. This defeats reverse coding by construction. - Deliberately ordinal and coarse, because the documentation does not support finer distinctions; coarseness attenuates toward the null, which is conservative. --- ## Research design: the estimator - Cox proportional-hazards model, estimated by partial likelihood, with robust variance clustered on the spacecraft. - It dominates a logistic-on-outcome model (which discards dwell time) and any mean-based model (which mishandles censoring and an unreliable mean in a heavy tail). - The baseline hazard is left unparameterized; the quantity of interest is the hazard ratio exp(beta_1). - Counting-process large-sample theory licenses the recurrent-event, time-dependent-covariate, and robust-variance extensions. --- ## Identification - The autonomy coefficient is identified off a within-stratum comparison of comparable episodes that differ in autonomy level: the operationalized Fogelian counterfactual. - Identifying assumption: conditional independence of autonomy and potential survival given complexity, distance, and age. Observational, not randomized. - Instrumental variables were considered and rejected: no candidate (era, budget) satisfies the exclusion restriction without affecting survival through other channels. - The principal residual confounder, program quality, is signed: a surviving protective estimate is an upper bound on autonomy's benefit. --- ## Base specification and robustness - Base model: the four-covariate Cox model, clustered robust variance, scaled-Schoenfeld proportional-hazards test with a fixed respecification rule, distance entered as a time-dependent covariate. - Five pre-registered robustness specifications: competing risks (recovery versus loss), shared frailty, mission-class stratification, Firth-penalized partial likelihood, and a reduced autonomy-plus-distance model. - The decision rule requires the sign of the autonomy effect to be stable across all five, not merely significant in the base model. --- ## Threats to validity - Internal: unobserved confounding (test rigor, operations experience), bounded by signed-bias reasoning and the frailty term; reverse coding, defeated by pre-flight scoring. - External: the sample over-represents complex, well-documented NASA and JPL missions; generalization is bounded to that class, and the tail is undersampled. - Construct: the coarse ordinal autonomy score and the constructed complexity index attenuate toward the null; both are validated against a second reader and the TRL anchor. - Statistical-conclusion: rare events mean limited power and wide intervals; pre-registration and Firth penalization address it. --- ## Analysis plan: six steps - One: assemble the episode dataset from the four named sources per the coding protocol. - Two: construct the survival object with fault entry as origin and mission-ending loss as the event. - Three: fit the base Cox model with clustered robust variance. - Four: test proportional hazards with scaled Schoenfeld residuals and respecify where it fails. - Five: fit the five robustness specifications. - Six: conduct inference on H0 versus H1 via the likelihood-ratio and Wald tests, with the likelihood-ratio test privileged under scarce events. --- ## The fixed decision rule - H0 is rejected for H1 if and only if the estimated hazard ratio is below one, its 95 percent confidence interval excludes one in the pre-registered base specification, and the sign is stable across all five robustness specifications. - A confidence interval that includes one fails to reject H0. - A point estimate below one with an interval spanning one is not read as support; suggestive-but-imprecise is not confirmatory. - A sign reversal once controls are added shows the raw association was confounding, not effect. --- ## Expected results (design-stage, illustrative only) - Design-stage plan. No hazard ratio is fitted on the assembled population; the result tables are specified and deliberately left unpopulated. - An H1-consistent shape would look like a hazard ratio near 0.6 with an interval of 0.4 to 0.9: roughly a forty percent lower loss hazard per autonomy level. - A fail-to-reject shape would look like a hazard ratio near 0.95 with an interval of 0.7 to 1.3. - These two numbers are decision-rule illustrations that bracket the interpretive range. They are not estimates. --- ## The tail subgroup, specified in advance - A subgroup restricted to the hardest episodes, at the greatest distance and shortest reaction time, tests whether autonomy's benefit concentrates in the tail even if the pooled estimate is modest or null. - The mechanism predicts a larger effect where the ground loop is slowest and onboard recovery has the most room to matter. - It is one comparison, named in advance for a theorized reason, reported with its weakened inferential status, the disciplined opposite of a post-hoc subgroup trawl. - The most likely nuanced outcome is a pooled null with a tail effect, itself a contribution. --- ## Confidence and uncertainty - Confidence in the design is high: the estimator, identification, and threat-responses rest on an established statistical literature. - Confidence in any directional result is deliberately withheld at the design stage; withholding it is what design-stage honesty requires. - The analysis is feasibility-limited, not precision-rich: effective sample size is governed by the event count, not the episode count. - Every null will be reported with the minimum detectable hazard ratio, so "no effect found" is never misread as "no effect exists." --- ## Why the argument holds - The goal: the first population-level, conditional hazard estimate of whether autonomy lowers mission-ending-loss risk after fault entry, an architecture-trade parameter or a credible failure to find one. - The phenomenon is genuine: safe-mode entry is routine and sometimes terminal, as Galileo, CloudSat, GLAS, and Dawn each show. - The stakes are concrete: autonomy investment and deep-space trades turn on this assumption, at real software, verification, and authority-shift cost. - The design reaches the mechanism: a conditional Cox hazard measures Fogel's counterfactual contrast directly. --- ## Why the argument holds, and what risk remains - It earns its place against the alternatives: the Cox model uses dwell time, handles censoring and clustering, tolerates a small event count; instrumental variables were considered and set aside. - What residual risk remains is contained: rare events, measurement error, and unobserved confounding are bounded by Firth penalization, second-reader coding, pre-flight scoring, and signed-bias reasoning. - Acknowledged, not resolved: unobserved confounding is bounded as an upper bound; rare events limit power; silent-episode under-recording has an uncertain sign. - The contribution is a measurement, not a system; the hazard ratio touches an architecture trade only as a conceptual input, stated in plain prose. --- ## Contribution restated: symmetric value - Under H1 supported: the estimate prices autonomy maturity in the currency of survival and supplies an architecture-trade parameter, most actionable for deep-space JPL missions. - Under H0 not rejected: the estimate relocates the autonomy case onto operational, cadence, and phase-specific grounds (entry, descent, and landing, where no ground loop exists), with a calibrated null. - Under the nuanced branch: a pooled null with a tail effect says autonomy's benefit concentrates where the average cannot see it. - In every branch the work converts an intuition into a measurement with stated uncertainty. The value does not depend on the data being kind. --- ## From design to execution - Stage one: code the episode inventory from NTRS, GAO, and JPL records, calibrated on documented post-mortems such as Dawn. - Stage two: anchor and second-reader-score the autonomy variable, hardening the TechPort TRL provenance. - Stage three: build the controls, fit the base and robustness models, test proportional hazards, pre-registered throughout. - Stage four: realize the power the event count allows, report the minimum detectable hazard ratio, and adjudicate H0 against the fixed rule. --- ## References - Cox (1972), Andersen and Gill (1982), Therneau and Grambsch (2000): the survival-analysis apparatus. - Castet and Saleh and collaborators (2009 through 2013): the validated spacecraft reliability-statistics tradition, imported and redirected. - Bernard (1999), Muscettola (1997), Pell (1998), Gao (2021): the autonomy demonstrations and surveys. - Fogel (1964), Leunig (2010): cliometric discipline. Taleb and colleagues (2014), Cirillo and Taleb (2020): tail-risk discipline. - Full numbered list of 134 references, all with clickable DOI or resolvable URL, appears in the dissertation backmatter. --- ## Defense questions to anticipate - How do you keep the autonomy score from being contaminated by the outcome it is meant to predict? - What is the minimum detectable hazard ratio given the realized event count? - If the pooled estimate is null but the tail subgroup is not, what do you conclude? - How do you defend conditioning, rather than an instrument, as the identification strategy? - Which single unobserved confounder most threatens the estimate, and how do you bound its direction?