--- marp: true theme: default paginate: true --- # Learning Curves for Onboard Autonomy Does each successive autonomous-operations flight demonstration lower the cost-to-field of the next? **Dissertation defense, design-stage artifact** **Candidate:** JPL_AUTONOMY_EDL_01 **Category:** Autonomous Systems and Robotics **Anchors:** W. Brian Arthur, Joel Mokyr **Date:** 2026-06-15 --- ## The contribution A belief that has guided two decades of investment deserves to be measured rather than trusted. This dissertation delivers the first fitted measurement of the experience-curve slope of onboard-autonomy qualification cost on cumulative flight-demonstrated heritage. - A single number, or a credible failure to find one. - It converts the NASA and JPL heritage-lowers-cost assumption from an assertion into a planning parameter. - The deliverable is the instrument, not a particular sign of the slope. Design-stage posture: no coefficient is fitted on the full dataset. Every expected result is illustrative and labeled as such. --- ## The hypotheses (H0 and H1) The contribution reduces to one falsifiable claim about a single coefficient, beta. - **H0 (null):** Per-episode autonomy development cost is flat with respect to the cumulative count of prior flight demonstrations within a capability class. Beta is statistically indistinguishable from zero. - **H1 (alternative):** Per-episode cost declines along a log-log experience curve as cumulative flight-demonstrated heritage accumulates. Beta is negative and statistically significant after controlling for capability class and decade. Falsifiable: a zero, positive, or not-distinguishable-from-zero slope rejects H1 and supports the flat-cost null. --- ## The problem today NASA and JPL invest in onboard autonomy and treat each flown capability as a heritage asset assumed to lower the next demonstration's cost. - Deep Space 1 Remote Agent (1999): first AI agent to control a NASA spacecraft. - EO-1 Autonomous Sciencecraft (2003 onward): onboard detection and replanning. - AEGIS (2010, 2016): autonomous science targeting on Opportunity and Curiosity. - Mars 2020: autonomous navigation on Perseverance; Ingenuity powered flight. The assumption drives portfolio sequencing but has never been stated as a measurable quantity. --- ## What is missing, and what it costs - **What is needed:** a cost-anchored, falsifiable learning rate per autonomy capability class, read as a number with an honest interval. - **What is absent:** no published study fits a Wright-type log-log model of capability-class autonomy qualification cost on cumulative heritage. - **What inaction costs:** build-or-wait and sequencing decisions stay made by assertion, with no quantitative complement to the ordinal, non-monetary TRL scale. --- ## The literature gap: two unjoined literatures - **Experience-curve economics.** The Wright log-log law is well validated for manufactured unit cost (Thompson 2012; Nagy et al. 2013; Farmer and Lafond 2016). It is about hardware unit cost, not the non-recurring engineering of qualifying a software-intensive capability. - **Spacecraft autonomy and TRL.** Documents demonstrations and maturity (Gao and Chien 2021; Mankins 2009; Olechowski et al. 2020). It describes heritage qualitatively, not as a cost curve. The contribution is the join: bring the validated instrument to the documented chronology. --- ## Theoretical framework: the experience curve The Wright experience curve states that unit cost falls by a constant percentage each time cumulative output doubles. In logarithms the relationship is linear, and its slope encodes a single learning rate. - The log-log form is the best-validated functional form across many technologies (Nagy et al. 2013; Lafond et al. 2018). - Causal content comes from organizational learning-by-doing, not static scale (Thompson 2012; von Hippel and Tyre 1995). - A time control is mandatory, because cumulative output and calendar time co-move (Wright versus Moore). --- ## Anchor 1: W. Brian Arthur Increasing returns and learning effects. - Learning effects are one of the self-reinforcing mechanisms of increasing returns: each use lowers the cost of the next. This is exactly what an experience curve measures (Arthur 1989). - This predicts the H1 direction: a negative slope is the expected signature when learning effects dominate. - Caution: increasing-returns systems are path-dependent and non-ergodic, so the realized slope reflects which classes received early investment (Arthur 1994, 2021). It bounds external validity. --- ## Anchor 2: Joel Mokyr Propositional versus prescriptive useful knowledge. - A technique becomes cheap to reproduce only when its propositional base is mature and its prescriptive knowledge is codified and reusable; otherwise it stagnates (Mokyr 2002). - Prediction: the cost-decline slope is steeper for capability classes whose knowledge is codified, for example through a shared flight-software substrate, and flatter where each project re-implements from scratch. - This is a testable moderator: codification predicts effect heterogeneity across classes. --- ## Data: four named sources One panel, assembled from four non-substitutable sources. - **NASA TechPort** project records and TRL histories: inventory, taxonomy, maturation covariate, timing. - **NTRS** autonomy reports: Remote Agent, EO-1, AEGIS, Ingenuity heritage chronology. - **NICM-class** parametric estimates: normalized development cost and cross-check (Stahl et al. 2010; Mrozinski et al. 2011). - **Published literature:** capability scope and the documented heritage links between episodes. --- ## Unit of analysis and capability classes The unit is the capability-class development episode: one mission or demonstration fielding one defined autonomy capability, assigned to exactly one primary class. - Onboard planning and scheduling. - Autonomous science target selection. - Autonomous surface or in-flight navigation. - Autonomous fault detection, isolation, and recovery. - Autonomous entry-descent-and-landing hazard handling. Multi-capability episodes are split, or assigned to the dominant class and flagged. --- ## The dependent variable: three-layer normalization Public autonomy-cost figures are reported inconsistently, so the dependent variable is built and graded, not lifted. - **Layer one:** extract the autonomy-specific non-recurring engineering portion where reported. Highest reliability. - **Layer two:** impute via a NICM-class parametric estimate of scope where it is not. Lower reliability, flagged. - **Layer three:** deflate every figure to constant-year dollars. Each observation carries a reliability flag, used to down-weight the weakest figures. The error is made explicit and auditable, not removed. --- ## The independent variable: forward-only heritage Cumulative within-class flight-demonstrated heritage: the count of prior same-class demonstrations that reached flight before the episode's development start. - Forward-only rule: heritage that arrives after development began cannot have lowered that cost, so it is not counted. - First-in-class is set to one before logging, anchoring the curve origin. - The rule is conservative: it undercounts heritage and biases the slope toward zero, making any rejection of the null more credible. --- ## Design and identification Two-way fixed-effects log-log regression: ln(Cost) = alpha + beta * ln(CumHeritage) + class FE + decade FE + error - **beta** is the experience-curve slope and the single parameter of interest. - **Class fixed effects** remove intrinsic cost and difficulty differences between classes. - **Decade fixed effects** remove economy-wide and tooling time trends. - Identification: beta is identified off within-class, within-decade variation in accumulated heritage, and only that. --- ## Why this estimator Ordinary least squares on the log-log form, not a non-linear or Bayesian estimator. - The log-log form is validated for prediction across technologies (Nagy et al. 2013; Rubin et al. 2015). - It returns a single interpretable slope; the implied learning rate is one minus two to the power beta. - It conserves the scarce degrees of freedom of a small panel. - Fixed effects, not random effects: the classes and decades are the specific, non-sampled categories of interest. --- ## Pre-registered robustness specifications Three, fixed before estimation, each targeting one named threat. 1. **Add the TRL maturation covariate** to separate heritage from the effect of starting at higher maturity. 2. **Broaden heritage to cross-class software-component reuse**, testing the downward bias from shared components. 3. **Inverse-imputation-error weighting**, so the least reliable cost figures contribute least. Decision rule: reject H0 only if beta is negative with its interval excluding zero in the baseline and in at least two of the three robustness specifications. --- ## Threats to validity - **Internal:** within-decade computing trend; selection of cheap episodes after heritage exists. Addressed by decade fixed effects, the maturation covariate, the cross-class specification, and the conservative counting rule. - **External:** generalizes to NASA and JPL deep-space and planetary autonomy in the window, not to commercial or terrestrial autonomy. - **Construct:** cost-accounting boundaries differ; the heritage count is a coarse proxy for reusable knowledge. - **Statistical-conclusion:** small sample, wide intervals; pre-registration controls specification search. --- ## Analysis plan: five steps 1. Build the episode inventory (TechPort and NTRS). 2. Construct cumulative within-class heritage counts (forward-only). 3. Assemble normalized cost (extraction, NICM imputation, deflation, with reliability flags). 4. Fit the baseline and the three pre-registered robustness specifications. 5. Report beta, its small-sample-robust interval, the implied learning rate, and the H0 decision, with diagnostic plots by class. --- ## Pre-analysis checks Two mandatory checks gate interpretation, both run unconditionally. - **Fixed-effects feasibility check:** any class or decade with a single observation contributes nothing to the within estimator; the effective sample is reported after singleton cells are removed. - **Influence diagnostic:** the slope is re-estimated with and without each high-leverage episode; a slope that depends on one episode is reported as not robust. These convert foreseeable small-panel failure modes into reported properties of the result. --- ## Expected results (design-stage, illustrative only) No coefficient has been fitted. The following are illustrations of how a result would be read, not estimates. - **If learning effects dominate:** beta would be a moderate negative number, an implied per-doubling cost reduction in the low tens of percent, steeper for classes with mature codified knowledge. - **If within-class heritage does not lower cost on its own:** beta would be indistinguishable from zero, which would not reject H0. Both outcomes are reportable and decision-relevant. --- ## Confidence and uncertainty Confidence is stated explicitly and tied to evidence. - **Direction of the mechanism:** moderate to high (Arthur's learning effect; broad Wright-form validation). - **Magnitude of the slope:** low by construction, because none is fitted; the magnitude is what the study exists to measure. - **Small panel:** the dominant statistical limit; a wide interval that contains zero is read as inconclusive, not as evidence for the null. - **Path dependence:** any slope is a path-contingent average, not a universal constant. --- ## How the argument fits together The whole study converts the heritage-lowers-cost assumption into a falsifiable, decision-relevant measurement. - The assumption runs unmeasured across the demonstration record, yet portfolio sequencing turns on it while the TRL scale tracks maturity rather than cost. - A within-class experience curve measures Arthur's learning effect directly, using the best-validated functional form and fixed effects that separate that effect from confounds. - What can go wrong is named and bounded in advance: pre-registration, influence diagnostics, conservative counting, and honest framing keep the inference within the evidence. **Residual risk:** small N, dependent-variable measurement error, within-decade trend, path dependence, each named and bounded. --- ## What a falsifying result looks like The contribution is falsifiable, and the falsification condition is fixed in advance. - **Outright falsification:** beta is zero or positive, or its interval contains zero across the baseline and robustness specifications. The flat-cost null is reported as the finding. - **Confound-driven attenuation:** an apparent slope collapses once the TRL covariate or the cross-class measure is added, indicating maturity or shared components, not within-class flight heritage. The study specifies in advance the conditions under which it declares its own claim unsupported. --- ## Implications under each branch - **If H1 holds:** the slope is a planning parameter. It makes the build-or-wait decision answerable by calculation, and supplies a cost-anchored complement to the TRL scale. - **If H0 holds:** the portfolio rationale shifts from heritage-driven cost reduction to capability value. The agency stops budgeting demonstrations as if each buys down the next. The most policy-relevant secondary finding is effect heterogeneity across classes, the Mokyr codification moderator. --- ## Contribution restated A single defensible number, or a defensible failure to find one, on a question the agency currently answers by assertion. - The instrument is the deliverable; its value does not depend on the sign of the slope. - Arthur names the mechanism and bounds external validity; Mokyr predicts where the slope steepens. - The design is complete; the construction steps to execution are named and reproducible. --- ## From design to execution The remaining work is construction, not a new research question. - Assemble the episode panel from TechPort and NTRS. - Execute the three-layer cost normalization; retain the imputation log. - Count heritage under the forward-only rule; retain the counting log. - Fit and report under the pre-registered rule; release code and panel. - Highest-value strengthening: small-N and few-clusters inference; Core Flight System reuse economics. --- ## Defense questions anticipated - How do you separate the autonomy learning effect from the general decline in software and computing cost? - Is the within-class count or the cross-class reusable-component stock the right experience measure? - With tens of episodes, what power do you have to reject the flat-cost null? - How do you treat episodes that field more than one capability class? - What single fitted result would make you accept H0? --- ## Selected references - Arthur, W. B. (1989, 1994, 2021). Increasing returns, path dependence, complexity economics. - Thompson, P. (2012). Unit cost and cumulative quantity; organizational learning-by-doing. - Nagy, B., Farmer, J. D., Bui, Q. M., Trancik, J. E. (2013). Statistical basis for predicting technological progress. - Farmer, J. D., Lafond, F. (2016); Lafond, F., et al. (2018). Predictability of technological progress. - Gao, Y., Chien, S. (2021). Autonomy for space robots. - Mankins, J. C. (2009); Olechowski, A. L., et al. (2020). Technology readiness levels. - Stahl, H. P., et al. (2010); Mrozinski, J., et al. (2011). Parametric space-cost models. Full list: 126 entries in the dissertation reference section, every DOI or URL resolvable. --- ## Thank you **Candidate:** JPL_AUTONOMY_EDL_01 Autonomous Systems and Robotics A defensible number, or a defensible failure to find one. Questions welcome.