{"claim": "The problem is real: NASA and JPL rely on an unmeasured heritage-lowers-cost assumption asserted across the autonomy demonstration record but never expressed as a measurable quantity. The Gao and Chien space-autonomy review situates the demonstrations in a longer arc and states the recurring heritage-lowers-cost claim that this study converts into a testable quantity, while reporting no cost figures.", "evidence": [{"source": "Y. Gao, S. Chien, 'Autonomy for Space Robots: Past, Present, and Future,' Current Robotics Reports (2021)", "doi_or_url": "https://doi.org/10.1007/s43154-021-00057-2", "grade": "A"}], "facet": "economics", "chapter": "ch3_literature_review", "subclaim": "real"}
{"claim": "A documented JPL autonomy reuse sequence the build-or-wait logic must speak to exists: AEGIS automated science targeting was first flight-demonstrated on the MER Opportunity rover and then reused onto ChemCam on MSL Curiosity, which the candidate names the central illustrative case for the heritage mechanism. This establishes that a real within-class heritage reuse episode is on the public record, grounding the problem as concrete rather than hypothetical.", "evidence": [{"source": "Estlin et al., AEGIS Automated Science Targeting for the MER Opportunity Rover, ACM TIST (2012)", "doi_or_url": "https://doi.org/10.1145/2168752.2168764", "grade": "A"}, {"source": "Francis et al., AEGIS autonomous targeting for ChemCam on Mars Science Laboratory, Science Robotics (2017)", "doi_or_url": "https://doi.org/10.1126/scirobotics.aan4582", "grade": "A"}], "facet": "economics", "chapter": "ch3_literature_review", "subclaim": "real"}
{"claim": "The problem is material: the only standing maturity instrument, the technology-readiness-level scale, is an ordinal non-monetary maturation index that tracks demonstrated readiness rather than the cost of advancing it, so it cannot substitute for a cost-of-heritage measure. This is exactly the instrument gap the study fills.", "evidence": [{"source": "Olechowski, Eppinger, Joglekar & Tomaschek, 'Technology readiness levels: Shortcomings and improvement opportunities,' Systems Engineering 23(4) (2020)", "doi_or_url": "https://doi.org/10.1002/sys.21533", "grade": "A"}, {"source": "Mankins, 'Technology readiness assessments: A retrospective,' Acta Astronautica 65(9-10) (2009)", "doi_or_url": "https://doi.org/10.1016/j.actaastro.2009.03.058", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "material"}
{"claim": "The problem is material at the program level: autonomy investment and portfolio sequencing turn on the heritage assumption, evidenced by recent high-profile autonomy infusions (the Ingenuity helicopter on Perseverance; autonomous robotics driving Perseverance progress) that the agency continues to fund on a heritage rationale it has not priced.", "evidence": [{"source": "Balaram, Aung & Golombek, 'The Ingenuity Helicopter on the Perseverance Rover,' Space Science Reviews (2021)", "doi_or_url": "https://doi.org/10.1007/s11214-021-00815-w", "grade": "A"}, {"source": "Verma et al., 'Autonomous robotics is driving Perseverance rover's progress on Mars,' Science Robotics (2023)", "doi_or_url": "https://doi.org/10.1126/scirobotics.adi3099", "grade": "A"}], "facet": "governance", "chapter": "ch7_discussion", "subclaim": "material"}
{"claim": "The design addresses a named causal mechanism rather than a generic correlation: a within-class log-log experience curve is the direct measurable trace of Arthur's learning-effect / increasing-returns mechanism, in which cumulative flight heritage leaves codified reusable design patterns and flight software that lower the cost to qualify the next same-class capability.", "evidence": [{"source": "W. B. Arthur, 'Competing Technologies, Increasing Returns, and Lock-In by Historical Events,' The Economic Journal (1989)", "doi_or_url": "https://doi.org/10.2307/2234208", "grade": "A"}], "facet": "mechanism", "chapter": "ch2_theoretical_framework", "subclaim": "mechanism"}
{"claim": "The log-log experience-curve form is a validated statistical basis for predicting technological cost decline, so fitting cost on cumulative experience is the discipline-standard operationalization of learning-by-doing, not an ad hoc functional choice; this grounds the method as the right trace of the mechanism.", "evidence": [{"source": "Nagy, Farmer, Bui & Trancik, 'Statistical Basis for Predicting Technological Progress,' PLOS ONE (2013)", "doi_or_url": "https://doi.org/10.1371/journal.pone.0052669", "grade": "A"}], "facet": "mechanism", "chapter": "ch2_theoretical_framework", "subclaim": "mechanism"}
{"claim": "Forward-only counting blocks look-ahead but does not make cumulative heritage exogenous; when log-cumulative-heritage is a monotone function of within-class demonstration order, heritage and ordinal rank are collinear and the slope carries no identifying variation separable from sequence position. Empirical credibility comes from a design that exhibits as-good-as-random variation in the regressor, not from adding a within-class rank covariate, which risks a bad-control problem since rank is a deterministic function of the same accumulation process.", "evidence": [{"source": "Angrist & Pischke, 'The Credibility Revolution in Empirical Economics,' Journal of Economic Perspectives 24(2):3-30 (2010)", "doi_or_url": "https://doi.org/10.1257/jep.24.2.3", "grade": "A"}, {"source": "Angrist & Pischke, Mostly Harmless Econometrics (Princeton Univ. Press, 2009)", "doi_or_url": "https://doi.org/10.1515/9781400829828", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Without instrument-shifted exogenous variation a regression coefficient does not identify a causal or structural parameter; an instrument recovers a local effect only under monotonicity and the exclusion restriction. Absent such variation in cumulative heritage, beta is a descriptive partial correlation, not a learning rate, so the learning-rate label must be downgraded or an instrument supplied.", "evidence": [{"source": "Imbens & Angrist, 'Identification and Estimation of Local Average Treatment Effects,' Econometrica 62(2):467-475 (1994)", "doi_or_url": "https://doi.org/10.2307/2951620", "grade": "A"}, {"source": "Angrist & Pischke, Mostly Harmless Econometrics (Princeton Univ. Press, 2009)", "doi_or_url": "https://doi.org/10.1515/9781400829828", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "A pooled slope estimated across heterogeneous groups is a weighted average of the underlying two-group comparisons with explicit weights; when one within-class pair (AEGIS Opportunity-to-ChemCam) supplies the only steep cost-decline transition it mechanically carries dominant weight, so the panel slope is that single comparison relabeled. The disciplined output is a Goodman-Bacon-style decomposition reporting each comparison's weight, and if a leave-one-out drop of that pair crosses the accept/reject boundary the correct finding is failure-to-reject.", "evidence": [{"source": "Goodman-Bacon, 'Difference-in-differences with variation in treatment timing,' Journal of Econometrics 225(2):254-277 (2021)", "doi_or_url": "https://doi.org/10.1016/j.jeconom.2021.03.014", "grade": "A"}, {"source": "Angrist & Pischke, Mostly Harmless Econometrics (Princeton Univ. Press, 2009)", "doi_or_url": "https://doi.org/10.1515/9781400829828", "grade": "A"}], "facet": "empirics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "Non-classical measurement error correlated with the regressor does not produce the attenuation-toward-zero of classical error; it can inflate, flip, or manufacture a slope. Because the NICM-class parametric imputation takes heritage- and complexity-correlated inputs, a negative beta produced on the imputed subset is consistent with the cost model reproducing its own heritage-cost coupling. Reliability weighting addresses error variance, not error correlation with the regressor, so the dispositive test is to fit beta on the directly-reported subset versus the imputation-dependent subset.", "evidence": [{"source": "Hyslop & Imbens, 'Bias from Classical and Other Forms of Measurement Error,' NBER Technical Working Paper t0257 (2000)", "doi_or_url": "https://doi.org/10.3386/t0257", "grade": "B"}, {"source": "Hausman et al., 'Using Instrumental Variables to Estimate Models with Mismeasured Regressors,' Handbook of Measurement Error Models (CRC Press)", "doi_or_url": "https://doi.org/10.1201/9781315101279-5", "grade": "B"}, {"source": "Angrist & Pischke, Mostly Harmless Econometrics (Princeton Univ. Press, 2009)", "doi_or_url": "https://doi.org/10.1515/9781400829828", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "CumHeritage is a deterministic forward sum of past selection-into-flight, so the variation surviving the two-way within transformation is still the class's own lagged selection history; the credibility-revolution standard places the first identification obligation on the regressor, and a within / fixed-effects transformation does not manufacture exogeneity. Absent a named exogenous shock (launch-manifest slip, mission-of-opportunity slot, directorate mandate, budget-line start/stop) used as an instrument, beta is a partial correlation of cost on its own lagged selection sequence.", "evidence": [{"source": "Angrist & Pischke, 'The Credibility Revolution in Empirical Economics,' Journal of Economic Perspectives 24(2):3-30 (2010)", "doi_or_url": "https://doi.org/10.1257/jep.24.2.3", "grade": "A"}, {"source": "Imbens & Angrist, 'Identification and Estimation of Local Average Treatment Effects,' Econometrica 62(2):467-475 (1994)", "doi_or_url": "https://doi.org/10.2307/2951620", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Selection-on-observables identifies a causal effect only under the conditional-independence assumption that heritage is as-good-as-random given class, decade and starting TRL. A realistic violator is propositional/codified maturity of a capability's underlying theory, which simultaneously lowers qualification cost and advances flight order, an omitted common cause the CIA forbids. Starting TRL is an ordinal non-monetary index that does not measure codified-propositional depth, and if TRL is itself raised by accumulating heritage it is a bad control whose inclusion biases beta.", "evidence": [{"source": "Angrist & Pischke, Mastering 'Metrics: The Path from Cause to Effect (Princeton Univ. Press, 2014)", "doi_or_url": "https://doi.org/10.2307/j.ctvcm4j72", "grade": "A"}, {"source": "Angrist & Pischke, Mostly Harmless Econometrics (Princeton Univ. Press, 2009)", "doi_or_url": "https://doi.org/10.1515/9781400829828", "grade": "A"}, {"source": "Olechowski et al., 'Technology readiness levels: Shortcomings and improvement opportunities,' Systems Engineering 23(4) (2020); Mankins, Acta Astronautica 65(9-10) (2009)", "doi_or_url": "https://doi.org/10.1002/sys.21533", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Under increasing returns the realized cost path is locked in by historical events and is not necessarily efficient, so a slope estimated off a single locked-in episode (the only documented same-capability second-platform pair) reflects historical selection rather than an efficient learning rate. The candidate's per-observation influence diagnostic is not the named leave-one-class-out (drop-AEGIS) re-estimation needed to test whether beta and the two-of-three decision rule survive removing that pair.", "evidence": [{"source": "W. B. Arthur, 'Competing Technologies, Increasing Returns, and Lock-In by Historical Events,' The Economic Journal (1989)", "doi_or_url": "https://doi.org/10.2307/2234208", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Arthur's lock-in dynamics extend to recombinant technologies, where combinatorial recombination of existing components conditions which technology becomes dominant and reuse paths cross categorical boundaries. The within-class forward-only count therefore scores cross-class software-component transfers as zero heritage; without a component-level reuse graph and a rank-concordance test, the cross-class robustness specification is not demonstrably a check on the same estimand.", "evidence": [{"source": "Zeppini & van den Bergh, 'Competing Recombinant Technologies for Environmental Innovation: Extending Arthur's Model of Lock-In,' Industry and Innovation (2011)", "doi_or_url": "https://doi.org/10.1080/13662716.2011.561031", "grade": "B"}, {"source": "W. B. Arthur, 'Competing Technologies, Increasing Returns, and Lock-In by Historical Events,' The Economic Journal (1989)", "doi_or_url": "https://doi.org/10.2307/2234208", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "In an increasing-returns process the surviving locked-in option need not be efficient, so a cheap second deployment can reflect selection-into-favorable-conditions rather than learning. Because NTRS carries documentation-survivorship bias toward documented successes, the panel contains no abandoned or stalled flew-once-never-reused autonomy class, leaving it silently conditioned on survivors with no falsifying observable to discriminate learning from selection.", "evidence": [{"source": "W. B. Arthur, 'Competing Technologies, Increasing Returns, and Lock-In by Historical Events,' The Economic Journal (1989)", "doi_or_url": "https://doi.org/10.2307/2234208", "grade": "A"}], "facet": "rival", "chapter": "ch7_discussion", "subclaim": "residual_risk"}
{"claim": "Retrospective learning rates are non-constant and correlate with the specific deployment programs that produced them, so a learning rate is a property of a realized program path rather than a transportable constant. This grounds the non-ergodicity threat: ordering the five capability classes by date-of-first-investment and testing whether per-class slope is monotone in funding sequence rather than ex-ante maturity is the falsification the design has not pre-committed.", "evidence": [{"source": "Wei, Smith & Sohn, Energy Policy (2017)", "doi_or_url": "https://doi.org/10.1016/j.enpol.2017.04.035", "grade": "A"}, {"source": "W. B. Arthur, 'Competing Technologies, Increasing Returns, and Lock-In by Historical Events,' The Economic Journal (1989)", "doi_or_url": "https://doi.org/10.2307/2234208", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Novel technologies are assembled combinatorially from existing ones, and the owner of the most-recombined foundational components captures disproportionate downstream value, so the within-class count miscounts the true reusable-knowledge stock. A recombination-stock measure counting components inherited from any prior class would test whether the within-class slope survives true combinatorial heritage; the cFS source in the corpus describes the framework but does not quantify reuse cost.", "evidence": [{"source": "Schrepel, synthesis of Arthur's complexity economics, Journal of Institutional Economics (2024)", "doi_or_url": "https://doi.org/10.1017/S1744137424000067", "grade": "A"}, {"source": "Zeppini & van den Bergh, 'Competing Recombinant Technologies,' Industry and Innovation (2011)", "doi_or_url": "https://doi.org/10.1080/13662716.2011.561031", "grade": "B"}, {"source": "McComas, NASA/GSFC's Flight Software Core Flight System, NASA STI (2013)", "doi_or_url": "http://hdl.handle.net/2060/20130013412", "grade": "B"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Conservation of attractive profits predicts that when one stage of a value chain becomes modular and good-enough, the scarce costly work migrates to an adjacent stage, so a dependent variable measured only as on-orbit qualification NRE is mis-segmented unless paired with a venue-share covariate coding the fraction of the close-the-loop job performed on-board versus delegated to ground. The coding substrate is retrievable in NTRS records documenting onboard-versus-ground task allocation.", "evidence": [{"source": "NASA NTRS, 'Lessons Learned from Autonomous Sciencecraft Experiment' (EO-1)", "doi_or_url": "https://ntrs.nasa.gov/citations/20090007670", "grade": "B"}, {"source": "NASA NTRS, 'Automated Targeting for the MER Rovers' (AEGIS lineage)", "doi_or_url": "https://ntrs.nasa.gov/citations/20150011960", "grade": "B"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "alternatives"}
{"claim": "A real documented sequencing/reuse decision exists against which an on-orbit-only slope could be contrasted with a total-cost-to-field accounting: AEGIS automated targeting for the MER/MSL rovers and EO-1 onboard autonomy with its operations consequences are both on the NTRS record. Because the build-or-wait decision the program faces is total cost-to-field, an on-orbit-only slope that ignores netted ground/downlink burden answers a narrower question than the portfolio faces.", "evidence": [{"source": "NASA NTRS, 'Automated Targeting for the MER Rovers' (AEGIS lineage)", "doi_or_url": "https://ntrs.nasa.gov/citations/20150011960", "grade": "B"}, {"source": "NASA NTRS, 'Onboard Autonomy on the Earth Observing One Mission'", "doi_or_url": "https://ntrs.nasa.gov/citations/20070035964", "grade": "B"}], "facet": "rival", "chapter": "ch7_discussion", "subclaim": "alternatives"}
{"claim": "Dietz's enterprise-ontology distinction between the documental layer and the ontological/coordination layer is the correct frame for the measurement charge: a layer-2 NICM parametric imputation and a layer-1 extracted line item are documental products of how cost happened to be recorded, and agreement at the documental layer does not establish that two figures denote the same essential construct, so ontological commensurability of any matched within-class cost pair must be certified before logging.", "evidence": [{"source": "The Transaction Axiom (Dietz 2006), in Enterprise Ontology", "doi_or_url": "https://doi.org/10.1007/3-540-33149-2_10", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Dietz's transaction-completeness criterion supplies the mechanism distinction the design needs: a transaction is complete only with an explicit production fact and an accept act by an authorized receiver. Mapped onto heritage, 'a prior flight existed' is a documental event whereas 'the successor project ingested and accepted the predecessor's codified artifacts' is the completed handoff, which grounds why counted heritage and transfer-realized heritage can diverge and why beta should be re-fit on a completed-transfer subset.", "evidence": [{"source": "The Transaction Axiom (Dietz 2006), transaction completeness / accept act, in Enterprise Ontology", "doi_or_url": "https://doi.org/10.1007/3-540-33149-2_10", "grade": "A"}], "facet": "mechanism", "chapter": "ch2_theoretical_framework", "subclaim": "mechanism"}
{"claim": "Interoperability at the documental layer does not guarantee interoperability at the coordination layer, so a scalar within-class count is a documental/calendar artifact unless each heritage edge is a coordination-layer transaction; a directed transfer graph whose edges record a named upstream artifact demonstrably re-fielded downstream would test whether CumHeritage is monotone in graph in-degree, and disagreement even on the AEGIS pair would show the regressor measures a class label rather than a heritage transfer.", "evidence": [{"source": "The Transaction Axiom (Dietz 2006), B/I/D layer model, in Enterprise Ontology", "doi_or_url": "https://doi.org/10.1007/3-540-33149-2_10", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Dietz's transaction axiom supplies a five-element completeness test (initiator role, executor role, promise act, production fact, accept act) that operationalizes heritage as a completed transfer rather than mere temporal precedence: producing project equals executor plus production fact, consuming project that re-qualifies equals initiator plus accept act. Re-cutting the panel to count only episodes where all five elements are evidenced, and testing whether the slope survives, distinguishes heritage from calendar adjacency.", "evidence": [{"source": "The Transaction Axiom (Dietz 2006), via crossref + completeness test, in Enterprise Ontology", "doi_or_url": "https://doi.org/10.1007/3-540-33149-2_10", "grade": "A"}], "facet": "mechanism", "chapter": "ch2_theoretical_framework", "subclaim": "mechanism"}
{"claim": "The heritage variable has a record-adjudicable but within-class, forward-only counting rule (credit only when a prior demonstration reached flight before that episode's development start; first-in-class set to 1), and by construction it excludes the reusable codified substrate that crosses class boundaries (cFS components, shared frameworks). The candidate concedes class assignment is a boundary call with moderate confidence, so the rule relocates the unresolved edge case from regime-adjudication to class-adjudication rather than dissolving it, like the functional approach that rises and stalls.", "evidence": [{"source": "McComas, Increasing flight software reuse with OpenSatKit (Core Flight System cFS), IEEE Aerospace (2018)", "doi_or_url": "https://doi.org/10.1109/aero.2018.8396631", "grade": "B"}, {"source": "Gangale, The Functional Approach: Its Rise and Stall, in How High the Sky? (Brill 2018)", "doi_or_url": "https://doi.org/10.1163/9789004366022_013", "grade": "B"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The cross-class reference-frame commensurability challenge is well-posed: EDL hazard-handling qualification is denominated against a one-shot irreversible event budget while onboard-planning qualification accrues against a long revisable operations timeline, so one demonstration is not a common unit across classes. Class fixed effects absorb level differences but do not render the cumulative-heritage axis dimensionally consistent; assuming a single Earth-referenced frame across incommensurable regimes injects a systematic sign-uncertain bias into the pooled slope.", "evidence": [{"source": "Gangale, The Architecture of Time, Part 2: The Darian System for Mars, SAE (2006)", "doi_or_url": "https://doi.org/10.4271/2006-01-2249", "grade": "B"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "A bright-line inclusion rule can be stated for four of the five named episodes because each autonomy function was a per-mission flight experiment with an identifiable owner: count flight-software NRE booked to the mission/experiment that first qualifies the onboard function on its own vehicle, exclude ground-system operations labor, exclude COTS adoption. This rule assigns an identical dollar boundary to Remote Agent, EO-1 ASE, AEGIS-Opportunity, and AEGIS-ChemCam because each has a distinct retrievable per-deployment software-qualification record.", "evidence": [{"source": "Bernard et al., Design of the Remote Agent experiment for spacecraft autonomy, IEEE Aerospace (1998)", "doi_or_url": "https://doi.org/10.1109/aero.1998.687914", "grade": "A"}, {"source": "Chien et al., Using Autonomy Flight Software to Improve Science Return on Earth Observing One, JACIC (2005)", "doi_or_url": "https://doi.org/10.2514/1.12923", "grade": "A"}, {"source": "Estlin et al., AEGIS Automated Science Targeting for the MER Opportunity Rover, ACM TIST (2012)", "doi_or_url": "https://doi.org/10.1145/2168752.2168764", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The boundary case the inclusion rule cannot adjudicate is Ingenuity, whose autonomy inherited cross-cutting flight infrastructure (the M2020/Perseverance flight stack and broader autonomous-robotics heritage) rather than originating as a fresh per-mission NRE, so its qualification dollar boundary is not assignable from a single appropriation line. Unlike the functional approach's definitional incoherence, this is a missing-record problem recoverable by disclosure, not a rule that proliferates with every new case.", "evidence": [{"source": "NASA NTRS, Cross-Cutting Flight Infrastructure Improvements on M2020 (id 20230006993)", "doi_or_url": "https://ntrs.nasa.gov/citations/20230006993", "grade": "B"}, {"source": "Verma et al., Autonomous robotics is driving Perseverance rover's progress on Mars, Science Robotics (2023)", "doi_or_url": "https://doi.org/10.1126/scirobotics.adi3099", "grade": "A"}, {"source": "Gangale, The Functional Approach: Its Rise and Stall, in How High the Sky? (Brill 2018)", "doi_or_url": "https://doi.org/10.1163/9789004366022_013", "grade": "B"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The Core Flight System is a GSFC-originated reusable flight-software framework positioned to amortize flight-software cost across many missions, a fixed-common-cost / public-good substrate; under a consistent venue rule its one-time development NRE should be excluded from each episode's per-mission regressand and only per-mission cFS integration counted. This exclusion does not mechanically manufacture the negative slope only if reported alongside the excluded codification term, since loading cFS NRE on the origin episode would steepen beta while silently excluding it removes the codified-knowledge mechanism, so the slope must be reported both ways.", "evidence": [{"source": "NASA NTRS, Core Flight System (cFS) Training (id 20205000691)", "doi_or_url": "https://ntrs.nasa.gov/citations/20205000691", "grade": "B"}, {"source": "NASA NTRS, Big Software for SmallSats: Adapting cFS to CubeSat Missions (id 20150021070)", "doi_or_url": "https://ntrs.nasa.gov/citations/20150021070", "grade": "B"}], "facet": "identification", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "A statistically insignificant result from a low-power instrument cannot be laundered into substantive evidence for the null; the candidate concedes no diagnostic manufactures power the data lack and reports a zero-spanning interval as inconclusive, but runs no false-negative Monte Carlo, so the demanded power simulation under a true 15-20% learning rate is a legitimate and currently-unmet design requirement before any interval-containing-zero is read as evidence of flat cost.", "evidence": [{"source": "Ziliak & McCloskey, The Cult of Statistical Significance (Univ. of Michigan Press)", "doi_or_url": "https://doi.org/10.3998/mpub.186351", "grade": "A"}], "facet": "empirics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "Substantive (decision) significance is a magnitude question distinct from statistical significance; the candidate frames the contribution as a slope, an interval, an implied rate and an accept-or-reject decision, gesturing at decision-relevance only illustratively while stating no learning rate is estimated. When the interval straddles trivial and decision-changing rates the honest contribution is a loss function over the slope tied to a real build-or-wait threshold, not a significance star on beta.", "evidence": [{"source": "Ziliak & McCloskey, The Cult of Statistical Significance (Univ. of Michigan Press)", "doi_or_url": "https://doi.org/10.3998/mpub.186351", "grade": "A"}], "facet": "economics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "A sign-and-interval-exclusion decision rule certifies direction, not magnitude precision, so it can fire on a wide-but-significant per-doubling interval whose bounds imply opposite sequencing calls, which is a significance star rather than oomph. The candidate applies a failure-to-reject discipline to wide intervals containing zero but states no symmetric magnitude-precision gate on wide intervals that exclude zero, the decision-indeterminate-but-significant cell the critique names.", "evidence": [{"source": "Ziliak & McCloskey, The Cult of Statistical Significance (Univ. of Michigan Press)", "doi_or_url": "https://doi.org/10.3998/mpub.186351", "grade": "A"}], "facet": "empirics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "Economists persuade by metaphor and story, so a conclusion that survives only under one flattering figure (the Wright/Henderson learning curve) is rhetorically rather than evidentially secured. The candidate's own Chapter 7 names the negotiated-budget rival (scale/funding and selection-on-cost) and concedes no instrument fully closes the channel, rating beta's freedom from selection low-to-moderate, so the learning interpretation rests partly on the chosen metaphor rather than on data that independently excludes the rival.", "evidence": [{"source": "McCloskey, The Rhetoric of Economics", "doi_or_url": "https://doi.org/10.2307/jj.36032609", "grade": "A"}], "facet": "rival", "chapter": "ch7_discussion", "subclaim": "residual_risk"}
{"claim": "A measurement whose incumbent target is unquantified and whose own interval is unfitted cannot move a decision-maker off a pre-existing magnitude; the contribution must be judged by whether its interval can overturn the assumed per-doubling reduction NASA banks on by assertion, not by clearing a significance threshold. The candidate must elicit the incumbent's implicit assumed slope in its own units and pre-register whether the fitted interval lies inside, excludes, or straddles it.", "evidence": [{"source": "Ziliak & McCloskey, The Cult of Statistical Significance (Univ. of Michigan Press)", "doi_or_url": "https://doi.org/10.3998/mpub.186351", "grade": "A"}], "facet": "economics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "Mokyr separates the reusable codified (propositional) knowledge stock from the prescriptive artifact and warns that a count of demonstrations is not the extensible self-correcting knowledge that drives cost decline; because cFS is documented as a platform- and project-independent reusable framework, a per-episode codification indicator (reusable component versus bespoke re-implementation) is constructible, and testing whether it diverges from the raw flight count is the instrument-validity gate before any flat-beta inference.", "evidence": [{"source": "Mokyr, The Gifts of Athena (Princeton Univ. Press, 2002)", "doi_or_url": "https://press.princeton.edu/books/paperback/9780691120133/the-gifts-of-athena", "grade": "A"}, {"source": "Stottler et al., On-board Autonomous Hybrid Spacecraft Subsystem Fault Detection, Proc. AMOS Conference (2022)", "doi_or_url": "https://amostech.com/TechnicalPapers/2022/Poster/Stottler_2.pdf", "grade": "C"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "mechanism"}
{"claim": "Mokyr's criterion is that cost falls only where prescriptive technique rests on a widening propositional base, so slope heterogeneity ordered by propositional maturity is the falsifiable prediction, not a secondary heterogeneity check; a pooled within-class beta that ignores the ordering confounds a non-learning class with an average over heterogeneous bases. The named classes are distinct documented autonomy lineages, so an ex-ante maturity rank can be assigned before cost figures are seen.", "evidence": [{"source": "Mokyr, The Gifts of Athena (Princeton Univ. Press, 2002)", "doi_or_url": "https://press.princeton.edu/books/paperback/9780691120133/the-gifts-of-athena", "grade": "A"}, {"source": "Lessons Learned in the Livingstone 2 on Earth Observing One Flight Experiment (2005)", "doi_or_url": "https://doi.org/10.2514/6.2005-7000", "grade": "B"}], "facet": "mechanism", "chapter": "ch2_theoretical_framework", "subclaim": "mechanism"}
{"claim": "Codified knowledge lowers the next demonstration's cost only if it diffuses cheaply to the next team, and the aerospace STI record shows producer-to-user diffusion is slow and tacit, while knowledge spillovers cluster near their institutional sources; a slope estimated on a same-team/same-codebase reuse case plausibly measures tacit co-location rather than a transferable parameter, so the design must contrast overlapping-team reuse against arms-length adoption.", "evidence": [{"source": "NASA/DOD Aerospace Knowledge Diffusion Research Project: US STI Policy, NTRS 19960052732", "doi_or_url": "https://ntrs.nasa.gov/citations/19960052732", "grade": "B"}, {"source": "Jaffe, Trajtenberg & Henderson, Geographic Localization of Knowledge Spillovers (1993)", "doi_or_url": "https://doi.org/10.2307/2118401", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Under Parasuraman, Sheridan and Wickens there is no single autonomy level for a system, only a vector of levels across four stages (information acquisition, information analysis, decision/action selection, action implementation), each with measurable performance consequences, and the meta-analytic record shows human-performance consequences differ sharply by stage. Because the candidate's capability class absorbs baseline difficulty rather than separating stage-level authority, same-class episodes can carry different stage vectors and the cumulative count sums non-comparable units, leaving the slope's unit of analysis unverified.", "evidence": [{"source": "Parasuraman, Sheridan & Wickens, A model for types and levels of human interaction with automation, IEEE Trans. SMC-A (2000)", "doi_or_url": "https://doi.org/10.1109/3468.844354", "grade": "A"}, {"source": "Onnasch, Wickens, Li & Manzey, Human Performance Consequences of Stages and Levels of Automation, Human Factors (2013)", "doi_or_url": "https://doi.org/10.1177/0018720813501549", "grade": "A"}], "facet": "identification", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The fitted specification carries no decision/action-stage authority-level regressor; NICM-class scope drivers are physical and programmatic scope variables that do not encode where on the four-stage vector authority was granted, yet the marginal V&V cost concentrates exactly at high decision/action authority where out-of-the-loop cases must be bounded. Any negative heritage slope is therefore confounded with an unmodeled level effect, and a level effect would be mislabeled as a learning effect unless authority level is entered as a competing regressor.", "evidence": [{"source": "Parasuraman, Sheridan & Wickens, A model for types and levels of human interaction with automation, IEEE Trans. SMC-A (2000)", "doi_or_url": "https://doi.org/10.1109/3468.844354", "grade": "A"}, {"source": "Onnasch, Wickens, Li & Manzey, Human Factors (2013)", "doi_or_url": "https://doi.org/10.1177/0018720813501549", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The EO-1 Autonomous Sciencecraft Experiment is documented as performing onboard autonomous science detection, replanning and retasking, a move toward closed-loop onboard decision authority, yet the design encodes it only as a heritage-count increment with no authority-level delta. For each within-class heritage pair, whether the successor operated at higher decision/action authority than its predecessor is a live levels-of-automation rival to the forward-only counting rule, since a successor at higher authority is a distinct level change rather than a heritage increment.", "evidence": [{"source": "NTRS: The Autonomous Sciencecraft Experiment onboard the EO-1 spacecraft (NASA/JPL)", "doi_or_url": "https://ntrs.nasa.gov/citations/20060043342", "grade": "B"}, {"source": "Parasuraman, Sheridan & Wickens (2000)", "doi_or_url": "https://doi.org/10.1109/3468.844354", "grade": "A"}], "facet": "rival", "chapter": "ch7_discussion", "subclaim": "residual_risk"}
{"claim": "V&V is the dominant qualification cost driver for space autonomous systems and that burden is autonomy-emphasized, so a cost-to-qualify regressand is not authority-neutral. Because the documented episodes occupy different action-stage authority (AutoNav executes maneuvers; AEGIS only selects targets; Remote Agent plans and executes), a higher-authority successor carries structurally heavier V&V at equal heritage, so action-stage authority must be entered as a covariate because it lives inside the cost construct.", "evidence": [{"source": "Cardoso et al., A Review of Verification and Validation for Space Autonomous Systems, Current Robotics Reports (2021)", "doi_or_url": "https://doi.org/10.1007/s43154-021-00058-1", "grade": "A"}, {"source": "Parasuraman, Sheridan & Wickens (2000)", "doi_or_url": "https://doi.org/10.1109/3468.844354", "grade": "A"}, {"source": "Sheridan & Verplank, Human and Computer Control of Undersea Teleoperators, MIT Man-Machine Systems Lab (1978)", "doi_or_url": "https://doi.org/10.21236/ada057655", "grade": "B"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The lumberjack effect means higher automation lowers routine cost while degrading failure recovery, pushing the un-handled exception out of development into operations, so the cost-to-field construct prices only development dollars and omits the expected operational cost of the off-nominal case. The AEGIS ChemCam successor hit the desired material >93 percent of the time versus 24 percent without onboard targeting, an envelope defined by a constrained objective function, so a within-class negative slope is not identified as learning unless the exception-handling envelope is held constant across the pair.", "evidence": [{"source": "Onnasch, Wickens, Li & Manzey, Human Performance Consequences of Stages and Levels of Automation, Human Factors (2013)", "doi_or_url": "https://doi.org/10.1177/0018720813501549", "grade": "A"}, {"source": "NASA NTRS: Results From the First Four Years of AEGIS Autonomous Targeting for ChemCam on MSL (id 20220001538)", "doi_or_url": "https://ntrs.nasa.gov/citations/20220001538", "grade": "B"}], "facet": "rival", "chapter": "ch7_discussion", "subclaim": "residual_risk"}
{"claim": "Under the four-stage decomposition the capability-class unit itself crosses stages: AEGIS target selection is acquisition/analysis/decision-dominant with no autonomous trajectory control, while AutoNav navigation and EDL hazard handling are action-implementation-dominant. Any within-class pair that joins an analysis/decision-stage demonstration with an action-implementation successor sums non-commensurable learning objects, and the within-class fixed effect does not purify the slope; only same-dominant-stage pairs survive a same-stage restriction.", "evidence": [{"source": "Parasuraman, Sheridan & Wickens, A model for types and levels of human interaction with automation, IEEE Trans. SMC-A (2000)", "doi_or_url": "https://doi.org/10.1109/3468.844354", "grade": "A"}, {"source": "NASA NTRS: Autonomous Navigation for Deep Space Missions (id 20130000290); Validation of Deep Space 1 AutoNav (id 20060032521)", "doi_or_url": "https://ntrs.nasa.gov/citations/20130000290", "grade": "B"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}