# Interrogation mind-map: JPL_AUTONOMY_EDL_01 Nodes: 123 | questions: 48 | grounded claims: 42 | gaps: 33 ## Questions - **[identification]** Forward-only counting blocks look-ahead but does not make cumulative heritage exogenous: first-of-kind episodes carry CumHeritage=1 AND the hard-thing-first cost premium, later episodes carry both higher heritage and the cheaper-because-attempted-later selection, and this confounder varies WITHIN class along the regressor so class/decade fixed effects cannot remove it. If you reorder each class by its own within-class demonstration index (1st/2nd/3rd) and regress log cost on that ordinal rank, does rank absorb the heritage coefficient? If ordinal position explains cost as well as log-heritage, beta is a relabeling of how far into the hard-thing-first sequence we are, not a learning rate, and no instrument has been offered to break the tie. (raised by angrist_pischke) - **[empirics]** Your Section 5.2 influence diagnostic is by construction the falsification test: you commit to dropping each influential episode and to calling a slope that moves on one episode 'not robust.' With tens of observations across five classes, the AEGIS Opportunity-to-ChemCam pair is the only documented within-class heritage reuse you name, so that class's slope is identified off effectively one cost-decline transition. What fraction of total identifying variation in beta comes from the single steepest within-class pair, and if dropping that one pair moves beta across your own accept/reject threshold for H0, have you not already conceded by your pre-registered rule that the contribution is descriptive rather than estimated? (raised by angrist_pischke) - **[measurement]** Your dependent variable is partly imputed by a NICM-class parametric model whose inputs (scope, complexity, plausibly heritage/maturity) correlate with cumulative heritage, so for episodes without separately disclosed autonomy NRE the cost is generated by a model that may already encode a heritage-cost relationship: non-classical measurement error correlated with your regressor, which can manufacture a negative slope rather than merely attenuate it. If you fit beta separately on the directly-reported (non-imputed) subset versus the imputation-dependent subset, do the two slopes agree in sign and magnitude, and if the negative slope appears only where NICM imputation supplied the cost, what licenses reading it as learning rather than as the cost model reproducing its own assumptions? (raised by angrist_pischke) - **[identification]** Beta is read off within-class, within-decade variation in cumulative heritage, but no source of as-good-as-random variation in that heritage is named. CumHeritage is the deterministic forward sum of which prior episodes were selected into flight, plausibly driven by the very qualification cost the dependent variable measures. Name the concrete exogenous shock that moves a class's cumulative heritage independently of its qualification cost, and exhibit from the TechPort/NTRS record at least one episode whose flight order was set by a budget line, mission-of-opportunity slot, or directorate mandate unrelated to its autonomy cost. Absent one, beta is a partial correlation between an outcome and its own lagged selection history, not a learning-curve slope. (raised by angrist_pischke) - **[identification]** You list reverse pathways as an internal-validity threat and claim the TRL maturation covariate and a cross-class robustness specification 'address' it. Adding a covariate is selection-on-observables, valid only if heritage is conditionally independent of the cost shock given class, decade, and TRL. State the exact conditional-independence assumption your design needs for beta to be causal; write down one realistic unobserved driver of both early demonstration and low cost (e.g. a capability whose underlying planning/estimation theory was already mature, so it was both flown first AND cheap to qualify); and show from the heritage chronology that this driver is fully captured by your covariates. If not, the TRL covariate is at best an incomplete control and at worst a bad control downstream of the same propositional maturity that drives flight order. (raised by angrist_pischke) - **[empirics]** Within a class your episodes are time-ordered, so cumulative within-class heritage rises near-monotonically with calendar year; decade fixed effects and ln(CumHeritage) therefore compete to absorb the same monotone secular trend (the Wright-vs-Moore observational-equivalence problem your own Section 2.1 cites Nagy et al. for). Report, from the assembled panel, the within-class residual correlation between ln(CumHeritage) and a continuous calendar-year control after partialling out the decade dummies, and the VIF or first-stage partial-R-squared of heritage net of time. If within class and decade heritage has almost no independent variation from calendar time, beta is not separately identified from a generic time trend and the design cannot distinguish learning-by-doing from contemporaneous computing/tooling improvements you already concede are a rival. (raised by angrist_pischke) - **[identification]** After capability-class and decade fixed effects, enumerate the surviving within-class, within-decade pairs that actually move ln(CumHeritage). If the AEGIS Opportunity-to-Curiosity/ChemCam reuse is the only documented same-capability-on-a-second-platform pair, is beta identified off more than a single increasing-returns episode, and would the decision rule (negative beta in baseline plus two of three robustness specs) survive a leave-one-class-out test that drops AEGIS? (raised by brian_arthur) - **[measurement]** Under combinatorial technology evolution the capability classes are themselves forming (AutoNav, Remote Agent planning, AEGIS targeting share recombined flight-software components, with cFS as the codified substrate), so the within-class count miscounts the true reusable-knowledge stock. Can you operationalize a component-level heritage graph (edges = documented flight-software/design-pattern reuse traceable in NTRS and cFS records) and show the within-class count and the cross-class component count rank the episodes the same way? If not, the within-class slope measures an analyst-imposed partition, and the cross-class spec is a different, possibly contradictory estimand rather than a robustness check. (raised by brian_arthur) - **[rival]** Arthur 1989's result is that the realized cost path of a locked-in technology reflects historical sequence, not efficiency: a cheap second deployment can occur because the class was selected into early, well-funded, reuse-friendly programs. What observable discriminates a genuine learning curve from a selection-into-favorable-conditions path? Specifically, do your data contain any abandoned or stalled autonomy class -- a capability that flew once and was never reused despite a candidate successor mission -- so the panel is not silently conditioned on survivors whose increasing returns already won? (raised by brian_arthur) - **[identification]** Under non-ergodicity the pooled within-class beta may be an artifact of early-investment sequence rather than a transferable learning rate. Order the five capability classes by date-of-first-investment and test whether per-class slope is monotone in that order; if slope rank tracks funding-sequence rank rather than ex-ante propositional-maturity rank, beta measures which basin JPL fell into and the build-or-wait use collapses. Does TechPort's start-date and funding record support that ordering test, and is it pre-committed? (raised by brian_arthur) - **[measurement]** The increasing-returns mechanism is combinatorial: a new capability is recombined from components of prior ones across classes, so the within-class heritage count scores cross-class transfers as zero heritage and can bias or invert the within-class slope. Can you build, from the cFS dependency graph and NTRS reuse statements, a recombination-stock measure counting components inherited from ANY prior class, and show whether the within-class slope survives once true combinatorial heritage replaces the in-class count? (raised by brian_arthur) - **[mechanism]** Arthur's fourth self-reinforcement mechanism, adaptive expectations, predicts cost falls because the institution expects reusability and pre-commits team, codebase, and review heritage before new code is written; if that channel dominates, the decline is keyed to institutional continuity (same JPL group, same flight-software lineage), not to the abstract count of prior in-class flights. Can you code each episode for flight-software-lineage and team continuity from authorship and cFS provenance, enter it alongside CumHeritage, and report whether the heritage slope survives the continuity control? (raised by brian_arthur) - **[measurement]** Build a rival within-episode cross-class reuse-stock index (flight software, design patterns, V&V artifacts inherited from any prior episode regardless of capability class, from the cFS dependency record and lessons-learned), regress cost on it, and pre-commit to reporting the reuse-stock slope side by side with the within-class heritage-count slope rather than burying reuse in a robustness check. If reuse-stock returns a significant negative slope while heritage-count is flat, the headline is measuring the wrong curve. (raised by christensen_c) - **[identification]** Re-cut the panel by job-defined clusters (close-the-perception-action-loop-without-ground-in-the-loop) instead of artifact-based capability classes, drawing the fixed effects around the operational job rather than the product category. What observable in TechPort and NTRS scope records distinguishes a job-based heritage link from an artifact-class one, and does the estimated slope survive, strengthen, or collapse when fixed effects are job-based? (raised by christensen_c) - **[economics]** Add an adoption-friction observable (schedule reserve consumed, descope events, or review-board findings attributable to the autonomy element per episode) and test whether it falls with heritage. A falling engineering cost-to-qualify is consistent with flat or rising organizational cost-to-adopt if each host mission's resource-allocation process rationally starves the low-margin autonomy line item; if qualification cost falls but adoption friction does not, the measured slope answers a build-or-wait question the program office does not face. Can the TechPort and lessons-learned record supply an adoption-friction measure, and does the heritage effect appear on it? (raised by christensen_c) - **[measurement]** Can you construct, per episode, an on-board execution share, the fraction of the operational job (close the perception-action loop without ground-in-the-loop latency) actually performed on-board versus delegated to ground/Earth/COTS, coded from documented CONOPS, ops-team staffing, ground-software scope, and downlink dependence in TechPort/NTRS and the DS1/EO-1/AEGIS lessons-learned? (raised by christensen_c) - **[identification]** Will you pre-commit to the discriminating regression, add per-episode on-board execution share alongside ln(CumHeritage), report whether the negative autonomy-NRE slope survives controlling for venue share, and report a slope that collapses or reverses once venue share enters as displacement (substitution artifact), not as a weakened learning effect? (raised by christensen_c) - **[rival]** Can you exhibit one real documented Autonomous Systems and Robotics sequencing decision (e.g., AEGIS Opportunity-to-ChemCam reuse, or an Ingenuity-class buy-or-defer) where your on-orbit-only slope would have recommended differently than a total-cost-to-field accounting that nets the displaced ground burden, drawing the ground-cost side from retrievable ops-team and downlink records? (raised by christensen_c) - **[measurement]** Three-layer regressand (layer-1 extracted autonomy NRE, layer-2 NICM-class parametric imputation, layer-3 deflated total) mixes documental artifacts with the essential construct. Certify ontological commensurability of a within-class layer-1-vs-layer-2 cost pair from source records, and rule out that layer-2 parametric imputation endogenously re-expresses the experience curve being fit (contaminating beta). (raised by dietz) - **[identification]** Capability-class unit is an analyst category, not an essential one; software components and design patterns admittedly cross classes. Map actual reusable software/verification artifact flow between episodes from NTRS heritage chronology and cFS provenance, show the fraction of real reuse that crosses a class boundary, and decide whether the within-class estimand is even supportable or whether the essential unit is the reusable-component lineage. (raised by dietz) - **[mechanism]** Heritage is a scalar count of prior in-class flight demonstrations, but the Mokyr/Arthur mechanism is a completed transfer: codified knowledge actually ingested and accepted by the successor (a completed handoff with an authorized receiver), not merely a prior flight existing. Operationalize transfer-realized heritage from TechPort lineage links and project docs and re-fit; show whether beta survives when count is replaced by realized transfer. (raised by dietz) - **[measurement]** Can the candidate exhibit, for the five episodes (Remote Agent, ASE/EO-1, AEGIS-Opportunity, AEGIS-ChemCam, Ingenuity/AutoNav), the actual completed reuse acts as a directed transfer graph (edge only where a NAMED upstream artifact was demonstrably re-fielded downstream), and show the scalar within-class CumHeritage count is monotone in the in-degree of that graph? If the graph and count disagree even on the AEGIS pair, the regressor measures a class label, not a heritage transfer. (raised by dietz) - **[identification]** Can the candidate certify, observation by observation, that the cost figure and the heritage edge it is regressed against share the SAME accounting boundary, i.e. that the reusing episode's measured NRE EXCLUDES the sunk qualification cost the producing episode already paid for the reused artifact, and produce a reconciliation table showing whose ledger the reused component's qualification cost lands on? If the producer's qualification cost silently reappears in the consumer's NRE, the negative slope is a double-counting artifact, not a learning curve. (raised by dietz) - **[mechanism]** Can the candidate operationalize 'heritage' as a completed transaction with all five elements present (identifiable producing project = executor delivering a documented reusable artifact = production fact, and a consuming project = initiator that accepted and re-qualified it = accept act), re-cut the panel to count ONLY episodes where all five are evidenced in TechPort/NTRS (dropping mere temporal precedence), and test whether the slope survives? If the slope is detectable on the forward-only count but vanishes on the completed-transfer subset, the estimand is calendar adjacency, not heritage. (raised by dietz) - **[governance]** Build the artifact that proves relevance: exhibit one real, documented Autonomous Systems and Robotics sequencing decision (AEGIS reuse, AutoNav infusion, NICM cost-cap call) that the fitted slope, had it existed, would have flipped or quantitatively bounded under the institution's stated constraints (mission cadence, directorate budget lines, decadal priorities, appropriations windows). If none, the number is interesting but not actionable. (raised by gangale) - **[measurement]** State the operational definition of the heritage unit with no functional ambiguity: can TechPort+NTRS adjudicate, by a stated rule and not analyst judgment, which prior demonstrations enter each episode's heritage count? Show the boundary case the rule cannot resolve (AEGIS Opportunity-to-Curiosity reuse vs a cFS component shared across navigation and planning), and say why the rule will not 'rise and stall' like a functional definition. (raised by gangale) - **[identification]** Before pooling capability classes, specify the cross-class normalization that maps a unit of EDL heritage (one-shot irreversible event budget) onto a unit of onboard-planning heritage (revisable operations timeline) so the cumulative-heritage axis is dimensionally consistent, and demonstrate on the panel that the fitted slope is invariant to that conversion. If a single Earth-referenced cost frame is assumed across classes, identify the systematic distortion injected into beta. (raised by gangale) - **[measurement]** State the venue/budget-line boundary of the autonomy-qualification regressand as a bright-line rule (not analyst judgment), applied identically to Remote Agent, EO-1 ASE, AEGIS-Opportunity, AEGIS-ChemCam, and Ingenuity; then exhibit the single boundary case the rule cannot adjudicate and explain why it does not 'rise and stall' like the functional approach to space delimitation. (raised by gangale) - **[identification]** By the stated rule, whose autonomy NRE is the cFS/OpenSatKit reusable substrate's development cost: loaded on the first flying episode, distributed across downstream users, or excluded as fixed common cost? Show from a retrievable cFS/GSFC funding or CONOPS record which appropriation line paid for the substrate, or mark a gap; and demonstrate the assignment does not mechanically manufacture the negative slope. (raised by gangale) - **[rival]** Demonstrate a unit of autonomy-qualification cost is denominated in one time-stable frame across the panel, or concede it is not, given NASA relocated where autonomy demand is paid (per-mission flight NRE -> ground operations -> shared reusable substrate funded outside missions); identify a real Autonomous Systems and Robotics sequencing (build-or-wait) decision the fitted slope would misinform if the denominator silently shifted frames, and state the conversion factor the design assumes equals one. (raised by gangale) - **[empirics]** Run a pre-registered power/false-negative Monte Carlo: draw synthetic panels from a TRUE 15-20% learning rate with your actual class/decade structure and report the share of draws whose interval still contains zero. A wide CI that fails to reject H0 may be a blind instrument, not flat cost. (raised by mccloskey) - **[economics]** State ex ante the decision-relevant magnitude (learning rate in constant-year dollars vs a representative cost-capped mission) that would flip a JPL build-or-wait call, then show from NICM-class figures the smallest slope that changes a real portfolio decision. If the CI spans trivial and decision-changing rates, report the loss function over the slope, not a star on beta. (raised by mccloskey) - **[measurement]** Split the panel on your own reliability flag: fit the slope on the audited subset where a real separately-reported autonomy NRE figure exists, and compare it to the imputed-heavy fit. If the negative slope lives only in imputed observations, the curve is an artifact of cost normalization, not heritage. (raised by mccloskey) - **[empirics]** Construct the joint distribution of (significant, decision-determinate) outcomes from actual class/decade degrees of freedom and NICM-class imputation error: what fraction of draws that reject H0 still leave the implied per-doubling cost reduction too unbounded for a program office to act on? If most significant rejections are decision-indeterminate, the contribution is a significance star, not oomph. (raised by mccloskey) - **[rival]** Swap the metaphor: code the same episodes not as units on a Wright/Henderson learning curve but as discrete, non-fungible engineering events whose cost is set by mission politics, funding windfalls, and PI continuity. Does any observable in the TechPort+NTRS corpus discriminate the experience-curve story from the rival 'sequence of bespoke negotiated budgets' story, or would both fit the same points equally? If observationally equivalent, the negative slope persuades by the chosen metaphor, not the data. (raised by mccloskey) - **[economics]** State the prior assumption your number replaces, in its own units. NASA justifies autonomy heritage investment 'by assertion'; what implicit per-doubling cost reduction does that assertion already bank on, and from the NICM-class figures is your fitted interval narrow enough to confirm or overturn that specific assumed magnitude? A measurement that cannot distinguish its own estimate from the assumption it claims to replace has added rigor's costume without rigor's payoff. (raised by mccloskey) - **[measurement]** Before fitting, construct a direct codification indicator from your own corpus (NTRS + cFS docs) coding each episode as documented reusable component vs bespoke re-implementation, and show empirically whether it diverges from the raw flight count across your panel. If they move together the count is a valid instrument; if they diverge a flat beta tells us nothing about learning. (raised by mokyr) - **[mechanism]** Assign each capability class an ex-ante propositional-maturity rank, fixed before any cost figure is seen (from Remote Agent V&V, EO-1/ASE, AEGIS second-platform reuse, the cFS substrate), then test whether class-specific slopes order themselves by that rank. A pooled within-class beta cannot distinguish 'autonomy does not learn' from 'I averaged a steep curve over a mature base with a flat one over an immature base'. (raised by mokyr) - **[identification]** Measure from project documentation whether the expected cost decline tracks team and institutional continuity (same JPL group, shared codebase, personnel overlap) rather than the abstract flight count, since AEGIS Opportunity-to-Curiosity reuse by an overlapping team is exactly where tacit transfer is most likely. If a steep slope appears only where people and code carry over, you measured tacit-knowledge co-location, not a transferable experience-curve parameter a portfolio office can apply to a new team. (raised by mokyr) - **[empirics]** State now, before estimation, the minimum effective within-class sample your decision rule requires to detect a codification x heritage interaction at your pre-committed significance level; and if the panel cannot support that interaction, will you withdraw the codification-moderator claim entirely rather than report it as a suggestive non-finding? Power analysis on the actual TechPort/NTRS episode count settles this. (raised by mokyr) - **[measurement]** Demonstrate that the NICM imputation inputs are orthogonal to your codification indicator; if a codified class is mechanically assigned a lower imputed cost by the cost model, your steeper-slope-where-codified finding is an artifact of the regressand's construction, not evidence of cumulative learning. The NICM input variables and your codification coding rule, cross-tabulated, settle this. (raised by mokyr) - **[mechanism]** Can you exhibit, from the NTRS lessons-learned record and cFS dependency documentation, at least one within-class successor episode whose development team had no personnel or codebase overlap with the predecessor and yet drew on the predecessor's codified components? If every documented cost reduction tracks intra-JPL personnel continuity, you are measuring a labor-hoarding effect inside one institution, not a Mokyrian access-cost decline, and the parameter does not transfer across centers or contractors. (raised by mokyr) - **[identification]** The IV is a scalar cumulative count of prior demonstrations within a 'capability class,' but the Parasuraman-Sheridan-Wickens model holds a capability is a vector of authority levels across four stages (information acquisition, analysis, decision/action selection, action implementation). Code each panel episode on the 1-10 level for each stage and test whether same-class episodes share the same stage-level vector; if within-class variance is large, the cumulative-heritage count sums non-comparable units. What does the coded vector data show? (raised by sheridan_verplank) - **[measurement]** You normalize cost by capability scope on a NICM-class basis, but the dominant cost driver of qualifying autonomy is the AUTHORITY LEVEL at the decision/action-implementation stages: a suggest-only system is cheap to verify, a decide-and-act-without-veto system incurs the bulk of cost in V&V of un-handled-exception and out-of-the-loop cases. Add decision/action-stage authority level as a regressor alongside cumulative heritage. Does the heritage coefficient survive? (raised by sheridan_verplank) - **[rival]** For each within-class heritage pair you call your cleanest evidence (e.g., EO-1 ASE bounded experiment to operational closed-loop retasking; AEGIS Opportunity to ChemCam; AutoNav ceding a larger drive fraction), extract from the documentation whether the successor operated at a HIGHER decision/action authority level than its predecessor. If cost rose or stayed flat precisely where authority rose, that is a rival your forward-only counting rule does not address. What does the authority-level delta show relative to the cost delta? (raised by sheridan_verplank) - **[measurement]** Qualification cost is the dependent variable, but V&V effort is a monotone function of the action-stage authority the system is licensed to exercise (a Level-7+ action-implementation autonomy committing an irreversible burn/landing demands categorically heavier V&V than a Level-2/3 advisory). Build an authority-weighted qualification-burden index per episode from the documented V&V record (Remote Agent V&V, AEGIS, AutoNav) and show the heritage slope is identified holding action-stage authority FIXED, not confounded by authority entangled inside the regressand itself. (raised by sheridan_verplank) - **[rival]** Onnasch et al. (2013) establish the lumberjack effect: higher automation lowers routine cost while degrading failure recovery, pushing the un-handled exception out of development into operations. The cost-to-field construct prices only development/qualification dollars and excludes the expected operational cost of the off-nominal case the autonomy does not handle. For the AEGIS Opportunity-to-ChemCam pair and the Remote Agent episode, state whether any documented descope of off-nominal coverage, deferred fault cases, or operations-phase anomaly attributable to the autonomy occurred, and show the falling slope is not partly successors qualifying a NARROWER exception envelope more cheaply rather than the same capability more cheaply. (raised by sheridan_verplank) - **[identification]** The five capability classes are artifact categories, but under PSW each episode automates a specific functional stage, and episodes inside one class can automate different stages (autonomous science target selection is largely information-analysis+decision; autonomous EDL hazard handling is decision+action-implementation). Re-tag every episode with its dominant PSW stage, report the cross-tabulation of capability class against dominant stage, and state which within-class heritage pairs survive a same-stage restriction - because pooling an acquisition/analysis-stage demonstration with an action-implementation-stage successor sums non-commensurable learning objects and the within-class fixed effect does not purify the slope. (raised by sheridan_verplank) ## Grounded claims - **[identification]** The objection is correct on Angrist-Pischke grounds and cannot be answered by the candidate's own design as documented. When log-cumulative-heritage is, by construction, a monotone function of within-class demonstration order, heritage and ordinal rank are collinear and the slope carries no identifying variation separable from sequence position. The credibility-revolution position is explicit that this class of problem is NOT solved by adding another covariate (a within-class rank control or an extra fixed effect) but only by a research design that exhibits as-good-as-random variation in the regressor: the analyst's first obligation is to identify the source of exogenous variation and the channel through which it moves the outcome. Absent an instrument that shifts heritage without shifting hard-thing-first sequence position (and satisfies an exclusion restriction), the coefficient is descriptive. Note further that conditioning on within-class rank to 'control for' sequence position risks a bad-control problem, since rank is itself a deterministic function of the same accumulation process; the diagnostic does not rescue identification, it relocates the assumption. The honest disposition is: report beta as a description of the cost-vs-sequence pattern and either supply an instrument / quasi-experiment or downgrade the learning-rate interpretation. - Angrist & Pischke, 'The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics,' Journal of Economic Perspectives 24(2):3-30 (2010) | https://doi.org/10.1257/jep.24.2.3 | grade A - Angrist & Pischke, Mostly Harmless Econometrics: An Empiricist's Companion (Princeton Univ. Press, 2009) | https://doi.org/10.1515/9781400829828 | grade A - Imbens & Angrist, 'Identification and Estimation of Local Average Treatment Effects,' Econometrica 62(2):467-475 (1994) | https://doi.org/10.2307/2951620 | grade A - **[empirics]** The objection holds and is decisive under the candidate's own pre-registered rule. A regression slope estimated across heterogeneous groups is a weighted average of the underlying two-group / two-period comparisons, so when one within-class pair (the AEGIS Opportunity-to-ChemCam transition) supplies the only steep cost-decline transition, that pair mechanically carries a dominant weight and the panel slope is, in effect, that single comparison relabeled as a pooled estimate. The Goodman-Bacon logic makes this transparent: the right move is to DECOMPOSE beta into its component comparisons and report each comparison's weight, rather than present the weighted average as if many independent transitions supported it. If a leave-one-out drop of that pair moves beta across the accept/reject boundary for H0, the candidate's own robustness criterion is violated and the correct reported finding is a failure-to-reject, i.e., the data cannot support a credible point estimate of a learning rate. Presenting an illustrative negative slope under those conditions overstates what the small-N panel identifies; the disciplined output is the decomposition plus the statement that identifying variation is concentrated in one pair. - Goodman-Bacon, 'Difference-in-differences with variation in treatment timing,' Journal of Econometrics 225(2):254-277 (2021) | https://doi.org/10.1016/j.jeconom.2021.03.014 | grade A - Angrist & Pischke, Mostly Harmless Econometrics: An Empiricist's Companion (Princeton Univ. Press, 2009) | https://doi.org/10.1515/9781400829828 | grade A - **[measurement]** The objection is correct and identifies a failure mode the candidate's reliability-weighted robustness spec does not fix. Classical (random, regressor-independent) measurement error attenuates a slope toward zero; the candidate's case is the non-classical one, where the error is correlated with the regressor because the NICM-class parametric imputation takes heritage/complexity-correlated inputs. Under non-classical error correlated with the regressor, the bias is NOT sign-preserving attenuation: it can inflate, flip, or manufacture a slope, so a negative beta produced on the imputed subset is consistent with the cost model reproducing its own heritage-cost coupling rather than with an estimated learning effect. Reliability weighting addresses error variance, not error correlation with X, so it cannot repair this. The dispositive test is exactly the split the question proposes: fit beta on the directly-observed (layer-one, non-imputed) autonomy-NRE subset versus the imputation-dependent subset. If the negative slope survives on the directly-reported subset with the same sign and comparable magnitude, the learning reading is defensible to that extent; if it appears only where NICM imputation supplied the cost, the candidate cannot license a learning interpretation and must report the imputation-induced artifact. This is a discipline-on-the-measurement question, not one the assembled space corpora settle. - Hyslop & Imbens, 'Bias from Classical and Other Forms of Measurement Error,' NBER Technical Working Paper t0257 (2000) | https://doi.org/10.3386/t0257 | grade B - Hausman et al., 'Using Instrumental Variables to Estimate Models with Mismeasured Regressors,' in Handbook of Measurement Error Models (CRC Press) | https://doi.org/10.1201/9781315101279-5 | grade B - Angrist & Pischke, Mostly Harmless Econometrics: An Empiricist's Companion (Princeton Univ. Press, 2009) | https://doi.org/10.1515/9781400829828 | grade A - **[identification]** The objection is correct and the design as documented does not answer it. Chapter 5 names what beta is identified OFF (within-class, within-decade surviving variation in CumHeritage) but never names a source of as-good-as-random variation IN that regressor; Section 5.2 itself lists 'reverse selection' as an unresolved rebuttal, conceding that low cost may cause the heritage state rather than the reverse. On the credibility-revolution standard the analyst's FIRST obligation is to identify the source of exogenous variation and the channel by which it moves the outcome; that obligation is on the regressor, and adding fixed effects or covariates does not discharge it. CumHeritage being a deterministic forward sum of past selection-into-flight means the variation that survives the two-way within transformation is still the class's own lagged selection history. A valid identifying shock would be an event that reorders flight demonstrations for reasons orthogonal to autonomy qualification cost (a launch-manifest slip, a mission-of-opportunity slot won on other grounds, a directorate flight mandate, a budget-line start/stop), used as an instrument satisfying exclusion and monotonicity, or exploited as a quasi-experiment. The candidate supplies none, and the assembled space corpora (AMOS, ACTA, Space Economy) return nothing that establishes such a shock for the named NASA/JPL autonomy episodes; this is a design obligation the candidate must meet, not a fact a citation settles. Absent it, beta is a descriptive partial correlation of cost on its own lagged selection sequence, and the learning-rate label is not licensed. The grounded disposition: either supply the instrument/quasi-experiment with its exclusion argument, or downgrade beta from 'learning rate' to 'cost-vs-accumulated-selection association.' - Angrist & Pischke, 'The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics,' Journal of Economic Perspectives 24(2):3-30 (2010) | https://doi.org/10.1257/jep.24.2.3 | grade A - Angrist & Pischke, Mostly Harmless Econometrics: An Empiricist's Companion (Princeton Univ. Press, 2009) | https://doi.org/10.1515/9781400829828 | grade A - Imbens & Angrist, 'Identification and Estimation of Local Average Treatment Effects,' Econometrica 62(2):467-475 (1994) | https://doi.org/10.2307/2951620 | grade A - JPL_AUTONOMY_EDL_01 dissertation, Chapter 5 (Research Design and Identification), Sections 5.2 and 5.4 | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ch5_research_design.md | grade C - **[identification]** The objection holds; the design substitutes a covariate for a design and the candidate's own text concedes the gap. The conditional-independence (ignorability / selection-on-observables) assumption beta requires is: ln(CumHeritage) is independent of the cost error epsilon conditional on capability-class effects, decade effects, and starting TRL, i.e. CIA: { potential cost(heritage) тлл heritage | gamma_c, delta_d, TRL }. Stated plainly, within a class-decade-TRL cell the episodes that happen to have higher accumulated heritage must be no cheaper-or-dearer to qualify for any reason other than the heritage itself. A realistic unobserved violator is exactly the one the question names and the candidate's own Mokyr lens supplies: propositional/codified maturity of a capability's underlying theory. A class whose planning-and-estimation theory was already codified (Mokyr's mature propositional base) is simultaneously (a) cheap to qualify, because the prescriptive engineering follows from settled theory, and (b) flown early, because settled theory lowers the perceived risk of attempting it first, hence selected into flight sooner and accumulating heritage faster. That single latent variable drives BOTH the regressor (flight order -> heritage) and the outcome (cost), which is precisely the omitted common cause the CIA forbids. The design does not show this is captured: starting TRL is an ordinal, non-monetary maturation index (Mankins; Olechowski et al.) that tracks demonstrated readiness, not the depth of the codified propositional substrate, so it cannot stand in for propositional maturity; the chronology in Ch4/Ch6 records TRL entry/exit but no measure of theory codification. Worse, on Angrist-Pischke's bad-control test TRL is suspect: if starting TRL is itself raised BY the same accumulating heritage/propositional maturity (an outcome of the treatment process), conditioning on it partials out part of the very heritage channel and biases beta, the textbook bad-control pathology. Section 5.2's own rebuttal list already flags 'the maturity confound' and Section 5.4 admits the TRL covariate 'mitigates but does not eliminate' reverse pathways, so the candidate has conceded that selection-on-observables, not a design, is what is on offer. Grounded disposition: either defend full ignorability given an explicit measure of propositional maturity (which the design lacks), or treat TRL as a potential bad control and report beta without claiming the covariate closes the selection-into-timing channel. The space corpora do not settle this; it is a design-and-assumptions question. - Angrist & Pischke, Mastering 'Metrics: The Path from Cause to Effect (Princeton Univ. Press, 2014) | https://doi.org/10.2307/j.ctvcm4j72 | grade A - Angrist & Pischke, Mostly Harmless Econometrics: An Empiricist's Companion (Princeton Univ. Press, 2009) | https://doi.org/10.1515/9781400829828 | grade A - Olechowski, Eppinger, Joglekar & Tomaschek, 'Technology readiness levels: Shortcomings and improvement opportunities,' Systems Engineering 23(4):395-408 (2020); and Mankins, 'Technology readiness assessments: A retrospective,' Acta Astronautica 65(9-10):1216-1223 (2009) | https://doi.org/10.1002/sys.21533 | grade A - JPL_AUTONOMY_EDL_01 dissertation, Chapter 5 Sections 5.2 and 5.4 (internal-validity threat); Chapter 2 (Mokyr propositional/prescriptive knowledge) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ch5_research_design.md | grade C - **[identification]** The candidate concedes the identifying variation is thin. Ch5 states beta is 'identified off within-class, within-decade variation in cumulative flight-demonstrated heritage' and rates identification confidence 'moderate, held back from high by the small effective sample'; Ch4 names the AEGIS Opportunity-then-ChemCam sequence as 'the cleanest within-class heritage pair' and 'the central illustrative case for the heritage mechanism.' This is consistent with Arthur's own warning that under increasing returns a slope estimated off a single locked-in episode reflects the outcome of historical selection rather than an efficient learning rate (Arthur 1989, lock-in by historical events). The candidate's Ch6 sec6.3.3 small-panel influence diagnostic partially answers the leave-one-out concern by pre-committing to report whether the verdict survives removal of any single observation and the joint removal of the two or three most influential points, but it is a per-observation influence check, not the named leave-one-CLASS-out test dropping AEGIS that the question demands. - JPL_AUTONOMY_EDL_01 dissertation, Ch5 sec5.2 (identification) and Ch4 sec on heritage construction / AEGIS pair | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ch5_research_design.md | grade C - JPL_AUTONOMY_EDL_01 dissertation, Ch6 sec6.3.3 small-panel influence diagnostic and sec6.3.4 fixed decision rule | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ch6_analysis_plan.md | grade C - W. B. Arthur, 'Competing Technologies, Increasing Returns, and Lock-In by Historical Events' (lock-in not necessarily efficient, retrieved via brian_arthur brain) | https://doi.org/10.2307/2234208 | grade A - **[measurement]** The candidate already concedes the within-class count is a coarse proxy and that reusable knowledge crosses class boundaries through shared flight-software substrate. Ch5 sec5.3 pre-registers a second robustness specification that broadens CumHeritage 'to count cross-class software-component reuse, not only within-class flight demonstrations,' arguing the within-class baseline understates heritage and biases the slope toward zero; Ch7 sec7.3.4 names the Core Flight System (cFS) as exactly the codified, reusable, plug-and-play substrate that should steepen the within-class slope. This treats cross-class reuse as a one-directional bias correction. It does NOT build the component-level heritage graph the question asks for, nor test rank-concordance between the within-class and cross-class orderings; Arthur's combinatorial-evolution and recombinant-lock-in literature is precisely the warrant that the reuse paths cross the analyst's partition and can reorder the episodes (recombinant extension of Arthur's lock-in model, Industry and Innovation 2011). - JPL_AUTONOMY_EDL_01 dissertation, Ch5 sec5.3 (second robustness spec: cross-class software-component reuse) and Ch7 sec7.3.4 (cFS codified substrate) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ch7_discussion.md | grade C - Zeppini & van den Bergh, 'Competing Recombinant Technologies for Environmental Innovation: Extending Arthur's Model of Lock-In', Industry and Innovation (2011) (retrieved via brian_arthur brain) | https://doi.org/10.1080/13662716.2011.561031 | grade B - **[rival]** The candidate explicitly recognizes the survivorship/selection threat but cannot fully close it. Ch4 admits NTRS 'carries a documentation-survivorship bias ... toward documented successes and against the abandoned attempts' and confines the claim to 'the population of documented flight demonstrations.' Ch7 sec7.3.3 ('Rival three: selection on cost') states the rival mechanism in Arthur's own terms -- 'the realized cost path reflects which classes were chosen for early investment, not only the intrinsic learnability of those classes' -- and rates confidence that beta is free of selection as 'low to moderate ... the rival the design controls least completely.' The named design responses (the TRL maturation covariate, the cross-class reuse spec, and the forward-only counting rule) address selection channels that run through maturity or shared components but, by the candidate's own admission, leave selection 'through a channel not captured by maturity or by cross-class reuse' inside beta. This matches Arthur exactly: in an increasing-returns market the dominant/surviving option need not be the efficient long-run choice (Arthur 1989). - JPL_AUTONOMY_EDL_01 dissertation, Ch4 (NTRS documentation-survivorship bias) and Ch7 sec7.3.3 'Rival three: selection on cost' (confidence low to moderate) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ch7_discussion.md | grade C - W. B. Arthur, 'Competing Technologies, Increasing Returns, and Lock-In by Historical Events' (locked-in path not necessarily efficient; outcome reflects historical sequence) | https://doi.org/10.2307/2234208 | grade A - **[identification]** The funding-order test is data-feasible but NOT pre-committed, which is the live vulnerability. The dissertation states the panel is assembled from 'NASA TechPort project records and technology-readiness histories,' which carry project start dates and funding records, so ordering the five classes by date-of-first-investment is constructible from the stated sources. The candidate already concedes the underlying problem at Toulmin strength in Section 5.5: 'the realized slope reflects which capability classes received early investment as much as it reflects any intrinsic learnability,' grounding non-ergodicity on Arthur 1989 (10.2307/2234208), Arthur 1994 (10.3998/mpub.10029), and Arthur's complexity-economics non-ergodicity, and corroborating it empirically with Wei, Smith and Sohn (2017), who find retrospective learning rates that vary and correlate with the specific deployment programs that produced them (10.1016/j.enpol.2017.04.035). HOWEVER, the only pre-committed pre-analysis checks in Section 6.3 are (i) the fixed-effects feasibility check and (ii) the small-panel influence diagnostic; the fixed decision rule (6.3.4) turns on sign and CI across baseline plus two of three robustness specs. NO funding-order monotonicity test, and no slope-rank-vs-funding-rank-vs-maturity-rank comparison, is among the pre-committed checks. The design's sole declared handle on path-contingency is cross-class slope heterogeneity read against the Mokyr codification ordering (5.5 qualifier; 7.4), which is a maturity-ordering test, not the adversary's funding-ordering test. So beta is reported as a planning input for the realized portfolio path while the specific falsification that would separate learnability-rank from funding-sequence-rank is named in principle but not pre-registered. - JPL_AUTONOMY_EDL_01 dissertation.md, Abstract / Ch1 (line 17) and Ch6.2 estimation procedure | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation.md, Section 5.5 (lines 858-874) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation.md, Section 6.3.1-6.3.4 (lines 955-975) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - W. B. Arthur, Competing Technologies, Increasing Returns, and Lock-In by Historical Events, Economic Journal (1989) [via hos-brian_arthur brain] | https://doi.org/10.2307/2234208 | grade A - M. Wei, S. Smith, M. Sohn, Energy Policy (2017) [cited as ref-114 in dissertation] | https://doi.org/10.1016/j.enpol.2017.04.035 | grade A - **[measurement]** The combinatorial-recombination mechanism is correctly attributed to Arthur and the design PARTIALLY anticipates it, but the specific cFS-dependency-graph recombination-stock variable the adversary demands is NOT buildable from the assembled panel. Arthur's technology-evolution account holds that novel technologies are assembled combinatorially from existing ones and that whoever owns the most-recombined foundational components captures disproportionate downstream value (hos-brian_arthur dossier; Schrepel synthesis 10.1017/S1744137424000067; recombinant-lock-in extension 10.1080/13662716.2011.561031), so the critique's mechanism is real. The candidate has already operationalized a blunt version: Section 6.5 specifies a robustness row 'baseline with cross-class heritage,' and Section 7.3 names confound-driven attenuation 'once the cross-class heritage measure is added' as a pre-named way the contribution can be weakened, conceding that the raw association may reflect 'shared components rather than within-class flight heritage.' To that extent a cross-class reuse-stock variable that could flip the result is contemplated. BUT the demand to BUILD that variable 'from the cFS dependency graph and NTRS reuse statements' is not supported by the panel: the only cFS source in the corpus is McComas 2013 (NASA STI 20130013412), which Section 3.6 characterizes as a framework description that 'does not measure the cost saved by reuse,' with the corpus footprint of cFS flagged as 'modest' and a needed 'focused sweep on Core Flight System reuse economics' deferred to the build phase. There is no component-level cFS dependency graph and no NTRS component-reuse statement set in the assembled evidence, so a true combinatorial recombination-stock measure cannot be constructed from current retrieval, and whether the within-class slope survives its substitution cannot be shown now. - hos-brian_arthur dossier (Arthur technology-evolution framework) and Schrepel synthesis of Arthur, Journal of Institutional Economics (2024) | https://doi.org/10.1017/S1744137424000067 | grade A - Competing Recombinant Technologies for Environmental Innovation: Extending Arthur's Model of Lock-In, Industry and Innovation (2011) [via hos-brian_arthur brain] | https://doi.org/10.1080/13662716.2011.561031 | grade B - JPL_AUTONOMY_EDL_01 dissertation.md, Sections 6.5 (line 118) and 7.3 falsification condition (line 114) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation.md, Sections 3.6 (lines 504-506) and ref-78; corpus.jsonl key mccomas2013 | http://hdl.handle.net/2060/20130013412 | grade B - **[measurement]** The premise that reuse appears only as an attenuation bias inside a robustness specification is partly inaccurate: the design already elevates the codified, reusable knowledge substrate (Core Flight System and shared autonomy frameworks) to a named theoretical construct, the 'Mokyr moderator', in a dedicated literature section (3.6) and a dedicated heterogeneity chapter (7.4), predicting steeper cost decline where reuse infrastructure is present. Christensen_c's stronger demand is nonetheless unmet: the current design counts heritage within-class and forward-only, deliberately undercounting cross-class software and design-pattern reuse to bias the slope toward zero, and it tests the substrate only as cross-class effect heterogeneity, not as a within-episode reuse-stock variable fit head-to-head against the heritage count with both slopes pre-committed. The candidate concedes the corpus is thinnest exactly on the substrate's cost effect, so the head-to-head christensen_c wants is constructible in principle from the cFS reuse record but is not currently specified or pre-registered. - JPL_AUTONOMY_EDL_01 dissertation.md, Sec 3.6 (lines 502-514) and ToC 3.6 / 7.4 | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation.md, proposition one (line 539) and navigation cross-class robustness specification (line 472) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation.md Sec 3.6 citing ref [78] (McComas, cFS) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - Hall of Shoulders dossier, christensen_c (disruption-as-falsifiable-theory; reusable-substrate vs artifact-count) | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/christensen_c/ | grade C - **[identification]** The jobs-to-be-done critique is correctly stated: segmenting by the artifact class (onboard planning, target selection, navigation, EDL hazard handling) rather than by the operational job the autonomy is hired to do is, in Christensen's frame, a product-category mis-segmentation, and AEGIS target selection and Perseverance AutoNav being different classes yet arguably the same close-the-loop-onboard job is a live instance of the boundary problem. The candidate's design does fix the capability classes from the bible and identifies the slope from within-class within-decade variation, so the boundary is load-bearing for identification, which means a job-based recut is a legitimate stress test. However, the empirical half of the question is unanswerable from the present record. - Hall of Shoulders dossier, christensen_c (JTBD test; Competing Against Luck) | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/christensen_c/ | grade C - JPL_AUTONOMY_EDL_01 dissertation.md, capability-class fixed effects (lines 191, 213, 391, 652) and class chronology (line 124) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - **[economics]** The conceptual distinction christensen_c draws is grounded and the candidate already half-recognizes it. The dependent variable is explicitly the recurring engineering cost to qualify the capability for flight, normalized on a NICM-class basis; it is a cost-to-qualify measure, not a cost-to-adopt measure. The Deep Space 1 Remote Agent lessons-learned record (ref [12]), which the candidate cites as the single most direct qualitative evidence in the corpus, reports that the impact of inserting system-level autonomy into a flight project was a major surprise with integration and verification effort far larger than anticipated, and the candidate names this as the 'recurring-engineering AND organizational cost' the dependent variable intends to capture, while conceding the narrative cannot populate ln(Cost). Christensen's RPV logic predicts that what gates fielding the next demonstration is whether the host project's values let it absorb a low-margin, schedule-risky insertion, not the engineering cost-curve, so a measured qualification-cost slope and an unmeasured adoption-friction slope can diverge. That divergence risk is real and is not currently instrumented in the design. - JPL_AUTONOMY_EDL_01 dissertation.md, dependent-variable definition (line 209) and desired-state framing (line 130) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation.md Sec 3.1 citing ref [12] (DS1 Remote Agent lessons-learned) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - Hall of Shoulders dossier, christensen_c (RPV framework; incumbent-survival / resource-allocation prediction) | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/christensen_c/ | grade C - **[measurement]** The measure is theory-mandated and constructible in principle. Conservation-of-attractive-profits predicts that when one stage of a value chain becomes modular and 'good enough', the scarce, costly work migrates to an adjacent stage the program's values can still absorb; segmenting by the job-to-be-done (close the loop without ground latency) rather than by satellite/instrument class is exactly what JTBD requires. So a dependent variable measured only as on-orbit qualification NRE is mis-segmented unless paired with a venue-share covariate. The coding substrate exists and is retrievable: NTRS holds 'Lessons Learned from Autonomous Sciencecraft Experiment' (EO-1, id 20090007670), 'Onboard Autonomy on the Earth Observing One Mission' (id 20070035964), and 'Automated Targeting for the MER Rovers' (AEGIS lineage, id 20150011960), each documenting onboard-versus-ground task allocation. The candidate has not built this split; it is therefore an open construction task, not an impossibility. - christensen_c thinker dossier (Christensen, Verlinden & Westerman 2002; The Innovator's Solution 2003), conservation of attractive profits; attractive profits migrate to wherever performance is still 'not good enough' | hos-christensen_c dossier | grade A - christensen_c thinker dossier, Jobs-to-be-done test: segment by the job and circumstance, not by product/satellite category (Competing Against Luck, 2016) | hos-christensen_c dossier | grade A - NASA NTRS, 'Lessons Learned from Autonomous Sciencecraft Experiment' (EO-1) | https://ntrs.nasa.gov/citations/20090007670 | grade B - NASA NTRS, 'Automated Targeting for the MER Rovers' (AEGIS lineage) | https://ntrs.nasa.gov/citations/20150011960 | grade B - **[identification]** The identification rationale is grounded in the theory the panelist owns: because attractive profits migrate within a value network, venue substitution moves within-decade and within-class, so decade fixed effects cannot absorb it, the substitution is a confounder operating at the same level as the treatment. Christensen's own methodological standard requires this test: he insists disruption is a falsifiable causal theory that predicts who wins and when and 'can be wrong', so a venue-share covariate that flips the slope is the disconfirming evidence the theory demands be reported as such. This grounds WHY the pre-commitment and the collapse-equals-displacement reporting rule are warranted. Whether the candidate makes the binding pre-registration is the candidate's to assert; the corpus settles the rationale, not the commitment. - christensen_c thinker dossier, 'Disruption is a falsifiable causal theory, not a label: it predicts who will win and when, and it can be wrong' (Christensen, Raynor & McDonald, 'What Is Disruptive Innovation?', HBR 2015) | hos-christensen_c dossier | grade A - christensen_c thinker dossier, conservation of attractive profits: value migrates within the value network to the adjacent not-good-enough stage (Christensen, Verlinden & Westerman 2002) | hos-christensen_c dossier | grade A - **[rival]** The decision artifact exists and is retrievable: AEGIS automated targeting for the MER/MSL rovers is documented in NTRS ('Automated Targeting for the MER Rovers', id 20150011960; 'Autonomous Exploration for Gathering Increased Science', id 20100033547), and EO-1 onboard autonomy with its operations consequences is documented ('Onboard Autonomy on the Earth Observing One Mission', id 20070035964; 'Lessons Learned from Autonomous Sciencecraft Experiment', id 20090007670). These establish that a real sequencing/reuse decision of the demanded kind is on the record. The build-or-wait decision the program faces is total cost-to-field, not on-orbit NRE alone, JTBD frames the relevant unit as the job (close the loop) summed across whatever venue discharges it, so an on-orbit-only slope that ignores netted ground/downlink burden answers a narrower question than the portfolio faces. - NASA NTRS, 'Automated Targeting for the MER Rovers' (AEGIS lineage; documented sequencing/reuse decision) | https://ntrs.nasa.gov/citations/20150011960 | grade B - NASA NTRS, 'Onboard Autonomy on the Earth Observing One Mission' (onboard-vs-ground task allocation on the record) | https://ntrs.nasa.gov/citations/20070035964 | grade B - christensen_c thinker dossier, JTBD: segment and measure by the job-to-be-done and its circumstance, not by product/satellite class | hos-christensen_c dossier | grade A - **[measurement]** Dietz's enterprise-ontology distinction between the documental (D) layer and the ontological/coordination layer is the correct frame for the measurement charge: a layer-2 NICM parametric imputation and a layer-1 extracted line item are documental products of how cost happened to be recorded, and Dietz's framework warns that agreement at the documental layer does not establish that two figures denote the same essential construct. The frame sharpens the question but does NOT itself certify commensurability of any matched pair; that requires the candidate's source cost records, which retrieval did not return. - The Transaction Axiom (Dietz 2006), in Enterprise Ontology; hall-of-shoulders dietz brain | https://doi.org/10.1007/3-540-33149-2_10 | grade A - **[mechanism]** Dietz's transaction completeness criterion supplies the mechanism distinction the candidate needs: a transaction is complete only with an explicit production fact AND an accept act by an authorized receiver; a missing accept act is an ungovernable (un-realized) transaction. Mapped onto heritage, 'a prior flight existed' is at most a documental event, whereas 'the successor project ingested and accepted the predecessor's codified artifacts' is the completed handoff with an authorized receiver. This grounds WHY counted heritage and transfer-realized heritage can diverge, but does not supply the TechPort lineage data needed to build the transfer-realized variable or re-fit beta. - The Transaction Axiom (Dietz 2006), transaction completeness / accept act; hall-of-shoulders dietz brain dossier | https://doi.org/10.1007/3-540-33149-2_10 | grade A - **[measurement]** Dietz's framework is the correct instrument for this challenge: the documental (D) layer can be perfectly populated (papers narrate reuse) while the coordination/ontological (B) layer transfer is unproven, which is exactly the gap between a class label and a completed transfer. The dietz dossier states the principle directly: 'interoperability at the documental layer does not guarantee interoperability at the coordination layer.' This grounds the diagnosis that a scalar within-class count is a D-layer/calendar artifact unless each edge is a B-layer transaction. - Dietz dossier (Hall of Shoulders, hos-dietz) citing the transaction axiom and B/I/D layer model | 10.1007/3-540-33149-2_10 | grade A - **[mechanism]** Dietz's transaction axiom supplies the exact five-element completeness test the question demands and certifies it as the correct operationalization of 'heritage as completed transfer': the atomic unit of coordination is a transaction (initiator/executor) through order-execution-result, and completeness is checked by naming the initiator role, executor role, promise act, production fact, and accept act, with an un-named executor or missing accept act constituting an ungovernable (incomplete) transaction. Mapped to heritage: producing project = executor + production fact; consuming project that re-qualifies = initiator + accept act. This grounds Mokyr's codification moderator as a transfer transaction, not mere precedence. - The Transaction Axiom (Dietz 2006), via Hall-of-Shoulders dietz brain (crossref + dossier completeness test) | https://doi.org/10.1007/3-540-33149-2_10 | grade A - **[governance]** A real, documented JPL autonomy reuse sequence the slope's build-or-wait logic must speak to demonstrably exists: AEGIS automated science targeting was first flight-demonstrated on the MER Opportunity rover (Estlin et al. 2012), then reused/infused onto ChemCam on MSL Curiosity (Francis et al. 2017), and re-targeted for SuperCam on Mars 2020. The candidate's own Ch.4 names exactly this Opportunity-to-ChemCam pair as 'the central illustrative case for the heritage mechanism.' The dissertation, however, motivates the slope only as input to 'the build-or-wait decision it informs' (Ch.5) and never runs the counterfactual on any one such record: it asserts relevance rather than exhibiting a decision the slope would have flipped or bounded under cadence/budget-line/decadal/appropriations constraints. The actionability claim is therefore unfalsified by the manuscript and the falsifier Gangale demands ('name the one decision it changes') is constructable from the public AEGIS record but not yet executed. - Estlin et al., AEGIS Automated Science Targeting for the MER Opportunity Rover, ACM TIST 2012 | https://doi.org/10.1145/2168752.2168764 | grade A - Francis et al., AEGIS autonomous targeting for ChemCam on Mars Science Laboratory, Science Robotics 2017 | https://doi.org/10.1126/scirobotics.aan4582 | grade A - JPL_AUTONOMY_EDL_01 dissertation, ch4_data_and_measurement.md (AEGIS Opportunity-to-ChemCam named central illustrative case) and ch5_research_design.md (slope motivated as build-or-wait input) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ | grade C - **[measurement]** The heritage variable has a stated record-adjudicable rule but it is a within-class, forward-only count, not a functional bright line, and the dissertation itself surfaces the boundary case it cannot resolve. Ch.4 defines the unit as cumulative within-class flight-demonstrated heritage, credited only when a prior demonstration 'reached flight before that episode's development start' (forward-only counting rule, first-in-class set to 1), adjudicated by NTRS chronology and published demonstration dates. Two failures follow. First, by construction the within-class rule excludes the reusable codified substrate that actually lowers requalification cost when it crosses class boundaries (cFS components, shared autonomy frameworks, verification artifacts) which is precisely Mokyr's propositional/prescriptive distinction; the AEGIS Opportunity-to-Curiosity reuse classifies as one within-class increment while a cFS component shared across navigation and planning has no consistent home and is dropped. Second, the candidate concedes the class assignment is 'a boundary call' with only 'moderate' confidence and that 'a single contested assignment can move a within-class estimate' so the rule does not escape the 'rise and stall' fate of a functional definition; it relocates the unresolved edge case from regime-adjudication to class-adjudication rather than dissolving it. cFS reuse as a real cross-class substrate is documented in the flight-software-reuse literature. - JPL_AUTONOMY_EDL_01 dissertation, ch4_data_and_measurement.md (forward-only within-class counting rule; 'boundary call', 'moderate' confidence; AEGIS pair as central case) and ch5 (within-class forward-only rule) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ | grade C - McComas, Increasing flight software reuse with OpenSatKit (core Flight System cFS), IEEE Aerospace 2018 | https://doi.org/10.1109/aero.2018.8396631 | grade B - Gangale, The Functional Approach: Its Rise and Stall, in How High the Sky? (Brill 2018), functional rule cannot adjudicate edge cases and stalls in practice | https://doi.org/10.1163/9789004366022_013 | grade B - **[identification]** The reference-frame commensurability challenge is well-posed and the dissertation does not meet it. The estimating equation pools episodes across five capability classes with a single normalized development-cost dependent variable and class fixed effects (gamma_c), identifying beta off within-class, within-decade variation. The candidate acknowledges class heterogeneity and offers a 'cross-class reuse measure' and the option of class-specific slopes / partial pooling, but provides no explicit conversion mapping a unit of EDL heritage onto a unit of onboard-planning heritage, and no invariance check showing the fitted slope is stable to that conversion. EDL hazard-handling qualification is denominated against a one-shot irreversible event budget while onboard-planning qualification accrues against a long revisable operations timeline (the Remote Agent planning lineage), so 'one demonstration' is not a common unit across the panel; class fixed effects absorb level differences in cost but do not render the cumulative-heritage axis dimensionally consistent, which is the distortion Gangale's two-frame-conversion idiom targets. This is the same problem Gangale formalized for off-Earth operations spanning two independent time references (the Darian/two-dimensional-time work): a single Earth-referenced frame assumed across incommensurable regimes injects a systematic, sign-uncertain bias into the pooled slope. - JPL_AUTONOMY_EDL_01 dissertation, ch5_research_design.md (log-log two-way FE specification; pooled across classes; 'cross-class reuse measure'; partial-pooling discussion) and ch4 (planning class origin = Remote Agent Experiment) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/chapters/ | grade C - Gangale, The Architecture of Time, Part 2: The Darian System for Mars, SAE 2006, temporal reference frames for off-Earth operations as designed infrastructure; conversion between independent frames | https://doi.org/10.4271/2006-01-2249 | grade B - **[measurement]** A bright-line inclusion rule CAN be stated for four of the five named episodes because each autonomy function was a per-mission flight experiment with an identifiable project/technology-program owner: Remote Agent flew as a designed spacecraft-autonomy experiment on DS1 (per-mission NRE inside the experiment scope); the EO-1 Autonomous Sciencecraft Experiment was an onboard autonomy-flight-software experiment whose cost is the experiment's own software NRE; AEGIS automated science targeting was deployed first on MER Opportunity and then ported to MSL/ChemCam as two distinguishable deployment efforts. Stated rule: count flight-software NRE booked to the mission/experiment that first qualifies the onboard function on its own vehicle; exclude ground-system operations labor and exclude COTS adoption. This rule assigns an identical dollar boundary to RA, EO-1, AEGIS-Opp, and AEGIS-ChemCam because each has a distinct, retrievable per-deployment software-qualification record. - Bernard et al., Design of the Remote Agent experiment for spacecraft autonomy (IEEE Aerospace 1998) | https://doi.org/10.1109/aero.1998.687914 | grade A - Sherwood/Chien et al., Using Autonomy Flight Software to Improve Science Return on Earth Observing One (JACIC) | https://doi.org/10.2514/1.12923 | grade A - Estlin et al., AEGIS Automated Science Targeting for the MER Opportunity Rover (ACM TIST) | https://doi.org/10.1145/2168752.2168764 | grade A - Francis et al., AEGIS autonomous targeting for ChemCam on Mars Science Laboratory: Deployment and results (Science Robotics 2017) | https://doi.org/10.1126/scirobotics.aan4582 | grade A - **[measurement]** The boundary case the rule CANNOT adjudicate is Ingenuity, whose autonomy did not originate as a fresh per-mission flight-software NRE but inherited cross-cutting flight infrastructure: the M2020/Perseverance flight stack and broader autonomous-robotics heritage were explicitly cross-cutting/reused rather than mission-bounded, so 'the dollar boundary of Ingenuity's autonomy qualification' is not assignable from a single appropriation line. UNLIKE gangale's functional-approach 'rise and stall' (which stalls because the RULE ITSELF cannot in principle adjudicate an edge case such as suborbital/VLEO flight), this regressand boundary case is a missing-RECORD problem, not a missing-rule problem: the rule is well-defined; the public funding ledger that would let it be applied to a reused substrate is simply not retrievable. It is therefore a data gap, recoverable by disclosure, not a definitional incoherence that proliferates with every new case. - NASA NTRS, Cross-Cutting Flight Infrastructure Improvements on M2020 (id 20230006993) | https://ntrs.nasa.gov/citations/20230006993 | grade B - Verma et al., Autonomous robotics is driving Perseverance rover's progress on Mars (Science Robotics 2023) | https://doi.org/10.1126/scirobotics.adi3099 | grade A - Gangale, T., The Functional Approach: Its Rise and Stall, in How High the Sky? (Brill 2018) | https://doi.org/10.1163/9789004366022_013 | grade B - **[identification]** cFS exists as a GSFC-originated reusable flight-software framework explicitly positioned to amortize flight-software cost across many missions (including small spacecraft), which is precisely the textbook fixed-common-cost / public-good substrate gangale names. Under the stated venue rule the consistent treatment is to EXCLUDE the substrate's one-time development NRE from every individual episode's per-mission regressand (it is a fixed common cost funded outside any single mission), and to count only the per-mission integration/qualification of cFS into that mission. This exclusion is defensible and does NOT mechanically manufacture the negative slope only if reported alongside the excluded-codification term: loading cFS NRE on the origin episode would inflate early cost and steepen beta, while silently excluding it removes the very codified-knowledge mechanism Mokyr-type accounts say drives the decline, so the slope must be reported both ways or the codification term carried explicitly. - NASA NTRS, Core Flight System (cFS) Training (id 20205000691) | https://ntrs.nasa.gov/citations/20205000691 | grade B - McComas et al. / cFS Community, The Core Flight System (cFS) Community: Providing Low Cost Solutions for Small Spacecraft (OpenAlex W record) | https://openalex.org/works?search=Core%20Flight%20System%20cFS%20Community%20Low%20Cost%20Small%20Spacecraft | grade C - NASA NTRS, Big Software for SmallSats: Adapting cFS to CubeSat Missions (id 20150021070) | https://ntrs.nasa.gov/citations/20150021070 | grade B - **[empirics]** The candidate's own design CONCEDES the power problem but does not quantify it. Section 6.3.1 states 'no diagnostic manufactures statistical power that the data do not contain,' and 6.4.3 commits only to REPORTING a wide interval containing zero as 'inconclusive rather than as positive evidence of a flat cost.' No Monte Carlo / false-negative simulation is run; power is handled by transparency, not calculation. McCloskey-Ziliak's sizeless-science critique is exactly that a 'fails to reject H0' from a low-power instrument cannot stand as evidence for the null, so the demanded power simulation is a legitimate and currently-unmet design requirement. - JPL_AUTONOMY_EDL_01 dissertation, Sections 6.3.1, 6.4.3 (lines 959, 997) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - Ziliak & McCloskey, The Cult of Statistical Significance (Univ. of Michigan Press) | https://doi.org/10.3998/mpub.186351 | grade A - **[economics]** The candidate frames the contribution as 'a fitted slope beta, its confidence interval, the implied learning rate, and an explicit accept-or-reject decision on the null' (Section 8.1, line 1159), significance, not oomph. Section 6.4.2 only gestures that a low-tens-of-percent rate 'would be decision-relevant in the build-or-wait sense' while stating 'no learning rate has been estimated and none is forecast.' No ex-ante decision-flipping magnitude threshold is derived from NICM figures and a cost cap, and no loss function over the slope is specified. This is precisely the substantive-significance gap McCloskey-Ziliak name; the demand for an ex-ante decision-relevant magnitude is settleable from the candidate's own cost data but is not met. - JPL_AUTONOMY_EDL_01 dissertation, Sections 6.4.2 and 8.1 (lines 991, 1159) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - Ziliak & McCloskey, The Cult of Statistical Significance (Univ. of Michigan Press) | https://doi.org/10.3998/mpub.186351 | grade A - **[measurement]** The candidate already carries a per-observation reliability flag recording which layer (direct / parametric-imputed / deflated-only) produced each cost figure (Section 4.2 measurement table, line 655; Section 4.3.4) and pre-registers an inverse-imputation-error WEIGHTING robustness specification (Section 5.3, line 830). That flag is exactly the variable needed for the audited-vs-imputed split McCloskey demands, but the design only DOWN-WEIGHTS imputed observations; it does not pre-commit to estimating the slope on the audited subset alone and comparing it to the imputed-heavy fit. The split is therefore feasible on the candidate's own apparatus but is not a registered analysis, and because the panel is explicitly not yet assembled (Section 4.6.5, line 765; 6.4.1, line 981) no audited-vs-imputed slope contrast exists to settle whether the effect is imputation-driven. - JPL_AUTONOMY_EDL_01 dissertation, Sections 4.2/4.3.4/5.3/4.6.5/6.4.1 (lines 655, 830, 765, 981) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - **[empirics]** The framework of the question is correct and is the candidate's own exposed flank: statistical rejection of a flat-cost null is not the same as a confidence interval tight enough to license a build-or-wait call, and a wide-but-significant per-doubling interval is a significance star, not oomph (Ziliak & McCloskey, sizeless-science critique). The dissertation's decision rule (sec. 6.3.4) is purely sign-and-exclusion: reject H0 iff beta is negative AND its interval excludes zero in the baseline and in at least two of three robustness specs. That rule certifies direction, not magnitude precision, so it can fire on an interval whose bounds imply opposite sequencing calls. The candidate half-concedes this: sec. 6.4 states a near-zero or wide interval is 'genuinely ambiguous' and commits to 'treat a wide interval that contains zero as a failure to reject rather than as evidence for the alternative' (lines 989, 997) -- but it nowhere applies the symmetric discipline to a wide interval that EXCLUDES zero, which is exactly the decision-indeterminate-but-significant cell McCloskey names. HOWEVER, the specific numeric fraction demanded cannot be computed: the artifact is design-stage, 'No coefficient is fitted on the full dataset; all expected results are explicitly illustrative' (line 10), the panel is unbuilt ('tens, not thousands' of episodes, line 753), and the NICM-class imputation error has no fitted magnitude (line 755). There are therefore no actual class/decade degrees of freedom and no actual imputation-error variance from which to construct the joint distribution. The principle is grounded; the requested number is unretrievable and is logged as a gap. - Ziliak, S. T. & McCloskey, D. N., The Cult of Statistical Significance (Univ. of Michigan Press) | https://doi.org/10.3998/mpub.186351 | grade A - JPL_AUTONOMY_EDL_01 dissertation, sec. 6.3.4 and 6.4 (lines 973-997) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation, lines 10, 753, 755 | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - **[rival]** The rival is not hypothetical -- it is already in the candidate's own Chapter 7, and the candidate concedes it is the channel the design closes least. The 'bespoke negotiated budgets' story decomposes into the dissertation's Rival two (scale/funding: 'larger, better-funded missions may field autonomy more efficiently... independent of any heritage', sec. 7.3.2) and Rival three (selection on cost / programmatic judgment: a beta 'that reflects what the agency chose to attempt rather than what heritage did to cost', sec. 7.3.3). The candidate's design responses are partial: decade fixed effects, NICM scope-normalization, the TRL maturation covariate, cross-class-reuse breadth, and the forward-only counting rule. Crucially, the candidate ADMITS no observable fully discriminates the negotiated-budget rival: 'Selection that operates through a channel not captured by maturity or by cross-class reuse... would remain in beta. There is no instrument in the design that fully closes this channel, and honesty requires saying so' (sec. 7.3.3), rating confidence that beta is free of selection as 'low to moderate' and naming it 'the rival the design controls least completely.' This is exactly McCloskey's point: with no instrument and no exogenous variation, the TechPort+NTRS observables cannot break the tie between learning and a sequence of negotiated budgets driven by mission politics/PI continuity, so the negative slope's learning interpretation rests partly on the chosen Wright/Henderson metaphor rather than on data that independently excludes the rival (McCloskey, Rhetoric of Economics: economics persuades by metaphor, and the honest move is self-awareness about which figure is doing the work). The discriminating observable McCloskey demands -- e.g. a PI-continuity or funding-windfall covariate, or an instrument for 'readiness' orthogonal to TRL -- is absent from the named corpus and is owed at build time. - McCloskey, D. N., The Rhetoric of Economics | https://doi.org/10.2307/jj.36032609 | grade A - JPL_AUTONOMY_EDL_01 dissertation, sec. 7.3.2-7.3.4 (lines 1084-1102) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - **[economics]** The critique lands and is grounded, but the adjudication the question asks for cannot be performed from the record, for two converging reasons the candidate's own text supplies. First, the incumbent assumption is stated only qualitatively: the abstract frames the belief as one 'long held but never measured' that the agency 'currently answers by assertion' (line 17), and Chapter 6 calls heritage-lowers-cost something NASA/JPL 'currently treat as self-evident' (sec. 6.4.3) -- nowhere is an implicit per-doubling magnitude attributed to the assertion, so there is no incumbent number stated in its own units to test against. Second, the study fits no interval: design-stage, no coefficient, illustrative figures only (lines 10, 979-981). The candidate even pre-concedes the McCloskey failure mode -- that a near-zero or wide interval is 'genuinely ambiguous' and must be reported as 'inconclusive rather than as positive evidence of a flat cost' (lines 989, 997) -- which is precisely the state of 'cannot distinguish its own estimate from the assumption it replaces.' So McCloskey's charge is correct in principle and partly self-certified by the candidate: a measurement whose incumbent target is unquantified and whose own interval is unfitted cannot yet move a decision-maker off a pre-existing magnitude. But the specific arithmetic the question requests -- the implicit assumed slope and whether the fitted NICM-based interval is narrow enough to overturn it -- is unretrievable, because neither the assumed magnitude nor the fitted interval exists in the corpus. Logged as a gap with a concrete remedy: at build time the candidate must elicit the incumbent's banked per-doubling reduction (the magnitude a sequencing decision implicitly assumes) and pre-register whether the fitted interval lies inside, excludes, or straddles it. - Ziliak, S. T. & McCloskey, D. N., The Cult of Statistical Significance (Univ. of Michigan Press) | https://doi.org/10.3998/mpub.186351 | grade A - JPL_AUTONOMY_EDL_01 dissertation, lines 10, 17, 979-981, 989, 997 and sec. 6.4.3 | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - **[measurement]** The methodological premise is sound and conceptually grounded: Mokyr distinguishes a reusable propositional/codified knowledge stock from the bare artifact, and warns that a count of demonstrations is not the same as the extensible, self-correcting knowledge that drives cost decline. A codification indicator is constructible in principle because cFS is documented as a platform- and project-independent reusable software framework, so 'delivered as a reusable cFS app/shared library' vs 'bespoke re-implementation' is an observable, codeable distinction. The instrument-validity test the panelist demands (does the codification indicator diverge from the flight count) is therefore the correct gate before any flat-beta inference. - Mokyr, The Gifts of Athena (2002), as captured in the mokyr Hall-of-Shoulders dossier | https://press.princeton.edu/books/paperback/9780691120133/the-gifts-of-athena | grade A - Stottler et al., On-board Autonomous Hybrid Spacecraft Subsystem Fault Detection (cFS described as reusable framework), Proc. AMOS Conference, 2022 | https://amostech.com/TechnicalPapers/2022/Poster/Stottler_2.pdf | grade C - **[mechanism]** The mechanism objection is valid and the proposed ex-ante ranking test is the right discriminator, because Mokyr's own criterion is that cost falls only where prescriptive technique rests on a widening propositional base, so slope heterogeneity ordered by propositional maturity is the falsifiable prediction, not a secondary heterogeneity check. A pooled slope that ignores this ordering genuinely confounds a non-learning class with averaging over heterogeneous bases. The ex-ante anchors the panelist names are real, identifiable capability classes in the public autonomy record (planetary onboard autonomy / ASE-type, V&V of autonomous systems, the cFS reuse substrate), so a maturity rank can be assigned before cost figures are seen. - Mokyr, The Gifts of Athena (2002), as applied in the mokyr Hall-of-Shoulders dossier review lens (propositional-vs-prescriptive falsifiable test) | https://press.princeton.edu/books/paperback/9780691120133/the-gifts-of-athena | grade A - Lessons Learned in the Livingstone 2 on Earth Observing One Flight Experiment (2005); A Review of Verification and Validation for Space Autonomous Systems (2021) | https://doi.org/10.2514/6.2005-7000 | grade B - **[identification]** The identification threat is real and well-grounded: Mokyr's access-cost argument holds that codified knowledge lowers the next demonstration's cost only if it diffuses cheaply to the next team, and the aerospace STI record shows that producer-to-user diffusion is slow and poorly understood, so a slope estimated on a same-team/same-codebase reuse case plausibly measures tacit co-location rather than a transferable parameter. The knowledge-spillover literature reinforces this: knowledge citations cluster near their geographic and institutional sources, so a slope driven by personnel/code overlap would not generalize to a new team, breaking the portfolio-office policy claim. The candidate must therefore separate a continuity-driven slope from a transferable one, e.g. by contrasting overlapping-team reuse against arms-length adoption. - NASA/DOD Aerospace Knowledge Diffusion Research Project: US Scientific and Technical Information Policy, NTRS 19960052732 | https://ntrs.nasa.gov/citations/19960052732 | grade B - Jaffe, Trajtenberg & Henderson, Geographic Localization of Knowledge Spillovers (1993), cited in the mokyr Hall-of-Shoulders dossier | https://doi.org/10.2307/2118401 | grade A - **[empirics]** PARTIAL. The dissertation has NOT pre-committed a minimum effective within-class sample for the codification x heritage interaction, and it explicitly declines to. Section 5.4 (statistical-conclusion validity) states 'no minimum-detectable-effect number is asserted here as a computed quantity, because the panel that would anchor it is not yet assembled.' On the withdrawal question the candidate's standing rule cuts the OTHER way from what Mokyr demands: Section 7.4 Qualifier pre-commits to report the moderator 'as suggestive heterogeneity where the data permit, not as a fitted moderation coefficient with its own confidence interval, unless a class is unusually well-populated,' and assigns the moderator LOW confidence because partitioning ~tens of episodes across five classes leaves 'any class with only one or two episodes contributing nothing to a within-class slope.' So the candidate (a) defers rather than supplies the power analysis, and (b) has pre-committed precisely to the suggestive-non-finding report Mokyr asks them to forswear. A formal interaction term (codification x ln-heritage) is never specified in the estimating equation, which carries only additive class and decade fixed effects; the moderator is operationalized as cross-class effect heterogeneity, not as an interaction coefficient with its own power budget. The honest reading: the central Mokyrian moderator is, on the candidate's own degrees-of-freedom accounting, at the edge of estimability and is currently decorative-until-proven rather than pre-committed-estimable. - JPL_AUTONOMY_EDL_01 dissertation, Ch5 Sec5.4 statistical-conclusion validity (power and minimum-detectable-effect paragraph) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation, Ch7 Sec7.4 Qualifier (Mokyr codification moderator) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - JPL_AUTONOMY_EDL_01 dissertation, Ch5 Sec5.1 (additive two-way structure vs class-by-decade saturation, degrees-of-freedom decision) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - **[identification]** The candidate cannot produce the coded stage-level vector data the question demands, because the dissertation never decomposes autonomy into the PSW four-stage authority vector: a full-text scan of dissertation.md returns ZERO occurrences of 'Parasuraman', 'Sheridan', 'four stages', 'authority level', or 'vector', while 'capability class' appears 112 times defined as 'the level at which heritage is counted and at which the fixed effects absorb baseline differences in cost and intrinsic difficulty.' The class is therefore an a-priori bin keyed to mission function (planning/scheduling, science-target selection, navigation, FDIR, EDL), NOT a level-homogeneous unit. The governing framework the candidate ignored is explicit that a single autonomy 'dial' is a category error: 'there is no single autonomy level for a system, only a vector of levels across the four stages, each with measurable consequences for situation awareness, workload, and recoverability' (PSW 2000), and the empirical meta-analysis (Onnasch et al. 2013) shows human-performance consequences differ sharply BY STAGE. Because the candidate's capability-class fixed effects absorb 'baseline difficulty' rather than separate stage-level authority, two episodes binned together (e.g., onboard planning in EO-1 ASE vs. a higher-authority closed-loop variant) can carry very different stage vectors, and the cumulative count sums non-comparable units, leaving the slope's unit-of-analysis uninterpretable. The candidate has no within-class stage-level variance result to offer because the coding was never done. - Parasuraman, Sheridan & Wickens, A model for types and levels of human interaction with automation, IEEE Trans. SMC-A (2000) | https://doi.org/10.1109/3468.844354 | grade A - Onnasch, Wickens, Li & Manzey, Human Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis, Human Factors (2013) | https://doi.org/10.1177/0018720813501549 | grade A - JPL_AUTONOMY_EDL_01 dissertation.md (full-text term scan, Ch.4 variable definitions) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - **[measurement]** The candidate's normalization cannot have removed an authority-level confound, because authority level is not a variable in the model at all. The fitted specification is ln(Cost_icd) = alpha + beta*ln(CumHeritage_icd) + gamma_c + delta_d + epsilon, with Cost normalized 'by capability scope on a NASA Instrument Cost Model-class basis' (NICM parametric scope drivers), capability-class fixed effects gamma_c, and demonstration/decade effects delta_d. There is NO decision/action-stage authority-level regressor (0 hits for 'authority level'). NICM-class scope drivers are physical and programmatic scope variables (the Stahl single-variable space-telescope cost models are the cited exemplar); they do not encode where on the four-stage vector authority was granted. The framework the candidate omitted predicts that the marginal V&V cost concentrates exactly at high decision/action authority, where the un-handled-exception and out-of-the-loop cases must be bounded and meaningful human control proven (PSW 2000; Onnasch et al. 2013). Because that regressor is absent, any negative heritage slope is confounded with an unmodeled level effect: the candidate cannot demonstrate the heritage coefficient survives the inclusion of decision-stage authority because no such competing-regressor test exists in the dissertation. This is an untested rival, and on the present model a level effect would be mislabeled as a learning effect. Whether beta in fact collapses is unknown from the record - the test was never run. - JPL_AUTONOMY_EDL_01 dissertation.md, Ch.4 construct table and NICM normalization passage | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - Parasuraman, Sheridan & Wickens (2000) and the supervisory-control program (plan/teach/monitor/intervene/learn) | https://doi.org/10.1109/3468.844354 | grade A - Onnasch, Wickens, Li & Manzey, Human Factors (2013) | https://doi.org/10.1177/0018720813501549 | grade A - **[rival]** The candidate has no authority-level delta to report for any heritage pair, because the within-pair analysis is conducted purely as a heritage-count increment with NICM scope normalization and contains no stage-level authority coding. The dissertation itself characterizes its cleanest pairs in heritage/reuse terms only: the AEGIS Opportunity-to-ChemCam pair is described as 'the same AEGIS capability, ported to a different instrument on a different rover... a reuse-and-extension of the Opportunity work rather than a fresh build' (Sec 3.2, 'the cleanest heritage pair'), and navigation is 'the deepest heritage chain' (Sec 3.3) - both framed by cumulative count, never by a decision/action authority delta. The primary record the candidate cites does flag exactly the authority transition the question raises: the EO-1 Autonomous Sciencecraft Experiment is documented as onboard autonomous science detection/replanning and retasking aboard EO-1 (NTRS 2004-2005), i.e., a move toward closed-loop onboard decision authority, yet the dissertation does not encode that as a higher decision-stage level. Therefore the levels-of-automation rival - that cost moved with AUTHORITY rather than with heritage in these very pairs - is left unaddressed by a forward-only counting rule. Because the candidate never extracted the authority-level delta, it cannot show whether cost rose where authority rose; the rival stands untested on the candidate's own strongest evidence. - JPL_AUTONOMY_EDL_01 dissertation.md, Ch.3 ToC (3.2, 3.3) and AEGIS deployment passage | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_AUTONOMY_EDL_01/dissertation.md | grade C - NTRS: The Autonomous Sciencecraft Experiment onboard the EO-1 spacecraft (NASA/JPL), ids 20060043342 / 20060042897 | https://ntrs.nasa.gov/citations/20060043342 | grade B - Parasuraman, Sheridan & Wickens (2000) | https://doi.org/10.1109/3468.844354 | grade A - **[measurement]** The co-movement the question alleges is real and grounded: V&V/qualification is the dominant cost driver for space autonomy and that burden is autonomy-emphasized, so a cost-to-qualify regressand is NOT authority-neutral. PSW establish that authority is a vector across four functional stages, and that high decision/action-implementation authority is precisely where qualification demand and out-of-the-loop risk concentrate; Sheridan & Verplank's original LOA scale frames decision/action authority as the graduated quantity at issue. The retrieved episodes occupy DIFFERENT action-stage authority (AutoNav controls the orbit/path autonomously; AEGIS only selects/sequences targets; Remote Agent plans/schedules), so within any 'capability class' a higher-authority successor carries structurally heavier V&V and will look more expensive at equal heritage, biasing the slope toward zero/positive; a successor held at equal-or-lower authority spuriously inflates the negative slope. CONSTRUCT-LEVEL VERDICT: the candidate must enter action-stage authority as a covariate (or stratify), because it lives inside the cost construct. NOT GROUNDED: no retrieved source supplies a quantified, authority-weighted per-episode burden index or a regression isolating the heritage slope from authority; that empirical artifact is absent from the corpus and must be constructed and defended by the candidate. - Cardoso et al., A Review of Verification and Validation for Space Autonomous Systems, Current Robotics Reports (2021) | https://doi.org/10.1007/s43154-021-00058-1 | grade A - Parasuraman, Sheridan & Wickens, A model for types and levels of human interaction with automation, IEEE Trans. SMC-A (2000) | https://doi.org/10.1109/3468.844354 | grade A - Sheridan & Verplank, Human and Computer Control of Undersea Teleoperators, MIT Man-Machine Systems Lab tech report (1978) | https://doi.org/10.21236/ada057655 | grade B - NASA NTRS: Validation and Verification of the Remote Agent for Spacecraft Autonomy (1999, id 20210003369); Autonomous Navigation for Deep Space Missions (2012, id 20130000290); Results From the First Four Years of AEGIS Autonomous Targeting for ChemCam on MSL (id 20220001538) | https://ntrs.nasa.gov/citations/20130000290 | grade B - **[rival]** The rival is warranted and partly evidenced. Onnasch's meta-analysis empirically confirms that higher degree-of-automation raises routine-condition performance and lowers workload while degrading the operator's ability to recover from automation failure (the lumberjack effect) - i.e., un-priced out-of-the-loop risk is shifted downstream into operations, exactly the cost the field-cost construct omits. The retrieved AEGIS record substantiates the scope-narrowing mechanism: the Opportunity-era AEGIS targeted the panoramic camera in an 'opportunistic' fashion under scientist-specified objectives, and the MSL/ChemCam successor reports hitting the desired material >93% of the time vs 24% without onboard targeting - an envelope DEFINED by a constrained objective function, not a same-scope re-qualification. RIVAL VERDICT: a within-class negative heritage slope is not identified as learning unless the exception-handling envelope is held constant across the pair; if envelope scope shrinks, the measured decline is a scope reduction, not a learning curve. NOT GROUNDED: retrieval did not return a source documenting a specific descope of off-nominal coverage, a deferred fault case, or an operations-phase anomaly causally attributable to the autonomy for either the AEGIS pair or Remote Agent (the RAX in-flight anomaly returned no citable record this turn); the candidate must produce envelope-scope evidence per pair before claiming learning. - Onnasch, Wickens, Li & Manzey, Human Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis, Human Factors (2013) | https://doi.org/10.1177/0018720813501549 | grade A - NASA NTRS: Results From the First Four Years of AEGIS Autonomous Targeting for ChemCam on Mars Science Laboratory (id 20220001538) | https://ntrs.nasa.gov/citations/20220001538 | grade B - NASA NTRS: Autonomous Exploration for Gathering Increased Science (id 20100033547); Two Years Onboard the MER Opportunity Rover (id 20130009128) | https://ntrs.nasa.gov/citations/20100033547 | grade B - **[identification]** GROUNDED and decisive. Under the PSW four-stage decomposition the documented episode descriptions cross-tag as follows, and the cross-tabulation shows the capability-class UNIT itself crosses stages: (a) Science target selection class - AEGIS (MER Opportunity and MSL ChemCam) analyzes onboard imagery and selects/sequences targets = information acquisition + information analysis + decision/action selection, with NO autonomous action-implementation on the platform's trajectory. (b) Navigation class - AutoNav determines state AND controls the path / executes maneuvers autonomously = information analysis + decision/action selection + ACTION IMPLEMENTATION. (c) Planning/scheduling class - Remote Agent plans, schedules, and controls the spacecraft = information analysis + decision/action selection (with execution authority). (d) EDL hazard handling = decision/action selection + ACTION IMPLEMENTATION (irreversible commit). The cross-tab thus places AEGIS-class episodes at acquisition/analysis-dominant while navigation and EDL episodes are action-implementation-dominant. CONSEQUENCE: any within-class pair that pairs an analysis/decision-stage demonstration with an action-implementation successor is summing non-commensurable learning objects; the cumulative-heritage count then aggregates incommensurable units and the within-class fixed effect does NOT purify the slope. Only pairs whose members share a dominant stage (e.g., AEGIS-MER -> AEGIS-MSL, both acquisition/analysis/decision; AutoNav DS1 -> AutoNav comet-flyby, both action-implementation) survive a same-stage restriction. This is the central Sheridan-Verplank objection: there is no single autonomy level for a system, only a vector across stages. - Parasuraman, Sheridan & Wickens, A model for types and levels of human interaction with automation, IEEE Trans. SMC-A (2000) | https://doi.org/10.1109/3468.844354 | grade A - NASA NTRS: Autonomous Exploration for Gathering Increased Science (id 20100033547); Results From the First Four Years of AEGIS Autonomous Targeting for ChemCam on MSL (id 20220001538) | https://ntrs.nasa.gov/citations/20100033547 | grade B - NASA NTRS: Autonomous Navigation for Deep Space Missions (id 20130000290); Using Autonomous Navigation for Interplanetary Missions: Validation of Deep Space 1 AutoNav (id 20060032521) | https://ntrs.nasa.gov/citations/20130000290 | grade B - NASA NTRS: Validation and Verification of the Remote Agent for Spacecraft Autonomy (id 20210003369) | https://ntrs.nasa.gov/citations/20210003369 | grade B ## Gaps - **[empirics]** The decisive empirical check the question demands cannot be computed and the candidate cannot supply it, because by the design's own repeated statement no panel is assembled and no coefficient has been fitted (Ch5 Section 5.0; Ch6 Sections 6.0-6.4 declare every number illustrative and the result tables specified-but-unpopulated). The within-class residual correlation between ln(CumHeritage) and a continuous calendar-year control net of decade dummies, and the VIF / first-stage partial-R-squared of heritage net of time, are numbers that exist only once the data are built; the grounded expert cannot manufacture them and refuses to invent them. The CONCEPTUAL force of the objection is established and conceded by the design: Section 5.2 cites Nagy et al. that where output grows exponentially the Wright cumulative-output form and the Moore calendar-time form become observationally similar, which is exactly why the candidate calls the decade control 'load-bearing'; and Section 5.4 admits a within-decade computing/tooling trend 'can survive' a decade-level control. On the credibility-revolution standard a before/after comparison with no untreated control is confounded by common time shocks, and here ln(CumHeritage) and calendar time are the same monotone series partitioned only by coarse decade bins, so the separability the candidate assigns to the decade dummies is asserted, not demonstrated. What is missing and unresolvable at the design stage is the QUANTITY: until the partial correlation and net-of-time variance share are reported, it is unknown whether decade fixed effects leave beta any identifying variation beyond a secular trend, and the design provides no commitment to report that specific diagnostic (it reports residual df and effective cell counts, not the heritage-net-of-time partial-R-squared). This is therefore an open empirical gap, not a settled claim: the candidate must, at build, report the heritage-vs-calendar-time partial correlation and VIF within class-decade cells, and if heritage carries near-zero independent variation the learning interpretation collapses into a generic time trend. (raised by angrist_pischke) - **[identification]** The design does not contain an explicit leave-one-class-out (drop-AEGIS) re-estimation, nor an enumeration of the surviving within-class within-decade contrasting pairs beyond the single AEGIS pair. The influence diagnostic is per-observation, not per-class, so the specific challenge -- whether beta and the two-of-three decision rule survive when the only documented same-capability second-platform pair is removed -- is unanswered by the current design and must be added as a pre-registered class-deletion robustness check. (raised by brian_arthur) - **[measurement]** No component-level heritage graph is operationalized from NTRS/cFS records, and no rank-concordance test between the within-class count and the cross-class component count is specified. Therefore the design cannot currently demonstrate that the cross-class spec is a robustness check on the same estimand rather than a distinct, possibly contradictory estimand; the discordance test the panelist demands is unanswered and would have to be added (build the reuse DAG, then a rank-correlation between the two orderings). (raised by brian_arthur) - **[rival]** The design provides no falsifying observable that discriminates learning (cost lowered by accumulated reusable knowledge) from selection-into-favorable-conditions (cost lowered because the surviving classes are the ones whose institutional/funding conditions made reuse cheap), and -- by the candidate's own statement that the study 'cannot recover the undocumented attempts' -- the panel contains no abandoned/stalled 'flew once, never reused despite a candidate successor' autonomy class. The panel is therefore silently conditioned on survivors of the increasing-returns race. The discriminating observable and the missing abandoned-class cases are unanswered; a censored-data or matched not-reused-control design would be required to address it. (raised by brian_arthur) - **[identification]** The candidate has NOT pre-committed to the funding-order monotonicity test (slope-rank vs date-of-first-investment-rank vs ex-ante propositional-maturity-rank). Although TechPort start-date/funding records make the test constructible, the design pre-registers only the fixed-effects feasibility check and the influence diagnostic, and offers only cross-class (maturity-ordered) heterogeneity as its path-contingency handle. Whether the per-class slope rank tracks funding sequence rather than learnability therefore remains untested and unpre-committed; the build-or-wait robustness of beta to the funding-basin objection is unresolved on current evidence. (raised by brian_arthur) - **[measurement]** A recombination-stock measure counting components inherited from ANY prior class cannot be built from current retrieval: the panel contains no cFS component-level dependency graph and no NTRS component-reuse statements, only McComas 2013's framework description, which the candidate concedes does not quantify reuse. The design contemplates only a coarse 'cross-class heritage' robustness row, not a dependency-graph-derived combinatorial stock. Whether the within-class slope survives, or inverts sign, under a true combinatorial heritage stock is therefore undetermined on the evidence retrieved this turn. (raised by brian_arthur) - **[mechanism]** REFUSED for lack of supporting evidence on the decisive empirical claim. Adaptive expectations is correctly one of Arthur's four self-reinforcement mechanisms (set-up/learning effects, coordination/network effects, scale economies, adaptive expectations; Arthur 1989, 10.2307/2234208), so the channel the adversary names is real and the dossier confirms the four-mechanism taxonomy. But the design carries NO flight-software-lineage or team-continuity variable: the panel's only flight-software-provenance source is McComas 2013 (a framework description, not a component-authorship or team-continuity record), and the dissertation specifies no authorship-coding or cFS-provenance coding step. There is therefore no retrievable basis to code institutional continuity per episode, to enter it alongside CumHeritage, or to report whether the heritage slope survives a continuity control. Whether the candidate is measuring the persistence of an institution rather than an artifact-level learning curve cannot be settled from current retrieval, and no number or finding can be asserted; this requires a build-phase provenance-and-authorship sweep the design has not performed. (raised by brian_arthur) - **[measurement]** No retrieved source establishes that the cFS dependency record plus lessons-learned can actually be assembled into a within-episode cross-class reuse-stock index at the resolution christensen_c demands, nor that the candidate has pre-committed to reporting the reuse-stock slope head-to-head against the heritage-count slope. The candidate itself flags the cFS corpus footprint as thin and its cost effect as unmeasured anywhere, so whether a reuse-stock regression would even be estimable, and whether it would return a significant negative slope while heritage-count stays flat, is unresolved by the present record. (raised by christensen_c) - **[identification]** No retrieved source identifies a TechPort or NTRS scope-record observable that distinguishes a job-defined heritage link (same operational job, close-the-perception-action-loop-without-ground-in-the-loop) from an artifact-class link, and nothing in the dissertation or the corpora reports a job-based re-clustering or a refit with job-based fixed effects. Whether the negative slope survives, strengthens, or collapses under job-based fixed effects is therefore unresolved; the candidate has not run it and no external source settles it. The empirical claim cannot be asserted without confabulation. (raised by christensen_c) - **[economics]** No retrieved source establishes that TechPort or the lessons-learned record can supply a per-episode adoption-friction measure (schedule reserve consumed, descope events, or review-board findings attributable to the autonomy element) in a form fittable against cumulative heritage, and nothing reports whether the heritage effect appears on such a measure. The DS1 lessons-learned attests organizational cost qualitatively for a single first-of-kind episode but is a narrative, not a panel variable. Whether qualification cost falls while adoption friction stays flat or rises is therefore unresolved by the present record and cannot be asserted. (raised by christensen_c) - **[rival]** No retrieved source supplies the netted total-cost-to-field accounting the question requires: the ground-cost side (per-episode ops-team staffing hours and downlink burden) quantified and netted against on-board autonomy NRE for AEGIS, DS1, or EO-1 was not found in AMOS, ACTA, Space Economy Papers, the christensen_c brain, or NTRS/OpenAlex this turn (NTRS returned 0 hits for both 'AEGIS ... operations cost' and 'EO-1 ... operations cost reduction' framings). The qualitative artifacts exist but the quantitative on-board-vs-ground cost split, the number that would let the on-orbit-only slope be contrasted against a total-cost-to-field slope on a real decision, is unbuilt and unretrievable. Per the question's own instruction, this side is marked a gap. Without it the rival (displacement vs learning) cannot be empirically adjudicated on a documented episode; it can only be motivated theoretically. (raised by christensen_c) - **[measurement]** No retrieved source contains the candidate's matched within-class cost pairs or the underlying TechPort/NTRS cost rows, so the demanded certification of layer-1-vs-layer-2 ontological commensurability and the audit of whether layer-2 NICM imputation endogenously embeds the experience-curve relationship in beta cannot be settled. AMOS, ACTA, and Space Economy returned 0 relevant hits; NTRS returned only the tangential 'Technology Cost and Schedule Estimation (TCASE) Final Report' (NTRS 20160000761) and OpenAlex returned nothing on the NICM CER substrate. Refused: empirical certification withheld pending access to the candidate's source-cost ledger. (raised by dietz) - **[identification]** No retrieved source provides an NTRS heritage chronology or cFS provenance map linking reusable software/verification artifacts across the candidate's development episodes, so the cross-class reuse fraction cannot be computed and the within-class-vs-component-lineage estimand question cannot be settled empirically. Targeted AMOS/ACTA queries on cFS reuse, flight-software provenance, and capability-class cost data returned 0 hits; NTRS returned no cFS-reuse records. Refused: cross-class reuse fraction not derivable from available retrieval. (raised by dietz) - **[mechanism]** No retrieved source contains TechPort lineage links or project documentation for this candidate's episodes recording whether successors actually ingested predecessor artifacts, so a transfer-realized heritage variable cannot be constructed and beta cannot be re-fit count-vs-realized. The Dietz frame establishes the distinction is real and testable, but the data to run the test was not returned. Refused: re-fit on transfer-realized heritage withheld pending TechPort lineage and project-ingestion records. (raised by dietz) - **[measurement]** The candidate cannot currently exhibit the demanded edge-by-edge transfer graph or test count-vs-in-degree monotonicity. The dissertation's own Section 3.2 concedes the AEGIS-Opportunity to AEGIS-ChemCam reuse is documented only in NARRATIVE terms ('the paper does not report the porting cost as a dollar figure'; 'Confidence that the cost consequence has been measured is zero'), and no fitted coefficient exists (design stage). Retrieval (OpenAlex, NTRS Francis et al. AEGIS-ChemCam 20220001538, cFS reuse reports 20090005965/20180001888) returns qualitative reuse narratives but NO named-artifact dependency edges with downstream re-fielding facts and NO in-degree vs CumHeritage comparison. The graph and the monotonicity test are unbuilt; the AEGIS pair is asserted as 'cleanest' on narrative reuse, not on a verified named-module transfer edge. (raised by dietz) - **[identification]** No accounting-boundary reconciliation exists and cannot be certified from the retrieved record. The dissertation's cost variable is a three-layer construction (extract autonomy-specific NRE where reported; NICM-class parametric IMPUTATION where not; deflate), with most figures imputed because, in the candidate's own words, the AEGIS, EO-1, and Remote Agent papers 'do not report the autonomy-specific development cost in a form comparable.' The candidate certifies the OPPOSITE of what the question demands: that per-episode autonomy NRE is largely unobserved and imputed. NICM methodology retrieval returned nothing in-corpus; the producer/consumer ledger-assignment of a reused component's qualification cost is undocumented. With imputed NRE there is no per-observation boundary audit, so the double-counting threat (producer's sunk qualification cost reappearing inside the consumer's NICM-imputed NRE) cannot be ruled out from evidence. Dietz's boundary-handshake test (make every inter-organizational boundary transaction explicit) names exactly the missing artifact, but the reconciliation table is absent. (raised by dietz) - **[mechanism]** The candidate has not re-cut the panel to the completed-transfer subset and cannot show whether the slope survives, because (a) no slope is fitted at all (design stage, 'No fitted coefficients are reported'), and (b) the CumHeritage variable is explicitly a forward-only cumulative COUNT of prior flight demonstrations, which the dissertation itself admits 'cannot distinguish a demonstration that codified reusable knowledge from one that produced a bespoke artifact.' Retrieval surfaced no TechPort/NTRS evidence pinning the five transaction elements (named executor + production fact + accepting initiator + re-qualification accept act) for even the AEGIS pair beyond a narrative 'build-on.' The forward-only-vs-completed-transfer comparison the question requires is therefore untested; the distinction between heritage and calendar adjacency that Dietz's axiom would force is acknowledged in the prospectus but not operationalized. (raised by dietz) - **[governance]** No retrieved source establishes a quantitative value of the candidate's fitted slope or a worked counterfactual in which that slope, applied to a specific AEGIS/AutoNav/NICM-class sequencing decision, flips or numerically bounds the actual outcome under stated institutional constraints. The dissertation is a research design (the panel is to be assembled, beta is to be estimated), so the actionability falsifier cannot be settled from current evidence; it can be constructed from the public AEGIS reuse record but has not been executed. The relevance claim is therefore neither confirmed nor refuted by retrieval. (raised by gangale) - **[identification]** No public, retrievable cFS/GSFC record was found this turn that names the specific appropriation/directorate budget line that actually paid for the cFS framework's development (NTRS cFS-funding and OpenAlex 'cFS appropriation funding directorate amortized cost' queries both returned no funding-line document). Without that record the candidate cannot DEMONSTRATE which line funded the substrate, only assert the exclusion treatment. The empirical demonstration that the chosen assignment does not manufacture the negative slope therefore cannot be completed from public sources as asked; it remains an unproven claim pending JPL/GSFC internal cost ledgers. (raised by gangale) - **[rival]** The retrieval this turn does not settle whether the 1999-2023 panel's autonomy-qualification cost is denominated in a single time-stable frame. There is documentary evidence that the LOCUS of autonomy work moved (per-mission flight experiments RA/EO-1/AEGIS -> a shared cross-mission substrate cFS and cross-cutting M2020 infrastructure), which establishes the displacement is an institutional frame change rather than a level shift. But no source retrieved provides the conversion factor between a per-mission-flight-NRE dollar and a shared-substrate/ground-operations dollar, nor a documented Autonomous Systems and Robotics sequencing decision with quantified build-vs-wait cost in both frames. The candidate must therefore CONCEDE the single-frame denomination is not demonstrated and that the design implicitly assumes a per-mission-to-substrate conversion factor of 1.0 that no retrieved record validates; absent that factor or a frame-specific deflator, decade fixed effects cannot absorb a frame change and the slope's sign on displaced episodes is not identified. (raised by gangale) - **[empirics]** No retrieved source provides the candidate's own power/false-negative simulation: neither the dissertation (which explicitly declines to quantify power) nor AMOS, ACTA, or Space-Economy corpora return a Monte Carlo of detection probability for a tens-of-observations two-way-fixed-effects autonomy-cost panel under a true 15-20% learning rate. The simulation must be executed by the candidate against the assembled panel's actual class/decade structure; it cannot be answered from existing retrieval. (raised by mccloskey) - **[economics]** No retrieved source supplies the decision-flipping magnitude: there is no representative JPL cost-capped-mission NICM figure plus build-or-wait threshold in the dissertation, AMOS, ACTA, or Space-Economy corpora from which to compute the smallest slope that changes a real portfolio decision, nor a specified loss function over beta. The candidate concedes the magnitude is unestimated. This is unanswerable from current retrieval and requires the candidate's own ex-ante threshold derivation. (raised by mccloskey) - **[measurement]** No retrieved source contains an audited-subset-vs-imputed-heavy slope comparison. The panel is explicitly unbuilt, the reliability flag exists but the registered specifications only weight rather than split, and no corpus supplies an empirical estimate of how much of the slope lives in imputed observations. The split is settleable only after the candidate assembles the panel and re-estimates; current retrieval cannot resolve it. (raised by mccloskey) - **[empirics]** The exact fraction of H0-rejecting draws that are decision-indeterminate cannot be retrieved or computed: the dissertation fits no coefficient, assembles no panel, and quantifies no NICM-class imputation-error variance, so the actual class/decade degrees of freedom and the joint (significant, decision-determinate) distribution do not exist in the record. The candidate must, at build time, add a magnitude-precision gate to the decision rule -- e.g. report the fraction of bootstrap/posterior draws whose implied per-doubling reduction crosses the program office's build-or-wait threshold -- before any significant slope is called decision-relevant. (raised by mccloskey) - **[economics]** The implicit per-doubling cost reduction that NASA's 'by assertion' justification already banks on is never quantified in the dissertation, and no interval is fitted, so it is impossible to adjudicate whether the study's estimate would confirm or overturn the incumbent magnitude. Remedy owed at build time: state the incumbent assumed slope in its own units (the per-doubling reduction a sequencing decision implicitly assumes), then pre-register a decision criterion on whether the fitted NICM-based interval lies inside, excludes, or straddles that magnitude -- otherwise the contribution cannot demonstrate decision-relevance against the assumption it claims to replace. (raised by mccloskey) - **[measurement]** The empirical question asked, whether the codification indicator and the raw flight count actually diverge across THIS candidate's specific panel of episodes, cannot be settled from retrieval. No retrieved source contains the per-episode coded data or the candidate's own coding of its panel. The public record holds lessons-learned and V&V papers (e.g. Livingstone 2 on EO-1; V&V reviews for space autonomous systems) but no quantified codification-vs-count comparison for this dataset. The divergence/convergence finding is an artifact only the candidate's own un-retrievable project data can produce; asserting either outcome would be confabulation. (raised by mokyr) - **[mechanism]** Whether THIS candidate's class-specific slopes actually order themselves by an ex-ante propositional-maturity rank cannot be answered from retrieval. No retrieved source contains the candidate's fitted per-class slopes, its cost figures, or a maturity-ordered slope test on its panel. Confirming that the slopes do (or do not) order by rank would require the candidate's own un-retrievable estimation output; asserting the result is confabulation. The retrievable record licenses the design of the test, not its outcome. (raised by mokyr) - **[identification]** Whether THIS candidate's estimated cost slope tracks team/codebase/personnel continuity rather than the flight count cannot be settled from retrieval. No retrieved source provides the AEGIS Opportunity-to-Curiosity (or any panel episode) team-continuity coding, codebase-overlap measure, or personnel-overlap data joined to cost. Public V&V/lessons-learned papers establish that the episodes exist and that tacit transfer is a live threat, but do not contain the continuity-vs-count decomposition for this dataset. The answer is a property of the candidate's own un-retrievable project records; asserting it would be confabulation. (raised by mokyr) - **[empirics]** No power analysis exists on the actual TechPort/NTRS episode count, and no pre-committed minimum effective within-class sample for the codification x heritage interaction is on the record; the candidate explicitly defers the minimum-detectable-effect to the unbuilt panel and has NOT committed to withdraw the moderator claim if the interaction proves unestimable (the standing commitment is the opposite: report it as suggestive heterogeneity). Retrieval this turn (dissertation Ch5-7) settles WHAT was committed but cannot supply the missing power number, because the panel is not assembled. The pre-commitment Mokyr demands is therefore absent from the corpus. (raised by mokyr) - **[measurement]** The corpus does not contain the demonstration Mokyr demands. Two ingredients of the requested cross-tabulation are absent. (1) The NICM-class imputation inputs ARE documented as scope drivers (mass, power, data rate, analogous subsystem parameters per Sec4.4/Sec4.5.4, parametric basis Stahl et al. ref 97), but (2) there is NO operationalized codification coding rule anywhere in the dissertation: the codification moderator is described qualitatively as the presence/absence of reuse infrastructure (Core Flight System, shared autonomy frameworks) by capability class, never as a coded per-episode indicator with stated coding inputs. With no codification variable defined, no NICM-input-vs-codification cross-tabulation can be exhibited, and the orthogonality claim cannot be tested from the present record. The candidate partially anticipates the circularity in spirit (the maturation/technology-readiness covariate is added precisely to separate a maturity effect from a heritage effect, Sec5.3 robustness spec 1), but maturity-orthogonality is not codification-orthogonality, and the specific artifact Mokyr names (NICM scoring a mature/codified class cheaper by construction, then that same cheapness being read back as a steeper learning slope) is neither ruled out nor cross-tabulated. This is a genuine measurement gap: the regressand-construction circularity remains live until a codification indicator is defined and shown orthogonal to the NICM scope inputs. (raised by mokyr) - **[mechanism]** No zero-team-overlap reuse case is exhibited in the corpus, and live retrieval did not surface one. The candidate's heritage variable is, by its own definition, a forward-only count of prior in-class flight demonstrations 'regardless of whether the codified artifact actually diffused' to an arms-length team (Sec4.4 CumHeritage definition), which is exactly the in-class-flight-count-not-diffusion measure Mokyr flags. Section 3.6 documents the cFS reuse substrate (McComas, NASA GSFC Core Flight System, ref 78) but states the literature 'says almost nothing about its cost consequence' and that 'the corpus footprint of the Core Flight System is modest.' A live NTRS API query for cFS reuse/heritage returned zero citations this turn, and the Space Economy and AMOS/ACTA brains returned nothing on cross-team codified reuse. The Mokyr-dossier backing (Jaffe, Trajtenberg & Henderson 1993 on the geographic localization of knowledge spillovers) actually SHARPENS the worry rather than resolving it: spillovers are empirically localized, so absent an exhibited arms-length case the documented cost reductions are equally consistent with intra-JPL personnel/codebase continuity (a labor-hoarding/co-location effect) as with a transferable access-cost decline. The candidate has flagged a build-phase 'focused sweep on Core Flight System reuse economics' (Sec7.5/Ch8) as the highest-value substantive strengthening, conceding the evidence is not yet in hand. Until a within-class successor with no personnel or codebase overlap is documented, the slope's transferability across centers and contractors is unestablished and the planning parameter offered to NASA may not generalize beyond JPL. (raised by mokyr) - **[measurement]** MEASUREMENT GAP (partial): the structural co-movement of authority and qualification cost is grounded, but no retrieved source supplies the candidate's required artifact - a quantified authority-weighted qualification-burden index per episode, nor a regression demonstrating the heritage slope is identified holding action-stage authority fixed. The candidate has not shown the slope is purified of authority confounding inside the regressand; this must be constructed and adversarially defended, not asserted. (raised by sheridan_verplank) - **[rival]** RIVAL GAP (partial): retrieval substantiates the lumberjack warrant and that AEGIS operates on a constrained objective-defined envelope, but did NOT return a source documenting a specific descope of off-nominal coverage, a deferred fault case, or an operations-phase anomaly causally attributable to the autonomy for the AEGIS Opportunity-to-ChemCam pair or for Remote Agent (the RAX in-flight deadlock returned no resolvable citable record this turn). The claim that the falling slope is NOT a scope reduction therefore remains unsettled; the candidate must hold exception-envelope scope constant across each within-class heritage pair and present the evidence. (raised by sheridan_verplank)