# Interrogation mind-map: JPL_MGMT_SMA_TECH_04

Nodes: 129 | questions: 46 | grounded claims: 43 | gaps: 40

## Questions

- **[identification]** Once environment, prominence, mass class, epoch, and subsystem random effects are conditioned out, how many mission-by-subsystem cells survive in strata containing BOTH a heritage-rich and a heritage-poor subsystem at comparable parts-class and test-fidelity? Report the overlap-cell count and per-stratum heritage variance, because near-zero overlap means the heritage coefficient is identified off extrapolation across non-overlapping strata. (raised by angrist_pischke)
- **[mechanism]** The falsification rule reads attenuation of b1 when parts-class and test-fidelity enter Model B as evidence for H1, but identical attenuation occurs if heritage operates THROUGH parts-class and test-fidelity as mediators rather than as confounders. What observable discriminates confounding from mediation, and can the heritage-to-parts-class and heritage-to-test-fidelity assignment relationships be estimated from the JPL archives so a negative heritage-to-test-fidelity link marks heritage as upstream cause rather than confounded correlate?
- **[measurement]** If as-run (and even as-planned) test scope is chosen in response to the heritage claim, a deep-heritage box deliberately given a thinner campaign, then test-fidelity is downstream of heritage assignment and conditioning on it is the bad-control trap. Can it be demonstrated, from parts-control-board and I&T-plan records, that planned test scope was fixed BEFORE the heritage determination at design review for the cells used, so test-fidelity is genuinely pre-determined and not a post-treatment channel improperly held fixed?
- **[identification]** Can you exhibit at least one dated policy switch (FCC 2022 five-year deorbit rule, NASA class A-D mission-classification tailoring revisions, EEE/PEM parts-policy threshold flips) under which otherwise-comparable subsystems on the same heritage rung had planned test-fidelity or required parts-class reassigned by the rule rather than the program, and report its first-stage strength as a shifter of the rigor regressors? If no such datable shifter exists in your records, concede the contribution is a conditional association, not a design-based estimate. (raised by angrist_pischke)
- **[measurement]** The heritage-depth variable is coded from a claim a board entered at design review, treated as data about the article; but upgrading a design-heritage box to a 'same-environment flight heritage' label can be an organizational act that retires embarrassment about a thin test budget, in which case the label and the rigor cut are co-produced espoused theory, not independent provenance, and H1's heritage attenuation under test-fidelity would be mechanical decoupling rather than a rival cause winning. What in the design measures the gap between heritage-as-espoused-at-the-board and the article's enacted as-built provenance (parts lots, environmental qualification), since two coders reconciling against 'the source document' only reproduce the espoused record without auditing it against behavior? (raised by argyris)
- **[identification]** Barring conditioning on the test pass/fail verdict is correct, but the more dangerous endogeneity is behavioral and upstream: the heritage label may cause the test-fidelity cut, an organizational routine in which 'it's heritage' is the undiscussable warrant for descoping qualification, making test-fidelity a mediator on the heritage-to-failure path rather than a competing confounder. What corpus evidence from design-review minutes or descope-decision records (not the candidate's own coded indices) would show the heritage claim was the spoken justification invoked to reduce test scope, and how does the nested Model A to Model B comparison change interpretation if that mediation is documented? (raised by argyris)
- **[rival]** The survivorship correction weights against under-documentation but does not touch a second, organizational selection: a board that quietly drops a heritage claim after an early on-orbit anomaly produces a review record that no longer carries the heritage label that licensed the failure, so the most diagnostic cases (heritage invoked, rigor cut, subsystem failed) are precisely the ones where the heritage attribution is scrubbed from the archive, near-miss re-encoding operating on the independent variable. What independent evidence (superseded review baselines, anomaly-board minutes, pre- versus post-anomaly heritage codings of the same subsystem) would detect heritage claims retired after the fact, and how would their presence bias the estimated heritage coefficient relative to the truth? (raised by argyris)
- **[measurement]** Test-fidelity is coded from the integration-and-test plan as-planned and as-run, but the as-run record is authored inside the same heritage banner as the heritage claim: a board that accepts heritage and thins qualification documents the thinned campaign as the planned-and-adequate scope, not as a reduction. From paired records on one article, find a case where the as-run test documentation calls the campaign nominal while an independent source (parts-stress waiver, delta-qual deviation, review-board action item, anomaly closure) shows scope was cut against the heritage argument. If the index reads only the plan's self-description, what observation reveals it is measuring espoused rather than enacted rigor? (raised by argyris)
- **[identification]** The survivorship correction fits a documentation-probability (IPW) model on observed mission characteristics. Dillon & Tinsley show NASA re-encodes a near-miss that produced no bad outcome as a success, suppressing the record. A heritage subsystem that suffered an early recoverable in-flight anomaly is such a near-miss, and its closure is written by actors with an interest in not impugning the heritage decision. From the closure records, show whether completeness and root-cause coding of an anomaly entry depend on whether the anomaly was survived versus mission-affecting, after conditioning on the IPW covariates. If documentation completeness is a function of the re-encoded outcome, the weights are fit on a probability endogenous to the correction target. What evidence would show documentation-probability is independent of near-miss re-encoding rather than driven by it? (raised by argyris)
- **[measurement]** Heritage-depth and test-fidelity coding are defended with chance-corrected inter-coder agreement on a common subset (Sec 3.5 / 4.5). But both coders read the same project documents, products of one program's behavioral world; high kappa can mean both coders are faithfully transcribing the board's self-sealing account of what counts as heritage. Agreement against a biased source is reliability without validity. Name an external behavioral anchor independent of the project's heritage narrative against which heritage-depth codes could be validated (e.g. as-built parts-lot and environmental-qualification facts from procurement and radiation-test records, not the heritage assessment matrix). On what fraction of audited cells does recorded heritage depth disagree with that external article-level anchor, and in which direction? (raised by argyris)
- **[identification]** Are the three regressors (heritage depth, parts-class choice, test-fidelity scope) co-determined outputs of a single program-pressure loop rather than independent drivers? Regress each standardized regressor on a measured pressure proxy (budget-margin-at-PDR, schedule-reserve-at-PDR, cost-growth-to-date) within strata; show the partial-correlation matrix of the three AFTER residualizing on pressure. If pressure predicts all three with shared sign, the Model A->Model B coefficient migration reallocates one pressure signal across three collinear proxies rather than adjudicating three independent drivers. (raised by forrester)
- **[mechanism]** Is the heritage regressor endogenous to lagged outcomes via a cross-mission learning loop the static proportional-hazards spec and launch-epoch dummies cannot represent? Epoch dummies absorb a secular TREND but not event-driven feedback that resets at each failure, not each decade. Can the assembled record date each subsystem's heritage-depth coding AND the standing heritage-caution posture at that date, so coded heritage depth can be tested for Granger-response to lagged same-lineage failure history? If coded heritage depth responds to lagged lineage failures, the regressor is endogenous to the outcome it is meant to explain. (raised by forrester)
- **[empirics]** Does a shared finite assurance-budget stock create a within-mission balancing loop that couples the three regressors at the cell level for reasons unrelated to delivered reliability? A deep-heritage claim on subsystem A consumes shared review-and-test budget unavailable to sibling subsystem B, mechanically lowering B's test-fidelity FLOW. From integration-and-test as-run records, can total mission assurance hours be measured as a stock and each subsystem's draw on it, to show whether a subsystem's test-fidelity is depressed when a sibling's heritage claim is deep? If present, per-cell coefficients confound article properties with an allocation game, and the cell-vs-aggregate distinction the design discards is exactly the one that matters. (raised by forrester)
- **[measurement]** Heritage depth is treated as a static level read off a design-review claim, but heritage is a STOCK accumulated through a flow (each successful flight adds, each rebuild-to-new-environment partially drains). A regression conditioning on the level cannot see the rate. Construct per subsystem cell the accumulated successful-flight-hours and the elapsed time since the last identical-configuration flight, and test whether realized failure hazard is predicted by the heritage LEVEL (ordinal rung) or by these FLOW-and-age quantities. If age-since-last-flight dominates the rung, the headline variable is mismeasured. (raised by forrester)
- **[identification]** A strong heritage claim lowers planned test scope, which lowers the probability the cell is fully documented and that an early anomaly is formally reported, which feeds back to make heritage-rich cells look more reliable in exactly the records being regressed on. This is a REINFORCING LOOP on the OUTCOME's observability, not a confounder to control. Estimate whether documentation/reporting probability is a decreasing function of heritage depth AFTER conditioning on realized early failure, and show the inverse-probability weighting breaks that loop rather than reweighting within it. If the loop survives, the heritage coefficient is partly an artifact of heritage suppressing its own failure record. (raised by forrester)
- **[mechanism]** The proportional-hazards model freezes every regressor at its design-review value, but failure hazard integrates over years of on-orbit residence; infant-mortality clearance and parts wear-out run on different clocks than the once-and-done test campaign. Decompose realized failure into the FAST early-window infant-mortality regime (where test-fidelity, a one-time flow, plausibly acts) and the SLOW wear-out regime (where parts-class, a property of an aging stock, acts), and show a single frozen-covariate hazard does not force a deep-heritage cell reliable in the fast regime to also count as reliable in the slow regime. If heritage's advantage lives in one timescale, a blended model has optimized the flow inside a system whose slow ceiling it never separated. (raised by forrester)
- **[measurement]** Name the catalog of record for the denominator and produce the reconciliation table BEFORE any coefficient: how many distinct JPL-class spacecraft launched in the epoch window per a fixed launch-manifest census; how many survive the three-way linkage (anomaly system + NTRS + JPL parts/heritage archives) to yield at least one fully-coded mission-by-subsystem cell; what fraction never enter the frame because the canonical-mission-identifier match or the subsystem crosswalk fails; and the per-source match rate at each linkage stage, not just the surviving-cell count. (raised by mcdowell)
- **[identification]** Inverse-probability-of-documentation weights are estimated from observed mission characteristics, yet the cells that drop out of linkage are the ones whose documentation is absent. On what observed covariates is the documentation-probability model fit, and can the candidate demonstrate from the launch-manifest frame that completely-unlinked spacecraft (zero coded cells) share enough covariate support with the documented subset that their documentation probability is identifiable rather than extrapolated? If heritage-rich flagships are the ones that survive linkage, show the covariate distribution of linked vs unlinked population on environment, prominence, mass class, and epoch. (raised by mcdowell)
- **[empirics]** What is the smallest documented unit the linkage can resolve, and how do operations-caused and minor anomalies that never triggered formal reporting bias the failure denominator differently across heritage-rich and heritage-poor subsystems? If flagship missions generate richer anomaly reporting than competed-line missions for the same physical event rate, the time-to-first-failure outcome measures reporting intensity confounded with prominence, not delivered reliability. Show the anomaly-reporting rate per subsystem-year as a function of mission prominence on the linked sample, so the panel can see whether the outcome floor is uniform across the strata the overlap region compares. (raised by mcdowell)
- **[measurement]** Before any coefficient, publish the linkage reconciliation ledger at the mission-by-subsystem cell the study estimates on: of all cells in the launch-manifest frame, what fraction resolve to a single agreed cell across all three source classes (anomaly system, NTRS, JPL archive), what fraction drop as unmatchable, and what fraction are many-to-one or one-to-many merges, with the per-cell match rate broken out by source pair. A heritage-versus-parts coefficient estimated on the clean-link subset is a coefficient on the well-documented survivors, not on the population the manifest defines. (raised by mcdowell)
- **[measurement]** Show, from the archives, the per-epoch documentation density and taxonomy version for each of the three indices (heritage matrix, EEE parts-class, integration-and-test conventions) across the launch-epoch window, and demonstrate that a fixed true heritage depth or parts-class maps to the same coded value in the earliest and latest epochs. A launch-epoch dummy absorbs a level shift in hazard but not a change in how the regressors themselves are measured; if a fixed true value does not map to a fixed code, the heritage-versus-parts comparison is confounded with instrument drift in the census. (raised by mcdowell)
- **[identification]** Heritage depth is coded from an operator-declared figure (the claim a board entered at design review). What is the independent check on that label? Reconcile the board's declared heritage depth against the as-built record (actual parts lots flown, actual fabrication supplier, actual prior-flight environment match) and report the discrepancy rate, the share of cells where declared same-environment flight heritage is contradicted by the as-built parts or environment. Without that reconciliation the heritage regressor is the unverified operator claim and the study tests the reliability of a label rather than the reliability of provenance. (raised by mcdowell)
- **[identification]** Commit to ONE declared DAG. If the archives show heritage depth predicts parts-class and test-fidelity (Section 6.2's own diagnostic), then heritage->parts-class->failure and heritage->test-fidelity->failure are directed paths, and entering b2,b3 into Model B conditions on mediators: under the back-door criterion that estimates a controlled direct effect, so the b1 attenuation sold as 'falsification toward H1' is mechanically guaranteed by mediation, not evidence of spuriousness. Name the declared DAG, the present arrows, and the conditional-independence implication that distinguishes 'mediator artifact' from 'genuine confounding' -- because the regression output is numerically identical in both worlds. (raised by pearl)
- **[mechanism]** If heritage's effect runs THROUGH parts-class and test-fidelity, and an unobserved 'program competence' (Section 6.2's third rival) confounds heritage<->failure, then parts-class+test-fidelity are a candidate fully-mediating set and the front-door criterion could identify the total heritage effect despite the unobserved confounder -- the estimand the selection-on-observables design cannot recover. Claim (a) FULL mediation (front-door applies, report the front-door estimate not a back-door coefficient race) or (b) PARTIAL mediation (b1 in Model B is neither total nor clean direct, and the falsification rule reads an uninterpretable residual)? Name, from the linkage data, the through-channel vs around-channel share of heritage's association with failure. (raised by pearl)
- **[empirics]** Overlap (positivity) is the wrong instrument for the threat you face. Even with perfect overlap at matched parts-class and test-fidelity, the headline test collapses if those matching variables are mediators, because forcing overlap on a mediator is conditioning on a mediator; and the confounder-vs-mediator ambiguity is INVISIBLE to a positivity/balance table -- it lives in arrow directions no balance diagnostic recovers. Which testable conditional-independence implication of your declared graph -- not a balance count -- will you check in the LLIS/NTRS/JPL data to license the direction of the heritage->parts-class and heritage->test-fidelity edges, and what observed independence would falsify the mediator reading and rescue the confounder-adjustment interpretation? (raised by pearl)
- **[identification]** Draw the selection diagram (not just the causal DAG): add square S-nodes for every mechanism that differs between the study population (documented JPL-class missions, fixed epoch) and the decision population (future heritage-discount decisions across the assurance portfolio). Section 6.1 issues an actionable review-board rule while 6.3 concedes the result is 'a statement about JPL-class missions.' Show which arrows the S-nodes sit on and demonstrate from the data that the standardized heritage-to-hazard coefficient is invariant across the launch-epoch and mass-class partitions already controlled for; if not invariant, the diagram is non-transportable and do(heritage-discount) is not licensed for the decision population. (raised by pearl)
- **[identification]** The IPW-of-documentation correction (4.3) treats missingness as recoverable by conditioning on observed mission characteristics, yet the same section states under-documented cells are 'disproportionately early-failed', the signature of a selection node pointed into by the outcome (failure), the one structure recoverability theory shows IPW cannot fix from observed covariates alone. Fit the documentation-completeness model both ways and report whether documentation probability still depends on realized early-failure AFTER conditioning on the full observed covariate set; if residual dependence is nonzero, name the external/surrogate margin (e.g., a population-level launched-spacecraft count by class/epoch) that restores recoverability. (raised by pearl)
- **[rival]** State the transport formula you would actually use, not the within-study coefficient. If a board applies the discount rule to a subsystem flown to a NEW radiation environment, that target cell is exactly where same-environment flight heritage is by construction absent, so the heritage rung the top category encodes does not exist in the target. From the heritage-depth-by-environment cross-tabulation, show whether any study cell shares the target's (heritage-depth, environment) profile; if none does, the query is an extrapolation across an S-node on the environment arrow and the estimate transports zero information. Commit to re-weighting, re-targeting the estimand to the overlapping sub-population, or declaring the decision-relevant cell non-identified. (raised by pearl)
- **[identification]** Is heritage depth a manipulable treatment with a well-defined potential outcome Y_i(heritage=k) at fixed parts-class and test-fidelity, given that the top category 'design+build+same-environment flight' is DEFINED to require equivalent parts and verified environmental qualification? Settle it on the assembled frame: report joint cell counts of heritage x parts-class x test-fidelity and show whether any off-diagonal cell (deep heritage at low parts-class; shallow heritage at full test) is non-empty. (raised by rubin)
- **[identification]** Can the mission-by-subsystem unit support a SUTVA-valid potential outcome when a shared EEE parts lot or shared power/data bus makes one subsystem's failure a direct function of another's parts assignment? Quantify the exposure: fraction of subsystem cells sharing a lot or bus with another cell in the same mission, and how the heritage-vs-parts contrast moves when interfering cells are excluded vs retained. If large, restate the estimand on the lot/bus unit rather than asserting SUTVA holds 'mostly'. (raised by rubin)
- **[measurement]** Are the overlap diagnostic and inverse-probability-of-documentation weighting outcome-blind, given that documentation completeness is itself caused by the outcome (early-failed cells may have heritage quietly dropped, thinner parts records, truncated test history)? Report, on the launch-manifest frame, the share of early-failed cells for which all three regressors are recoverable from strictly pre-launch documents (design-review heritage matrices, as-procured parts records, as-planned I&T plans dated before commissioning), and show the heritage-vs-parts contrast on that pre-launch-only subset against the full weighted sample. (raised by rubin)
- **[measurement]** On the assembled mission-by-subsystem frame, within each single heritage rung, partition cells by which underlying claim (design-only, design+build, design+build+parts, same-environment flight) produced that rung and report the within-rung-across-version dispersion of the realized infant-mortality outcome. If a fixed rung maps to materially different outcomes by sub-version, the rung is not one treatment level (SUTVA no-hidden-versions limb) and b1 averages over incommensurable interventions. Show the within-rung-across-version outcome spread before any coefficient is read. (raised by rubin)
- **[identification]** State the auditable blinding seal, not the intention: were heritage-depth and test-fidelity coders shown source excerpts with all post-commissioning anomaly, end-of-life, and mission-outcome text physically redacted, and can you produce, on a held-out subset, the agreement between blind-coded and unblinded-coded heritage rungs? If blind and unblinded coding diverge, the indices were constructed with knowledge of outcomes and the design-before-outcomes objectivity claim fails. (raised by rubin)
- **[identification]** The survivorship IPW makes 'being fully documented' a second treatment with its own assignment mechanism. On the launch-manifest frame, show the covariate balance the documentation-propensity weights achieve on that second assignment: after weighting, do thinly-documented cells overlap fully-documented cells on environment, prominence, mass class, epoch, and subsystem, or is there a region with documented cells and no undocumented donors (or the reverse)? Without overlap the weights extrapolate and the survivorship fix re-imports the selection it claims to remove. (raised by rubin)
- **[measurement]** Inter-coder agreement defends reliability, not construct validity. Name an independent operationalization of delivered reliability-margin per article (as-run delta-qualification test margin, radiation dose-derating ratio, post-delivery rework-discrepancy counts) NOT read from the same JPL heritage/parts/test archive, and state the convergent-divergent statistic (MTMM-style correlation, not kappa) plus the value that would force abandoning the index. (raised by shadish_cook_campbell)
- **[empirics]** Instrumentation drift lives in the REGRESSORS: if heritage-matrix granularity and test-plan documentation depth deepened over epochs, a fixed true heritage level is coded as a higher ordinal later purely because the paper trail thickened, correlating heritage with epoch. An outcome-side epoch dummy cannot difference this out. What instrument-anchored measurement-invariance test (e.g., re-code held-out early- and late-epoch subsystems against one fixed rubric blind to launch date, report coded-level-by-epoch interaction) will show the ordinals carry the same meaning across epochs, and at what interaction level do you concede drift rather than construct change? (raised by shadish_cook_campbell)
- **[identification]** Heritage depth may be a proxy for the prominence control: Section 1.4 defines the strongest heritage as same-environment flight on flagship-class (high-prominence) missions, and the overlap section concedes the field always pairs deep heritage with high parts-class and full test. Conditioning on prominence can bleed the heritage signal or act as a bad control if prominence is coded from the same archives. What does the overlap diagnostic return as the count of mission-by-subsystem cells where heritage varies WHILE prominence, parts-class, and test-fidelity sit in a common stratum, and below what cell count do you declare the heritage coefficient non-identified rather than reporting an attenuated estimate as if it answered H0 vs H1? (raised by shadish_cook_campbell)
- **[identification]** Epoch dummies absorb both the history threat (secular technology change) and the cross-epoch variation that identifies a heritage discount; name the surviving threat and show, on the assembled mission-by-subsystem frame, what fraction of the residual heritage-vs-parts contrast is within-epoch (identifying) versus between-epoch (confounded), with within-epoch cell counts. (raised by shadish_cook_campbell)
- **[rival]** The sampling frame (Section 6.3) is the high-documentation flagship/competed-science stratum, but the policy lever (1.4: a weak heritage claim substituted to cut test scope) is pulled in the thin-documentation, smaller-class, faster-cadence stratum. State the UTOS frame and produce the explicit generalization warrant (surface similarity, ruling out irrelevancies, a named causal mechanism) licensing transfer to the under-documented classes the conclusion is for. (raised by shadish_cook_campbell)
- **[empirics]** Pre-specified branches (overlap-trimming full vs trimmed, RE vs FE-by-subsystem, three heritage re-codings, payload-excluded re-runs, finer epoch dummies) each read the same H1-vs-H0 verdict on a modest JPL-class sample (Section 5.4); this is a forking-paths / multiplicity threat to statistical-conclusion validity. Commit to ONE designated primary specification and ONE trimming rule that decide the falsification BEFORE the robustness fan-out, and state the decision rule for when the primary returns H1 but a pre-specified branch returns H0. (raised by shadish_cook_campbell)
- **[empirics]** Re-estimate the heritage-vs-rigor head-to-head not on the standardized mean-hazard coefficient but on the tail object that causes mission loss: the early-window infant-mortality mass of the mixture-Weibull, or an upper quantile of the per-mission failure-count distribution. Does heritage's apparent advantage live in the Gaussian-ish bulk while parts-class and test-fidelity dominate the early-failure tail, or does the ranking survive a tail-scored test? (raised by taleb)
- **[identification]** Build the documentation-probability model both ways and test whether the probability a cell is fully documented depends on its realized early-failure status AFTER conditioning on environment, prominence, mass class, and epoch. If ruin removes its own evidence (early latent-defect failures leave thinner/recoded archives), the missingness is outcome-dependent (MNAR), not covariate-MAR, the IPW corrects the wrong selection mechanism, and the sensitivity bounds must span outcome-dependent missingness. Show the bounds widen and report where the heritage-vs-rigor verdict flips. (raised by taleb)
- **[mechanism]** Treat the as-run test-fidelity index as a continuous dose and estimate the CURVATURE of the infant-mortality hazard in test-fidelity within the deep-heritage stratum versus the no-heritage stratum: is the hazard's response to cutting test scope sharply convex (accelerating harm) precisely where heritage is invoked to justify the cut? A second-order convexity test settles whether a heritage discount is a benign mean shift or a fragility-transfer toward the absorbing early-failure regime, which an additive proportional-hazards mean-coefficient comparison structurally cannot distinguish. (raised by taleb)
- **[empirics]** Section 5.4 concedes a 'modest' JPL-class population. Demanded: leave-one-mission-out and leave-one-cell-out influence distribution for the standardized contrast (b1 minus the larger of b2,b3); empirical fraction of single deletions that reverse the falsification decision; max |dfbeta| as a share of the contrast. If one deletable observation flips the verdict, the point comparison is not the planning quantity. (raised by taleb)
- **[identification]** Section 4.3 inverse-weights by an estimated documentation probability and bounds undocumented cells 'across plausible bounds.' Demanded: the breakdown bound, the smallest fraction of adversarially-placed undocumented early-failure cells (values set to maximally favor H0) required to overturn an H1 verdict; compare to the observed share of unlinked early-failed spacecraft. If the breakdown fraction is below that observed share, the correction does not bound the conclusion against ruin in the data. (raised by taleb)
- **[governance]** Sections 6.2 and 1.4 frame the danger as a review board substituting a weaker heritage claim and cutting test scope, an asymmetry where whoever issues the heritage discount bears no on-orbit consequence, yet the estimand is purely associational. Demanded: from parts-control-board and design-review records, code which actor's heritage assertion authorized each test-scope reduction and whether that actor was downstream-accountable for the realized failure, then test whether the heritage-to-failure association concentrates in the skin-out-of-the-game cells. If the effect lives there, the claim is about incentive misalignment, not provenance. (raised by taleb)

## Grounded claims

- **[identification]** The methodological standard the candidate must meet is real: a regression coefficient on heritage is only identified by within-stratum heritage contrasts that actually exist in the data (common support / overlap). Where strata lack a heritage-rich AND heritage-poor unit at comparable controls, the coefficient is filled in by the model's functional form (extrapolation), not by a comparison. This is the selection-on-observables / overlap requirement that Angrist-Pischke's design-based program insists on diagnosing before estimation. BUT: no retrieved source reports the candidate's actual overlap-cell count or per-stratum heritage variance, so the empirical answer to whether the head-to-head test is identified on THIS JPL population cannot be asserted from retrieval.
    - Angrist & Pischke, Mostly Harmless Econometrics: An Empiricist's Companion (regression under selection-on-observables; the credibility of a regression rests on where the as-good-as-random variation comes from) | https://doi.org/10.1515/9781400829828 | grade A
    - Angrist & Pischke, The Credibility Revolution in Empirical Economics, J. Economic Perspectives 24(2):3-30 (the analyst's first obligation is to identify the source of exogenous variation; design over functional-form assumptions) | https://doi.org/10.1257/jep.24.2.3 | grade A
- **[mechanism]** The critique is methodologically correct and the discriminating step is identifiable in principle: nested-coefficient attenuation is observationally identical under confounding (controls share a common prior cause with heritage) and under mediation (heritage causes the controls, which cause the outcome), so coefficient shrinkage alone cannot distinguish the two. The discriminating observable is the heritage-to-mediator ASSIGNMENT relationship: if a deeper heritage claim predicts a LOWER planned test scope / parts-class, that directional first-stage marks the variable as caused BY heritage (mediator, post-treatment) rather than a confounder, which bears directly on whether conditioning on it is even legitimate. This is the same logic by which Angrist-Pischke treat conditioning on outcomes-of-treatment ('bad controls') as bias-reintroducing and insist the assignment mechanism be specified. BUT: whether the candidate can actually estimate these heritage-to-mediator relationships from the JPL archives, and their sign, is not established by any retrieved source.
    - Angrist & Pischke, Mostly Harmless Econometrics (bad controls: conditioning on variables themselves caused by the treatment reintroduces bias; assignment mechanism must be specified before estimation) | https://doi.org/10.1515/9781400829828 | grade A
    - Angrist, Imbens & Rubin, Identification of Causal Effects Using Instrumental Variables, JASA 91(434):444-455 (causal identification requires explicit statement of the assignment mechanism, not coefficient behavior alone) | https://doi.org/10.1080/01621459.1996.10476902 | grade A
- **[measurement]** The bad-control logic invoked is exactly the Angrist-Pischke standard: a regressor that is itself caused by the treatment whose effect is being estimated must not be held fixed, because conditioning on a post-treatment variable reintroduces bias and changes the estimand. Classifying test-fidelity as a clean pre-launch input is only defensible if planned test scope was set independently of (and not in response to) the heritage determination; if the heritage claim drives a thinner planned campaign, test scope is post-treatment and conditioning on it biases the heritage coefficient at the heart of the test. Establishing temporal/causal pre-determination from the design-review record is therefore the necessary and correct demonstration. BUT: no retrieved source establishes whether the JPL records actually show planned test scope was fixed before the heritage determination for the cells used; that documentary fact is outside retrieval.
    - Angrist & Pischke, Mostly Harmless Econometrics (bad controls = conditioning on outcomes/consequences of treatment; this reintroduces bias and is to be avoided) | https://doi.org/10.1515/9781400829828 | grade A
    - Angrist & Pischke, Mastering 'Metrics: The Path from Cause to Effect (the empiricist's checklist; selection-on-observables defensible only when conditioning set excludes post-treatment variables) | https://doi.org/10.1515/9781400851272 | grade A
- **[identification]** No dated policy-switch instrument exists in the candidate's archive. The dissertation contains zero occurrences of 'FCC', 'deorbit', 'discontinuity', or a dated parts-policy shifter, and 'first stage' appears once (record-linkage, not an IV first stage). The candidate concedes on the record: 'without an instrument or a natural experiment, no causal reading is warranted' and 'the honest statement of the result is therefore a conditional association under unconfoundedness, with the residual-confounding risk reported rather than assumed away.' Section 8.4.2 lists a future identification upgrade (parts-lot disruption, supplier exit, mandated requalification) as 'a target for future identification rather than a property it currently possesses.' On the panelist's own discipline (Mostly Harmless Econometrics; Angrist-Imbens-Rubin LATE), selection-on-observables yields association, not design-based identification. CONCESSION SECURED: the contribution is a conditional association, not a design-based estimate.
    - JPL_MGMT_SMA_TECH_04 dissertation (own corpus, jpl_mgmt_sma_tech_04.db / dissertation.md, Sec 8.3.1 rebuttal and Sec 8.4.2) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - Angrist & Pischke, Mostly Harmless Econometrics (selection-on-observables yields association, not design-based identification absent quasi-experimental variation) | https://doi.org/10.1515/9781400829828 | grade A
    - Angrist, Imbens & Rubin, Identification of Causal Effects Using Instrumental Variables (IV requires nonzero first stage + exclusion + monotonicity) | https://doi.org/10.1080/01621459.1996.10476902 | grade A
- **[empirics]** On the attenuation diagnostic, the candidate's records report a coefficient-stability procedure and a bounded sensitivity analysis but NOT the two specific quantities demanded. The dissertation states 'coefficient stability is examined by adding the control blocks one at a time, so a heritage coefficient that moves sharply when a single control enters is identified as fragile rather than reported as a finding,' and it runs a 'bounded sensitivity analysis' using a formal selection-bias / unmeasured-confounding apparatus (ref [83]). However, the corpus contains ZERO occurrences of 'Oster', 'partial R', 'proportional selection', or 'selection on unobservables'. Therefore the candidate can produce a block-wise stability table and a sensitivity bound, but the specific (a) partial R-squared of parts-class+test-fidelity conditional on heritage and controls and (b) an Oster-style proportional-selection / coefficient-stability bound (the implied delta of selection on unobservables needed to overturn the verdict) are NOT in the current records. PARTIAL: the apparatus to compute them exists; the reported numbers do not. The candidate also already distinguishes 'underpowered' from 'H0 confirmed', mitigating the non-result-misread-as-falsification concern.
    - JPL_MGMT_SMA_TECH_04 dissertation (own corpus, Sec 5.5.1 / 6.5.3 coefficient stability; Sec 5.3/5.5 bounded sensitivity analysis citing ref [83]) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - Angrist & Pischke, Mostly Harmless Econometrics (a coefficient change across nested models is uninterpretable without the partials; 'bad controls' caution) | https://doi.org/10.1515/9781400829828 | grade A
- **[measurement]** The measurement objection is well-founded in Argyris's own terms: the espoused-theory/theory-in-use distinction is the sharpest tool for auditing the gap between what an institution declares and what its behavior reveals (Argyris & Schon, Theory in Practice, 1974), and that gap is empirically real in space organizations where declared values diverge from inferable theory-in-use. A heritage label coded off a review record captures espoused theory; the article's as-built parts lots and environmental-qualification record are the theory-in-use. The candidate must add a behavioral audit channel that pairs the board's heritage assertion against the as-built record so that heritage-as-property-of-the-article is separable from heritage-as-justification; coder reconciliation against the source document cannot do this because it only certifies the espoused record. Independent corroboration that declared values and enacted behavior diverge systematically is documented in 'What Leaders Say versus What They Do' (Mor Barak, Luria & Brimhall, Group & Organization Management, 2021).
    - Argyris & Schon, Theory in Practice (1974); applied via hos-argyris dossier | https://ntrs.nasa.gov/api/citations/20030093634/downloads/20030093634.pdf | grade A
    - Mor Barak, Luria & Brimhall, 'What Leaders Say versus What They Do,' Group & Organization Management (2021) | https://doi.org/10.1177/1059601121992889 | grade A
- **[identification]** Argyris's organizational-defensive-routine construct predicts exactly the mediation structure the question raises: threatening information (an inadequate test budget) is filtered out before it reaches decision-makers, the filtering is undiscussable, and a fluent warrant ('it's heritage') is produced to protect against embarrassment or threat; the CAIB found precisely this pattern at NASA, where engineers' safety concerns could not get onto the decision agenda. Because the warrant is an enacted, spoken justification, it is auditable in the behavioral record. The candidate therefore cannot resolve mediation versus spuriousness from regression alone: the discriminating evidence is descope-justification language in design-review and waiver minutes showing heritage invoked as the licensing reason for the rigor cut. If that ordering (heritage label, then descope) is documented, the nested Model A-to-Model B coefficient drop is mediation (an effect transmitted through test-fidelity), not confounding controlled away, which inverts the interpretation of the attenuation. The candidate's Section 6.2 concession that mediation and spuriousness are distinguishable only by checking whether heritage predicts test-fidelity is consistent with this, but the auditable artifact, not the coded index, is what settles the direction.
    - Columbia Accident Investigation Board Report, Vol. One (2003); Argyris, 'Reinforcing Organizational Defensive Routines,' Human Resource Management (1986); via hos-argyris dossier | https://ntrs.nasa.gov/api/citations/20030093634/downloads/20030093634.pdf | grade A
    - Mahler, Organizational Learning at NASA: The Challenger and Columbia Accidents (2009) | https://doi.org/10.1353/book3701 | grade A
- **[rival]** The mechanism the question names is documented: Dillon and Tinsley's near-miss work isolates that when a near-miss does not produce a bad outcome, organizations systematically re-encode it as a success, an evaluation bias that destroys the signal double-loop learning needs. Applied to the independent variable, the same defensive re-encoding operates after an anomaly: the heritage attribution that licensed a cut is the embarrassing element and is the one most likely to be scrubbed from the superseded record, so failed-heritage cases lose their heritage label asymmetrically. This is selection on the IV, not just the outcome, and it biases the estimated heritage coefficient toward zero (attenuation) because the cases that would most strongly link heritage to failure are the ones whose heritage code is removed, making heritage look safer than it is. Detection requires evidence outside the as-finalized review record: versioned/superseded review baselines, anomaly-board (MRB/anomaly-resolution) minutes, and pre-anomaly versus post-anomaly heritage codings of the same subsystem, exactly the auditable artifacts Argyris's enacted-behavior level demands rather than the declared final record.
    - Dillon & Tinsley, 'Near-Miss Evaluation Bias as an Obstacle to Organizational Learning: Lessons from NASA' (2006) | https://ntrs.nasa.gov/api/citations/20060047554/downloads/20060047554.pdf | grade A
    - Columbia Accident Investigation Board Report, Vol. One (2003) | https://ntrs.nasa.gov/api/citations/20030093634/downloads/20030093634.pdf | grade A
- **[measurement]** The diagnostic Argyris demands is grounded: espoused theory (the board's declared scope) and theory-in-use (the rigor actually delivered to the article) can diverge, and a 'culture' or rigor claim must be validated at the level of enacted behavior, not declared values, because that is the level at which Columbia was actually lost. The observation that would reveal the index is measuring espoused rather than enacted rigor is a within-article disagreement: the as-run plan codes a component present/nominal while an independent artifact authored OUTSIDE the heritage narrative (a parts-stress waiver, delta-qual deviation, or anomaly-closure action item) records that the same component was waived or de-scoped. The candidate's own design half-concedes this: 4.3.5 makes as-run the primary measure 'because it reflects what the article actually received,' but the as-run record is still a heritage-banner artifact, so as-run is enacted scope only if no second, independent source contradicts it. The defensive-reasoning signature to look for is a closure or waiver whose premise (heritage retires this risk) is asserted but never tested and cannot be independently checked. NO retrieved source supplies a specific paired-record case (named article, the nominal as-run entry, and the contradicting waiver/deviation): the study is pre-registered and not yet executed (8.4.1, 'the immediate next step is execution'), so the empirical case demanded does not exist to cite.
    - Argyris dossier (hall_of_shoulders/argyris), citing Argyris & Schon, Theory in Practice (1974) and Argyris, Overcoming Organizational Defenses (1990) | file:///D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/argyris/ | grade C
    - Mor Barak, Luria & Brimhall, 'What Leaders Say versus What They Do,' Group & Organization Management (2021), per Argyris dossier | https://doi.org/10.1177/1059601121992604 | grade A
    - JPL_MGMT_SMA_TECH_04 dissertation, Sec 4.3.5 Test-program fidelity | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[identification]** Argyris's challenge is grounded and the candidate's own text concedes its force. Dillon & Tinsley establish that when a near-miss produces no bad outcome organizations systematically re-encode it as a success, an evaluation bias that destroys the signal a learning system needs; this makes documentation probability a function of the re-encoded OUTCOME, not only of the mission characteristics (prominence, mass, orbit, epoch, subsystem) the IPW model conditions on. The candidate's IPW model (6.2.4) assumes documentation is missing-at-random given those covariates and explicitly admits the failure mode Argyris names: 'if documentation completeness depends on the outcome itself in a way the covariates do not capture, for example if a catastrophic early failure caused records to be sealed or never written, then the weights are insufficient.' Near-miss re-encoding is precisely an outcome-dependent, behaviorally-driven selection mechanism, not missing-at-random on covariates, so the weighting probability is endogenous to the thing being corrected. The evidence that WOULD show independence is a falsification test the candidate has not run: condition on the IPW covariates and test whether closure completeness and root-cause specificity are statistically independent of survived-vs-mission-affecting status; independence (a null) would support MAR, dependence would confirm endogenous re-encoding. The right structural move is double-loop: surface and make testable the actors' interest in the record being incomplete, rather than modeling documentation as an exogenous function of mission traits. NO retrieved source reports the result of that conditional-independence test on actual closure records, so the empirical direction of the dependence cannot be asserted.
    - Dillon, R. L., & Tinsley, C. H. (2006), 'Near-Miss Evaluation Bias as an Obstacle to Organizational Learning: Lessons from NASA' (retrieved via Argyris brain source record) | https://ntrs.nasa.gov/api/citations/20060047554/downloads/20060047554.pdf | grade B
    - Columbia Accident Investigation Board report (2003) and Mahler, Organizational Learning at NASA (2009), per Argyris dossier | https://ntrs.nasa.gov/api/citations/20030093634/downloads/20030093634.pdf | grade B
    - JPL_MGMT_SMA_TECH_04 dissertation, Sec 6.2.4 The documentation-probability model and the weights it produces; Sec 4.4 survivorship as a measurement problem | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[measurement]** The reliability-without-validity critique is grounded and the external anchor Argyris requests is partly available in the candidate's own source structure. Chance-corrected inter-coder agreement (the candidate's warrant in 4.5) certifies that two analysts transcribe the SAME document consistently; it cannot certify that the document measures the article rather than the board's account of it, because Argyris's espoused/theory-in-use distinction is precisely that a self-sealing organizational claim can be transcribed at high fidelity by both coders while diverging from enacted reality. The candidate codes heritage depth from 'the design-review heritage assessment matrix, which states what the project actually claimed' (4.3.3), an espoused-claim source. A valid external anchor must be article-level and authored outside the heritage narrative: the candidate's own design already names two such sources whose facts the four-rung heritage ladder claims to summarize, parts control records (4.3.4, read 'directly from parts control records,' the basis of the parts-class variable) and the NTRS parts-stress and radiation-hardness-assurance reports (4.1.2, the rigor source), plus same-environment flight facts (orbit/radiation environment) which are matters of record independent of the matrix's claim. Validating the matrix's same-environment top rung against the actual prior-flight orbit/radiation environment, and validating a design-plus-build claim against the as-built parts-lot, is the construct-validity audit kappa cannot supply. NO retrieved source reports the fraction of audited cells on which board-recorded heritage depth disagrees with such an external anchor, or its direction, because the audit has not been run (pre-registered study, 8.4.1).
    - Argyris dossier (hall_of_shoulders/argyris), citing Argyris & Schon, Theory in Practice (1974); Argyris, Overcoming Organizational Defenses (1990) | file:///D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/argyris/ | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation, Sec 4.3.3 Heritage depth, Sec 4.3.4 EEE parts-class, Sec 4.1.2 NTRS reliability/parts-stress/radiation-hardness reports | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation, Sec 4.5 Record linkage and inter-coder measurement reliability | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[measurement]** The objection is methodologically valid: an ordinal heritage rung is a LEVEL, but the decision-relevant behavior lives in the underlying STOCK (accumulated successful flight history) and the FLOW/age that drains it (time since last identical-configuration flight). Forrester's core tenet is that the state of a system lives in its stocks, which change only through flows; a covariate frozen at a level cannot recover the accumulation or its decay. Sterman's bathtub-dynamics experiments empirically demonstrate that even expert subjects systematically fail to infer a stock's trajectory from its flows ('stock-flow failure'), which is exactly the inference a level-only regression silently asks the data to do. Therefore the candidate must regress hazard on the flow-and-age quantities (accumulated flight-hours, elapsed time since last identical flight) and show the heritage rung adds signal beyond them; if the age term dominates, the rung is a mismeasured proxy for an accumulation. NOTE: the JPL heritage matrices and flight logs needed to compute the cell-level flight-hours and age values were NOT retrievable this turn (AMOS/ACTA returned zero on heritage/reliability), so the empirical 'does age dominate the rung' result is asserted as a required test, not as a settled finding.
    - Sterman, 'Bathtub Dynamics: initial results of a systems thinking inventory' (System Dynamics Review, 2000), documents stock-flow failure: subjects cannot infer accumulation behavior from flows, justifying formal stock-flow modeling over level intuition. | https://doi.org/10.1002/sdr.198 | grade A
    - forrester dossier (Hall of Shoulders brain, hos-forrester), 'Stocks, flows, and accumulation: the state of a system lives in its stocks; stocks change only through flows; behavior over time is governed by accumulation, not by the instantaneous level.' | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/forrester/ | grade C
- **[identification]** The structure described is a reinforcing (positive) feedback loop endogenous to the data-generating process: heritage depth lowers test/reporting intensity, which lowers failure observability, which inflates measured reliability of heritage-rich cells, which justifies the next heritage claim. Forrester's apparatus holds that system behavior is dominated by closed loops of causation and that counterintuitive outcomes arise precisely because analysts see local cause-and-effect but not the loop; the canonical orbital instance is the Kessler collisional cascade, an empirically established self-sustaining reinforcing loop. Meadows' leverage-point hierarchy ranks the GAIN AROUND REINFORCING LOOPS (point 7) above buffer sizes and parameter tweaks, which means a survivorship/IPW correction that merely reweights observations WITHIN the loop is a low-leverage parameter fix, whereas the candidate's claim requires acting on the loop's gain, i.e. modeling the reporting probability as an explicit function of heritage and proving the weights restore the suppressed failures. Decision: the candidate must (a) estimate P(documented/reported | heritage, realized-early-failure) and show it decreases in heritage after conditioning on failure, and (b) show IPW recovers the unobserved failure mass rather than redistributing observed mass. NOTE: whether IPW actually opens this specific loop in the JPL launch-manifest records is NOT settleable from retrieved sources (no JPL reporting-probability data retrievable; AMOS survivorship-bias query returned zero), so this is graded as a required identification test, not a verified result.
    - forrester dossier (hos-forrester), 'Feedback loops: system behavior is dominated by closed loops of causation; reinforcing loops amplify; counterintuitive outcomes arise because actors see local cause-and-effect but not the loop structure' (Forrester, Industrial Dynamics, 1961). | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/forrester/ | grade C
    - Meadows, 'Leverage Points: Places to Intervene in a System' (1999), ranks gain around reinforcing loops (pt 7) and strength of balancing loops (pt 8) far above parameter/buffer adjustments (pts 11-12), distinguishing acting ON a loop from adjusting within it. | https://doi.org/10.4324/9781849773386-15 | grade A
    - Kessler & Cour-Palais, 'Collision frequency of artificial satellites: the creation of a debris belt' (JGR, 1978), empirical archetype of an endogenous self-sustaining reinforcing loop in the space domain. | https://doi.org/10.1029/JA083iA06p02637 | grade A
- **[mechanism]** The timescale-separation objection is well-founded on three grounded points. (1) The bathtub hazard model itself separates a decreasing-hazard infant-mortality regime from an increasing-hazard wear-out regime; an assessment validating these trends in electronics confirms the two regimes are distinct physical clocks, so a single frozen-covariate hazard that does not distinguish them can credit a cell as reliable in the early window while saying nothing about its wear-out ceiling. (2) A standard proportional-hazards model holding covariates at their design-review value is a known limitation; the survival-analysis literature treats time-varying covariates and time-varying coefficients precisely to handle regressors whose effect is not constant over follow-up, which is the formal remedy the candidate's single frozen-covariate model omits. (3) Forrester's apparatus and the contemporary orbital literature both insist on separating a fast operational loop from a slow capacity/wear constraint: Colombo, Martinez, Letizia et al. explicitly distinguish the slow capacity timescale from the fast traffic-deconfliction timescale, the same fast/slow decomposition the candidate must apply to test-fidelity (a one-time flow) versus parts-class (an aging stock). The dossier states the standing demand directly: if the fast loop stabilizes each encounter while the slow ceiling is still overshooting, the analyst 'has optimized the flow inside a system whose ceiling [they] ignored.' Decision: the candidate must split the hazard into an early-window (test-fidelity-sensitive) regime and a wear-out (parts-class-sensitive) regime and demonstrate heritage's apparent advantage is not confined to one clock; a blended single-covariate PH model conflates the two and may credit heritage in the wrong regime.
    - Jais et al., 'An Assessment of Validity of the Bathtub Model Hazard Rate Trends in Electronics' (IEEE Access, 2021), examines the infant-mortality (decreasing) vs wear-out (increasing) hazard-rate regimes as distinct phases of the bathtub model in electronic parts. | https://doi.org/10.1109/access.2021.3050474 | grade A
    - Zhang et al., 'Time-varying covariates and coefficients in Cox regression models' (Annals of Translational Medicine, 2018), establishes that a Cox PH model with covariates frozen at baseline is mis-specified when effects vary over follow-up, and gives the time-varying-covariate/coefficient remedy. | https://doi.org/10.21037/atm.2018.02.12 | grade A
    - Colombo, Martinez, Letizia et al., 'Space capacity management and its interaction with space traffic management' (Acta Astronautica, 2025), distinguishes the slow capacity-management timescale from the fast traffic-deconfliction timescale; the canonical fast/slow stock-vs-flow separation. (retrieved via acta-brain / hos-forrester) | https://doi.org/10.1016/j.actaastro.2025.01.069 | grade A
- **[measurement]** The candidate NAMES the catalog of record correctly: the sampling frame is built from launch manifests, not surviving-mission documentation, with a three-stage linkage (stage 1: missions matched by a canonical mission identifier reconciled from launch designation, project name and launch date; stage 2: subsystems mapped to a common functional taxonomy via a fixed crosswalk with two-coder adjudication of ambiguous cases; stage 3: anomaly time-and-root-cause assignment). This design satisfies McDowell's F1 'catalog as ground truth' and F6 'reconciliation as the discipline's hygiene' at the DESIGN level. However, the dissertation is explicitly a design-stage analysis plan: it states 'the dataset is still being assembled' and 'the numbers below are illustrative expectations, not measured results.' No launch-count denominator, no per-source stage-by-stage match rate, and no unlinked-fraction has been computed. The reconciliation table McDowell demands first does not yet exist in the candidate's record.
    - Candidate prospectus, JPL_MGMT_SMA_TECH_04, 'Does Flight Heritage Buy Reliability?', Sections 3.5 (Record linkage), 4.3 (Survivorship correction, step 1), and 5 (Analysis plan: 'illustrative expectations used to define falsification thresholds, not measured results') | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/prospectus.md | grade C
    - mcdowell dossier (Hall of Shoulders brain hos-mcdowell), frameworks F1 and F6 | file:///D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/mcdowell/ | grade B
- **[identification]** The candidate specifies the covariates the documentation-probability model would use (mission environment, mission prominence, mass class, launch epoch, subsystem type) and acknowledges the support problem in principle: heritage and parts-class are 'best documented for JPL flagship and competed science missions and thinner for the smallest classes,' and overlap is elevated to a first-class reported result rather than a formality. This concedes McDowell's structural point. But the candidate cannot demonstrate identifiability: IPW reweights cells it observes and cannot recover completely-unlinked spacecraft that contribute zero coded cells if those lie outside the covariate support of the linked subset. The requested linked-vs-unlinked covariate distribution on environment, prominence, mass class, and epoch has not been computed, because the frame is unassembled. The candidate's own text frames documentation completeness as itself an outcome to be modeled, which is the correct posture but is not yet an executed identifiability check.
    - Candidate prospectus, Sections 3.3 (Controls), 3.4 (Coverage and limitations), 4.3 (Survivorship correction, step 2 IPW), 2.5 (overlap as the gate that decides whether the question is answerable); dissertation.md Rubin-program passage (documentation-probability model as missing-data-as-assignment-mechanism) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/prospectus.md | grade C
    - Rosenbaum & Rubin, 'The central role of the propensity score in observational studies for causal effects,' Biometrika 70(1):41-55, 1983 (cited as ref [15]); Rubin, 'For objective causal inference, design trumps analysis,' Annals of Applied Statistics 2(3):808-840, 2008 (ref [16]) | https://doi.org/10.1093/biomet/70.1.41 | grade A
- **[empirics]** The candidate explicitly concedes the mechanism McDowell names: 'Anomaly records undercount minor anomalies that did not trigger formal reporting, which biases the outcome toward more severe events,' and operations-caused and environment-caused anomalies are retained but flagged so sensitivity to their inclusion can be tested. The smallest documented unit is the mission-by-subsystem cell, with anomalies assigned a time relative to commissioning and a root-cause class. This is a real acknowledgement that the outcome is a documentation construct. But the candidate accepts the severity-truncation as 'acceptable because the hypotheses concern reliability-relevant failures' and does NOT establish that reporting completeness is uniform across prominence strata. The decisive empirical object McDowell demands, anomaly-reporting rate per subsystem-year as a function of prominence on the linked sample, is not produced; without it, a prominence-correlated reporting gradient maps directly onto the heritage signal (flagships are both heritage-rich and richly reported), so the measured failure floor may be a documentation artifact dressed as on-orbit behavior. The descriptive reliability base the candidate relies on (Castet/Saleh, Tafazoli, Saleh/Castet) is itself built from formally reported on-orbit failures and inherits the same reporting-intensity exposure.
    - Candidate prospectus, Sections 3.2 (Unit of analysis), 3.4 (Coverage and limitations), 3.5 (Record linkage, stage 3) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/prospectus.md | grade C
    - Castet & Saleh, 'Satellite and satellite subsystems reliability: Statistical data analysis and modeling,' Reliability Engineering and System Safety 94(11):1718-1728, 2009 (candidate ref [1]); Tafazoli, 'A study of on-orbit spacecraft failures,' Acta Astronautica 64(2-3):195-205, 2009 (ref [4]); Saleh & Castet, 'Health scorecard of spacecraft platforms,' Acta Astronautica 68(7-8):1153-1166, 2011 (ref [5]) | https://doi.org/10.1016/j.ress.2009.05.004 | grade A
- **[measurement]** PARTIAL / REFUSE-ON-NUMBER. The candidate's dissertation specifies the linkage PROTOCOL but reports no reconciliation-ledger numbers. Sec 4.5 fixes a three-stage linkage at the mission-by-subsystem cell: stage-1 mission matching by a canonical mission identifier reconciled from launch designation, project name, and launch date, with a containment rule requiring agreement on at least two of the three identifying fields and adjudication of residual ambiguity against the launch manifest as the authoritative record of what flew and when; stage-2 a fixed, frozen subsystem crosswalk with two-coder adjudication (crosswalk and adjudication log = Appendix B); stage-3 anomaly assignment by time-since-commissioning and root-cause class. The frame is built from launch manifests, not surviving-mission documentation (Sec 4.4), so early-failed cells enter the denominator before documentation completeness corrupts it, and inverse-probability-of-documentation weighting is applied. BUT the per-cell single-agreed / dropped-unmatchable / many-to-one-or-one-to-many fractions and the per-source-pair match rate McDowell demands are NOT reported: Sec 4.6 states the PRACA and JPL archival sources are access-controlled and not yet assembled, and Sec 4.4 states design confidence is 'moderate and contingent rather than high' until the data are assembled. The ledger must be produced from the assembled frame; it cannot be asserted from retrieval.
    - JPL_MGMT_SMA_TECH_04 dissertation, Sec 4.5 Record linkage and inter-coder measurement reliability (three-stage linkage; canonical mission identifier; two-of-three-field agreement; adjudication against the launch manifest; frozen subsystem crosswalk and Appendix B adjudication log) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation, Sec 4.4 (launch-manifest sampling frame vs surviving-mission documentation; inverse-probability-of-documentation weighting; survivorship as first-order measurement threat; design confidence moderate and contingent until the data are assembled) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - McDowell dossier (Collegium Hall of Shoulders), F1/F6 catalog reconciliation and definitional rigor: reconcile competing counts (operator claims, official catalog, independently tracked population) and state the tracking floor; a census built on the documented subset is not the population | file:///D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/mcdowell/ | grade C
- **[measurement]** PARTIAL / REFUSE-ON-NUMBER. The candidate has ANTICIPATED the instrument-drift concern but supplies no per-epoch documentation-density or taxonomy-version census, and no same-true-value-same-code demonstration. The measurement table (Sec 4.3.1) sources heritage depth from design-review-era JPL heritage assessment matrices, EEE parts-class from JPL parts control board records cross-checked against NTRS parts-stress/NEPP records, and test-fidelity from integration-and-test plans plus NTRS qualification/acceptance and RHA summaries. Sec 4.1 / 4.4 concede the documentation-completeness gradient varies by program class and over time (best for flagship/competed missions, thinner for the smallest classes). Launch epoch enters only as a control (year or epoch dummies) to absorb secular technology change (Sec 4.3.6), and Sec 5 pre-specifies a finer epoch-dummy diagnostic plus a stricter same-environment re-coding to test whether a heritage effect is a calendar trend in disguise. BUT McDowell's specific demand, a per-epoch census of documentation density and taxonomy VERSION for each of the three indices and a demonstration that a fixed true heritage depth / parts-class maps to the same code in earliest and latest epochs, is NOT in the dissertation. The candidate himself states confidence in the epoch diagnostic is 'moderate, bounded by the number of well-documented missions per epoch.' The epoch-by-epoch taxonomy-version census and same-true-value-same-code mapping cannot be asserted from retrieval and must be produced from archive metadata.
    - JPL_MGMT_SMA_TECH_04 dissertation, Sec 4.3.1 measurement table (heritage from design-review-era heritage assessment matrices; parts-class from parts control board records cross-checked vs NTRS/NEPP; test-fidelity from I&T plans + NTRS qualification/RHA) and Sec 4.3.6 (launch-epoch control to absorb secular technology change) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation, Sec 5 epoch-rival diagnostic (finer epoch dummies + stricter same-environment re-coding; confidence that the epoch diagnostic separates a secular trend from a heritage effect is moderate, bounded by the number of well-documented missions per epoch) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - McDowell dossier (Collegium Hall of Shoulders), F4 trend accounting + tracking-floor logic: the instrument that measures the census drifts in resolution over the window; a dummy absorbs a level shift, not a change in how the regressors are measured | file:///D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/mcdowell/ | grade C
- **[identification]** GROUNDED METHODOLOGICAL VERDICT (settles the theory, not the data): Pearl's critique is correct as identification theory. If heritage causes parts-class and test-fidelity -- exactly the structure Section 6.2 of this dissertation proposes as a possibility ('may operate as mediators rather than confounders') -- then conditioning on them in Model B estimates a controlled direct effect, not a confounding-adjusted total effect, and the resulting b1 attenuation is mechanically produced by mediation. This is the textbook Table 2 fallacy: a single regression coefficient on the exposure cannot be read simultaneously as the confounder-adjusted effect of heritage and as evidence of spuriousness, and the mediator and confounder data-generating worlds yield numerically identical coefficients (Westreich & Greenland 2013). Only a graph-implied conditional-independence (d-separation) test, not the coefficient race, can distinguish them, and that test requires the assembled dataset -- which the dissertation states it does not yet have.
    - Hall of Shoulders dossier: Judea Pearl (collegium thinker brain 'pearl'), citing Pearl, Causality: Models, Reasoning and Inference (2nd ed., 2009) | https://doi.org/10.1017/cbo9780511803161 | grade A
    - Westreich D, Greenland S, 'The Table 2 Fallacy: Presenting and Interpreting Confounder and Modifier Coefficients,' American Journal of Epidemiology | https://doi.org/10.1093/aje/kws412 | grade A
    - JPL_MGMT_SMA_TECH_04 dissertation.md, Section 6.2 (estimation procedure) and 6.4 (falsification rule) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[mechanism]** GROUNDED METHODOLOGICAL VERDICT: The dichotomy is real and correctly posed. The unobserved 'program competence' confounder the design fears is precisely the case where front-door identification, not back-door adjustment, is the licensed tool: if parts-class and test-fidelity fully mediate heritage and are themselves unconfounded with failure given measured covariates, the front-door criterion recovers the total heritage effect even with heritage<->failure confounded by an unobserved cause. The candidate's selection-on-observables (back-door) design discards this strategy that its own causal structure hands it. Under partial mediation, b1 in Model B is neither the total effect (the indirect path is blocked) nor a clean controlled direct effect that the falsification rule can interpret, so the H1 verdict reads an uninterpretable residual. Which case holds is empirically settleable only by decomposing the heritage->failure association into through-channel and around-channel portions on assembled data.
    - Hall of Shoulders dossier: Judea Pearl (collegium thinker brain 'pearl'), 'd-separation and the back-door / front-door criteria' and 'Review lens' sections | https://doi.org/10.1017/cbo9780511803161 | grade A
    - Pearl J, 'The Causal Foundations of Structural Equation Modeling' | https://doi.org/10.21236/ada557445 | grade B
    - VanderWeele TJ, Vansteelandt S, 'Mediation Analysis with Multiple Mediators,' Epidemiologic Methods | https://doi.org/10.1515/em-2012-0010 | grade A
- **[empirics]** GROUNDED METHODOLOGICAL VERDICT: Pearl is correct that positivity/overlap and balance diagnostics address extrapolation and finite-sample exchangeability but are blind to edge DIRECTION; a balance table is symmetric in the confounder and mediator graphs and cannot adjudicate between them. The direction of heritage->parts-class and heritage->test-fidelity must be licensed by a graph-implied conditional-independence (d-separation) test, which is the only partly-testable content a DAG exposes. Concretely: under the mediator graph (heritage -> parts-class/test-fidelity -> failure) heritage and failure are NOT d-separated by the empty set and the mediators lie on the open causal path, whereas under the pure-confounder graph the mediators are non-descendants of heritage; the observable signature is whether heritage statistically predicts parts-class and test-fidelity at all (a descendant test) and whether the heritage-failure association is blocked when the intermediates are conditioned. An observed independence that would falsify the mediator reading is heritage independent of parts-class and of test-fidelity in the data -- if heritage does not predict them, they cannot be mediators of heritage, rescuing the confounder-adjustment interpretation. This d-separation check, not any balance count, is the licensing test.
    - Hall of Shoulders dossier: Judea Pearl (collegium thinker brain 'pearl'), DAG / d-separation sections | https://doi.org/10.1017/cbo9780511803161 | grade A
    - Tennant et al. / methodological guidance, 'Drawing Credible Directed Acyclic Graphs for Causal Inference' (preprint) | https://doi.org/10.31234/osf.io/u4yta_v4 | grade B
    - Westreich D, Greenland S, 'The Table 2 Fallacy,' American Journal of Epidemiology | https://doi.org/10.1093/aje/kws412 | grade A
- **[identification]** The methodological apparatus the question demands is well-founded: a within-study effect P(Y|do(X)) is distinct from a transported effect, and transport is licensed only when the mechanism is invariant across the differing-population markers (S-nodes) on the relevant arrows. Pearl's framework makes the assumptions explicit, partly machine-checkable via conditional-independence implications, and frames an invariance claim a cross-domain holdout can falsify, so the epoch/mass-class invariance test the candidate is asked to run is the correct and falsifiable license condition. GROUNDED ONLY ON THE METHODOLOGY: I cannot assert any empirical invariance result from the candidate's frame, because no retrieved corpus contains the candidate's standardized heritage coefficient or its values across epoch dummies / mass-class strata.
    - Hall of Shoulders Pearl dossier (selection diagrams, do-calculus, invariance-enables-transport thesis; 'a stated invariance claim that a cross-domain holdout can falsify'; Pearl, Causality 2nd ed. 2009) | https://doi.org/10.1017/cbo9780511803161 | grade A
- **[identification]** The structural diagnosis the question presupposes is correct as causal theory: when the selection/missingness node is pointed into by the outcome variable itself (here, documentation completeness depending on realized early-failure), the data are not missing-at-random and inverse-probability weighting on observed covariates cannot in general recover the target contrast; recoverability then requires an external or surrogate margin rather than selection-on-observables. The candidate's own statement that under-documented cells are 'disproportionately early-failed' is precisely this non-recoverable signature. GROUNDED ONLY ON THE METHODOLOGY: I cannot report whether residual dependence on early-failure persists after conditioning, nor validate any specific external margin, because the documentation-completeness model and its fit are not in any retrieved corpus.
    - Hall of Shoulders Pearl dossier (do-calculus completeness; missing-data recoverability; Causal Hierarchy Theorem, lower-rung data cannot answer higher-rung queries without explicit causal assumptions; Pearl & Mackenzie, The Book of Why 2018) | https://doi.org/10.1017/cbo9780511803161 | grade A
- **[rival]** The disjunction the question forces is the correct decision-theoretic frame: under a transport problem, if the target cell lies outside the study population's support (no study cell shares the target's heritage-depth-by-environment profile, i.e. an S-node on the environment arrow), the within-study estimate transports zero information and the only admissible moves are (a) a transport/re-weighting formula when an overlapping bridging cell exists, (b) re-targeting the estimand to the overlapping sub-population, or (c) declaring the decision-relevant cell non-identified. Pearl's apparatus supplies exactly these three exits and no fourth. GROUNDED ONLY ON THE METHODOLOGY: I cannot state which exit applies, because whether any study cell overlaps the target (heritage-depth, environment) profile is an empirical fact about the candidate's cross-tabulation that no retrieved corpus contains.
    - Hall of Shoulders Pearl dossier (do-operator vs seeing; transportability and external validity as the open methodological frontier for space SSA/STM/autonomy; explicit non-identification when adjustment set fails the criterion) | https://doi.org/10.1017/cbo9780511803161 | grade A
- **[identification]** Rubin's challenge is methodologically valid and partly anticipated by the design, but the empirical demand cannot be answered: the requested joint cell table does not exist. SUTVA requires the treatment be a single, well-defined intervention with no hidden versions; a 'treatment' that is definitionally a composite makes the estimand incoherent before estimation (Rubin 1978; Imbens-Rubin 2015). The dissertation's own top heritage level is defined as 'design+build+same-environment flight,' coded against prior-flight orbit and radiation environment (Ch4.3.3), which bundles environmental qualification into the heritage construct, and it frames the test as a nested-coefficient comparison of conditional associations rather than a single manipulable causal contrast (Ch2.5.1, Ch2.6). The design names overlap as a physical coexistence condition to be reported as a first-class result and concedes that adequate overlap on the assembled population is 'unknown at the design stage and is precisely what the diagnostic will establish' (Ch2.5.2). But the dissertation states four times that no dataset is assembled and 'No estimated coefficient is presented as a finding anywhere in the document' (Ch1 line 173; Ch2 line 345 'a specification, not a result ... deliberately left unmeasured'; Ch5 line 844). Therefore the off-diagonal cell counts Rubin demands are not retrievable from any source and asserting them would be confabulation.
    - Rubin (1978), Bayesian Inference for Causal Effects: The Role of Randomization, Ann. Statist. (SUTVA: no interference, no hidden versions of treatment / single well-defined intervention) | https://doi.org/10.1214/aos/1176344064 | grade A
    - Imbens & Rubin (2015), Causal Inference for Statistics, Social, and Biomedical Sciences (overlap must be demonstrated before outcomes analyzed; SUTVA limbs) | https://doi.org/10.1017/cbo9781139025751 | grade A
    - JPL_MGMT_SMA_TECH_04 dissertation.md Ch4.3.3 (heritage depth ordinal top level = design+build+same-environment flight, coded against prior-flight orbit/radiation) and Ch2.5.1-2.6 (nested-coefficient estimand; overlap as physical condition reported as a result; 'unknown at the design stage') | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation.md lines 173, 345, 844 ('a specification, not a result'; dataset 'still being assembled'; 'No estimated coefficient is presented as a finding anywhere in the document') | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[identification]** The SUTVA / interference challenge is correct in principle and the design acknowledges the exact mechanism, but the empirical interference fraction and the coefficient-contrast movement do not exist and cannot be asserted. Under the potential-outcomes framework, no-interference means one unit's treatment does not change another unit's potential outcome; a shared parts lot or shared power bus is a textbook interference structure that makes the unit-level estimand incoherent, not merely noisy (Rubin 1978; Imbens-Rubin 2015; Rubin dossier: 'debris is interference incarnate ... SUTVA fails by construction'). The dissertation explicitly identifies the power subsystem and shared power bus as a common-cause node that can propagate a single fault across subsystems (Ch3.1, citing the electrical-power-subsystem multi-state analysis) and proposes to 'record lot-level and bus-level commonality so that violations can be flagged and the affected cells examined separately,' rating the residual interference risk 'moderate' (Ch2.5.1, Ch5.4.1). Rubin's objection that flagging-and-quarantine treats a definitional incoherence as a nuisance is well-grounded. But the requested quantity, the fraction of cells sharing a lot or bus and the contrast under exclusion vs retention, requires the assembled mission-by-subsystem frame, which the dissertation states does not yet exist; no number is retrievable, so it is refused.
    - Rubin (1978), Bayesian Inference for Causal Effects (no-interference limb of SUTVA; interference renders unit-level estimand incoherent) | https://doi.org/10.1214/aos/1176344064 | grade A
    - Rubin hall-of-shoulders dossier (Synthesis + Challenge 2: 'SUTVA fails by construction ... debris is the paradigm case of interference'; verified canonical Rubin/Holland/Rosenbaum DOIs via Crossref) | file:///D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/rubin/dossier.md | grade A
    - JPL_MGMT_SMA_TECH_04 dissertation.md Ch3.1 (shared power bus as common-cause node) + Ch2.5.1/Ch5.4.1 (record lot/bus commonality, flag and examine affected cells separately; residual interference risk 'moderate') | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation.md line 844 (mission-by-subsystem dataset 'still being assembled'; chapter reports the procedure, not estimates) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[measurement]** The documentation-endogeneity challenge is valid and the design names the corresponding threat, but the pre-launch-only recoverability share and its contrast are not computed and cannot be asserted. Rubin's design discipline requires the analysis be blinded to outcomes and that selection corrections not be fit on outcome-contaminated covariates (Rubin 2008 design-trumps-analysis; Imbens-Rubin 2015). The dissertation's own warrant for inverse-probability-of-documentation weighting concedes the documentation probability depends on observed characteristics and, 'most seriously, missions and subsystems that failed early may have produced thin documentation, and heritage claims that were quietly dropped after an early failure may not be archived as heritage at all,' biasing the comparison in a direction that 'flatters heritage' (Ch5.3.1 Grounds). Its stated outcome-blindness defenses are exactly the pre-launch artifacts Rubin asks for: provenance coded from the design-review heritage assessment matrix that 'predates the outcome,' parts-class from as-procured parts-control-board records, and test-fidelity from as-planned I&T scope, with the documentation-probability model frozen in pre-registration before outcomes are seen (Ch4.3.3-4.3.5, Ch5.4.1 'reverse documentation' threat, Ch5.3.2). This means the design is constructed to permit the pre-launch-only recoverability test Rubin demands; what it does not provide is the executed share, because IPW weights are fit on covariates (prominence, mass class, epoch, environment, agency, subsystem type) that are themselves partly outcome-correlated, and the confidence in the unobservables channel is only 'moderate,' resting on a bounded sensitivity analysis rather than a recovered number (Ch5.3.1 Qualifier, Ch5.3.2). The frame is unassembled (lines 173/345/844), so the falsifiable recoverability share is unretrievable and is refused.
    - Rubin (2008), For Objective Causal Inference, Design Trumps Analysis, Ann. Appl. Statist. (design/analysis-plan fixed while blind to outcomes) | https://doi.org/10.1214/08-aoas187 | grade A
    - Imbens & Rubin (2015), Causal Inference for Statistics, Social, and Biomedical Sciences (selection correction / overlap; missing-data view of causal inference) | https://doi.org/10.1017/cbo9781139025751 | grade A
    - JPL_MGMT_SMA_TECH_04 dissertation.md Ch5.3.1-5.3.2 (documentation-probability model; early-failed cells produce thin docs and dropped heritage labels; bias 'flatters heritage'; IPW fit on observed characteristics; unobservables channel only 'moderate', bounded sensitivity) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation.md Ch4.3.3-4.3.5 + Ch5.4.1 (provenance-at-design-review coding rule that 'predates the outcome'; parts-class as-procured; test-fidelity as-planned; reverse-documentation residual risk 'low') | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[measurement]** Rubin's no-hidden-versions audit is methodologically correct and the dissertation concedes its premise, but the demanded within-rung-across-version dispersion table cannot be produced and asserting any number would be confabulation. SUTVA requires each treatment level be a single well-defined intervention with no hidden versions; if one ordinal rung is reachable by physically distinct articles encoding different claims, that rung is not one treatment level and the coefficient is an average over incommensurable interventions (Rubin 1978; Imbens-Rubin 2015; Rubin dossier SUTVA limb). The candidate's own Section 1.4-1.5 states the four heritage claims 'are not interchangeable' and that the decision-relevant failure mode is exactly the substitution of a weaker claim for a stronger one (dissertation line 135), and codes heritage as an ordinal depth measure (none, design-only, design+build, design+build+same-environment), conceding that within the upper rungs distinct provenance is collapsed to one ordinal value (lines 135, 145, 561, 1473). The design's only response is a robustness re-coding that requires same-environment flight for the top category to test whether the heritage association is driven entirely by same-environment cases (lines 804, 947), which is NOT the requested within-rung outcome-dispersion partition. Decisively, the mission-by-subsystem frame is unassembled and the result tables are 'specified here in structure but left unpopulated' (line 959), with 'No estimated coefficient... presented as a finding anywhere in the document' (rubin_r1 verification, lines 173/345). The within-rung-across-version outcome spread therefore does not exist in any retrieved source and is refused.
    - Rubin (1978), Bayesian Inference for Causal Effects: The Role of Randomization, Ann. Statist. (SUTVA: no hidden versions; a treatment level must be a single well-defined intervention) | https://doi.org/10.1214/aos/1176344064 | grade A
    - Imbens & Rubin (2015), Causal Inference for Statistics, Social, and Biomedical Sciences (SUTVA limbs; a label spanning heterogeneous interventions yields an ill-defined average effect) | https://doi.org/10.1017/cbo9781139025751 | grade A
    - Rubin hall-of-shoulders dossier (SUTVA: 'Potential outcomes are well-defined only under... no hidden versions'; rung-as-average diagnosis) | file:///D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/rubin/dossier.md | grade A
    - JPL_MGMT_SMA_TECH_04 dissertation.md Sec 1.4-1.5 lines 135/145/561/1473 (four heritage claims 'not interchangeable'; ordinal depth coding collapses distinct provenance) and lines 804/947 (only a same-environment re-coding robustness check, not a within-rung dispersion table) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation.md line 959 ('result tables are specified here in structure but left unpopulated, because populating them would require executing the procedure on the assembled dataset') | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[identification]** Rubin's demand to convert asserted outcome-blindness into an auditable seal is valid, the design supplies a partial procedural seal, but the specific seal Rubin asks for (physical redaction of outcome text) is NOT specified and the blind/unblind held-out concordance cannot be produced, so the objectivity claim is unproven and the empirical demand is refused. The design-trumps-analysis standard requires the design, including index construction, be fixed while blind to outcomes (Rubin 2008; Imbens-Rubin 2015). The dissertation operationalizes this two ways: (i) two independent coders code heritage-depth and test-fidelity and record inter-coder agreement (line 863; Sec 4.5 'measurement reliability is reported, not assumed', lines 593-605), and (ii) an eight-step procedure whose dependency order makes Steps 1-5 a function of predictors only, with Step 6 the first to join predictors to outcomes, so 'every decision that an analyst could otherwise have tuned to manufacture a result has already been fixed and recorded' (lines 860-875). It also codes provenance from the design-review heritage matrix that 'predates the outcome' and parts/test from as-procured/as-planned records (rubin_r1_c3 evidence). But two gaps defeat the seal as Rubin frames it. First, the dissertation specifies inter-CODER agreement (coder vs coder), NOT blind-vs-unblinded agreement (redacted-source coding vs full-file coding); the two are different audits and only the latter tests outcome-contamination. Second, the dissertation does not state that source excerpts were presented with post-commissioning anomaly, end-of-life, and outcome text physically redacted; it relies on the document's DATE (design-review-era) rather than on REDACTION to enforce blindness, and it concedes a 'reverse documentation' threat in which a failed subsystem could be retrospectively recoded as less heritage-rich (line 776). Most decisively, coding has not been executed on the assembled frame (line 959; frame 'still being assembled'), so no held-out blind/unblind concordance number exists. The seal is therefore asserted procedurally but not demonstrated; the requested concordance is unretrievable and refused.
    - Rubin (2008), For Objective Causal Inference, Design Trumps Analysis, Ann. Appl. Statist. (design/index construction must be sealed while blind to outcomes) | https://doi.org/10.1214/08-aoas187 | grade A
    - Imbens & Rubin (2015), Causal Inference for Statistics, Social, and Biomedical Sciences (objectivity rests on the design being completed before outcomes are examined) | https://doi.org/10.1017/cbo9781139025751 | grade A
    - JPL_MGMT_SMA_TECH_04 dissertation.md line 863 + Sec 4.5 lines 593-605 (two independent coders on heritage and test-fidelity indices; inter-CODER agreement recorded) and lines 860-875 (eight-step outcome-blind dependency order; Steps 1-5 predictors only, Step 6 first joins outcomes) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
    - JPL_MGMT_SMA_TECH_04 dissertation.md line 776 ('reverse documentation' threat: a failed subsystem could be retrospectively recoded as less heritage-rich) and line 959 (procedure not yet executed; tables unpopulated) - no held-out blind/unblind concordance exists, and blindness rests on document date not physical redaction | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[identification]** Rubin's elevation of the documentation-probability model to a second assignment mechanism requiring its own overlap and balance is correct, the design recognizes the threat, but the demanded balance/overlap diagnostic on the documentation propensity does not exist and is refused. Inverse-probability weighting is only valid where positivity/overlap holds on the weighting model's covariates; without common support the weights extrapolate into strata with no donors (Imbens-Rubin 2015; Rubin dossier 'Overlap, positivity, and the limits of e[xtrapolation]'). The dissertation builds the sampling frame from launch manifests rather than surviving records precisely to admit early-failed and thin-documentation cells (lines 752, 1507), estimates a documentation-probability model and inverse-weights by it, and concedes survivorship is the first-order threat: 'missions and subsystems that failed before producing complete documentation are underrepresented' and heritage 'quietly dropped after an early failure' biases the comparison in a direction that flatters heritage (lines 1161, 776; rubin_r1_c3). It also names overlap as a first-class condition to be reported, conceding the physical possibility that JPL practice always pairs deep heritage with high parts-class and full test so that 'no within-stratum comparison... exists' and that adequate overlap is 'unknown at the design stage' (line 1153; rubin_r1 Ch2.5.2). But the dissertation reports the overlap/balance diagnostic for the HERITAGE treatment, and does not present a SECOND balance table showing that, after documentation weighting, thinly-documented cells overlap fully-documented cells on environment, prominence, mass class, epoch, and subsystem. The documentation propensity is treated as a correction to be fit and frozen, not as an assignment mechanism whose own positivity is demonstrated. Because the frame is unassembled and tables are 'left unpopulated' (line 959), the documentation-propensity balance and the existence of a no-donor region cannot be computed from any retrieved source; the demand is refused.
    - Imbens & Rubin (2015), Causal Inference for Statistics, Social, and Biomedical Sciences (overlap/positivity must hold on the weighting model's covariates; weights extrapolate without common support) | https://doi.org/10.1017/cbo9781139025751 | grade A
    - Rosenbaum & Rubin (1983), The central role of the propensity score in observational studies for causal effects, Biometrika (any assignment-probability model used for weighting requires demonstrated balance on the conditioning covariates) | https://doi.org/10.1093/biomet/70.1.41 | grade A
    - Rubin hall-of-shoulders dossier ('Overlap, positivity, and the limits of extrapolation'; weighting past common support imports selection) | file:///D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/rubin/dossier.md | grade A
    - JPL_MGMT_SMA_TECH_04 dissertation.md lines 752/1507 (frame from launch manifests to admit early-failed cells), lines 1161/776 (survivorship first-order threat; documentation depends on outcome; bias flatters heritage), line 1153 (overlap reported for heritage; possibility of no within-stratum comparison) - no second balance table on the documentation propensity, frame unassembled (line 959) | file:///D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/dissertation.md | grade C
- **[measurement]** GROUNDED on the methodological standard, REFUSED on the candidate-specific statistic. SCC's tradition does require exactly what is demanded: an operation (a 'sustainability' score, 'heritage-depth' ordinal) must be shown to instantiate the named construct and not a correlated proxy, via a second independent operationalization that should move with the first if the construct is real, with a pre-stated divergence that would falsify it (shadish_cook_campbell dossier; Cook & Campbell 1979). An admissible second operationalization must come from a different method/source than the coding archive: an environment- or article-state measure that is rigorous about the state but does not inherit the documentation practice of the producing program (cf. Anselmo & Pardini 2022, which measures environment state rigorously yet by design cannot attribute that state to the producing intervention). However, NO retrieved source supplies, for THIS candidate's JPL heritage/test data, a specific multitrait-multimethod convergent correlation value, nor an empirically justified abandonment threshold; those are candidate-design facts no queried corpus settles, so no number is asserted.
    - shadish_cook_campbell dossier (Hall of Shoulders brain), 'a proxy ... is an operation. Demonstrate construct validity ... Give me a second, independent operationalization that should move together with the first ... and tell me what divergence would falsify it.' | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/shadish_cook_campbell | grade A
    - Cook & Campbell, Quasi-Experimentation: Design and Analysis Issues for Field Settings (1979), catalogues construct-validity threats and convergent/divergent operationalization | https://doi.org/10.1207/s15327752jpa4601_16 | grade A
    - Anselmo & Pardini, Using the space debris flux to assess the criticality of the environment in low Earth orbit (2022), independent state metric that measures rigorously but does not inherit the producing intervention's documentation | https://doi.org/10.1016/j.actaastro.2022.05.045 | grade B
- **[empirics]** GROUNDED on the threat, REFUSED on the candidate-specific threshold. Instrumentation is a named SCC threat that explicitly includes calibration/measurement changes over a multi-year record (the dossier flags Ozone Monitoring Instrument calibration drift over its record as instrumentation, alongside history), and SCC treats validity threats as an enumerable checklist that a before-after or dummy-adjusted comparison fails to neutralize unless the design addresses the specific live threat (shadish_cook_campbell dossier; Cook & Campbell 1979). This legitimates the demand for an instrument-anchored, blind re-coding invariance check rather than an outcome-side dummy. But NO retrieved source supplies a coded-level-by-epoch interaction magnitude or a pre-set concession level for THIS heritage/test-fidelity instrument; that threshold is a candidate-design decision no queried corpus settles, so none is asserted.
    - shadish_cook_campbell dossier (Hall of Shoulders brain), 'instrumentation (changes in the Ozone Monitoring Instrument's calibration over its multi-year record)'; 'threats to internal validity as an enumerable rival-cause checklist ... history, maturation, selection ...' | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/shadish_cook_campbell | grade A
    - Cook & Campbell, Quasi-Experimentation (1979), instrumentation as a threat to validity in field settings | https://doi.org/10.1207/s15327752jpa4601_16 | grade A
- **[identification]** GROUNDED on the construct-irrelevance/bad-control risk, REFUSED on the candidate-specific overlap count. SCC's construct-validity threat fires precisely when a composite may instantiate a correlated proxy rather than the named construct, and the dossier states this directly for composite indicators ('does the composite instantiate disclosure willingness rather than sustainability'); the policy-instrument literature it cites makes the same proxy point ('does the incentive target sustainability or a correlated proxy', Colvin, Karcz & Wusk 2021). If heritage depth is partly a proxy for prominence, conditioning on prominence is a construct-validity/bad-control hazard, validating the demand for a within-stratum overlap count and a pre-set non-identification rule before estimation. But NO retrieved source supplies the candidate's actual count of mission-by-subsystem cells with heritage variation inside a common prominence/parts-class/test stratum, nor a defensible minimum-cell non-identification threshold; these are candidate-dataset facts no queried corpus settles, so none is asserted.
    - shadish_cook_campbell dossier (Hall of Shoulders brain), construct-validity threat that a composite may instantiate a correlated proxy rather than the named construct | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/shadish_cook_campbell | grade A
    - Colvin, Karcz & Wusk, Policies for incentivizing orbital debris assessment and remediation (2021), MIT Science Policy Review, 'does the incentive target sustainability or a correlated proxy' | https://doi.org/10.38105/spr.16gdw8z5d4 | grade B
    - Space sustainability rating: designing a composite indicator to incentivize satellite operators (2025), Acta Astronautica, composite indicator construct-validity exemplar | https://doi.org/10.1016/j.actaastro.2025.03.034 | grade A
- **[identification]** The threat is correctly named: a fixed effect that differences out a shared secular trend (history) also differences out any treatment variation that is coded along that same axis, so an epoch dummy cannot simultaneously be the control series for history AND leave the heritage discount identified unless the heritage-vs-parts contrast survives WITHIN epoch. In the Campbellian frame this is the difference between a genuine quasi-experimental control series (a contemporaneous nonequivalent comparison class: same epoch, same environment, same prominence, heritage-poor vs heritage-rich subsystems) and a dummy that absorbs trend and treatment together. The methodological demand is legitimate and standard: an interrupted-time-series / nonequivalent-group design earns inference by differencing out shared history via a comparison series while preserving the treatment contrast, not by collapsing both into one fixed effect. GROUNDING LIMIT: the actual partition (fraction within- vs between-epoch) and the within-epoch cell counts live in the candidate's data, not in any retrievable source, so no number is asserted here; the candidate must produce them.
    - Shadish, Cook & Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference (2002), dossier extract | https://doi.org/10.1198/jasa.2005.s22 | grade A
    - Bernal, Cummins & Gasparrini, Interrupted time series regression for the evaluation of public health interventions (2017), retrieved via thinker brain | https://doi.org/10.1093/ije/dyw098 | grade A
    - Cook & Campbell, Quasi-Experimentation: Design and Analysis Issues for Field Settings (1979), retrieved via thinker brain | https://doi.org/10.1207/s15327752jpa4601_16 | grade A
- **[rival]** The external-validity demand is exactly the SCC generalized-causal-inference standard: generalization is not a free gift from a representative sample but a reasoned argument built from surface similarity, ruling out irrelevancies, making discriminations, interpolation/extrapolation, and causal explanation, stated over the UTOS frame (units, treatments, observations, settings). The critique is structurally correct: when the frame by construction excludes the population the conclusion targets, 'the mass-class and orbit controls bound it' is an assumed (interpolation/extrapolation) warrant, not an argued one, and SCC require the transfer to be argued rather than assumed, especially across a population shift (here, flagship-documentation to thin-documentation regimes). A regime/stratum shift is a generalized-causal-inference problem demanding an explicit transfer argument before the verdict can be claimed for the new domain. GROUNDING LIMIT: whether the candidate's specific surface-similarity claims and named mechanism actually hold across the documentation strata is a property of the dissertation, not of any retrieved source, so it is not asserted here.
    - Review: Experimental and Quasi-Experimental Designs for Generalized Causal Inference (Evaluation and Program Planning), retrieved via thinker brain | https://doi.org/10.1016/j.evalprogplan.2004.01.006 | grade B
    - Moon to Mars: strategic frameworks for space traffic management in cislunar and cismartian environments (2025), retrieved via thinker brain | https://doi.org/10.1016/j.actaastro.2024.12.056 | grade B
    - Shadish, Cook & Campbell (2002), dossier extract | https://doi.org/10.1198/jasa.2005.s22 | grade A
- **[empirics]** The multiplicity / researcher-degrees-of-freedom threat is real and is a statistical-conclusion-validity threat in SCC terms: a many-branch robustness fan-out reading the same verdict is convergence-by-selection, not a falsification test, unless convergence is required in advance. SCC's own remedy is the matching one the question demands: multiple operationalism is a strength only when paired with explicit pre-registration of which patterns would falsify the hypothesis, so the analyst cannot read a defensible verdict off whichever branch agrees. The methods literature names the same hazard precisely: undisclosed flexibility in data collection and analysis (researcher degrees of freedom) inflates false positives, and the 'garden of forking paths' shows the multiplicity bites even without conscious fishing whenever the analysis is contingent on the data. Therefore designating one primary specification + one trimming rule + a pre-committed cross-branch decision rule is the correct fix and is consistent with the candidate's own falsification framing. GROUNDING LIMIT: whether the verdict actually flips across the candidate's branches on this sample is a fact about the dissertation's runs, not about any retrieved source, and is not asserted here.
    - Shadish, Cook & Campbell dossier (multiple-operationalism + pre-registration of falsifiers) | https://doi.org/10.1198/jasa.2005.s22 | grade A
    - Simmons, Nelson & Simonsohn, False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis (2011), OpenAlex | https://doi.org/10.1037/e519702015-014 | grade A
    - Uncertainty Quantification for Space Situational Awareness and Traffic Management (2019), retrieved via thinker brain | https://doi.org/10.3390/s19204361 | grade B
- **[empirics]** Taleb's critique is methodologically valid: under fat tails and a documented mixture distribution, the mean (and a standardized mean-hazard coefficient) is dominated by a few extreme realizations, the historical record undersamples the tail, and sample means are unreliable, so a predictor ranking by mean hazard can legitimately invert between bulk and the early-failure tail subpopulation. The candidate's own anchor literature (Castet and Saleh) establishes that a mixture-Weibull, capturing a distinct early-failure (infant-mortality) subpopulation, fits the on-orbit data better than a single distribution, so a decision keyed to the mission-ending early-failure draw must be scored on that subpopulation, not the average hazard. However, whether heritage's advantage actually survives or inverts under tail-scoring is an EMPIRICAL verdict requiring a re-estimation on the assembled mission-by-subsystem data; the dissertation is an explicit design-stage analysis plan ('no number in this section is a result'; 'not estimated coefficients') and has not executed it, and no retrieved corpus reports this head-to-head, so the verdict is refused.
    - Hall of Shoulders Taleb dossier (Extremistan / sample means unreliable / mean dominated by extreme realizations) | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/taleb | grade A
    - Castet & Saleh, 'Single versus mixture Weibull distributions for nonparametric satellite reliability,' Reliability Engineering and System Safety 95(3):295-300, 2010 (candidate ref [5]) | https://doi.org/10.1016/j.ress.2009.10.001 | grade A
    - Candidate dissertation ch6_analysis_plan.md / prospectus.md (design-stage, 'no number is a result', verdict not executed) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/chapters/ch6_analysis_plan.md | grade C
- **[identification]** Taleb's identification concern is valid in principle: his program holds that ruin removes its own evidence, the historical record systematically undersamples extreme events, and treating the absence of catastrophe as evidence of safety is the classic fragility-hiding error; this is precisely outcome-dependent (MNAR) censoring rather than covariate-conditional MAR. The candidate's IPW correction is explicitly estimated from observed mission characteristics (prominence, mass class, orbit class, epoch, subsystem type) and thus assumes missingness is MAR conditional on those covariates; the candidate even concedes the under-documented cells are 'disproportionately early-failed,' which is the outcome-dependent mechanism Taleb names. If documentation probability still depends on realized early-failure status after conditioning, the IPW is correcting the wrong selection mechanism and the stated sensitivity bounds (which the plan derives under the documentation-probability model, not an MNAR model) are too narrow. But whether the dependence on realized early-failure status persists after conditioning, by how much the MNAR bounds widen, and where the verdict flips are EMPIRICAL quantities requiring the both-ways model on assembled data; the dissertation has not executed this (design-stage) and no retrieved source reports it, so those magnitudes are refused.
    - Hall of Shoulders Taleb dossier (record undersamples extreme events; absence of catastrophe is not evidence of safety; ruin removes its own evidence) | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/taleb | grade A
    - Taleb et al., 'The Precautionary Principle (with Application to the Genetic Modification of Organisms),' arXiv:1410.5787, 2014 (fat tails + non-localizable exposure; absence-of-evidence reasoning) | https://doi.org/10.48550/arXiv.1410.5787 | grade A
    - Candidate dissertation ch5_research_design.md / ch6_analysis_plan.md (IPW from observed covariates; under-documented cells 'disproportionately early-failed'; design-stage, not executed) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/chapters/ch5_research_design.md | grade C
- **[mechanism]** Taleb's second-order objection is structurally valid: fragility is defined as a concave (disproportionately harmed) second-order response to the dose of a stressor, and the antifragility program insists on detecting fragility from the curvature of the system's response to a dose rather than from a forecast or a mean association. An additive proportional-hazards index measures average (first-order) associations and therefore cannot, by construction, recover the curvature of the infant-mortality hazard in test-fidelity, nor whether that curvature differs (steepens) inside the deep-heritage stratum, which is exactly the fragility-transfer signal that distinguishes a benign mean shift from a push toward the absorbing early-failure regime. Demonstrating the convexity therefore requires a flexible (non-additive, interacted, dose-curvature) specification the candidate's stated additive index does not provide. But whether the as-run test-fidelity hazard is in fact sharply convex within the deep-heritage stratum versus the no-heritage stratum on the assembled data is an EMPIRICAL result; the dissertation is a design-stage plan with no executed estimates and no retrieved corpus reports this curvature, so the empirical convexity verdict is refused.
    - Hall of Shoulders Taleb dossier (fragile = concave to the stressor; detect fragility from second-order/convex response to the dose, not forecast) | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/taleb | grade A
    - Taleb, 'Antifragile: Things That Gain from Disorder,' 2012 (convexity/concavity of response to dose as the fragility measure) | https://doi.org/10.1080/14697688.2013.829244 | grade B
    - Candidate dissertation ch5_research_design.md / ch6_analysis_plan.md (additive proportional-hazards index; design-stage, convexity not estimated) | D:/Claude_Code/brain/collegium/candidates/dissertations/JPL_MGMT_SMA_TECH_04/chapters/ch6_analysis_plan.md | grade C
- **[empirics]** The demand is methodologically warranted but cannot be satisfied from retrieval. The correct instruments exist and are standard: case-deletion influence diagnostics and dfbeta (Belsley-Kuh-Welsch; Cook), which quantify how a single observation moves each coefficient. Taleb's warrant is direct: in Extremistan the variance and mean of a process are dominated by a few extreme realizations, so a small non-ergodic sample's coefficient is unreliable and a point comparison can be flipped by one high-leverage cell. So the question's framing is sound. BUT no retrieved source contains JPL_MGMT_SMA_TECH_04's assembled frame, its dfbeta values, the fraction of sign-reversing single deletions, or the max |dfbeta| share. Those specific empirical quantities are unsupported by any source retrieved this turn and are therefore not asserted.
    - Belsley, Kuh & Welsch, 'Regression Diagnostics: Identifying Influential Data and Sources of Collinearity' (NBER working-paper precursor 'Linear Regression Diagnostics') | https://doi.org/10.3386/w0173 | grade A
    - Cook & Weisberg, 'Residuals and Influence in Regression' | https://doi.org/10.2307/1269506 | grade A
    - Hall of Shoulders taleb dossier (Incerto synthesis; The Black Swan / Antifragile) | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/taleb | grade C
- **[identification]** The critique of IPW as a thin-tailed repair to non-ignorable missingness is correct and citable. IPW corrects bias only under missing-at-random conditional on observables; it cannot recover a non-ignorable (MNAR) tail, and its performance degrades sharply when the estimated weights are highly variable, precisely the regime of rare, disproportionately-undocumented catastrophic early failures. Taleb's warrant: the historical record systematically undersamples extreme events, so treating documented survival as evidence of safety is the fragility-hiding error, and the missing cells are exactly the absorbing-tail the test is about. A breakdown / tipping-point bound is the right object, not a plausible-range sensitivity. BUT no retrieved source contains the candidate's launch-manifest frame, the computed breakdown fraction, or the observed share of unlinked early-failed spacecraft. Those numbers are unsupported and not asserted.
    - Kang & Schafer, 'Demystifying Double Robustness...' and Robins et al. comment, Statistical Science; Sterne-style guidance 'Accounting for missing data... multiple imputation is not always the answer' | https://doi.org/10.1214/07-sts227 | grade A
    - Robins, Sued, Lei-Gomez & Rotnitzky, 'Comment: Performance of Double-Robust Estimators When Inverse Probability Weights Are Highly Variable' | https://doi.org/10.1214/07-sts227d | grade A
    - Hampel, 'Robust estimation: A condensed partial survey' (breakdown-point literature) | https://doi.org/10.1007/bf00536619 | grade A
    - Hall of Shoulders taleb dossier (The Black Swan; non-naive precautionary principle, Taleb et al. arXiv:1410.5787) | https://arxiv.org/abs/1410.5787 | grade C
- **[governance]** The demand is theoretically warranted: skin in the game is, in Taleb's formulation, both an ethical and a statistical filter, removing the actors whose assertions impose risk they do not bear; an associational heritage-to-failure regression that never instruments who issued the discount and whether they bore the downside cannot separate an incentive-asymmetry mechanism from a provenance mechanism. The accountability literature confirms that authority-without-downstream-consequence is a measurable, distinct construct. BUT no retrieved source contains JPL_MGMT_SMA_TECH_04's parts-control-board or design-review records, any coding of asserting-authority vs. downstream-accountability, or any test of effect concentration in the skin-out-of-the-game cells. That observable does not exist in retrieval and is not asserted.
    - Hall of Shoulders taleb dossier (Skin in the Game, 2018, within the Incerto) | D:/Claude_Code/brain/collegium/hall_of_shoulders/brains/taleb | grade C
    - Bovens, 'Analysing and Assessing Public Accountability: A Conceptual Framework' / 'Does Public Accountability Work? An Assessment Tool' | https://doi.org/10.1111/j.1467-9299.2008.00716.x | grade A

## Gaps

- **[identification]** The specific number demanded, the count of mission-by-subsystem overlap cells in strata holding both a heritage-rich and heritage-poor subsystem at comparable parts-class and test-fidelity, plus the per-stratum heritage variance, is not present in any retrieved source. Without that figure from the candidate's own assembled frame, retrieval cannot settle whether the heritage coefficient is identified off real within-stratum contrast or is silent extrapolation (a reshuffling of controls). The candidate must produce the overlap-cell tabulation. (raised by angrist_pischke)
- **[mechanism]** Whether the JPL parts-control-board and I&T-plan archives actually permit estimation of the heritage-to-parts-class and heritage-to-test-fidelity assignment relationships, and what sign that heritage-to-test-fidelity link takes, is not present in any retrieved source. The general inferential point (mediation vs confounding indistinguishable by attenuation alone; assignment-relationship sign discriminates) is grounded, but the candidate's empirical capacity to run that discriminating estimate on this data is unestablished by retrieval. (raised by angrist_pischke)
- **[measurement]** The documentary question, whether parts-control-board and I&T-plan records show planned test scope was fixed before the heritage determination at design review for the specific cells used, cannot be answered from any retrieved source. The principle (pre-treatment status is required for test-fidelity to be a clean control, else it is a bad control) is grounded; the empirical verification against the JPL archive is a gap only the candidate's source records can close. (raised by angrist_pischke)
- **[identification]** RD design demand unanswered. The candidate's archive contains ZERO occurrences of 'discontinuity', 'running variable', 'bandwidth', 'bunching', 'McCrary', or 'regression discontinuity'. Mission risk classification (A-D) and prominence/mass tiers are used only as covariate controls, never as a running variable with a threshold-assigned rigor regime. The records do not identify a running variable, do not count mission-by-subsystem cells in a usable bandwidth around any cutoff, and run no manipulation/bunching test for programs just below a classification cutoff. The candidate cannot, from current records, exploit the threshold-assigned variation in rigor as an RD design, nor justify discarding it; the panelist's claim that an RD sits latent in the parts-control / I&T records is not engaged anywhere in the dissertation. REFUSE: no source in the candidate's corpus settles this; asserting an RD design would be confabulation. (raised by angrist_pischke)
- **[measurement]** No corpus retrieved this turn supplies a quantified, space-domain estimate of how often a board upgrades a design-heritage box to a flight-heritage label that the as-built parts/environmental-qualification record does not support, nor a paired design-review-assertion-versus-as-built dataset that would measure the divergence magnitude. The mechanism (espoused vs enacted) is grounded, but the empirical size of the espoused-enacted heritage gap for the article class is not settled by available evidence. (raised by argyris)
- **[identification]** No retrieved source this turn provides a coded sample of actual descope-decision or design-review-waiver minutes in which 'heritage' is verbatim the justification for reducing qualification test scope, with the temporal ordering established case by case. The mediation mechanism is grounded in CAIB/Argyris, but the specific documentary corpus that would let the candidate fix the heritage-then-descope causal ordering empirically was not located in the brains or vault gap-fill this turn. (raised by argyris)
- **[rival]** No retrieved source this turn quantifies the rate or directional magnitude of post-anomaly heritage-label scrubbing in versioned spacecraft review baselines, nor supplies a paired pre-/post-anomaly heritage-coding dataset that would let the candidate estimate the attenuation bias on the heritage coefficient. The re-encoding mechanism is grounded (Dillon & Tinsley), but its measured effect size on a heritage IV in the candidate's article population is not settled by available evidence. (raised by argyris)
- **[measurement]** No retrieved source identifies a specific paired-record case on a single article where as-run test documentation describes the campaign as nominal while an independent parts-stress waiver, delta-qual deviation, review-board action item, or anomaly closure shows scope was cut against the heritage argument. The candidate's study is pre-registered and unexecuted (8.4.1), the JPL parts-stress/waiver and anomaly archives were not retrieved this turn, and the space corpora (AMOS, ACTA) returned zero hits for heritage-driven test-scope reduction. The empirical existence-claim cannot be asserted from evidence in hand. (raised by argyris)
- **[identification]** No retrieved source reports, from actual anomaly-closure records, whether closure completeness and root-cause coding depend on survived-vs-mission-affecting status after conditioning on the IPW covariates. The candidate has not executed this conditional-independence test (study pre-registered, 8.4.1), and no corpus retrieved this turn supplies the closure-record data. The direction and significance of the dependence, and therefore whether documentation-probability is independent of near-miss re-encoding, cannot be asserted from evidence in hand. (raised by argyris)
- **[measurement]** No retrieved source reports the fraction of audited cells on which the board's recorded heritage depth disagrees with an external, article-level anchor (as-built parts-lot, procurement/radiation-test records, prior-flight orbit/radiation environment), nor the direction of that disagreement. The validity audit against an out-of-narrative anchor has not been executed (study pre-registered, 8.4.1), and the underlying procurement/radiation-test records were not retrieved this turn. The disagreement fraction and direction cannot be asserted from evidence in hand. (raised by argyris)
- **[identification]** REFUSED on the affirmative empirical claim. The grounded retrieval supports the conceptual frame: Forrester's systems-dynamics review lens demands the candidate 'reproduce the trouble using only the system's internal feedback structure, with no exogenous shock,' and treats shared program state as a stock whose flows co-emit downstream choices. But no retrieved source reports the partial-correlation matrix, the within-stratum regression of regressors on a PDR-pressure proxy, or the surviving independent variance for THIS candidate's assembled JPL archive. The space corpora (AMOS, ACTA, Space Economy) returned zero hits on heritage/parts/test-fidelity confounding by schedule-cost pressure, and OpenAlex gap-fill returned only off-topic works. Whether residualized variance survives is a property of the candidate's dataset, which retrieval cannot inspect; asserting a number would be confabulation. The objection is structurally valid and the candidate must run the residualization-then-partial-correlation test; the result is unknown from any source retrieved this turn. (raised by forrester)
- **[mechanism]** REFUSED on the affirmative empirical claim. The retrieval grounds the mechanism that motivates the objection: Forrester's data-generating rule can itself be a stock that outcomes update (Industrial Dynamics 1961; Counterintuitive Behavior of Social Systems, 1971, doi:10.1007/bf00148991), and his explicit endogeneity test asks whether trouble can be reproduced from internal feedback without an external shock. That establishes the objection is well-posed in systems-dynamics terms. But no retrieved source dates this candidate's per-subsystem heritage coding, documents the standing heritage-caution posture time series, or reports a Granger test of coded heritage depth on lagged lineage failures. AMOS and ACTA returned zero hits on heritage endogeneity / cross-mission learning in a hazard model. Whether the archive carries the timestamps to run the test, and what the test would show, is unknown from any source retrieved this turn; asserting endogeneity-confirmed or -refuted would be fabrication. (raised by forrester)
- **[empirics]** REFUSED on the affirmative empirical claim. The retrieval strongly grounds the conceptual machinery: Forrester's stocks/flows formalism treats assurance hours as a depletable stock changed only through flows, and his shared-finite-resource framing (the orbital-commons analogues in the dossier: monopolistic competition in limited orbital space, doi:10.1007/s10640-025-00959-1; capacity-vs-traffic management, doi:10.1016/j.actaastro.2025.01.069) is the same structure: one actor's draw on a finite shared stock imposes an uncompensated cost on siblings, a balancing loop that couples decisions for allocation reasons rather than article reliability. That makes the SUTVA objection more than contamination: it is a within-mission negative-coupling loop on a shared stock. But no retrieved source measures THIS candidate's mission assurance-hour stock, the per-subsystem draw, or the depression of a subsystem's test-fidelity when a sibling's heritage claim is deep. AMOS returned zero hits on assurance-budget allocation across subsystems; Space Economy returned zero on the shared-resource test-fidelity tradeoff. Whether the I&T as-run records resolve assurance hours to the subsystem cell, and what the allocation test would reveal, is unknown from any source retrieved this turn. (raised by forrester)
- **[identification]** The JPL-data-specific empirical results these questions demand were NOT retrievable this turn. AMOS and ACTA returned zero on heritage/reliability/bathtub/survivorship queries (those corpora are SSA/SDA/astrodynamics, not spacecraft-reliability-engineering), and no JPL heritage-assessment matrices, flight logs, or launch-manifest reporting-probability records were available. Consequently three sub-claims remain unsettled and are asserted only as required tests, not findings: (Q1) whether age-since-last-flight actually dominates the heritage rung in the JPL cells; (Q2) whether the documentation/reporting probability is empirically a decreasing function of heritage depth after conditioning on early failure, and whether the candidate's IPW actually opens that loop versus reweighting within it; (Q3) whether heritage's measured advantage empirically localizes to one timescale in the JPL records. The methodological frame is grounded; the candidate-specific numbers are absent and must not be fabricated. (raised by forrester)
- **[measurement]** The executed reconciliation table is absent. The candidate has not produced: (a) the count of distinct JPL-class spacecraft launched in the epoch window per a fixed launch-manifest census; (b) the per-source match rate at stage 1 (canonical-mission-identifier match across anomaly system / NTRS / JPL archives), stage 2 (subsystem-crosswalk success), and stage 3 (anomaly time-and-root-cause codability); (c) the fraction of launched spacecraft that never enter the frame at all. Because the dataset is unassembled, the denominator currently IS the subset the three archives co-document, which is precisely the failure McDowell's question is designed to expose. No retrieved source supplies these numbers, so no count can be asserted. (raised by mcdowell)
- **[identification]** The identifiability of the survivorship correction is not demonstrated. Missing: (a) the explicit covariate list and functional form of the fitted documentation-probability model; (b) the covariate distribution of the LINKED versus the UNLINKED (zero-coded-cell) population on environment, prominence, mass class, and epoch from the launch-manifest frame; (c) any evidence that completely-unlinked spacecraft share covariate support with the documented subset rather than lying outside it. If the heritage-rich flagships are the survivors, the IPW weights are an extrapolation from a catalog the candidate has not independently verified, and no retrieved source closes this. The candidate's sensitivity analysis over bounded assumptions for undocumented cells is a partial mitigation, not the requested identifiability demonstration, and it too is unexecuted. (raised by mcdowell)
- **[empirics]** The candidate has not produced the anomaly-reporting rate per subsystem-year as a function of mission prominence on the linked sample, which is the diagnostic that would show whether the outcome floor (time-to-first-failure / infant-mortality) is uniform across the heritage-rich and heritage-poor strata the overlap region compares. The candidate accepts severity truncation as acceptable rather than measuring whether reporting completeness itself varies with prominence; if it does, reporting intensity is confounded with prominence and with heritage, and the failure 'measurement' is partly a documentation artifact. No retrieved source supplies the prominence-by-reporting-rate gradient, so the differential-reporting threat to the outcome side stands unrebutted on this round. (raised by mcdowell)
- **[identification]** The declared-versus-as-built heritage discrepancy rate, the share of mission-by-subsystem cells where a board's declared same-environment flight heritage is contradicted by the as-built parts lots, fabrication supplier, or prior-flight environment match, is NOT present in any retrieved source. The dissertation conceptually concedes the gap (a heritage label can attach to a design rebuilt with different parts lots, modified for a new environment, integrated by a different team, or tested less; 'same as before' can be true at the block-diagram level and false at the parts-list and test-matrix level) and operationalizes a PARTIAL hedge by coding the top heritage level against the prior flight's orbit and radiation environment (Sec 4.3.3) plus a stricter same-environment re-coding robustness check. But that codes the DECLARED environment match more strictly; it is not an independent reconciliation of the declared label against the as-built record. Heritage is coded from the design-review heritage assessment matrix (the operator claim), and the as-built JPL parts-lot / supplier records are access-controlled and not yet assembled, so no discrepancy rate exists to retrieve. Until the candidate reconciles declared heritage against the as-built record and reports the contradiction rate, the heritage regressor remains, in McDowell's terms, the unverified operator claim, and whether H0/H1 is about provenance at all cannot be settled from retrieval. (raised by mcdowell)
- **[identification]** The DATA half of Q1 cannot be answered. Pearl demands the candidate state, from the archives, which conditional-independence implication of the declared graph holds and thereby licenses 'mediator artifact' vs 'genuine spuriousness.' But the dissertation explicitly states (Ch.6 synthesis, Ch.7) that results are 'expected directions and illustrative magnitudes, clearly labeled as not yet executed on the full assembled dataset; the contribution is the pre-registered specification, not estimated coefficients.' No heritage->parts-class and heritage->test-fidelity arrow evidence has been estimated, so no conditional-independence implication can be reported as holding or failing. The candidate must (a) draw and freeze ONE DAG and (b) pre-register the specific d-separation test (e.g., heritage independent of failure given the full mediator set under the mediator graph vs not under the confounder graph) before assembly, and report it on the assembled LLIS/NTRS/JPL data. Until then the headline b1-attenuation test is non-identified as stated. (raised by pearl)
- **[mechanism]** The DATA half of Q2 cannot be answered. Pearl demands the through-channel vs around-channel SHARE of heritage's failure association and a commitment to full vs partial mediation from the mission-by-subsystem linkage. The dissertation is a design-stage plan with no assembled dataset and no estimated coefficients (stated explicitly in Ch.6/Ch.7), so no mediation decomposition exists to report a share. Worse, the candidate cannot currently claim front-door identification even in principle, because front-door requires that the mediator set be UNCONFOUNDED with the outcome given measured covariates -- and 'program competence' plausibly confounds the mediator->failure leg too, which would void front-door as well. The candidate must (a) commit to full or partial mediation as a pre-registered structural assumption, (b) state whether front-door's mediator-outcome unconfoundedness is defensible against program competence, and (c) report the channel-share decomposition on the assembled data. None is available pre-assembly. (raised by pearl)
- **[empirics]** The DATA half of Q3 cannot be answered. Pearl demands the specific conditional-independence test the candidate WILL run in the LLIS/NTRS/JPL data and the observed independence that would falsify the mediator reading. The methodology of the right test is grounded (claim c3: a d-separation/descendant test of whether heritage predicts parts-class and test-fidelity), but the candidate has assembled no dataset, so no independence can be reported as observed or refuted -- the design itself states the data is 'not yet executed on the full assembled dataset.' The candidate must pre-register, before assembly: (1) the frozen DAG with directed edges; (2) the exact conditional-independence implication chosen as the licensing test (heritage-vs-mediator association as the descendant test; heritage-failure association conditioned on the intermediate set as the path-blocking test); (3) the decision rule mapping the observed independence to the mediator vs confounder reading. Absent the assembled data none of these is reportable, and the overlap/balance machinery in Ch.5-6 does not substitute for it. (raised by pearl)
- **[identification]** No retrieved source (AMOS, ACTA, Space Economy, or the Pearl brain) contains the candidate's assembled mission-by-subsystem frame, the standardized heritage-to-hazard coefficient, or its values across the candidate's epoch dummies and mass-class strata. The empirical core of Q1, demonstrating from the data that the heritage coefficient IS or IS NOT invariant across those partitions, cannot be settled by retrieval and must not be asserted. The methodological license condition is grounded (pearl_r2_c1); the empirical invariance verdict is a confabulation risk and is refused. (raised by pearl)
- **[identification]** No retrieved source contains the candidate's documentation-completeness (IPW) model, the launch-manifest frame, or the test of whether documentation probability still depends on realized early-failure after conditioning on the full covariate set. The empirical core of Q2, reporting the residual dependence and naming the specific recoverability-restoring external/surrogate margin (e.g., a population-level launched-spacecraft count by class/epoch) as validated for THIS frame, cannot be settled by retrieval. The structural diagnosis is grounded (pearl_r2_c2); the empirical fit result is refused. (raised by pearl)
- **[rival]** No retrieved source contains the candidate's heritage-depth-by-environment cross-tabulation. The empirical core of Q3, showing whether any study cell shares the target's (heritage-depth, new-radiation-environment) profile, and therefore which of the three exits (re-weight / re-target / declare non-identified) the decision-relevant cell actually triggers, cannot be settled by retrieval. The admissible-exits frame is grounded (pearl_r2_c3); the cell-level overlap verdict is refused. (raised by pearl)
- **[identification]** EMPIRICAL DEMAND UNANSWERABLE FROM SOURCE (Q1): the joint cell counts of heritage-depth x parts-class x test-fidelity, and whether any off-diagonal cell (deep heritage at low parts-class, or shallow heritage at full test-fidelity) is non-empty, do not exist. The dissertation states the mission-by-subsystem frame is 'still being assembled' and that 'No estimated coefficient is presented as a finding anywhere in the document' (lines 173/345/844). Until the frame is assembled and the cross-tabulation computed, the manipulability-of-heritage question is unresolved: the design CONCEDES the top heritage category is definitionally bundled with parts and qualification and offers only a robustness re-coding, not a demonstration of a non-empty off-diagonal. No source retrieved this turn supplies the counts; asserting any would be confabulation. (raised by rubin)
- **[identification]** EMPIRICAL DEMAND UNANSWERABLE FROM SOURCE (Q2): the fraction of subsystem cells sharing an EEE parts lot or a power/data bus with another cell in the same mission, and the movement of the heritage-vs-parts contrast when interfering cells are excluded vs retained, do not exist. The design proposes to record lot/bus commonality and examine affected cells separately (residual interference risk rated 'moderate') but reports no interference fraction and no estimand restated on the lot/bus unit, because the frame is unassembled (line 844). The question of whether the chosen unit can support a SUTVA-valid potential outcome is therefore empirically open. No retrieved source settles it. (raised by rubin)
- **[measurement]** EMPIRICAL DEMAND UNANSWERABLE FROM SOURCE (Q3): the share of early-failed cells for which heritage, parts-class, and test-fidelity are all recoverable from strictly pre-launch documents (design-review heritage matrices, as-procured parts records, as-planned I&T plans dated before commissioning), and the heritage-vs-parts contrast on that pre-launch-only subset vs the full weighted sample, do not exist. The design is built to permit this test (provenance-at-design-review rule; as-procured/as-planned coding; pre-registered documentation-probability model) and concedes the unobservables channel is only 'moderate', but it reports no recoverability share and no pre-launch-only contrast, because the launch-manifest frame is unassembled (lines 173/345/844). Whether the survivorship correction and overlap claim are genuinely outcome-blind is therefore unproven on this design. No retrieved source supplies the falsifiable share. (raised by rubin)
- **[measurement]** EMPIRICAL DEMAND UNANSWERABLE FROM SOURCE (Q1): the within-rung-across-version dispersion of the infant-mortality outcome (cells inside one heritage rung partitioned by which of the four claims produced the rung, with the realized-outcome spread across those sub-versions) does not exist. The dissertation concedes the four claims are 'not interchangeable' and collapses them to an ordinal rung, but offers only a same-environment re-coding robustness check, not a within-rung outcome-dispersion partition, and the frame is unassembled with result tables 'left unpopulated' (line 959). Whether each heritage rung is one well-defined treatment level or an average over incommensurable interventions is therefore empirically open; no retrieved source supplies the dispersion, and asserting any spread would be confabulation. (raised by rubin)
- **[identification]** EMPIRICAL DEMAND UNANSWERABLE FROM SOURCE (Q2): the auditable blinding SEAL Rubin requires is not demonstrated. The design specifies inter-CODER agreement (coder vs coder) and an outcome-blind step-dependency order keyed to document DATE, but does NOT specify physical redaction of post-commissioning anomaly/end-of-life/outcome text, and cannot produce held-out blind-coded vs unblinded-coded heritage-rung concordance because coding is unexecuted on an unassembled frame (line 959). The dissertation itself names a 'reverse documentation' contamination threat (line 776). Whether the heritage and test-fidelity indices were genuinely constructed blind to outcomes is therefore unproven on this design; no retrieved source settles it. (raised by rubin)
- **[identification]** EMPIRICAL DEMAND UNANSWERABLE FROM SOURCE (Q3): the covariate balance and overlap of the DOCUMENTATION-PROBABILITY (survivorship IPW) model treated as a second assignment mechanism do not exist. The dissertation reports overlap for the heritage treatment and freezes a documentation-probability correction, but presents no second balance table showing whether, after weighting, thinly-documented cells overlap fully-documented cells on environment, prominence, mass class, epoch, and subsystem, and identifies no no-donor region. The frame is unassembled and tables are unpopulated (line 959). Whether the survivorship fix extrapolates past common support, re-importing the selection it claims to remove, is therefore empirically open; no retrieved source supplies the documentation-propensity balance. (raised by rubin)
- **[measurement]** No retrieved source supplies a specific multitrait-multimethod convergent-divergent correlation value for the candidate's heritage-depth ordinal against an independent reliability-margin measure, nor a defensible abandonment threshold. The required value is a candidate-design quantity; absent it, the falsification number cannot be asserted from retrieval. (raised by shadish_cook_campbell)
- **[empirics]** No retrieved source supplies a coded-level-by-epoch interaction value or a pre-set drift-concession threshold for the candidate's heritage and test-fidelity ordinals under a single fixed blind rubric. The invariance test is endorsed in form, but the numeric concession level is a candidate-design quantity not settled by retrieval. (raised by shadish_cook_campbell)
- **[identification]** No retrieved source supplies the candidate's within-stratum overlap cell count (heritage varying while prominence, parts-class, and test-fidelity are held in a common stratum) or a pre-set minimum-cell non-identification threshold. The bad-control/proxy risk is grounded in form, but the concrete count and cutoff are candidate-dataset quantities not settled by retrieval. (raised by shadish_cook_campbell)
- **[identification]** No retrieved source supplies the candidate's own on-data quantities: the fraction of the residual heritage-vs-parts contrast that is within-epoch versus between-epoch, and the surviving within-epoch cell counts. The threat is named and the remedy is grounded, but whether history is controlled or confounded in THIS frame cannot be settled from retrieval; the candidate must report the partition and the within-epoch cell counts. (raised by shadish_cook_campbell)
- **[empirics]** The empirical verdict is unsettled and refused: whether, on the assembled JPL-class mission-by-subsystem data, heritage's advantage lives in the bulk while parts-class and test-fidelity dominate the early-failure tail mass (mixture early-subpopulation weight / upper failure-count quantile), or whether the mean-coefficient ranking survives a tail-scored re-estimation. No executed result exists (dissertation is a pre-registered design) and no retrieved source reports it; cannot be asserted without fabrication. (raised by taleb)
- **[identification]** Refused empirical verdict: whether documentation probability depends on a cell's realized early-failure status after conditioning on environment, prominence, mass class, and epoch (i.e., MNAR vs MAR), how far the outcome-dependent (MNAR) sensitivity bounds widen beyond the candidate's covariate-conditional IPW bounds, and where within those wider bounds the heritage-vs-rigor verdict flips. No both-ways model has been run (design-stage dissertation) and no retrieved source supplies these numbers; cannot be asserted. (raised by taleb)
- **[mechanism]** Refused empirical verdict: whether the infant-mortality hazard's response to cutting test-fidelity is sharply convex (accelerating harm) within the deep-heritage stratum and whether that curvature exceeds the no-heritage stratum (a fragility transfer toward the absorbing early-failure regime). No curvature/convexity has been estimated (additive design-stage model, no executed results) and no retrieved source reports it; cannot be asserted without fabrication. (raised by taleb)
- **[empirics]** No retrieved source supplies the candidate's actual leave-one-out / leave-one-cell-out influence distribution, the empirical fraction of single deletions that reverse the H0/H1 verdict, or the maximum |dfbeta| as a share of the (b1 minus max(b2,b3)) contrast. These are facts about the candidate's specific assembled frame and cannot be produced from the corpora. The candidate must run and report the diagnostic on the actual frame; absent that, the pre-registered point-comparison rule cannot be certified against single-observation fragility. (raised by taleb)
- **[identification]** No retrieved source supplies the candidate's adversarial breakdown fraction (smallest share of undocumented early-failure cells set to favor H0 that overturns H1) on the launch-manifest frame, nor the observed share of unlinked early-failed spacecraft to compare it against. The settling comparison the question demands cannot be computed from the corpora. The candidate must estimate the breakdown bound on the actual launch-manifest frame and place it against the observed unlinked-early-failure share; absent that, the survivorship correction is not demonstrated to bound the conclusion against the missing tail. (raised by taleb)
- **[governance]** No retrieved source supplies a coding of the candidate's parts-control-board / design-review records that ties each test-scope reduction to the actor whose heritage assertion authorized it and to whether that actor bore the realized on-orbit downside, nor any test of whether the heritage-to-failure association concentrates in the no-downside cells. The skin-in-the-game observable the question demands is not present in any corpus and must be constructed from the candidate's own program records; absent it, the regression as specified cannot adjudicate incentive-misalignment vs. provenance. (raised by taleb)