{"claim": "A single latency scalar read off outcome-records can conflate three causally distinct generative processes (Model I rational pause, Model II organizational-routine queue clearing, Model III bureaucratic bargaining resultant), which carry opposite reform implications; the decomposition logic is sound and process type should be hand-coded before the coefficient is trusted.", "evidence": [{"source": "Jones, C. M. (2010), Bureaucratic Politics and Organizational Process Models, Oxford Research Encyclopedia of International Studies", "doi_or_url": "https://doi.org/10.1093/acrefore/9780190846626.013.2", "grade": "A"}, {"source": "Roberts (2025), simulating Allison's three foreign-policy decision-making models as distinct processes", "doi_or_url": "https://doi.org/10.1017/S1049096525101650", "grade": "A"}, {"source": "Hall, J. L. (2016), Columbia and Challenger: Organizational failure at NASA, Space Policy", "doi_or_url": "https://doi.org/10.1016/j.spacepol.2016.11.001", "grade": "B"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Authorizing-office workload is the operating variable of a Model II organizational-congestion mechanism and proximity to the appropriations calendar is the clock of a Model III bargaining game; if both are channels of the same dynamics through which latency would act, treating them as exogenous nudges mis-specifies them, so a seat-by-seat exclusion argument plus a falsification test (e.g., no-year-funded programs) is warranted.", "evidence": [{"source": "Jones, C. M. (2010), Oxford Research Encyclopedia of International Studies (Model II SOP/routine output; Model III bargaining resultants among seated players)", "doi_or_url": "https://doi.org/10.1093/acrefore/9780190846626.013.2", "grade": "A"}, {"source": "Hall, J. L. (2016), Columbia and Challenger: Organizational failure at NASA, Space Policy", "doi_or_url": "https://doi.org/10.1016/j.spacepol.2016.11.001", "grade": "B"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "The bureaucratic-politics common-cause rival is live: in Model III a contested multi-center program can co-produce both long latency and high cost growth as joint residues of one coalition fight with neither causing the other, and era fixed effects absorb common shocks but not program-specific bargaining intensity, so an independent documentary proxy for bargaining intensity is the correct falsification against over-attribution.", "evidence": [{"source": "Jones, C. M. (2010), Oxford Research Encyclopedia (Model III pulling and hauling, where you stand depends on where you sit)", "doi_or_url": "https://doi.org/10.1093/acrefore/9780190846626.013.2", "grade": "A"}, {"source": "Hall, J. L. (2016), Columbia and Challenger: Organizational failure at NASA, Space Policy", "doi_or_url": "https://doi.org/10.1016/j.spacepol.2016.11.001", "grade": "B"}], "facet": "rival", "chapter": "ch7_discussion", "subclaim": "alternatives"}
{"claim": "Under a continuous treatment, parallel trends identifies treatment-on-the-treated parameters at a given dose but does not license comparing them across dose levels because selection-into-dose generates bias; reading a cross-dose slope as a marginal effect therefore requires a strictly stronger stateable assumption, and absent it the defensible estimand is ATT-at-observed-dose, with the disaggregated ATT(g,t) plus transparent aggregation weights the disciplined alternative.", "evidence": [{"source": "Callaway, Goodman-Bacon & Sant'Anna, Difference-in-Differences with a Continuous Treatment (2024 working paper)", "doi_or_url": "https://doi.org/10.2139/ssrn.4716682", "grade": "B"}, {"source": "Callaway & Sant'Anna, Difference-in-Differences with multiple time periods, Journal of Econometrics (2021)", "doi_or_url": "https://doi.org/10.1016/j.jeconom.2020.12.001", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "The correct test for the anticipation/baseline-gaming worry is pre-treatment placebo ATT(g,t); a flat pre-trend is necessary but not sufficient (conventional pre-trends tests have low power and conditioning on passing a pre-test can itself distort the estimate), and the practitioner must state the canonical assumptions, name which are relaxed, choose clean controls, and report sensitivity to parallel-trends/anticipation violations rather than a single point estimate.", "evidence": [{"source": "Roth, Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends, American Economic Review: Insights (2022)", "doi_or_url": "https://doi.org/10.1257/aeri.20210236", "grade": "A"}, {"source": "Roth, Sant'Anna, Bilinski & Poe, What's Trending in Difference-in-Differences? Journal of Econometrics (2023)", "doi_or_url": "https://doi.org/10.1016/j.jeconom.2023.03.008", "grade": "A"}], "facet": "identification", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "A TWFE/2x2 estimator under staggered timing equals a weighted average of all two-group/two-period comparisons, and when effects grow over time comparisons using already-treated units as controls enter with negative weights and can drive the estimate toward zero or the wrong sign; instrumenting does not escape this, so the 2SLS estimand must itself be decomposed and shown convex, with the Callaway-Sant'Anna corrective (exclude already-treated controls, estimate ATT(g,t), aggregate transparently) the standard it must meet.", "evidence": [{"source": "Goodman-Bacon, Difference-in-differences with variation in treatment timing, Journal of Econometrics (2021)", "doi_or_url": "https://doi.org/10.1016/j.jeconom.2021.03.014", "grade": "A"}, {"source": "Callaway & Sant'Anna, Difference-in-Differences with multiple time periods, Journal of Econometrics (2021)", "doi_or_url": "https://doi.org/10.1016/j.jeconom.2020.12.001", "grade": "A"}], "facet": "empirics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "The dissertation is design-stage and reports no estimates from the assembled panel (Table 6.1 is the illustrative coefficient table deliberately left unpopulated); the Goodman-Bacon decomposition is named only as a diagnostic to be reported once estimated, and wild-cluster bootstrap plus alternative clustering are pre-committed so conclusions are not driven by a few large modern programs.", "evidence": [{"source": "PHD-08 dissertation.md, Section 1.7 (line 158), list of tables (line 53), Section 4.2 (line 549); Hall-of-Shoulders callaway_santanna dossier", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Goodman-Bacon, Difference-in-differences with variation in treatment timing, Journal of Econometrics (2021)", "doi_or_url": "https://doi.org/10.1016/j.jeconom.2021.03.014", "grade": "A"}], "facet": "empirics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "The panel documents the five-band era-regime partition (pre-formal-baseline, program-management formalization, NPR 7120.5 life-cycle-review, joint-confidence-level, Standing-Review-Board) and the coarse-resolution coverage of early decades, and pre-commits to report descriptive latency and outcome distributions per era regime before estimation, but the era-disaggregated estimates are not produced (the bands are illustrative, fixed at assembly).", "evidence": [{"source": "PHD-08 dissertation.md, Appendix C era-regime table (lines 1368-1380), Appendix B coverage table, Section 6 describe-before-estimate (line 831)", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Callaway & Sant'Anna, Difference-in-Differences with multiple time periods, Journal of Econometrics (2021)", "doi_or_url": "https://doi.org/10.1016/j.jeconom.2020.12.001", "grade": "A"}], "facet": "rival", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "The dissertation confines the target population to NASA programs with documented baselines 1958-2026 and explicitly refuses generalization to other agencies or commercial programs; cancelled/censored cases (Constellation; SLS upper stage plus second mobile launcher) enter only as program-record anecdotes, the panel is unbalanced by construction with some outcomes censored, and NTRS coverage-density bias over-represents better-documented (larger, modern) programs, so transportability is acknowledged and partially bounded by design but not demonstrated.", "evidence": [{"source": "PHD-08 dissertation.md, Sections 1.6-1.7 (lines 144-158), 4.2 (line 549), 4 known biases (line 525)", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Yoon (2024), nonparametric censored-dependent-variable panel estimator (noted sensitivity lane)", "doi_or_url": "https://doi.org/10.2139/ssrn.4953073", "grade": "B"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Complex program behavior is dominated by closed loops of causation, so a latency -> trouble relationship must be presumed bidirectional until tested (trouble is itself an inflow to review activity that extends latency); the canonical project literature operationalizes this as rework, schedule-pressure, and ripple-effect loops, and a single-equation FE+IV slope cannot by construction distinguish an open-loop from a closed-loop coefficient, so an asymmetric cross-lag test is required.", "evidence": [{"source": "Forrester, Industrial Dynamics (1961) / Urban Dynamics", "doi_or_url": "https://doi.org/10.2307/214050", "grade": "A"}, {"source": "Lyneis et al., Quantifying the impacts of rework, schedule pressure, and ripple effect loops on project schedule performance, System Dynamics Review", "doi_or_url": "https://doi.org/10.1002/sdr.1551", "grade": "A"}, {"source": "Lyneis & Ford, System dynamics applied to project management: a survey, assessment, and directions, System Dynamics Review", "doi_or_url": "https://doi.org/10.1002/sdr.377", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "mechanism"}
{"claim": "Complex social systems are insensitive to most chosen policy levers and tend to counteract them (policy resistance), because actors perceive local short-term cause-and-effect but not the closed-loop delayed structure; the domain compensating channel is documented (cutting authorized review time raises schedule pressure, which increases rework and ripple-effect loops, partially restoring elapsed schedule), so a measured latency reduction need not compress the closed-loop outcome and only a post-regime trajectory test can reveal which way it went.", "evidence": [{"source": "Forrester, Counterintuitive Behavior of Social Systems (1971)", "doi_or_url": "https://doi.org/10.1007/bf00148991", "grade": "A"}, {"source": "Forrester, Urban Dynamics (low-leverage policy deepens the targeted condition)", "doi_or_url": "https://doi.org/10.2307/214050", "grade": "A"}, {"source": "Lyneis et al., Quantifying the impacts of rework, schedule pressure, and ripple effect loops, System Dynamics Review", "doi_or_url": "https://doi.org/10.1002/sdr.1551", "grade": "A"}], "facet": "governance", "chapter": "ch7_discussion", "subclaim": "mechanism"}
{"claim": "Cost growth and schedule slip are stocks (accumulations) while per-phase latency is a flow; Sterman's bathtub-dynamics result shows even expert subjects misinfer an accumulation's trajectory from a contemporaneous flow because flow-to-stock delays are not mentally simulable, so if the dominant latency-to-outcome delay exceeds the phase window the within-phase contemporaneous beta mis-assigns to phase p an accumulation it only seeded, making an impulse-response lag estimate mandatory before the phase-as-unit choice is justified.", "evidence": [{"source": "Sterman, Bathtub Dynamics: initial results of a systems thinking inventory, System Dynamics Review", "doi_or_url": "https://doi.org/10.1002/sdr.198", "grade": "A"}, {"source": "Forrester, Industrial Dynamics (delays / limits-of-mental-model principle)", "doi_or_url": "https://doi.org/10.2307/214050", "grade": "A"}, {"source": "Lyneis et al., Quantifying the impacts of rework, schedule pressure, and ripple effect loops, System Dynamics Review", "doi_or_url": "https://doi.org/10.1002/sdr.1551", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "A reinforcing latency -> rework -> latency loop is a closed internal feedback structure that a static level-on-level regression with additive fixed effects cannot represent or detect; Forrester's endogeneity test demands reproducing the troublesome behavior from internal feedback alone, and the canonical space-domain reinforcing-loop archetype is the self-sustaining collisional (Kessler) cascade, so if the narrative posits a loop, latency is a flow inside it rather than an exogenous regressor and treating it as exogenous biases beta.", "evidence": [{"source": "Forrester, Industrial Dynamics (information feedback loops and material/information delays govern system behavior)", "doi_or_url": "https://doi.org/10.1002/sdr.284", "grade": "A"}, {"source": "Kessler & Cour-Palais, Collision frequency of artificial satellites: the creation of a debris belt (1978)", "doi_or_url": "https://doi.org/10.1029/JA083iA06p02637", "grade": "A"}], "facet": "mechanism", "chapter": "ch2_theoretical_framework", "subclaim": "mechanism"}
{"claim": "Because cost growth and schedule slip are stocks built from their flows, a phase-end stock regressed on latency risks confounding a real rate effect with mechanical integration over a longer elapsed window; the discriminating test is to regress the dated within-phase accrual rate on latency rather than the end-of-phase stock, since if latency predicts the instantaneous accrual rate the effect is real whereas if it predicts only the end stock the result may be an integration artifact of elapsed time.", "evidence": [{"source": "Sterman, Bathtub Dynamics: initial results of a systems thinking inventory (2000), System Dynamics Review", "doi_or_url": "https://doi.org/10.1002/sdr.198", "grade": "A"}, {"source": "Forrester, Industrial Dynamics (a stock is the integral of its net flow; rate-vs-accumulation decomposition)", "doi_or_url": "https://doi.org/10.1002/sdr.284", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "An actor-level (within-program) estimate need not compose into the system-level behavior of interest (cadence collapse, chronic overrun) because the shared authorization queue integrates all programs; the space literature documents the congestion mechanism (coordination burden scales super-linearly with population, an un-priced congestion externality), so an office-by-year queue-congestion test is required to rule out a fallacy of composition.", "evidence": [{"source": "Colombo, Martinez, Letizia et al., Space capacity management and its interaction with space traffic management, Acta Astronautica (2025)", "doi_or_url": "https://doi.org/10.1016/j.actaastro.2025.01.069", "grade": "A"}, {"source": "Weinzierl, Space, the Final Economic Frontier, Journal of Economic Perspectives (2018)", "doi_or_url": "https://www.aeaweb.org/articles?id=10.1257/jep.32.2.173", "grade": "B"}], "facet": "empirics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "The dissertation is design-stage and reports no executed estimates; the two-resolution latency design (coarse milestone-to-milestone full-span, fine key-decision-point modern-only) is described as a falsification test, but the overlap-window correlation, the coarse-minus-fine level gap in months, and any drift test are absent, so comparability is asserted via era fixed effects and a codebook mapping rather than demonstrated by a stable level offset; the constant-comparable-units standard the candidate adopts is the Maddison / Penn World Table tradition.", "evidence": [{"source": "PHD-08 dissertation.md Sec 1.7, 1.8, 4.3.1-4.3.2, Ch6 (no executed estimates)", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Feenstra, Inklaar & Timmer, The Next Generation of the Penn World Table, American Economic Review (2015)", "doi_or_url": "https://doi.org/10.1257/aer.20130954", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The dissertation concedes the documentary baseline definition drifted across eras (an informal 1965 target and a 70% joint-confidence-level 2015 commitment are not the same object) and names it Limitation 1, proposing era fixed effects plus a reserves-to-baseline conservatism check, but reports no original-vs-reset baseline distribution for the re-baseline subset and fixes no by-era baseline-selection rule; because baseline is the shared denominator of cost growth and schedule slip and harder, longer-latency phases are more likely to be re-baselined, an era fixed effect cannot net out a distortion correlated with the regressor.", "evidence": [{"source": "PHD-08 dissertation.md Sec 4.6 (changing documentary baseline), Sec 4.7 Limitations 1 & 3, Sec 1.8 (denominators), Ch6 (no executed estimates)", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Feenstra, Inklaar & Timmer, The Next Generation of the Penn World Table, American Economic Review (2015)", "doi_or_url": "https://doi.org/10.1257/aer.20130954", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Maddison's periodization standard requires phases be defined by documented breaks in a measured series rather than by political/reform-calendar dates; drawing era cuts at administrative-reform dates makes the era dummies co-determined with the authorization-regime variation, so the fixed effects can absorb the treatment, and the result must be shown invariant across at least two defensible periodizations because apparent long-horizon trend breaks frequently revert.", "evidence": [{"source": "Bolt & van Zanden (2017), The Maddison Project: Historical GDP Estimates Worldwide, Journal of World-Historical Information / Maddison Project Database", "doi_or_url": "https://doi.org/10.5195/jwhi.2017.46", "grade": "C"}, {"source": "Maddison, A. (1982), Phases of Capitalist Development, Oxford University Press (ISBN 9780198284505) -- the source in which Maddison defined phase boundaries by measured breaks in growth and productivity rates, not by political events", "doi_or_url": "https://www.worldcat.org/isbn/9780198284505", "grade": "C"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Maddison's growth-accounting discipline requires splitting deflated cost growth into an accumulation (standing-cost) component close to an identity with elapsed time and a total-factor-productivity residual carrying the substantive claim that faster decisions are a real lever; the space-economy value literature locates the productivity (value-added-per-input) question rather than a revenue scalar, confirming the substantive claim lives in the residual.", "evidence": [{"source": "Paravano, Rosseau, Locatelli, Weinzierl & Trucco (2024), Toward the LEO economy: A value assessment of commercial space stations, Acta Astronautica 228", "doi_or_url": "https://doi.org/10.1016/j.actaastro.2024.11.060", "grade": "A"}, {"source": "Maddison Project Database 2020 (Bolt, J. & van Zanden, J. L.) -- the living continuation of Maddison's growth-accounting framework covering GDP, capital, and productivity by country from antiquity to 2018, embodying the factor-decomposition discipline Maddison developed in Dynamic Forces in Capitalist Development (Oxford 1991)", "doi_or_url": "https://www.rug.nl/ggdc/historicaldevelopment/maddison/releases/maddison-project-database-2020", "grade": "C"}], "facet": "mechanism", "chapter": "ch2_theoretical_framework", "subclaim": "mechanism"}
{"claim": "Maddison's rule for an unbalanced panel that is dense for the modern segment and sparse for 1958-1980 is the conjectural-but-explicit estimate: report per-decade observation counts and the point-identified-versus-bounded share with explicit uncertainty bands, and re-estimate on the pre-1980 segment alone, so honesty about how little is measured in the thin window replaces a false-precision long-run claim; the space-weather economic-cost review models stating explicitly that a rigorous assessment is in its infancy.", "evidence": [{"source": "Eastwood, Biffis, Hapgood, Green, Bisi et al. (2017), The Economic Impact of Space Weather: Where Do We Stand?, Risk Analysis 37(2):206-218", "doi_or_url": "https://doi.org/10.1111/risa.12765", "grade": "A"}, {"source": "Maddison, A. (2007), Contours of the World Economy 1-2030 AD: Essays in Macro-Economic History, Oxford University Press (ISBN 9780199227211) -- the work in which Maddison applied the conjectural-but-explicit estimate standard to reconstruct GDP from 1 AD to 2030 with documented uncertainty", "doi_or_url": "https://www.worldcat.org/isbn/9780199227211", "grade": "C"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Realized strategy is the joint product of deliberate and emergent streams and patterns can be realized in the absence of, or despite, prior apex intention, so a recorded authorization event may ratify a commitment that crystallized earlier rather than constitute the decision itself; the construct-validity threat is real and a measured commitment-to-authorization divergence distribution is required before the recorded authorization can be treated as the decision.", "evidence": [{"source": "Mintzberg & Waters, Of strategies, deliberate and emergent, Strategic Management Journal (1985)", "doi_or_url": "https://doi.org/10.1002/smj.4250060306", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Structured observation of practicing executives shows managerial work is fragmented, fast-paced, interruption-driven, and verbally mediated, which is precisely the decision activity that leaves no dated artifact, and NASA's standardized review machinery is documented to record compliance rather than learning, so two programs with identical documentary entries can have radically different real deliberation and the latency variable's claim to index decision speed rather than records-keeping density requires an independent process-derived validation.", "evidence": [{"source": "Mintzberg, Managerial Work: Analysis from Observation, Management Science (1971)", "doi_or_url": "https://doi.org/10.1287/mnsc.18.2.b97", "grade": "A"}, {"source": "Dillon, Tinsley & Rogers, Near-Miss Evaluation Bias as an Obstacle to Organizational Learning: Lessons from NASA, NASA NTRS (2006)", "doi_or_url": "https://ntrs.nasa.gov/citations/20060047554", "grade": "A"}], "facet": "mechanism", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Mintzberg's five configurations are each coordinated by a distinct dominant mechanism, and NASA is a hybrid (machine-bureaucratic procurement/safety/review grafted onto a professional-bureaucratic engineering core with adhocratic mission teams); because review-time means conformance under standardization but obstructs mutual adjustment under adhocracy, a single pooled latency slope can average effects of opposite sign and opposite economic meaning, so it is not interpretable until stratified by the dominant configuration of each program-phase.", "evidence": [{"source": "Mintzberg, Structure in Fives: Designing Effective Organizations (1983)", "doi_or_url": "https://doi.org/10.2307/2393181", "grade": "A"}, {"source": "Madsen & Dillon, Near-Miss Evaluation Bias as an Obstacle to Organizational Learning: Lessons from NASA (NTRS 20060047554, 2006)", "doi_or_url": "https://ntrs.nasa.gov/citations/20060047554", "grade": "A"}], "facet": "mechanism", "chapter": "ch2_theoretical_framework", "subclaim": "mechanism"}
{"claim": "Managerial-work realism predicts that documentary trigger and authorization timestamps are ratification artifacts bracketing a process that substantively opened and closed earlier, and because standardized review machinery records ceremony rather than substance, the ratification lag plausibly scales with administrative ceremony which itself scales with phase difficulty, biasing beta; validating the clock requires a hand-coded oral-history/correspondence sample the dissertation does not present.", "evidence": [{"source": "Mintzberg, Managerial Work: Analysis from Observation (Management Science, 1971)", "doi_or_url": "https://doi.org/10.1287/mnsc.18.2.b97", "grade": "A"}, {"source": "Madsen & Dillon, Near-Miss Evaluation Bias as an Obstacle to Organizational Learning: Lessons from NASA (NTRS 20060047554, 2006)", "doi_or_url": "https://ntrs.nasa.gov/citations/20060047554", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "Era fixed effects that absorb reorganizations soak up precisely the regime variation that would discriminate cause from symptom: configuration mismatch (forcing a machine-bureaucratic logic onto adhocratic novel work) makes long latency and high cost co-products of one ill-fitting structure rather than latency causing cost, and Marshall Space Flight Center history documents recurring shifts between centralized program control and decentralized engineering expertise that are exactly the regime variation era-FE discards, so the era-FE strategy is conservative against the causal claim.", "evidence": [{"source": "Mintzberg, Structure in Fives (configuration mismatch: running an innovative undertaking as a machine bureaucracy)", "doi_or_url": "https://doi.org/10.2307/2393181", "grade": "A"}, {"source": "Dunar & Waring, Power to Explore: A History of the Marshall Space Flight Center 1960-1990 (NASA SP-4313, NTRS 20000031366, 1999)", "doi_or_url": "https://ntrs.nasa.gov/citations/20000031366", "grade": "B"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "Institutions (the rules of the game) must be distinguished from organizations (the players), and a fixed-effects design that absorbs the rule regime into era effects and the standing organization into program effects risks differencing out exactly the institutional variation it claims to measure, leaving residual variation that may be organizational or idiosyncratic; the coefficient cannot speak to the institutional mechanism unless residual latency is shown to track a coded register of documented rule changes.", "evidence": [{"source": "North, D. C. (1990), Institutions, Institutional Change and Economic Performance", "doi_or_url": "https://doi.org/10.1017/CBO9780511808678", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "mechanism"}
{"claim": "Formal labels and operative rules routinely come apart, so a documentary (de jure) latency measure cannot be assumed to track the de facto rule-in-use without validation: across 1042 space arrangements invoking global-commons language does not produce significantly different operational rules, and a formally polycentric architecture fails to promote the norms it nominally encodes, both instances of the de jure / de facto gap North's method is built to expose.", "evidence": [{"source": "Pic, Evoy & Morin (2023), Outer Space as a Global Commons: An Empirical Study of Space Arrangements, International Journal of the Commons 17", "doi_or_url": "https://doi.org/10.5334/ijc.1271", "grade": "A"}, {"source": "Morin & Couette (2025), The Missing Ingredients for a Polycentric Governance System of Orbital Debris, Global Environmental Politics 25", "doi_or_url": "https://doi.org/10.1162/glep_a_00775", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The rules of the game determine which actions are routed to an authorizing office, so an instrument drawn from office workload is endogenous to the very rule regime under test rather than an exogenous shock, and the appropriations cycle is itself a path-dependent institution that can move cost and schedule through channels independent of latency; a causal claim therefore requires identifying the transaction-cost mechanism and an empirical exclusion test (zero partial association of workload with outcomes conditional on latency) before it is licensed.", "evidence": [{"source": "North & Weingast (1989), Constitutions and Commitment, Journal of Economic History 49(4):803-832", "doi_or_url": "https://doi.org/10.1017/S0022050700009451", "grade": "A"}, {"source": "North, D. C. (1990), Institutions, Institutional Change and Economic Performance (path dependence / increasing returns)", "doi_or_url": "https://doi.org/10.1017/CBO9780511808678", "grade": "A"}], "facet": "rival", "chapter": "ch5_research_design", "subclaim": "alternatives"}
{"claim": "Transaction cost is a property of the rules of the game rather than a portable constant, and institutional change is incremental and path-dependent, so a within-NASA latency-to-overrun estimate is bounded by NASA's enabling rules and is not entitled to portable-law status without a stated scope condition that names the enabling rule and bounds the inference.", "evidence": [{"source": "North, D. C. (1990), Institutions, Institutional Change and Economic Performance, Cambridge University Press", "doi_or_url": "https://doi.org/10.1017/CBO9780511808678", "grade": "A"}], "facet": "governance", "chapter": "ch7_discussion", "subclaim": "residual_risk"}
{"claim": "The dissertation defines eras at institutional-regime (multi-decade) granularity and concedes the appropriations-calendar instrument and the funding-instability index are both built from the same NASA requested-versus-appropriated budget record, defending exclusion by saying the funding-instability control absorbs the direct-funding channel; on a causal graph, conditioning on a control built from the same fiscal source the instrument exploits does not establish instrument independence, and the design-stage document reports no executed within-era independence test, so the fiscal-regime back-door is conceded but unresolved.", "evidence": [{"source": "PHD-08 dissertation.md, Sec 1.3 periodization, Sec 4.5 / 5.4 instrument validity", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Pearl, Causality: Models, Reasoning and Inference (2nd ed., 2009) (back-door / d-separation criterion)", "doi_or_url": "https://doi.org/10.1017/cbo9780511803161", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "The dissertation predicts the schedule-slip coefficient largest and most robust because authorization latency is itself a component of elapsed schedule, and operationalizes latency from the same lifecycle-review milestones (KDPs, PCRs, confirmation reviews, NPR 7120.5, SRB) that bound the phase intervals from which schedule slip is computed, but supplies no set-theoretic decomposition proving the authorization sub-interval is disjoint from the engineering-execution sub-interval; by the candidate's own component-of-elapsed-schedule language the latency interval is a sub-interval of phase time, so on the current operationalization the schedule-slip arrow risks being an identity rather than an estimate.", "evidence": [{"source": "PHD-08 dissertation.md Sec 5.3 mechanism statement, Sec 4.3 measurement, Sec 4.4 outcomes", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Majerowicz & Shinn, schedule-delay vs cost-overrun analysis (standing-cost channel; NASA instrument/mission schedule-growth literature)", "doi_or_url": "https://doi.org/10.1109/aero.2014.6836219", "grade": "B"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "The candidate names the reverse-causation threat and answers it by measuring latency early in each phase plus the instruments, but this defends only against trouble that emerges after phase entry, not Pearl's sharper variant of trouble foreseen at phase entry (immature TRL, contested baseline) which the candidate's own control vector flags; because program fixed effects are program-level and time-invariant they cannot absorb a phase-varying anticipated-difficulty signal, and the dissertation exhibits no dated event trail establishing the latency-trigger precedes the first risk signal, so the temporal-precedence assumption is asserted not demonstrated.", "evidence": [{"source": "PHD-08 dissertation.md Sec 5.5.1 internal validity, Sec 4.5 controls, Sec 1.5 / 5.3 identification", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Pearl, Causality: Models, Reasoning and Inference (2nd ed., 2009) (d-separation / temporal precedence)", "doi_or_url": "https://doi.org/10.1017/cbo9780511803161", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "The candidate's mechanism is an explicit mediation (latency -> standing-cost accrual + requirements drift, and latency -> schedule slip -> cost), but Pearl's front-door criterion fails here because standing-cost accrual, requirements churn, and workforce idle time are themselves driven by technical difficulty, so difficulty arrows directly into the mediator and is an unblocked common cause of mediator and outcome; with the back-door blocked only by the contested IV, both identification doors are closed on the current graph, and the candidate never drew the mediated DAG or attempted front-door identification.", "evidence": [{"source": "PHD-08 dissertation.md Ch3 mechanism statement & Sec 5/Ch8", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Pearl, Causality: Models, Reasoning and Inference (2nd ed., 2009) (front-door criterion)", "doi_or_url": "https://doi.org/10.1017/cbo9780511803161", "grade": "A"}, {"source": "Bellemare, Bloem & Wexler, The Paper of How: Estimating Treatment Effects Using the Front-Door Criterion, Oxford Bulletin of Economics and Statistics (2024)", "doi_or_url": "https://doi.org/10.1111/obes.12598", "grade": "B"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "The candidate builds latency at two resolutions because documentary density is era-correlated (early decades sparser and coarser, baselines defined less formally), which is a measurement error co-determined with looser baselines and larger measured overruns, i.e. non-classical outcome-correlated error rather than mean-zero classical noise; because the coarse error and the outcome are both functions of era documentary density, conditioning on era fixed effects is conditioning on a node downstream of both the mismeasured latency and the outcome (collider-adjacent) and does not purge it, and the candidate treats the two resolutions only as a robustness check, never as an errors-in-variables problem.", "evidence": [{"source": "PHD-08 dissertation.md Ch3/Ch4 data construction & Sec 5.9 robustness battery", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Pearl, Causality: Models, Reasoning and Inference (2nd ed., 2009) (non-classical error and collider/selection bias under adjustment)", "doi_or_url": "https://doi.org/10.1017/cbo9780511803161", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "A DAG's exclusion restriction implies a testable vanishing partial correlation between instrument and outcome residual given treatment and controls (an overidentification-style independence test when two instruments serve one endogenous regressor), but the candidate's pre-registered validity battery (two resolutions, cadence definitions, FE structures, heterogeneity-robust estimators, Goodman-Bacon, wild-bootstrap, first-stage strength, imperfect-instrument bounds) omits it; the IV defense rests on first-stage F (relevance, not exclusion) and on sensitivity bounds that weaken under assumed violation rather than test-and-falsify the graph, so the causal claim rests on an exclusion restriction that is bounded-against but not falsification-tested.", "evidence": [{"source": "PHD-08 dissertation.md Sec 5.9 pre-registration & validity battery table", "doi_or_url": "file:///D:/Claude_Code/brain/collegium/candidates/dissertations/PHD-08/dissertation.md", "grade": "C"}, {"source": "Pearl, Causality: Models, Reasoning and Inference (2nd ed., 2009) (testable d-separation implications of a DAG)", "doi_or_url": "https://doi.org/10.1017/cbo9780511803161", "grade": "A"}], "facet": "empirics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "Megaproject cost overruns are empirically fat-tailed and dominated by a handful of catastrophic realizations, the same Extremistan regime in which sample mean and variance are dominated by a few extremes and the historical record undersamples the tail; tail-index estimation via the Hill estimator on upper order statistics is the established pre-estimation diagnostic, and if the tail exponent alpha is at or below 2 the second moment does not exist, so program-clustered Gaussian intervals are computed around a moment that is not defined and the mean-regression apparatus must be replaced rather than reported.", "evidence": [{"source": "Flyvbjerg, Five things you should know about cost overrun, Transportation Research Part A (2018)", "doi_or_url": "https://doi.org/10.1016/j.tra.2018.07.013", "grade": "A"}, {"source": "Flyvbjerg, Big is Fragile: An Attempt at Theorizing Scale (2016)", "doi_or_url": "https://doi.org/10.48550/arxiv.1603.01416", "grade": "B"}, {"source": "Beirlant et al., Tail Index Estimation and an Exponential Regression Model, Extremes (1999)", "doi_or_url": "https://doi.org/10.1023/a:1009975020370", "grade": "A"}], "facet": "measurement", "chapter": "ch4_data_and_measurement", "subclaim": "residual_risk"}
{"claim": "A leave-extreme-out jackknife on the headline latency coefficient is the correct falsification design and should be pre-registered with an ex-ante instability threshold, because under fat tails the aggregate is dominated by a few realizations so a coefficient stable only because the tail programs are retained is reporting the influence of those few points, not a population effect; refusing to pre-register converts a falsification test into a post-hoc robustness narrative.", "evidence": [{"source": "Flyvbjerg, Big is Fragile: An Attempt at Theorizing Scale (2016)", "doi_or_url": "https://doi.org/10.48550/arxiv.1603.01416", "grade": "B"}], "facet": "empirics", "chapter": "ch6_analysis_plan", "subclaim": "residual_risk"}
{"claim": "For a ruin-bearing fat-tailed outcome the conditional-mean estimand is mis-targeted and must be re-specified on the tail, because the decisive distinction is between survivable repeated risks and absorbing ruin (a self-reinforcing Kessler cascade is effectively irreversible on operational timescales); quantile regression at the 90th/95th percentile or a tail-index regression directly estimates whether latency moves the upper tail differently from the center, so a null mean effect with a positive tail effect is a rejection of H0, not a confirmation, and the mean-based decision rule must be re-anchored on the tail estimand.", "evidence": [{"source": "Lewis, Understanding long-term orbital debris population dynamics, J. Space Safety Engineering (2020)", "doi_or_url": "https://doi.org/10.1016/j.jsse.2020.06.006", "grade": "A"}, {"source": "Koenker & Hallock, Quantile Regression, Journal of Economic Perspectives (2001)", "doi_or_url": "https://doi.org/10.1257/jep.15.4.143", "grade": "A"}], "facet": "identification", "chapter": "ch6_analysis_plan", "subclaim": "alternatives"}
{"claim": "The policy claim compress latency to reduce overrun is, as specified, a within-survivor association silent on the worst latency outcome: cancellation is an absorbing barrier where latency's cost is effectively infinite, and because all three dependent variables are defined only against a baseline that ruined programs never set, the ruin outcome is censored out of every regression and the survivor record undersamples the extreme realizations that dominate the process, so the contribution must be restated with an explicit scope restriction to surviving programs or re-specified with a ruin-inclusive outcome (cancellation hazard) before any causal policy claim is licensed.", "evidence": [{"source": "Taleb et al., The Precautionary Principle (with Application to the Genetic Modification of Organisms) (ruin vs repeated-risk distinction; absorbing-barrier exposure)", "doi_or_url": "https://doi.org/10.48550/arxiv.1410.5787", "grade": "A"}], "facet": "rival", "chapter": "ch7_discussion", "subclaim": "alternatives"}
{"claim": "The bureaucratic-politics lens predicts pooled latency is not one variable because decisions are political resultants of bargaining among players in distinct positions, so a single program/era fixed effect cannot separate a beta that is a weighted mix of seats with opposite mechanisms; partitioning each authorization interval by the owning seat and reporting a within-seat versus between-seat beta is the required Model III decomposition the candidate has not supplied.", "evidence": [{"source": "Jones, C.M., Bureaucratic Politics and Organizational Process Models, Oxford Research Encyclopedia", "doi_or_url": "https://doi.org/10.1093/acrefore/9780190846626.013.2", "grade": "A"}], "facet": "rival", "chapter": "ch7_discussion", "subclaim": "alternatives"}
{"claim": "When an authorizing office is congested the clearing order is a bargaining resultant by political standing, so workload can reach outcomes through coalition standing (queue priority) rather than solely through imposed latency, making a winner-versus-loser heterogeneity test (does workload predict cost/schedule differently for portfolio winners than for deprioritized losers) a legitimate identification challenge to the workload instrument's exclusion restriction.", "evidence": [{"source": "Jones, C.M., Bureaucratic Politics and Organizational Process Models, Oxford Research Encyclopedia (Model III pull-and-haul, relative power, deadlines, deals)", "doi_or_url": "https://doi.org/10.1093/acrefore/9780190846626.013.2", "grade": "A"}], "facet": "identification", "chapter": "ch5_research_design", "subclaim": "residual_risk"}
{"claim": "A reorganization is a Model III intervention that changes who must say yes, so a re-seating event study (clearance layer eliminated, authority delegated downward, review office stood up or disbanded) is the most direct natural experiment available to test whether latency is a causal lever moving with the seat or a passive correlate of administrative regime; era fixed effects that absorb reorganizations discard exactly this identifying variation.", "evidence": [{"source": "Jones, C.M., Bureaucratic Politics and Organizational Process Models, Oxford Research Encyclopedia (Model III reorganizations re-seat players / abolish clearances)", "doi_or_url": "https://doi.org/10.1093/acrefore/9780190846626.013.2", "grade": "A"}], "facet": "governance", "chapter": "ch7_discussion", "subclaim": "residual_risk"}