Measuring the Attribution Gap[cite: 1]

A staged empirical protocol – companion to The Attribution Gap (v8)[cite: 1]
Working paper – measurement companion, v2.[cite: 1]

Firm-level claims only; the firewall of the theory paper is held (no reliance on, and no connection to, any monetary or early-warning line of work).[cite: 1]

0. Purpose and scope[cite: 1]

This document specifies how to test the firm-level claims of The Attribution Gap.[cite: 1] It shows that contribution, dependency, and attribution can be measured from sources that do not contaminate each other; it reduces the theory’s predictions to observable signatures in records firms already hold; and it sequences the work as a staged ladder, cheap-first, so each expensive stage is justified by a cheaper one.[cite: 1] It does not price the absolute value of any capability, and it does not claim that one study proves the theory.[cite: 1] It is a programme: recognition, then pattern, then cause.[cite: 1]

1. Three claims, with different burdens[cite: 1]

The theory contains three distinct empirical claims.[cite: 1] They should be separated, because they cost very different amounts and the cheapest already carries the surprise.[cite: 1]

• Claim A – the dependency-attribution divergence. Firms increasingly fail to acknowledge capabilities they remain structurally dependent on, and shadow, under-invest in, or cut them.[cite: 1] Requires only two independently measured quantities – technical dependency and declared attribution – tracked over time, plus the can’t-kill-but-unlisted contradiction.[cite: 1] No value estimate, no circularity.[cite: 1] Surprising on its own (the efficient-firm view predicts the least depended-on are dropped).[cite: 1] This is the phenomenon.[cite: 1]
• Claim B – the destroyed capability was a true diamond, not lock-in. A highly central application may be central because it is genuinely valuable, or because it is a bottleneck, locked-in, or central only through poor architecture.[cite: 1] Establishing value requires the contribution estimator (§4) behind the Study-0 reliability gate; this is the only place lineage ≠ value bites.[cite: 1] This is the object.[cite: 1]
• Claim C – the divergence is incentive-driven, not neglect-driven. Claim A is consistent with two causes: entropy passively lost over time (decay by neglect), or entropy actively maintained because naming the dependency threatens captured rent (decay by incentive).[cite: 1] Distinguishing them is the paper’s central novelty.[cite: 1] The rival «decay by neglect» is therefore not merely a confound to control (§7) – it is Claim C’s null hypothesis.[cite: 1] This is the mechanism, and the real target.[cite: 1]

The full theory is AABAC.[cite: 1] The programme leads with A (cheap, clean, surprising), treats C as the claim that carries the novelty, and treats B as the value upgrade.[cite: 1]

The contrast class for Claim C. The mechanism does not predict that obscuring always wins.[cite: 1] Sometimes an owner does step forward – «I will own the important thing; I will put my budget and my name on it.»[cite: 1] That stewardship case is not a counterexample; it is the contrast class that makes C falsifiable.[cite: 1] C predicts that the probability obscuring wins rises with the contribution deficit and the blame-exposure of the capability: high-deficit capabilities are obscured; low-deficit ones can find a willing owner.[cite: 1] The variation between obscuring-predominant and stewardship-predominant cases — and whether it tracks the deficit — is itself the test.[cite: 1]

2. The unit of analysis: the application-capability pair[cite: 1]

The theory concerns capabilities, the traceable technical unit is the application.[cite: 1] They are related but not identical: one application may serve several capabilities; one capability may span several applications; an application may be replaced while its capability survives; a capability may be lost while some application remains.[cite: 1] The operational unit is therefore the application-capability pair a capability and the application(s) performing its information transformations – coded through an explicit mapping:[cite: 1]

application → information transformation ↔ business capability[cite: 1]

Outcomes (§6) are coded at the pair level.[cite: 1] Where the mapping is one-to-one, application and capability coincide; where it is not, the pair preserves technical traceability without pretending the two are the same thing, and the mapping is reported alongside the result.[cite: 1]

3. The anti-circularity design, and the two reliability gates[cite: 1]

Two independent sources. The two quantities being compared come from sources that cannot contaminate each other.[cite: 1]

Quantity	What it is	Source (independent of the other)
Technical dependency[cite: 1]	what transforms what; the upstream source of a piece of information; fan-out[cite: 1]	logs, lineage, integration/interface maps, the architecture model machine-read, indifferent to what anyone declared[cite: 1]
Declared attribution[cite: 1]	which applications stakeholders name as important; criticality ratings; budget, headcount, ownership allocated[cite: 1]	elicitations, inventories, budget and resourcing records, RACIs – the human and political layer[cite: 1]

Because the first is read from the technical estate and the second from human declarations, the gap between them cannot be an artifact of estimating one from the other.[cite: 1] This protects against dependency-attribution coupling.[cite: 1] It does not, by itself, protect a second coupling, which the design must also gate:[cite: 1]

• Gate 1 – Study-0 (contribution reliability), for Claim B. Before any contribution number is used, two or more assessors score the same capabilities from the same evidence and must reach acceptable agreement (§4).[cite: 1] Below threshold, contribution is not yet a measurement and only Claims A and C proceed.[cite: 1]
• Gate 2 – the dependency-tracing reliability gate, for the shadow set. For the visible estate, dependency is machine-read and clean.[cite: 1] For the shadow set that carries the discovery-gap prediction, dependency is reconstructed by tracing – effortful, partial, survivorship-prone.[cite: 1] So for shadow capabilities, «independent machine-read dependency» is in fact a reconstruction, and if the same analyst both traces dependency and estimates contribution, the two-source independence does not protect against dependency-contribution coupling.[cite: 1] Two safeguards, symmetric to Study-0: (i) role separation – the analyst who traces dependency for a shadow capability is not the one who estimates its contribution; (ii) a tracing reliability gate – independent analysts reconstruct the dependency of the same shadow capabilities from the same evidence and must reach acceptable agreement before those reconstructions are used.[cite: 1]

4. The contribution estimator and its scoring protocol (Claim B)[cite: 1]

Contribution is estimated from consequence-variables, each anchored in the technical estate or in operational record, and each independent of attribution (never drawn from budget, headcount, or recognized credit): process criticality; unique transformations performed; substitution availability; replacement cost and time; failure consequence (revenue, service, safety, compliance); tacit-knowledge concentration.[cite: 1]

To make this a measurement rather than an opinion, the protocol fixes:[cite: 1]

• Scales – each variable scored on an anchored ordinal scale (e.g., 0-4), with a written rubric per level.[cite: 1]
• Evidence requirement – each score carries a scoring note citing its lineage/log/operational source; an unevidenced variable is recorded as missing, not zero.[cite: 1]
• Aggregation – unweighted sum or mean first, to avoid premature weighting; report sensitivity to alternative weights.[cite: 1]
• Reliability statistic – ICC for the continuous aggregate, or Krippendorff’s a for the ordinal items.[cite: 1]
• Threshold – pre-declared (e.g., a ≥ 0.67 tentative, ≥ 0.80 strong).[cite: 1] Below it, Claim B does not proceed.[cite: 1]

5. The observable signatures[cite: 1]

Five independently sourced measurements of the same gap, at different points in the lifecycle.[cite: 1] Each notes which claim it serves.[cite: 1]

• Signature 1 – the lists (divergence over time). (Claim A.)[cite: 1] Technical-dependency centrality (lineage) versus declared attribution (importance rating, budget), at two or more time points.[cite: 1] Expected: the most dependency-central capabilities are attributed less and less, appearing centrally early, then marginally, then absent while remaining load-bearing.[cite: 1] The cross-sectional form is the can’t-kill-but-unlisted contradiction (absent from the asserted list, yet stakeholders refuse decommissioning: «it is the source of X»).[cite: 1] The longitudinal widening is the strongest and hardest line (needs the measurement at two times).[cite: 1] Confounders: age, scope, benign undocumentation, inventory reorganization.[cite: 1]
• Signature 2 – the RACIs (responsibility claimed vs. accountability accepted). (Claim A; supports C.)[cite: 1] For the depended-on-but-under-attributed set, the gap between how many parties claim responsibility and how many accept accountability.[cite: 1] Construction: let p_k be the proportion of respondents (or of accountability statements in the record) naming candidate owner k; report H = -Σ_k p_k log p_k when the distribution is available.[cite: 1] A simpler first-pass measure, used before the full distribution exists, is E = (responsibility claimants) + (accepted accountable owners + ε) – high when many claim responsibility and few accept accountability.[cite: 1] Expected: the most important capabilities are widely claimed-as-responsible and widely refused-as-accountable – the attribution gap seen in the ownership ledger.[cite: 1] Confounders: legitimately shared ownership; matrixed structures.[cite: 1]
• Signature 3 – the decisions (the business-case investment gap). (Claim A; outcomes.)[cite: 1] At a rationalization or modernization decision, the budget/ownership/accountability committed in the signed business case versus the structural dependency.[cite: 1] Expected: the wider the gap, the less investment is committed despite a structural dependency – diamonds cut or left to decay, zombies kept alive.[cite: 1] The build-vs-buy and internalization wedges and the (probabilistic) ownership-silence outcome live here.[cite: 1] Confounders: real redundancy; genuine end-of-life; strategic exit; in acquisitions, the capability-attributable portion only (exclude goodwill, synergy, control premium, growth options).[cite: 1]
• Signature 4 – raised-and-ignored («we said it fifty times»). (Claim C – the primary discriminator.)[cite: 1] The count of times a dependency was explicitly surfaced in emails, minutes, documentation, escalations, calls versus the count of times it was subsequently not acted on, over a defined window.[cite: 1] Expected: critical dependencies are surfaced repeatedly and systematically not acted on.[cite: 1] Why it discriminates: neglect predicts low surfacing (genuinely unknown); incentive predicts high surfacing with persistent non-action (known, repeatedly, and obscured or ignored anyway).[cite: 1] You cannot attribute to «lost over time» what was raised, documented, and explained dozens of times and still dropped.[cite: 1] This is the cleanest single test of C.[cite: 1] Confounders: legitimate deprioritization for resource reasons (control by holding budget rank constant); record availability.[cite: 1]
• Signature 5 – accountability-avoidance (volunteering, budget and credit held constant). (Supports C and the silence outcome.)[cite: 1] Holding budget and credit constant, the rate of volunteering for / attendance at ownership of a high-dependency task (e.g., mission-critical in production, fifty downstream dependents, high blame-exposure) versus a low-dependency task (e.g., a greenfield pilot, no dependents, high potential credit).[cite: 1] Expected: fewer take the high-dependency task despite equal budget and credit.[cite: 1] Why the design is clean: holding budget and credit constant isolates accountability-avoidance from mere underfunding, and turns the ownership-silence scene into a rate measurable across many meetings rather than a single anecdote.[cite: 1] Confounders: skill/fit differences (control by task-class matching); seniority.[cite: 1]

6. Outcome definitions and time windows[cite: 1]

The destructive outcomes are coded separately, at the application-capability-pair level, within pre-declared windows (12/24/36 months after the ownership decision).[cite: 1] Without a window, almost any eventual event could be read as confirming the theory.[cite: 1]

• Non-investment – requested investment denied or absent within the window.[cite: 1]
• Shadowing – continues operating without recognized budget, named ownership, or roadmap status.[cite: 1]
• Elimination – formally removed or replaced.[cite: 1]
• Scar formation – fragmentation or bridging introduced after an attempted cut (the «one became five»).[cite: 1]

7. The decay-by-neglect rival, and the incremental-variance test (Claim C)[cite: 1]

The chief rival to «decay by incentive» is «decay by neglect»: the divergence is just stale documentation, not actively maintained.[cite: 1] This is Claim C’s null hypothesis, and it must be measured directly, not waved away.[cite: 1] Neglect variables: documentation age; architecture-team capacity; update frequency; staff turnover; technical debt; CMDB/inventory quality; recent M&A; reorganization.[cite: 1]

The test: whether attribution distortion and the responsibility-accountability asymmetry explain additional variance in the divergence and in the destructive outcomes beyond the neglect variables.[cite: 1] If they do not, the discovery gap is stale documentation and Claim C fails.[cite: 1] Signature 4 is the sharpest single discriminator, because high surfacing with persistent non-action is exactly what neglect cannot produce.[cite: 1]

8. Data sources and provenance[cite: 1]

All already exist; the constraint is willingness to share, not absence.[cite: 1] Technical dependency – logs, lineage tooling, CMDB/integration maps, architecture models.[cite: 1] Declared attribution – historical inventories, criticality surveys, budget/headcount records (often multi-year).[cite: 1] Responsibility vs. accountability – RACIs and ownership registers.[cite: 1] Decisions – business cases, investment-committee minutes, rationalization roadmaps, postmortems.[cite: 1] Surfacings – email and meeting records, escalation logs.[cite: 1] The retrospective stages are feasible because the same underlying quantities were recorded at the time, for other reasons.[cite: 1]

9. The staged ladder[cite: 1]

Cheap-first; the epistemic ceiling of each stage is stated.[cite: 1]

• Stage 0 – the gates. Study-0 (contribution reliability, §4) and the dependency-tracing reliability gate with role separation (§3).[cite: 1] Establishes: that contribution and shadow-dependency are measurements.[cite: 1] Cannot establish: anything about the gap.[cite: 1]
• Stage 1 – blind recognition survey, two arms. A short anonymous instrument asking whether the pattern is recognized.[cite: 1] Arm 1: enterprise architects (the author can reach hundreds).[cite: 1] Arm 2: decision-makers – CIOs, finance, portfolio owners – the people who actually authorize the cuts.[cite: 1] The second arm matters because architects hold a professional identity around «the org kills good systems and nobody owns them,» so architect recognition partly measures a shared worldview, and the theory even casts the architect as the function the mechanism defeats; recognition by the authorizers is far stronger and partly breaks the in-group selection.[cite: 1] Establishes: recognizability and face validity – the justification for the data work.[cite: 1] Cannot establish: truth (shared experience can be shared folklore).[cite: 1]
• Stage 2 – management postmortem interviews. Qualitative interviews with decision-makers on closed cases: was the capability affirmed important; who would own it; why was it cut.[cite: 1] Establishes: the mechanism in their own words («no one would be accountable»).[cite: 1] Cannot establish: incidence or magnitude; subject to hindsight.[cite: 1]
• Stage 3 – retrospective quantitative. Signatures 1-5 on historical records across firms or periods, with the neglect controls and incremental-variance test (§7) and the confounders of §10.[cite: 1] Establishes: that A and C (and, if Stage 0 passed, B) are present, patterned, and associated with shadowing and elimination.[cite: 1] Cannot establish: causation against an unobserved common driver.[cite: 1]
• Stage 4 – prospective, including the decisive comparison. Carry the metrics forward; and compare clarity-only versus revaluation-then-clarity on matched portfolios.[cite: 1] This comparison is decisive but ethically and practically constrained – an organization may refuse to knowingly expose a portfolio to a treatment the theory predicts is destructive.[cite: 1] Realistic alternatives: stepped-wedge rollout; matched portfolios entering treatment at different times; natural experiment across business units; difference-in-differences; interrupted time series; regression discontinuity where a governance threshold determines which capabilities receive revaluation.[cite: 1] Two requirements:[cite: 1]
  1. The revaluation treatment must use the same Study-0-gated estimator (§4).[cite: 1] [F-10] sidesteps using contribution to measure the gap, but it operationalizes contribution as the treatment; if the revaluation method is ungated, the decisive test inherits an ungated input.[cite: 1]
  2. The evaluator must be external to the local incentive equilibrium, operationalized as: owns no competing application; receives no budget from the rationalization outcome; does not report to a claimant organization; has no performance objective tied to the preferred solution; uses pre-declared valuation criteria; cannot personally capture the released budget.[cite: 1]

10. Threats to validity[cite: 1]

• Circularity (dependency vs attribution) – handled by the two-source design (§3); contribution-side evidence never drawn from attribution-side signals.[cite: 1]
• Dependency-contribution coupling and survivorship (shadow set) – handled by role separation and the tracing reliability gate (§3); residual survivorship (lost shadow capabilities under-recorded) is partly remedied by tracing from the visible.[cite: 1]
• Folklore and in-group selection (survey) – Stage 1 measures recognition, not truth, and the decision-maker arm discounts architect-only sympathy.[cite: 1]
• The obsolescence disguise (a falsifiability tension, stated openly) – the theory predicts entropy-driven cuts are recorded as «retiring legacy,» i.e., the records mislabel them as obsolescence; yet the performance-vs-informational test must separate the two using those same records.[cite: 1] The escape is that the responsibility-accountability / ownership-silence signal (Signatures 2 and 4) is orthogonal to the obsolescence disguise and should survive it.[cite: 1] So that test rests, by design, on the orthogonal signal rather than on the cut’s stated reason.[cite: 1]
• Selection – firms that share historical data may differ systematically; report selection and compare sharers and non-sharers on observables.[cite: 1]
• Hindsight – Stage 2 reconstructions are corroborative only.[cite: 1]
• Lineage ≠ value – constrains Claim B only; handled by §4.[cite: 1] Claims A and C are unaffected.[cite: 1]
• The counterfactual – capabilities never built cannot be measured directly; anchored only on the rejected-build-then-buy proxy.[cite: 1]

11. Which stage tests which claim[cite: 1]

• Claim A – Signatures 1-3, Stage 3 (longitudinal Signature 1 = [F-2], the most exposed line).[cite: 1]
• Claim B – Stage 0 gate + the value component of Signature 1 and the discovery-gap enrichment ([F-5]).[cite: 1]
• Claim C – Signature 4 (primary), Signature 5, and the incremental-variance test over neglect variables (§7), plus the stewardship contrast class.[cite: 1]
• Ownership-silence ([F-4]) – Stage 2 (qualitative) and Signatures 2, 5 (quantitative).[cite: 1]
• Discovery gap / scars ([F-5], [F-8]) – Stage 3, with Gate 2 for the shadow set.[cite: 1]
• Internalization / build-vs-buy ([F-6], [F-7]) – Stage 3, Signature 3, capability-attributable portion only.[cite: 1]
• Loss is informational, not performance ([F-9]) – Stage 3 via §7 controls; discrimination rests on Signatures 2 and 4 (§10).[cite: 1]
• The governance experiment ([F-10]) – Stage 4.[cite: 1]

12. Discipline[cite: 1]

The firewall holds: this protocol tests the firm-level theory on its own terms and connects to no monetary or early-warning work; any eventual convergence is a separate study whose evidential value depends on this independence.[cite: 1] Witness is motivation; measurement is proof – the author’s practice supplies face validity (Stage 1); Stages 3-4 are what survive a skeptic.[cite: 1] Lead with the cheap and the clean: Claim A on independent sources, established as low on the ladder as possible, before any value claim or expensive access.[cite: 1] Firm and individual identities are protected; only aggregate, anonymized results are published; access to historical records is each organization’s to grant.[cite: 1]

---

Appendix – Blind recognition survey (instrument sketch, two arms)[cite: 1]

Short, anonymous; each item on a frequency scale (never / rarely / sometimes / often / almost always) with an optional free-text example.[cite: 1] The architect arm and the decision-maker arm share most items, reworded for the respondent’s vantage.[cite: 1]

1. You have found applications not on a portfolio’s list of important systems that stakeholders refused to let you decommission because something critical depended on them.[cite: 1]
2. Over repeated assessments, genuinely load-bearing source systems were rated less important over time, even as dependence on them held or grew.[cite: 1]
3. In a rationalization decision, everyone affirmed a capability mattered, but when asked who would own and be accountable for it, the answer was effectively silence.[cite: 1]
4. A capability everyone depended on was widely claimed as a responsibility but refused as an accountability.[cite: 1]
5. An internal build was rejected on a fully-loaded business case, and a comparable capability was later acquired externally.[cite: 1]
6. Cutting one application produced several loosely-coupled legacy fragments plus an unused replacement («one became five»).[cite: 1]
7. A critical dependency was raised, documented, and explained repeatedly before being ignored. (How many times, in your sharpest example: 1/2-3/4-9/10+/dozens?) – the raised-and-ignored item, the discriminator for incentive-over-neglect.[cite: 1]
8. At equal budget and recognition, you have seen people decline ownership of a mission-critical, high-dependency capability while volunteering for a low-dependency, high-visibility one. – the accountability-avoidance item.[cite: 1]

A high «often / almost always» rate establishes that the pattern is recognized and worth the retrospective study; it does not, by itself, establish that the pattern is true.[cite: 1] Agreement from the decision-maker arm – the people who authorize the cuts – counts for far more than agreement from architects alone.[cite: 1]

End of measurement companion (v2). To instantiate the retrospective stages on specific portfolios, the consequence-variable rubric (§4), the neglect-variable list (§7), and the outcome windows (§6) are fixed first, both reliability gates are cleared (§3), and only then are historical records drawn – anonymized, with the owning organization’s consent.[cite: 1]