This document is a field notes paper: a structured conceptual contribution grounded in direct practitioner observation, prior to formal empirical validation. It extends the framework introduced in Human Intelligence Debt: A Socio-Technical Metric for Measuring the Human Cost of Imperfect Data Flows and developed in The Harvester Multiplication Problem. Readers unfamiliar with the core definitions — Human Intelligence Contribution Ratio (HICR), Human Intelligence Contribution Target (HICT), and Human Intelligence Debt (HI-Debt) — are encouraged to read those papers first. The next stage of this series will be a working paper incorporating measurement methodology, sector-level pilot data and an empirical research protocol.
A Game-Theoretic Reading of Why Rational Agents Consistently Produce Irrational Architectures
Prefatory Note
The first paper in this series established that Human Intelligence Debt accumulates when organisations fail to convert technological potential — measured by the Human Intelligence Contribution Target (HICT) — into actual improvements in the proportion of operators performing genuinely human roles (HICR). The second paper identified the structural mechanism: capability fragmentation, in which incompatible systems generate coordination overhead that absorbs human cognitive capacity without producing genuine information.
This paper addresses the question that both predecessors left open: why does fragmentation persist and compound despite the evident costs? The answer is not that organisations are badly managed, or that the people involved lack competence or good intentions. The answer is that the incentive structure of enterprise architecture creates a game in which the individually rational move for every player — at every level, with every motivation — is the move that produces more systems, more dependencies, and higher Human Intelligence Debt.
This paper examines that game from two directions: top-down, through the structural logic of rationalization initiatives, and bottom-up, through the behavioral dynamics of individual agents operating within fragmented environments.
Part 1 — The Rationalization Paradox
How Simplification Initiatives Structurally Produce Complexity
One of the most persistent puzzles in enterprise architecture is this: organisations regularly launch rationalization initiatives — with genuine mandate, real budget and capable people — and end the process with more systems than they started with. Not because the initiative failed in the conventional sense. But because the structural logic of top-down rationalization produces this outcome almost regardless of intent or competence.
Understanding why requires looking at two distinct cases: the rationalization led by someone who does not fully understand the system, and the rationalization led by someone who does.
Case A: The Incompetent Rationalizer
The rationalizer who does not deeply understand the operational reality of the organisation will prune by visible cost rather than by functional understanding. They will identify systems that appear redundant, that have low adoption metrics, that generate complaints, or that are expensive to license. They will decommission them.
What they will not see — because it is not visible from the outside — is the web of informal dependencies that have accumulated around those systems over years. A compliance report that is generated nowhere else. A master data extract that three downstream systems depend on. A calculation logic that exists only in that system’s proprietary engine and has never been documented.
When the system goes offline, the organisation discovers these dependencies through failure: production stops, a regulatory submission cannot be filed, a warehouse cannot process movements. The pain is immediate and visible. The connection to the rationalization decision is direct and undeniable.
The organisational memory of this event is long. The next time a rationalization initiative is proposed, the institutional response is not «let us do it better this time.» It is «the last time we did this, we stopped selling.» The organisation has been immunized against rationalization by the trauma of a poorly executed one.
The rational response to this immunization, at the operational level, is prudence: before decommissioning anything, build a bridge system to cover the dependency. Before removing the legacy platform, create a parallel environment to ensure continuity. Before cutting the integration, maintain a fallback. Each of these prudential measures is individually rational. Their collective effect is that the rationalization initiative ends with more systems than it began with — and the original system still running.
The migration debt spiral described in the previous paper illustrates this precisely: a decommissioning initiative that could not safely eliminate a single system ended with five systems where one existed before, plus the original system still active, because each discovered dependency generated a new tactical solution that could not itself be safely removed.
Case B: The Competent Rationalizer
The rationalizer who deeply understands the organisation faces a different problem. They see the full landscape: which systems are genuinely redundant, which integrations are unnecessary, which capabilities are duplicated. They have the knowledge to rationalize.
But they also see something else: the backlog of genuine value that the organisation has never captured because its architecture was too fragmented to deliver it. Fields that are not being harvested because no harvester covers them. Capabilities that customers need but no system currently provides. Decisions that are being made manually because no integration exists to automate them.
The rational question a competent rationalizer faces is not purely «how do I reduce the number of systems?» It is «where is the highest-value intervention?» And the answer, in most cases, is that expanding the harvestable surface generates more value than reducing the number of harvesters.
Eliminating a system is technically complex, politically costly and slow. The dependencies are real, the migration risk is real, and the organisational resistance is real. Adding a capability that covers unmet need is faster, produces visible results quickly, and builds political support.
The competent rationalizer therefore tends to rationalize incrementally — removing one harvester while adding another to cover what was being missed — rather than dramatically reducing system count. The net effect on total system count, especially in the early phases of the initiative, is often neutral or slightly positive.
This is not incompetence. It is the correct local decision given the constraints. But its structural consequence is the same: the number of systems does not decrease as fast as the theoretical potential of the rationalization would suggest.
The Structural Conclusion
The critical insight is that both cases — competent and incompetent top-down rationalization — produce the same structural tendency: more systems over time, not fewer.
This is not the result of bad management. It is the result of rational behavior within an architectural environment where the cost of adding is always lower than the cost of removing. Until that asymmetry is addressed at the incentive level — until decommissioning is as easy, as fast and as low-risk as adoption — rationalization initiatives will continue to produce complexity faster than they remove it.
Human Intelligence Debt does not accumulate because organisations are badly managed. It accumulates because the system rewards addition and penalizes subtraction, consistently, across all levels of competence and all levels of intent.
Part 2 — The Agent Paradox
How Productive and Unproductive Behavior Both Generate Human Intelligence Debt
If top-down rationalization tends structurally to increase system complexity, one might expect that bottom-up dynamics — driven by individual agents working closer to operational reality — would compensate. In practice, the opposite occurs. Bottom-up dynamics produce the same structural outcome through a completely different mechanism.
Understanding this requires examining two types of agent behavior: the productive agent and the blocking agent.
The Productive Agent
The productive agent is the person in the organisation who genuinely wants to deliver results. They encounter a fragmented informational environment: systems that do not speak to each other, data that cannot be accessed, processes that require manual reconciliation before any decision can be made. They have agency, technical capability and motivation.
Their rational response is to build a workaround. A tool that extracts what they need from the systems that exist and assembles it into something usable. A script that automates the reconciliation they were doing manually. An agent, a dashboard, a custom integration, a local database. It works. They deliver.
The organisation notices. This person produces results while others are still waiting for the backlog to clear. The organisation becomes dependent on their output — and implicitly, on their tool. When they leave, the tool becomes a black box that no one fully understands but no one can safely decommission. It has become a new harvester in the field, covering a strip that no official system covers, operating on logic that exists nowhere in the documentation.
The productive agent did not intend to create Human Intelligence Debt. They intended to deliver value. The debt was a structural byproduct of rational behavior in an environment that made unofficial solutions easier than official ones.
The Blocking Agent
The blocking agent is the person — or team, or unit — whose rational interest is not in maximising organisational output but in controlling a flow. This control may derive from political positioning, from budget ownership, from risk aversion, or simply from the incentive structure of their role. They are not necessarily malicious. They are rational within their incentive landscape.
Their tool — whether inherited, acquired or built — becomes a chokepoint. Data passes through them. Decisions require their validation. Integrations depend on their cooperation. The organisation cannot bypass them without bypassing their system, and the system is too embedded to bypass safely.
The blocking agent does not need to build a new system to generate Human Intelligence Debt. They need only to ensure that their existing system remains indispensable. Every request for integration that enters their backlog and stays there, every API that is promised and never delivered, every data sharing agreement that is perpetually under review — these are the mechanisms by which a blocking agent converts a governance role into a structural dependency.
The Structural Symmetry
The critical observation is that both agents — the productive one and the blocking one — end up in the same structural position: owners of a system that the organisation cannot easily remove.
The productive agent’s system cannot be removed because it delivers value that nothing else delivers. The blocking agent’s system cannot be removed because it controls a flow that nothing else controls. The organisation’s dependency on both is real. The Human Intelligence Debt generated by both is real. And the route to that dependency — through diametrically opposite motivations — is structurally identical.
In game-theoretic terms, both agents have reached a dominant strategy equilibrium: regardless of what others do, the individually rational move is the one that maximises personal indispensability. The productive agent maximises it through value creation. The blocking agent maximises it through flow control. The payoff matrix is different; the equilibrium position is the same.
The Integration Layer That Becomes a Silo
There is a third agent pattern that deserves explicit attention: the integrator.
Organisations frequently respond to fragmentation by introducing an integration layer — a middleware platform, an iPaaS solution, a data orchestration team, an enterprise architecture function, or a systems integrator engagement. The intention is to reduce the coordination cost between existing systems.
In the early phase, this works. The integrator reduces friction, enables data flows that did not previously exist, and absorbs coordination work that was previously consuming human effort.
But the integrator who succeeds becomes a dependency. The organisation’s data flows now route through the integration layer. The integration logic — transformation rules, routing decisions, validation logic — accumulates inside the integrator’s domain. The integrator, whether deliberately or by structural gravity, becomes a new silo: a third party that the two original silos now both depend on.
When the integration layer is a vendor, the organisation has converted an internal fragmentation problem into an external dependency problem. When it is an internal team, the team faces the same incentive dynamic as every other agent: their influence within the organisation is proportional to the indispensability of their systems. Rationalising their own layer reduces their influence. Expanding it increases it. The rational choice, consistently, is expansion.
The Systemic Equilibrium
What emerges from both top-down and bottom-up dynamics is not a management failure or a cultural problem. It is a structural equilibrium in which the rational behavior of every agent — regardless of their competence, their intentions or their organisational role — tends toward the same outcome: more systems, more dependencies, more coordination overhead, and higher Human Intelligence Debt.
The game being played has a clear payoff structure:
- Adding a system: immediate visible value, fast delivery, political support, low personal risk.
- Removing a system: slow, technically complex, politically costly, high personal risk if something breaks.
- Maintaining indispensability: protects budget, headcount and organisational influence regardless of whether the underlying value is genuine.
- Architectural coherence: long payback period, benefits distributed across the organisation, credit diffuse and hard to claim.
No individual player needs to be aware of this structure for the equilibrium to hold. It self-reinforces through the accumulated outcomes of individually rational decisions made across thousands of operational moments.
This equilibrium cannot be broken by better methodology. Agile frameworks, lean management, change management programmes and integration governance all operate within the same incentive structure that produces the problem. They improve the efficiency of movement within the equilibrium. They do not change the equilibrium itself.
Breaking the equilibrium requires changing the underlying cost asymmetry: making decommissioning as easy as adoption, making architectural coherence as rewarded as delivery speed, and making the total cost of system proliferation as visible as the immediate benefit of adding a new tool.
Until that asymmetry is addressed, the field will continue to fill with harvesters. Not because anyone decided it should. Because everyone — rationally, locally, individually — decided to add one more.
This work is produced by the AI Integrity Management working group at The Integral Management Society, a Swiss non-profit association bringing together senior specialists from adaptive systems, complex systems, artificial intelligence, mission-critical operations and governance.