Opening proposition
The idea in brief
A controller observes a few variables and uses them to keep a system near a target. In a fixed machine, success means that the chosen variables captured enough of the machine’s behaviour.
Living, institutional and algorithmic systems are not fixed. They adapt under repeated control. Controller-moulding proposed that this adaptation can soften or eliminate behaviours the controller cannot perceive. The system comes to inhabit the low-dimensional world presupposed by the dashboard.
A performance measure can therefore become accurate for a disturbing reason: it has helped remove everything that made it incomplete. The map acquires predictive power by remaking the territory.
Full exposition
Control changes when the system can adapt
Classical control problems often begin with a plant—a machine or process whose behaviour is taken as given—and a controller designed to regulate it. Better sensing and modelling allow the controller to respond more accurately. When the controlled system is a school, firm, organism, market or online community, that separation breaks down. The plant observes the controller too.
People alter behaviour to satisfy metrics. Organisations reorganise around reporting categories. Biological systems remodel under repeated stress. Recommendation systems train on responses partly created by earlier recommendations. The controller is therefore not acting on a stable object; it is applying a selective environment to a system capable of changing its own response modes.
How the moulding occurs
Suppose a controller sees only a few aggregate variables. Behaviours that improve those variables are rewarded or preserved, while behaviours that matter in unmeasured ways receive no protection. Over many cycles, resources flow toward the visible modes. Invisible variation becomes costly, fragile or institutionally unintelligible and begins to disappear.
Eventually the controller’s simple model predicts the system unusually well. This success can be mistaken for proof that the chosen variables captured the system’s natural structure. Controller-Moulding offers another explanation: repeated governance has compressed the system until the variables became sufficient. The model fit is real, but it is historically produced.
The diagnostic signature
Ordinary learning by a controller should improve prediction while leaving the underlying variety available. Moulding predicts a different joint pattern: unexplained variation falls, performance on the measured variables rises, and the repertoire of behaviours outside those variables contracts. The system becomes easier to control and less able to surprise.
The strongest audit therefore looks backward. Which practices, roles, species, strategies or kinds of person vanished while the dashboard became more accurate? Did measurement improve, or did the measured categories acquire power over survival? Counterfactual tests can help: when the metric is briefly removed, does lost behaviour return, or has the system been structurally rebuilt around it?
Governance by preserving unruly dimensions
The idea does not imply that all simple control is oppressive. Simplicity can be necessary for coordination. The danger lies in confusing governability with truth. A controller should be evaluated partly by whether it preserves valuable dimensions it cannot presently measure.
Practical safeguards include rotating metrics, maintaining qualitative channels, protecting experimental niches, tracking diversity independently of performance and periodically redesigning the controller from observations made outside its own categories. The deeper lesson is that a successful map may have conquered its territory. Accuracy alone cannot tell us whether the world was understood or made obedient.
A controller is also an environment
A controller observes selected variables and uses them to reward, punish or correct a system. In textbook examples, the controlled plant is described as if its basic organisation remains fixed. Living and social systems do not oblige. Employees learn what the dashboard notices, organisms adapt to repeated interventions, platforms reorganise around ranking signals, and students allocate effort to what examinations can register. The controller becomes part of the environment to which the system adapts.
Controller-Moulding names the long-run consequence: the system may change until the controller’s simplified representation becomes increasingly accurate. This is subtler than gaming a metric. Gaming leaves an important reality intact while producing a misleading number. Moulding can eliminate or marginalise the unmeasured activities themselves. After sufficient adaptation, the measured variable predicts behaviour well because the world has been reorganised to make it predictive.
From measurement error to ontological change
At first, a coarse controller misdescribes the system. A school values curiosity, patience, explanation and collaboration, while an examination captures only some testable outputs. Repeatedly attaching advancement to the test creates selection pressure. Teachers devote more time to examinable forms of knowledge, students prefer strategies with visible scores, and administrators remove activities that do not contribute to measured results. The original measurement error shrinks—not because the test improved, but because the school changed.
This produces a dangerous kind of validation. Later data show a strong relationship between test performance and what the institution now does. Designers may conclude that the original metric was insightful. The historical sequence tells a different story: the metric first omitted important dimensions, then helped make those dimensions scarce. Accuracy at the end cannot by itself establish adequacy at the beginning.
How to distinguish moulding from learning
Adaptation to control is not necessarily bad. A thermostat is useful because a heating system responds predictably, and professional standards often make practice more reliable. The diagnostic issue is whether improvement comes from better performance on an underlying purpose or from contraction of the purpose into the controller’s vocabulary. A hospital that reduces preventable harm has learned; a hospital that stops recording difficult cases so its visible outcomes improve has gamed; a hospital that gradually abandons forms of care not represented in funding categories has been moulded.
The strongest empirical signature is historical dependence. Introduce two different simple controllers to initially similar systems. If each population reorganises until its own controller becomes more predictive—and the resulting systems diverge in the dimensions they retain—then controller accuracy is partly endogenous. Removing the controller should not instantly restore the old variety, because skills, roles, infrastructure and expectations have already adapted.
The ratchet and the loss of counterevidence
Controller-moulding can reinforce itself. As unmeasured practices disappear, fewer people remember why they mattered, fewer examples remain available to challenge the metric, and the costs of supporting them appear increasingly exceptional. The controller then looks simpler and more successful. This success justifies wider application, which further reduces the unmeasured remainder. The system is not merely trapped by incentives; it is losing the evidence needed to imagine an alternative description.
That ratchet explains why replacing a metric can be difficult even after its shortcomings are acknowledged. The organisation no longer contains the capacities that the new measure would need to observe. Restoring them requires investment before they can generate legible results, creating an awkward interval in which the old controller still appears more accurate. Reform must therefore rebuild variation as well as change measurement.
Preserving dimensions the controller cannot see
A robust design does not try to measure everything. It deliberately protects activities whose value may be real but temporarily illegible. Multiple evaluators with nonidentical criteria, rotating metrics, protected discretionary budgets, qualitative review and periodic removal of incentives can prevent one representation from becoming the sole survival environment. These arrangements preserve counterfactual variety: evidence about what the system could be under a different controller.
At a personal scale, the idea applies when tracking tools reshape the activity being tracked. Counting words can help someone write, but it can also turn writing into word production; sleep scores can improve habits while teaching a person to experience rest through a dashboard. The practical question is not whether measurement influences behaviour—it always does to some degree—but whether the influence is making the simplification true by deleting valued parts of the activity.
A thought experiment
A school is managed by one examination score. At first, the score poorly represents teaching. Over years, schedules, staffing, student strategies and curricula reorganize around the examination. The score now predicts school activity extremely well—not because measurement improved, but because unmeasured forms of learning were selected out.
Consequences
What the idea changes
Successful control would cease to be strong evidence that a model was originally adequate. Simplicity can be an outcome imposed on the system. This reframes dashboards, workplace metrics, recommendation systems and even biological homeostasis.
Ways to think with it
- It linked measurement, long-run adaptation and apparent model accuracy in one mechanism.
- It predicted declining unexplained variation together with declining behavioural diversity.
- It suggested auditing what disappeared as a controller became more successful.
Further reading
The intellectual neighbourhood
Notes on the idea’s provenance and editorial review are kept separately in the editorial appendix.