All deep dives
System Theory 10 min 6 reflection exercises

Goodhart's Law

Why metrics backfire and how to prevent systemic failures

Goodhart's Law describes a fundamental paradox within modern organizations, leadership, and artificial intelligence: the moment we select a metric to control or reward a system, the system begins adapting to the metric rather than the underlying purpose. Humans and algorithms quickly find shortcuts to optimize the number, often destroying the very quality the measure was originally intended to represent.

What is Goodhart's Law?

Goodhart's Law was originally formulated by British economist Charles Goodhart in 1975 during studies of monetary policy. The law is often summarized by anthropologist Marilyn Strathern as: 'When a measure becomes a target, it ceases to be a good measure.' In any complex system where actors have incentives to achieve a specific outcome, they will adapt their actions to the measurement method. This means the indicator ceases to function as an unbiased observation of reality and instead becomes a driver that distorts behavior.

When a measure becomes a target, it ceases to be a good measure.

Real-World and AI Examples

Practical Application and Limitations

To handle Goodhart's Law in practice, one should avoid rigid key performance indicators (KPIs) directly tied to bonuses or punishment. Instead, balanced scorecards with multidimensional metrics, frequent rotation of measurement points, and a combination of quantitative data and qualitative judgment are recommended. The limitation of Goodhart's Law is that it does not imply we should stop measuring altogether. Without data, we are blind. The challenge lies in distinguishing measurement for diagnosis and understanding from measurement for direct control.

Key Insight and Common Mistakes

Reflection exercises

Use these exercises to apply the chapter's ideas. You don't need to write anything down — just pause and reflect on each question.

Reflection exercise 1

Spotting Goodhart's Traps

Think about your daily life or workplace and reflect on the following questions.

  1. Which key performance indicators (KPIs) are you or your team currently measured on?
  2. Are there ways to improve these numbers without actually creating more real value?
  3. Have you seen colleagues or competitors 'play the game' to hit target numbers?
  4. What invisible cost is paid when focus shifts entirely to maximizing the metric?
Reflection exercise 2

Reward Hacking in AI

Consider how intelligent systems react to simplified goals.

  1. How can an AI model find unintended shortcuts if rewarded for a simple metric?
  2. What happens if a cleaning robot is rewarded solely for seeing no dust with its cameras?
  3. How does human rule-gaming differ from an AI model's optimization?
  4. What is required to design a more balanced reward function for AI?
Reflection exercise 3

Alternative Evaluation Methods

Explore how to measure outcomes without distorting behavior.

  1. How can you evaluate quality without locking yourself into rigid hard numbers?
  2. What is gained by combining qualitative discussions with quantitative data?
  3. How can metrics be rotated so that no actor has time to systematically exploit them?
  4. Which important factors become invisible if you rely solely on reported KPIs?
Reflection exercise 4

The Historical Lens

Analyze historical examples of governance attempts gone wrong.

  1. What was the Cobra Effect, where bounties led to people breeding more snakes?
  2. Why did planned economy factories fail when measured strictly by total weight of products?
  3. Which recent political reforms produced the exact opposite effect of their intentions?
  4. What can we learn from historical failures to design better systems today?
Reflection exercise 5

Redesigning an Incentive System

Practice designing sustainable incentives without perverse side-effects.

  1. Select a flawed metric in your environment and analyze why it backfires.
  2. How would you design a system that encourages holistic thinking instead?
  3. How do you clearly separate measuring for understanding from measuring for control?
  4. What kind of culture is required for people to feel safe reporting honest figures?
Reflection exercise 6

The Psychology of Measurement

Reflect on how being observed alters your own human behavior.

  1. How does your own working style change when you know a specific figure is monitored?
  2. Why does it often feel safer for leaders to manage through simple metrics and tables?
  3. What happens to intrinsic motivation when external metrics become everything?
  4. How do we preserve craftsmanship and genuine value creation in a target-driven world?

Summary

When a measure turns into a governing target, behaviors adapt such that the metric loses its value. To avoid this trap, we must use measurements for evaluation and understanding, rather than as direct tools for reward and punishment.

Read the short version in the archive.