The same observed behavior can arise from very different learned mechanisms.
A model may achieve excellent test performance while relying on features, representations, or strategies that differ substantially from those we intended.
In this chapter
2.1 The Rashomon Effect
Many models can achieve essentially indistinguishable performance while implementing different solutions.
2.2 Underspecification
Training objectives often fail to uniquely determine the behavior we want outside the observed training distribution.
2.3 Adversarial Features
Models can rely on predictive features that are valid statistically but misaligned with the features humans expect.
2.4 Shortcut Learning
Learners frequently discover easier predictive strategies than the mechanism we intended them to learn.
2.5 Why Interpretability Follows
If behavior does not uniquely identify mechanism, we need another channel for discovering which solution was actually learned.