Foundations
Section 2.5

Why Interpretability Follows

Underdetermination creates the need for another communication channel.

Underdetermination gives us a direct motivation for interpretability.

Behavior is insufficient

Observed performance narrows the space of possible mechanisms but often does not identify one.

Interpretation as additional information

Interpretability provides another channel through which the learned system can communicate which solution it actually implemented.

From observation to control

Once the relevant mechanism becomes observable, we can ask whether it is also controllable.

Interpretable Intelligence synthesizes work from across the field, including research from Guide Labs. Relevant authors and results are cited throughout.