# Interpretability

> Research aimed at understanding what is happening inside an AI model, and why it gives its answers.

Source: https://www.superintelligence.com.au/glossary/interpretability/

Interpretability is the effort to understand how an AI model reaches its outputs by looking inside it. A modern model is made of billions of numbers, and no one writes down rules for what they do. The system is trained rather than programmed, so even its builders cannot simply read off why it said what it said.

The field tries to change that. One approach studies individual parts of the network, looking for patterns that correspond to recognisable ideas, like a place, a rule of grammar or a kind of deception. Another traces the steps a model takes on a particular answer.

It matters for safety because it might let people check what a system is actually doing, rather than trusting what it says about itself. For example, it could help spot a model that behaves well in tests but pursues a different aim in practice.

Progress has been real but partial. Researchers can explain some small circuits and features, and they cannot yet give a complete account of a large model. Views differ on whether it will scale up in time to be useful for the most capable systems.

Related: https://www.superintelligence.com.au/glossary/alignment.md, https://www.superintelligence.com.au/glossary/parameters.md, https://www.superintelligence.com.au/glossary/transformer.md, https://www.superintelligence.com.au/glossary/reasoning-model.md