A Multimodal Automated Interpretability Agent
Authors: name: Tamar Rott Shaham; name: Sarah Schwettmann; name: Franklin Wang; name: Achyuta Rajaram; name: Evan Hernandez; name: Jacob Andreas; name: Antonio Torralba
Published: 2024-04-22 17:55:11

Abstract: This paper describes MAIA, a Multimodal Automated Interpretability Agent.
MAIA is a system that uses neural models to automate neural model understanding
tasks like feature interpretation and failure mode discovery. It equips a
pre-trained vision-language model with a set of tools that support iterative
experimentation on subcomponents of other models to explain their behavior.
These include tools commonly used by human interpretability researchers: for
synthesizing and editing inputs, computing maximally activating exemplars from
real-world datasets, and summarizing and describing experimental results.
Interpretability experiments proposed by MAIA compose these tools to describe
and explain system behavior. We evaluate applications of MAIA to computer
vision models. We first characterize MAIA's ability to describe (neuron-level)
features in learned representations of images. Across several trained models
and a novel dataset of synthetic vision neurons with paired ground-truth
descriptions, MAIA produces descriptions comparable to those generated by
expert human experimenters. We then show that MAIA can aid in two additional
interpretability tasks: reducing sensitivity to spurious features, and
automatically identifying inputs likely to be mis-classified.