Rethinking Interpretability in the Era of Large Language Models
Authors: name: Chandan Singh; name: Jeevana Priya Inala; name: Michel Galley; name: Rich Caruana; name: Jianfeng Gao
Published: 2024-01-30 17:38:54

Abstract: Interpretable machine learning has exploded as an area of interest over the
last decade, sparked by the rise of increasingly large datasets and deep neural
networks. Simultaneously, large language models (LLMs) have demonstrated
remarkable capabilities across a wide array of tasks, offering a chance to
rethink opportunities in interpretable machine learning. Notably, the
capability to explain in natural language allows LLMs to expand the scale and
complexity of patterns that can be given to a human. However, these new
capabilities raise new challenges, such as hallucinated explanations and
immense computational costs.
  In this position paper, we start by reviewing existing methods to evaluate
the emerging field of LLM interpretation (both interpreting LLMs and using LLMs
for explanation). We contend that, despite their limitations, LLMs hold the
opportunity to redefine interpretability with a more ambitious scope across
many applications, including in auditing LLMs themselves. We highlight two
emerging research priorities for LLM interpretation: using LLMs to directly
analyze new datasets and to generate interactive explanations.