Title: Unravelling the Mechanisms of Manipulating Numbers in Language Models
Primary area: interpretability and explainable AI
Keywords: interpretability, probing, representations, numbers, language models, factuality

Abstract: Recent work has shown that different large language models (LLMs) converge on similar and accurate input embedding representations for numbers. These findings conflict with the documented propensity of LMs to produce erroneous outputs when dealing with numeric information. In this work, we aim to shed light on how language models manipulate numbers and quantify the lower bounds of accuracy of these mechanisms. 

We find that despite surfacing errors, different language models learn interchangeable representations of numbers that are systematic, accurate and universal across their hidden states and the types of input contexts. This allows us to create universal probes for each LLM and to trace information --- including the causes of output errors --- to specific layers in LLMs' architecture. Our results show ways for more robust applications of probing methods and may inform future, addressed refinements of LLMs' architectures.

TL;DR: We show that language models use interchangeable, universal and systematic representation of numbers, allowing accurate tracing of information -- and its accuracy and robustness -- within their computation.