Looking Inward: Language Models Can Learn About Themselves by
  Introspection
Authors: name: Felix J Binder; name: James Chua; name: Tomek Korbak; name: Henry Sleight; name: John Hughes; name: Robert Long; name: Ethan Perez; name: Miles Turpin; name: Owain Evans
Published: 2024-10-17 17:24:10

Abstract: Humans acquire knowledge by observing the external world, but also by
introspection. Introspection gives a person privileged access to their current
state of mind (e.g., thoughts and feelings) that is not accessible to external
observers. Can LLMs introspect? We define introspection as acquiring knowledge
that is not contained in or derived from training data but instead originates
from internal states. Such a capability could enhance model interpretability.
Instead of painstakingly analyzing a model's internal workings, we could simply
ask the model about its beliefs, world models, and goals. More speculatively,
an introspective model might self-report on whether it possesses certain
internal states such as subjective feelings or desires and this could inform us
about the moral status of these states. Such self-reports would not be entirely
dictated by the model's training data.
  We study introspection by finetuning LLMs to predict properties of their own
behavior in hypothetical scenarios. For example, "Given the input P, would your
output favor the short- or long-term option?" If a model M1 can introspect, it
should outperform a different model M2 in predicting M1's behavior even if M2
is trained on M1's ground-truth behavior. The idea is that M1 has privileged
access to its own behavioral tendencies, and this enables it to predict itself
better than M2 (even if M2 is generally stronger).
  In experiments with GPT-4, GPT-4o, and Llama-3 models (each finetuned to
predict itself), we find that the model M1 outperforms M2 in predicting itself,
providing evidence for introspection. Notably, M1 continues to predict its
behavior accurately even after we intentionally modify its ground-truth
behavior. However, while we successfully elicit introspection on simple tasks,
we are unsuccessful on more complex tasks or those requiring
out-of-distribution generalization.