Societal Alignment Frameworks Can Improve LLM Alignment
Authors: name: Karolina Stańczak; name: Nicholas Meade; name: Mehar Bhatia; name: Hattie Zhou; name: Konstantin Böttinger; name: Jeremy Barnes; name: Jason Stanley; name: Jessica Montgomery; name: Richard Zemel; name: Nicolas Papernot; name: Nicolas Chapados; name: Denis Therien; name: Timothy P. Lillicrap; name: Ana Marasović; name: Sylvie Delacroix; name: Gillian K. Hadfield; name: Siva Reddy
Published: 2025-02-27 13:26:07

Abstract: Recent progress in large language models (LLMs) has focused on producing
responses that meet human expectations and align with shared values - a process
coined alignment. However, aligning LLMs remains challenging due to the
inherent disconnect between the complexity of human values and the narrow
nature of the technological approaches designed to address them. Current
alignment methods often lead to misspecified objectives, reflecting the broader
issue of incomplete contracts, the impracticality of specifying a contract
between a model developer, and the model that accounts for every scenario in
LLM alignment. In this paper, we argue that improving LLM alignment requires
incorporating insights from societal alignment frameworks, including social,
economic, and contractual alignment, and discuss potential solutions drawn from
these domains. Given the role of uncertainty within societal alignment
frameworks, we then investigate how it manifests in LLM alignment. We end our
discussion by offering an alternative view on LLM alignment, framing the
underspecified nature of its objectives as an opportunity rather than perfect
their specification. Beyond technical improvements in LLM alignment, we discuss
the need for participatory alignment interface designs.