Alignment Studio: Aligning Large Language Models to Particular
  Contextual Regulations
Authors: name: Swapnaja Achintalwar; name: Ioana Baldini; name: Djallel Bouneffouf; name: Joan Byamugisha; name: Maria Chang; name: Pierre Dognin; name: Eitan Farchi; name: Ndivhuwo Makondo; name: Aleksandra Mojsilovic; name: Manish Nagireddy; name: Karthikeyan Natesan Ramamurthy; name: Inkit Padhi; name: Orna Raz; name: Jesus Rios; name: Prasanna Sattigeri; name: Moninder Singh; name: Siphiwe Thwala; name: Rosario A. Uceda-Sosa; name: Kush R. Varshney
Published: 2024-03-08 21:26:49

Abstract: The alignment of large language models is usually done by model providers to
add or control behaviors that are common or universally understood across use
cases and contexts. In contrast, in this article, we present an approach and
architecture that empowers application developers to tune a model to their
particular values, social norms, laws and other regulations, and orchestrate
between potentially conflicting requirements in context. We lay out three main
components of such an Alignment Studio architecture: Framers, Instructors, and
Auditors that work in concert to control the behavior of a language model. We
illustrate this approach with a running example of aligning a company's
internal-facing enterprise chatbot to its business conduct guidelines.