Backpack Language Models (2023)

Abstract: We present Backpacks: a new neural architecture that marries strong modeling performancewith an interface for interpretability and control. Backpacks learn multiple non-contextual sense vectors for each word in a vocabulary, and represent a word in a sequence as a context-dependent, non-negative linear combination ofsense vectors in this sequence. We find that, after training, sense vectors specialize, each encoding a different aspect of a word. We can interpret a sense vector by inspecting its (non-contextual, linear) projection onto the output space, and intervene on these interpretable hooks to change the model’s behavior in predictable ways. We train a 170M-parameter Backpack language model on OpenWebText, matching the loss of a GPT-2 small (124Mparameter) Transformer. On lexical similarity evaluations, we find that Backpack sense vectors outperform even a 6B-parameter Transformer LM’s word embeddings. Finally, we present simple algorithms that intervene on sense vectors to perform controllable text generation and debiasing. For example, we can edit the sense vocabulary to tend more towards a topic, or localize a source of gender bias to a sense vector and globally suppress that sense.

## introduction consider the prefix the ceo believes that ___, and the problem of debiasing a neural language model's distribution over he/she. intuitively, the bias for he originates in the word ceo, because replacing ceo with nurse flips the observed bias. a successful intervention to debias ceo must reliably apply in all contexts in which the word ceo appears; ideally we would want to make a non-contextual change to the model that has predictable effects in all contexts. in general, in all aspects of interpretability and control, it is desirable to make interventions with a tractable interface (e.g., noncontextual representations) that apply globally. ## the backpack architecture in this section, we define the general form of the backpack architecture. we then show how continuous bag-of-words word2vec (cbow) @xcite and self-attention-only networks @xcite are special cases of backpacks. ## language modeling with backpacks in this section, we define a neural autoregressive language model parameterized by a backpack. we use the standard softmax parameterization of the probability over the next token in a sequence, with a weight matrix @xmath0 recall (section 2.1) that backpack representations o j are defined by sense vectors c(x) and contextualization weights α j . in section 3.1 we describe a parameterization of c for the predictive sense vectors in equation (1), and in section 3.2 we describe a parameterization of a for the contextualization weight network in equation (3). when o j is parameterized by a backpack, we call a model of the form given by equation ( 9 ) a backpack lm. ## emergent structure in sense vectors backpack language model sense vectors are not trained using a supervised notion of word sense, but implicitly specialize to encode different shades of a word's predictive use. in this section, we qualitatively examine sense vectors (section 5.1) and quantitatively demonstrate their effectiveness in computing lexical similarity and relatedness (section 5.2). taken together, this suggests that sense vectors can provide a high-level interface for intervention, which we explore in section 6. ## sense vectors for control in this section, we demonstrate several proof-ofconcept methods that leverage sense vectors for controlling lm behavior. ## related work representation learning in nlp. learning probabilistic models of text for use in representation learning and identifying resulting structure has a long history in nlp, from non-contextual word vectors @xcite @xcite @xcite to contextual networks @xcite @xcite @xcite . deep averaging networks @xcite are not backpacks; they first perform averaging and then nonlinear computation. ## discussion in this section, we address a few natural questions about the expressivity and interpretability of backpacks, highlighting the limits of our knowledge. ## conclusion non-contextual word2vec embeddings initiated modern deep learning research in nlp, and have fascinating geometric structure. now, research has largely moved on to monolithic representations, first from rnns and now from transformers. our work suggests that we can have both rich lexical structure and interventions, and strong contextual performance, in a single model.