Venue: NAACL 2021

# Game-theoretic Vocabulary Selection via the

Shapley Value and Banzhaf Index

Roma Patel Brown University &Marta Garnelo DeepMind &Ian GempDeepMind &Chris Dyer &Yoram Bachrach DeepMind

Work done during an internship at DeepWind. Corresponding email: romapatel@brown.edu or yorambac@google.com.

###### Abstract

The input vocabulary and their learned representations are crucial to the performance of neural NLP models. Using the full vocabulary results in less explainable and more memory intensive models, with the embedding layer often constituting the majority of model parameters. It is thus common to use a smaller vocabulary to lower memory requirements and construct more interpretable models.

We propose a vocabulary selection method that views words as members of a team trying to maximize the model's performance. We apply power indices from cooperative game theory, including the Shapley value and Banzhaf index, that measure the relative importance of individual team members in accomplishing a joint task. We approximately compute these indices to identify the most influential words.

Our empirical evaluation examines multiple NLP tasks, including sentence and document classification, question answering and textual entailment. We compare to baselines that select words based on frequency, TF-IDF and regression coefficients under L1 regularization, and show that this game-theoretic vocabulary selection outperforms all baselines on a range of different tasks and datasets.

+
Footnote †: Work done during an internship at DeepWind. Corresponding email: romapatel@brown.edu or yorambac@google.com.

## 1 Introduction

Most state-of-the-art NLP methods use neural networks that require a pre-defined vocabulary to vectorise and encode text. In large text datasets, the vocabulary size can grow to hundreds of thousands of words, and having an embedding space over the entire vocabulary results in models that are expensive in memory and compute, and hard to interpret.

Many of the words in the vocabulary are not crucial to task performance, and can be removed without a significant drop in final task performance. It is common to use heuristics such as frequency or TF-IDF to reduce vocabulary size. After filtering to obtain a smaller vocabulary, "out-of-vocabulary" (OOV) words are replaced with an unknown word token \(<\)UNK\(>\). This reduction in vocabulary size has many advantages. Models with reduced vocabulary are more easily interpretable and achieve increased transparency Adadi and Berrada (2018); Samek et al. (2019), require less memory, can be used in resource constrained settings, and are less prone to overfitting Sennrich et al. (2015); Shi and Knight (2017); L Hostis et al. (2016); Chen et al. (2019). However, reducing the vocabulary size with a heuristic such as frequency is often not optimal. For example, Figure 1 shows the top ranked words according to frequency (blue), that are largely unimportant for the sentiment task at hand.

We consider the **vocabulary selection problem:** given a target vocabulary size \(k\) (or equivalently, a target memory footprint or a "budget" of model parameters for the embedding layer), what is the optimal word subset we should use as our vocabulary? Our solution's output, based on the Shapley

Figure 1: An example sentence from SST-2 Socher et al. (2013), as well as the distribution of heuristic values based on vocabulary selection algorithms. Frequency and TF-IDF weight stopwords (right) higher whereas a game-theoretic Shapley-based approach tends to value task-specific words (left) more.



value, is also shown in Figure 1, demonstrating that it focuses on words relevant to the task.

**Our Contribution:** We use game theoretic principles to propose a vocabulary selection method. We cast the vocabulary selection problem as a _cooperative game_, which considers subset of words as a "team" whose goal is to solve the NLP task at hand. We define the performance of a team as the performance of a model that uses only those words as its vocabulary. Our method applies solution concepts from game theory to determine the relative importance of each word in achieving the goal. Specifically, we consider the Shapley value Shapley (1953) and Banzhaf index Banzhaf III (1964), key concepts in game theory, that are used as "power-indices" for measuring the individual contribution of team members to the success of the team. We approximate these indices by sampling subsets of words and training a model on each subset to contrast model performance when including and omitting a target word.

We evaluate our approach against baselines such as TF, TF-IDF and ranking using logistic regression coefficients under L1 regularization. We evaluate on a range of datasets and task structures: single-sentence classification, pairwise-sentence classification and document classification. While our method is _significantly more demanding computationally_ than these simple baselines, we empirically demonstrate that it outperforms these baselines on all tasks, offering better tradeoffs between the vocabulary size and the model's performance.

## 2 Method

We assume a dataset \(D\) and a training method \(M\) for training on \(D\) and producing a model \(f_{\theta}\) where \(\theta\) are tuned model parameters. The model is evaluated on a validation set \(T\) to estimate how well the model generalizes to the true data distribution. An evaluation metric (for example the model accuracy or \(F_{1}\) score, as evaluated on the validation set) for each model \(f_{\theta}\) is denoted by \(q(f_{\theta})\), thus allowing an assessment of the performance of a subset of words. We first briefly discuss preliminaries from cooperative game theory Chalkiadakis et al. (2011).

### Preliminaries: Cooperative Game Theory

Cooperative game theory investigates settings where multiple players work together in teams. A (transferable-utility) cooperative game consists of a set \(A=\{a_{1},\ldots,a_{n}\}\) of players and a characteristic function \(v:2^{A}\rightarrow\mathbb{R}\) mapping any subset of players \(C\subseteq A\) (called a "team" or "coalition") to a real value \(v(C)\) indicating the performance of the team when working together.

The **Shapley value**Slapley (1953), denoted \(\phi(v)=(\phi_{1},\ldots\phi_{n})\), reflects each player's individual contribution to the success of the team, adhering to fairness axioms Dubey (1975). 1 Similarly, the **Banzhaf index**Banzhaf III (1964), denoted \(\beta(v)=(\beta_{1},\ldots\beta_{n})\), measures impact of individuals on the success of a team, using different axioms Dubey and Shapley (1979); Strafin Jr (1988).

Footnote 1: The Shapley value has also been used to examine power in team formation Aziz et al. (2009); Mash et al. (2017); Bachrach et al. (2020), combinatorial tasks Ueda et al. (2011); Banarse et al. (2019), pricing and auctions Bachrach (2010); Kamboj et al. (2011); Blocq et al. (2014) or political settings Bilbao et al. (2002); Bachrach et al. (2011); Filmus et al. (2019), or feature importance for model explainability Lundberg and Lee (2017).

Consider quantifying the individual contribution of a player \(a_{i}\in A\) in a game with the characteristic function \(v\). Examine the player \(a_{i}\) and a coalition \(C\subseteq A\setminus\{a_{i}\}\) that does not contain that player. The **marginal contribution** of \(a_{i}\) to the coalition \(C\) is defined as \(m(a_{i},C)=v(C\cup\{a_{i}\})-v(C)\), i.e. the increase in value arising from adding \(a_{i}\) to the coalition \(C\). Similarly, denote the set of _permutations_ over then \(n\) players as \(\Pi\) (i.e. each \(\pi\in\Pi\) is a bijection \(\pi:A\to A\)), and denote the predecessors of \(a_{i}\in A\) in the permutation \(\pi\) as \(b(a_{i},\pi)\). The marginal contribution of \(a_{i}\) in the permutation \(\pi\) is defined as \(m(a_{i},\pi)=v(b(a_{i},\pi)\cup\{a_{i}\})-v(b(a_{i},\pi))\), i.e. the increase in value arising from adding \(a_{i}\) to the players appearing before it in the permutation \(\pi\).

The Banzhaf index \(\beta_{i}\) of player \(a_{i}\) is the marginal contribution of player \(a_{i}\) averaged over all possible coalitions that do not contain that player:

\[\beta_{i}=\frac{1}{2^{n-1}}\sum_{C\subset A|i\in C}v(C\cup\{i\})-v(C)\]

The Shapley value \(\pi_{i}\) of a player \(a_{i}\) is the marginal contribution of that player, averaged across all permutations:

\[\phi_{i}=\frac{1}{n!}\sum_{\pi\in\Pi}v(b(a_{i},\pi)\cup\{a_{i}\})-v(b(a_{i}, \pi))\]

The Banzhaf index of \(a_{i}\) can be viewed as the expected increase in performance under uncertainty about the participation of other players in the team 

-- if each of the other players has an equal probability of joining the team or not joining it, how much value to we expect to add when \(a_{i}\) joins the team. Similarly, the Shapley value can be viewed as the expected increase in team value that \(a_{i}\) would yield when players join the team in a random order. 2

Footnote 1: [https://www.cs.ucaballiance.edu/~citation/](https://www.cs.ucaballiance.edu/~citation/)

Footnote 2: An equivalent formula for the Shapley value is: \(\phi_{i}=\sum_{C\subseteq A|i\in C}\frac{|C|(A|-|C|-1)!}{|A|}v(C\cup\{i\})-v(C)\), showing the different weights the indices give to different size coalitions.

### Our Approach: Vocabulary Selection by Comparing Power Indices

Given the entire vocabulary \(V\) and a budget of \(k\) words to use, our method selects a subset \(V^{\prime}\subset V\) where \(|V^{\prime}|=k\), optimizing the performance \(q(f_{\theta}^{V^{\prime}})\) of a model \(f_{\theta}^{V^{\prime}}\) trained using a vocabulary consisting only of the words in \(V^{\prime}\).

We view each word as a player and each subset of words \(C\subseteq V\) as a team, and construct a cooperative game. The characteristic function \(v:V\rightarrow\mathbb{R}\) maps a subset of words (partial vocabularies) to the performance obtained when training a model with only these words a vocabulary. Formally, we define the performance \(v(C)\) of the team \(C\subseteq V\) to be the performance \(q(f_{\theta}^{C})\) of an NLP model \(f_{\theta}^{C}\) with the words in \(C\) as its input vocabulary. 3

Footnote 3: For example, for text classification we may define \(v(C)\) to be the model’s accuracy when using \(C\) as the vocabulary.

Given a vocabulary \(C\subseteq V\), evaluating \(v(C)\) requires training a model \(f_{\theta}\) on dataset \(D\) using only the words in \(C\) as the vocabulary 4, and measuring its performance on the validation set \(T\) to obtain \(v(C)=q(f_{\theta}^{C})\). We compute the Shapley value \(\phi_{i}\) or Banzhaf index \(\beta_{i}\) of any word \(w_{i}\in V\) (see Section 2.1). Words with high values are ones that have a larger positive influence on performance, whereas words with lower values are ones that do not impact task performance when they are removed. 5

Footnote 4: For example in a text classification task, one could train a neural network classifier \(f_{\theta}^{G}\) on the dataset \(D\), replacing all the words in \(V\setminus C\) with the UNK token.

Footnote 5: The direct formulas for the Shapley or Banzhaf indices enumerate over _all_ possible world subsets or permutations, which is intractable. Hence, we use an approximation algorithm [76, 8].

Observe that the Banzhaf index \(\beta_{i}\) is the expected marginal contribution \(m(a_{i},C)\) for a coalition \(C\) sampled uniformly at random from the set \(\{C\subseteq V|a_{i}\in C\}\), and the Shapley value \(\phi_{i}\) is the expected marginal contribution \(m(a_{i},\pi)\) for a permutation \(\pi\) sampled uniformly at random from \(\Pi\). We can approximate these by taking a sample of coalitions or permutations, and examining \(a_{i}\)'s average marginal contribution in the sample. For the Shapley value, the sample consists of permutations of words in the vocabulary, where for each permutation \(\pi\) we train two models on vocabularies that differ by a single word \(w\). The performance difference between the two models is then the marginal contribution of the word \(w\). For the Banzhaf index, we directly construct the vocabulary by flipping a fair coin per word to determine its inclusion in the vocabulary. The power index is approximated as the average marginal contribution of the word across the samples. Finally, we select the \(k\) words with the highest power index as our vocabulary \(V^{\prime}\). This is shown in Algorithms 1, 2.

```
1:Inputs: NLP dataset \(D\) with full vocabulary \(V\)
2:for each word \(w\) in \(V\)do
3:\(\beta_{w}\gets 0\) (initialise Banzhaf index estimate)
4:for i=1 to \(S\) (number of sampled coalitions)do
5:\(C_{1}\leftarrow\emptyset\)
6:for j=1 to \(|V|\)do
7:\(s\leftarrow\text{Uniform}(\{0,1\})\)
8:if\(s=1\) and \(w_{j}\neq w\)then
9:\(C_{1}\gets C_{1}\cup\{w_{j}\}\)
10:endif
11:endfor
12:\(C_{2}\gets C_{1}\cup\{w\}\) (random coalition including \(w\))
13:\(f_{\theta}^{C_{1}}\leftarrow\text{TrainModel}(C_{1})\) (Train on vocabulary \(C_{1}\))
14:\(f_{\theta}^{C_{2}}\leftarrow\text{TrainModel}(C_{2})\) (Train on vocabulary \(C_{2}\))
15:\(m(w,C_{1})\gets q(f_{\theta}^{C_{2}})-q(f_{\theta}^{C_{1}})\)
16:\(\beta_{w}\leftarrow\beta_{w}+m(w,C_{1})\)
17:endfor
18:\(\beta_{w}\leftarrow\frac{1}{\beta}\beta_{w}\) (average marginal contributions)
19:endfor
20:Rank words in \(V\) based on Banzhaf estimates \(\beta_{w}\)
21:Return top \(k\) words in ranking
```

**Algorithm 1**Banzhaf Vocabulary Selection 



### Datasets and Tasks

We consider three different task structures.

**Single Sentence Classification:** the task requires a model to encode the words of a given sentence and output a classification based on properties of sentences (for e.g., sentiment or acceptability). We evaluate on a sentiment-analysis task using the SST-2 dataset Socher et al. (2013) and a corpus acceptability task using the CoLA dataset Warstadt et al. (2019); Wang et al. (2018). The sentiment analysis task contains 9.6k sentences labelled with a positive or negative sentiment, while the acceptability task contains 8.5k sentences labelled with an acceptability judgement about whether or not it is a grammatically correct English sentence.

**Entailment and Question Pair Classification:** this task requires a model to encode two sentences and output a classification based on the relation between them. We evaluate on a textual entailment task using the SNLI dataset Bowman et al. (2015a) and a question pair classification task using the QQP dataset Wang et al. (2018). SNLI contains 550k sentence pairs and requires models to encode two different sentences, a premise and a hypothesis, and predict one of three relations between them: an entailment, a contradiction or a neutral relation. The QQP task contains 364k pairs and requires models to encode two different text inputs, a question and an alternate question composed of different words, and to predict whether or not the two questions correspond to the same answer.

**Document Classification:** this task requires models to encode an input document or article, and predict a class based on properties of the document. We evaluate on the AG-News and Yelp datasets Zhang et al. (2015). The AG-News dataset contains the title and description of 120,000 news articles in four categories (the prediction target is the category). The Yelp dataset contains 130,000 million samples with text reviews, with the prediction target being the polarity of the review (positive or negative). The number of words in each text instance (document) are significantly larger than in the single sentence classification task, requiring models to capture phenomena like co-reference and temporal order that are prevalent in longer texts.

### Methodology

Our method in Section 2.2 is agnostic to the specific model and training procedure: we simply assume we have access to an algorithm that trains on a dataset \(D\) and produces a trained model \(f_{\theta}\) whose quality \(q(f_{\theta})\) is evaluated on a validation set \(T\).

We perform our empirical evaluation using both an LSTM classifier and a logistic regression classifier. Our method trains many models with different vocabularies to select the final vocabulary \(V^{\prime}\). We then evaluate the quality of the chosen reduced vocabulary \(V^{\prime}\) by training a final model \(f^{V^{\prime}}\) which uses only the vocabulary \(V^{\prime}\) and evaluate the performance of \(f^{V^{\prime}}\) on a held out test \(T^{\prime}\).

To maximize performance, one should use the same architecture during the vocabulary selection process as the evaluation. However, words that are strong features for one architecture are likely to also be strong features for another architecture. Hence, we can select the the vocabulary using one architecture even if we intend to use this vocabulary for another architecture. As our vocabulary selection procedure trains many models, we use logistic regression models during the vocabulary selection process. We show it still significantly outperforms baselines, and allows faster and more efficient computation of the Shapley value. We then evaluate the quality of the vocabulary using an LSTM model.

**Training logistic regression models:** To train the logistic regression classifier in the single-text case, we represent each sentence or document as the set of words that occur in that text sample. For the pairwise-sentence case, we similarly represent each paired input with _three times_ the number of word features, using a one hot encoding indicating that the word occurred only in the first sentence (e.g. question), only in the second sentence (e.g. answer) or whether it occurred in both sentences. This model is far simpler than state-of-the-art text classification models, but we find it is a good-enough proxy for the Shapley computation step, and much more economical computationally.

#### Evaluating the Selected Vocabulary's Quality

To train the LSTM classifier, we encode words using an embedding layer of size 100. These embeddings are fed one at a time to an LSTM encoder with a hidden layer size of 100, and the output of the LSTM encoder is fed into a feedforward neural network yielding the final classification Deng and Liu (2018) over some number of classes.

Our experiments show that even when using the simple logistic regression for the vocabulary selection process we achieve a significant performance improvement over baselines, as evaluated with an LSTM model. In other words, the vocabulary qual

ity improvement transfers to more complex models.

#### Baselines

We contrast the performance of our approach (Algorithm 1 based on the Banzhaf index and Algorithm 2 based on the Shapley value) with several baselines. We first consider ranking by term frequency (TF), i.e selecting the most frequently occuring words in the dataset. We also consider ranking words by TF-IDF scores Ramos et al. (2003), which is commonly used for web search. As a stronger baseline we consider ranking words based on their regression coefficients, a method used for estimating feature importance Ellis (2010); Nimon and Oswald (2013). In this baseline, we train a logistic regression model with \(L_{1}\) regularization on the dataset \(D\) (the regularization encourages the model to have low weights, setting the weight of many features to zero when the regularization is strong enough); we then rank features by the absolute coefficient of each feature in the trained model. We refer to this as the \(L_{1}\)**baseline**. 6

Footnote 6: In logistic regression with \(L_{1}\) regularization, the regression coefficients and derived word ranking depend on the degree of regularization and the initialization. Methods like GLMpath Friedman et al. (2010) obtain the entire \(L_{1}\) path of the GLM at the cost of fitting a single model. In the spirit of stability selection Meinshausen and Bühlmann (2010), to alleviate stochasticity we average 20 training runs of the \(L_{1}\)-regularized model, averaging coefficients to obtain the ranking over words (still cheaper computationally than our approach).

Our approach for calculating the Banzhaf index or Shapley value is based on a random sample of coalitions, and achieving a good accuracy requires taking many samples, especially when ranking a vocabulary with many words. To keep the required compute manageable while achieving a reasonable approximation, we first apply a **pre-filtering** step, selecting a large vocabulary (but not the full vocabulary) by applying the TF heuristic, then selecting the final small vocabulary from this large vocabulary using our approach. For instance, with a target vocabulary size of 100 words, we first filter out all but the 1,000 most frequent words and then rank based on the Shapley value (and contrast the performance of this method with selecting the top 100 words based solely on TF or TF-IDF score). When comparing against the \(L_{1}\) baseline, we similarly apply an \(L_{1}\) based pre-filtering.

## 4 Empirical Results

We analyze the performance of our method and the baselines across a range of target vocabulary sizes, investigating which method achieves a better trade-off between vocabulary size and model quality.

### Vocabulary size and model quality tradeoffs

Figure 3 contrasts our method with the TF and TF-IDF baselines in the SST-2 dataset. It shows that for all methods, increasing the allowed vocabulary size improves the model quality (at the cost of an increased number of parameters or memory).

The figure indicates that both the Banzhaf and Shapley algorithms offer a significantly better tradeoff between vocabulary size and model quality -- they produce a better performing model at all the tested vocabulary sizes (the performance gap is especially pronounced for smaller vocabulary sizes).

Interestingly, the performance of both the Banzhaf and Shapley is very similar. Although they both select words with high marginal contributions, they rely on different power indices. To determine whether they select the same words, we examined the words selected at a target vocabulary size of \(|V^{\prime}|=100\). Figure 2 shows the top words according to the different methods. The top 100

Figure 3: Performance of Shapley (red), Banzhaf (green), TF (blue) and TF-IDF (orange) on AG-News.

Figure 2: Example of top ranked words on the AG-News based on various methods: Shapley (**S**), Banzhaf (**B**), **TF** and **TF-IDF**.

 

words under the Banzhaf and Shapley algorithms intersected on less than 70% of the words, so although they have similar performance, there are non-negligible differences in the words they select.

Figure 3 relates to single sentence classification. Figure 4 shows similar results for the two other types of tasks: pairwise sentence classification and document classification. Similarly to the previous figure, these results indicate that our approach achieves a significantly better tradeoff between vocabulary size and model accuracy. This indicates that our proposed approach offers advantages across a wide set of NLP tasks.

Table 1 shows the performance of an LSTM classifier across all tasks and datasets for the various methods. It shows a consistent improvement over the baselines in all the tasks for both the Banzhaf and Shapley methods (which have very similar performance in all the datasets).

**Comparison with the \(L_{1}\) baseline:** Section 3.2 considered the stronger baseline of ranking by regression coefficients in an \(L_{1}\) regularized logistic regression. The high-level motivation of this baseline is similar to our approach in that words are ranked based on their influence as measured by training a model; however, the \(L_{1}\) method trains a single model (or has a computational cost similar to training one or few models), whereas a power index computation relies on training a _sample_ of models. Figure 5 shows our approach outperforms the \(L_{1}\) baseline.

**Comparison with subword approaches:** Subword embeddings [8] is a recent approach which considers tokens that can be parts of words, resulting in a less sparse vocabulary and having features shared across words. Such approaches are flexible and allow choosing a target vocabulary size. Our approach can also work with subword embeddings: after computing some set of subwords over the vocabulary, we can still filter out less important subwords to improve task performance. We evaluated whether applying our approach on top of using subword embeddings can still lead to improved performance. We first run a byte-pair encoding (BPE) algorithm [9, 6] over each input vocabulary for a dataset. This algorithm operates by merging together the most frequent sequence of adjacent

\begin{table}
\begin{tabular}{l|c c c} \hline \hline
**Task \& Dataset** & **Method** & **Vocab** & **Acc** \\ \hline SST-2 & TF-IDF & 17,539 & 80.2 \\ [2] & Frequency & & 80.3 \\  & Banzhaf & & 81.7 \\  & **Shapley** & & **81.9** \\ \hline COLA & TF-IDF & 9007 & 63.5 \\ [3] & Frequency & & 63.7 \\  & Banzhaf & & 63.9 \\  & **Shapley** & & **64.2** \\ \hline SNLI & TF-IDF & 42,392 & 83.9 \\ [3] & Frequency & & 83.9 \\  & Banzhaf & & 84.1 \\  & **Shapley** & & **84.3** \\ \hline QQP & TF-IDF & 117,303 & 80.8 \\ [3] & Frequency & & 81.2 \\  & **Bankzhaf** & & **81.9** \\  & **Shapley** & & **81.9** \\ \hline \hline AG-NEWS & TF-IDF & 159,697 & 79.6 \\ [3] & Frequency & & 78.5 \\  & Banzahf & & 79.9 \\  & **Shapley** & & **80.2** \\ \hline YELP & TF-IDF & 458,705 & 84.5 \\ [3] & Frequency & & 83.9 \\  & Banzhaf & & 86.7 \\  & **Shapley** & & **87** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance of vocabulary selection methods across datasets and tasks, at a target vocabulary size of \(|V^{\prime}|=750\) words (column 3 is initial vocabulary size). Note performance is lower than state-of-the-art methods, as results are based on a significantly reduced vocabulary size (and using a simple LSTM architecture, with no hyperparameter tuning).

Figure 4: Performance of a Shapley, TF and TF-IDF in additional task structures.

Figure 5: Performance on SST-2 of Shapley (red), **Logistic-L1** (green), TF (blue) and TF-IDF (orange).

 

tokens in each iteration. We do this for a total number of 10,000 merges, resulting in a smaller vocabulary that now composed of subwords. We then apply Shapley, Banzhaf, TF and TF-IDF rankings of these subword tokens, as we have done in the word-level experiments. Figure 6 shows that we have improved performance over the baselines in the subword case as well.

## 5 Discussion

The results in Section 4 show that a game theoretic approach to vocabulary selection can achieve better tradeoffs between the vocabulary size and model performance than heuristics such as TF and TF-IDF based selection, or a method based on regression coefficients in an \(L_{1}\) regularized logistic regression. This advantage comes at the cost of having a significantly higher computational cost of selecting the vocabulary. Following the expensive selection step, we now have the benefit of a smaller model which is more interpretable and explainable, has a reduced memory consumption and potentially less prone to overfitting. We have proposed several ways to mitigate the compute load of selecting the vocabulary: applying a heuristic pre-filtering step and using logisitic regression models rather than the full model while estimating power indices.

## 6 Related Work

We proposed a vocabulary selection method for NLP tasks, using cooperative game theory. We discuss related work on model compression, tailoring the vocabulary in NLP tasks and using subword embeddings, and approximating game theoretic solutions and using them for explainable AI.

**Model compression:** Using the full vocabulary to train models limits the applicability of models in memory-constrained or computation-constrained scenarios Faruqui et al. (2015); Yogatama et al. (2015). Earlier work discusses methods for compressing model size. These yield models that are less expensive in memory and compute, and that are also more easily interpretable. Model compression methods include matrix compression methods such as sparsification of weights in a matrix Wen et al. (2016), Bayesian inference for compression Molchanov et al. (2017); Neklyudov et al. (2017), feature selection methods such as ANOVA Girden (1992), precision reduction methods Han et al. (2015); Hubara et al. (2017) and approximations of the weight matrix Tjandra et al. (2017); Le et al. (2015). Our method relies on game theoretic principles; it filters our vocabulary words, and can thus operate with any NLP architecture (i.e. the method is agnostic to the model architecture used). Further, the interpretability in our case stems from having few features, clearly highlighting the most impactful features in the dataset.

**Vocabulary selection methods and subword and character level embeddings:** earlier work examined selecting a vocabulary for an NLP task. Some alternatives drop out words Chen et al. (2019), whereas character-level methods that attempt to represent the input text at the level of individual characters Kim et al. (2015); Lee et al. (2017); Ling et al. (2015) while subword methods attempt to tokenize words into parts of words in a more efficient way Sennrich et al. (2015); Kudo and Richardson (2018).

**Character level embedding** methods decompose words to allow each individual character to have its own embedding. This reduces the vocabulary size to the number of characters, much smaller than the number of words in the full vocabulary. However, this is not applicable for some character-free languages (e.g. Chinese, Japanese, Korean). Also, such methods have reduced performance, and typically use larger embedding sizes than word embedding models to obtain reasonable quality Zhang et al. (2015); Kim et al. (2015).

In contrast, **subword embeddings** have shown improved performance for several NLP tasks. Such methods typically merge pairs of frequent character sequences, to get a more optimal token vocabulary from an information-theoretic viewpoint. Byte-pair encoding (BPE) algorithms construct subword vo

Figure 6: Performance of Shapley (green), TF (blue) and TF-IDF (orange) when considering **subword embeddings** on the AG-News dataset.

 

cabulary that is less sparse, and increases shared features between words 7, allowing better propogation of semantic meaning. As shown in Section 4, our method can operate on top of subword embeddings, and achieve good tradeoffs between the model size and performance.

Footnote 6: The word “_sadder_” could be split into “_sad_” and “_er_’, where the ending “_er_” has a similar meaning in other circumstances — “_faster_”, “_nearer_” etc.

**Cooperative game theory and applications for explainable AI:** we use concepts from game theory, viewing words as players in a game whose goal is to improve model performance. Such settings have been a key topic of study in game theory since the 1950s Weintraub (Weintraub, 1992). Many solution concepts have been proposed, examining issues such as stability and fairness. Power indices such as the Banzhaf index Banzhaf III (Banzhaf III, 1964) and Shapley value Shapley (1953) to measure the relative impact players have on the outcome of the game. It is computationally hard to calculate them even in simple games Matsui and Matsui (2001); Elkind et al. (2007). We have applied a Monte-Carlo sampling approximation based on existing methods Fatima et al. (2008); Bachrach et al. (2010).

Our use of the Shapley value is akin to recent explainable AI methods, that attempt to allow AI models to provide human readable insights to explain their decisions Adadi and Berrada (2018); Samek et al. (2019). For example, power indices (such as the Shapley value) have been used to explain individual model predictions Datta et al. (2016); Lundberg and Lee (2017), by estimating the contribution of individual features on each prediction. This can be done for linear models Lundberg and Lee (2017) as well as tree-based models Lundberg et al. (2020).

Explainable AI methods typically take a trained model and a given instance as input, and perturb the features of the instance, using the same model to output predictions for many perturbed inputs. In contrast, our goal is not to understand the predictions of a given model, but to select an small input vocabulary set for a task, focusing on the most relevant part of the input space and yielding simpler and more interpretable models. Further, we train many models to estimate contributions, rather than perturbing the inputs for a single model.

## 7 Conclusion

We proposed a vocabulary selection method based on cooperative game theory and empirically showed improvements over baselines in multiple NLP tasks. Our approach, with its task-specific vocabulary, offers an improved model size and quality tradeoffs.

Several questions remain open for future research on better vocabulary selection. Could alternative power indices, apart from what we have shown using the Shapley and Banzhaf indeces, achieve better performance? Is there a way to better combine our methods with subword embeddings? Moreover, given that our method is computationally demanding during vocabulary construction time, an interesting problem is to explore ways to speed up this process; both theoretically, through a different power index calculation, and practically, through better parallelization.

## 8 Acknowledgements

We would like to acknowledge Thore Graepel for formative advice and discussions at early stages of the project. We would also like to thank Richard Everett, Thomas Anthony, Andrea Tacchetti and Tom Eccles for helpful questions and comments throughout the project, and Ellie Pavlick for suggestions on robustly evaluating NLI models. We would also like to acknowledge the anonymous reviewers and area chairs, whose feedback and suggested changes for additional experiments were hugely helpful in making the paper stronger, more robust, and clearer than before.

## References

* Adadi and Berrada (2018) Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: A survey on explainable artificial intelligence (xai). _IEEE Access_, 6:52138-52160.
* Aziz et al. (2009) Haris Aziz, Oded Lachish, Mike Paterson, and Rahul Savani. 2009. Power indices in spanning connectivity games. In _International Conference on Algorithmic Applications in Management_, pages 55-67. Springer.
* Bachrach (2010) Yoram Bachrach. 2010. Honor among thieves: collusion in multi-unit auctions. In _AAMAS_.
* Bachrach et al. (2011) Yoram Bachrach, Edith Elkind, and Piotr Faliszewski. 2011. Coalitional voting manipulation: A game-theoretic perspective. In _IJCAI_ 

Yoram Bachrach, Richard Everett, Edward Hughes, Angeliki Lazaridou, Joel Z Leibo, Marc Lanctot, Michael Johanson, Wojciech M Czarnecki, and Thore Graepel. 2020. Negotiating team formation using deep reinforcement learning. _AIJ_, 288.
* Bachrach et al. (2010) Yoram Bachrach, Evangelos Markakis, Ezra Resnick, Ariel D Procaccia, Jeffrey S Rosenschein, and Amin Saberi. 2010. Approximating power indices: theoretical and empirical analysis. _JAAMAS_, 20(2):105-122.
* Banarse et al. (2019) DJlan Banarse, Yoram Bachrach, Siqi Liu, Guy Lever, Nicolas Heess, Chrisantha Fernando, Pushmeet Kohli, and Thore Graepel. 2019. The body is not a given: Joint agent policy learning and morphology evolution. In _AAMAS_.
* Banzhaf III (1964) John F Banzhaf III. 1964. Weighted voting doesn't work: A mathematical analysis. _Rutgers L Rev._, 19:317.
* Bilbao et al. (2002) Jesus Mario Bilbao, Julio R Fernandez, Nieves Jimenez, and Jorge J Lopez. 2002. Voting power in the european union enlargement. _EJOR_, 143(1):181-196.
* Blocq et al. (2014) Gideon Blockq, Yoram Bachrach, and Peter Key. 2014. The shared assignment game and applications to pricing in cloud computing. In _AAMAS_.
* Bowman et al. (2015a) Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015a. A large annotated corpus for learning natural language inference. _arXiv preprint arXiv:1508.05326_.
* Bowman et al. (2015b) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015b. A large annotated corpus for learning natural language inference. In _EMNLP_.
* Chalkiadakis et al. (2011) Georgios Chalkiadakis, Edith Elkind, and Michael Wooldridge. 2011. Computational aspects of cooperative game theory. _Synthesis Lectures on Artificial Intelligence and Machine Learning_, 5(6):1-168.
* Chen et al. (2019) Wenhu Chen, Yu Su, Yilin Shen, Zhiyu Chen, Xifeng Yan, and William Wang. 2019. How large a vocabulary does text classification need? a variational approach to vocabulary selection. _arXiv preprint arXiv:1902.10339_.
* Datta et al. (2016) Anupam Datta, Shayak Sen, and Yair Zick. 2016. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In _IEEE SP_, pages 598-617.
* Deng and Liu (2018) Li Deng and Yang Liu. 2018. _Deep learning in natural language processing_. Springer.
* Dubey (1975) Pradeep Dubey. 1975. On the uniqueness of the shapley value. _IJGT_, 4(3):131-139.
* Dubey and Shapley (1979) Pradeep Dubey and Lloyd S Shapley. 1979. Mathematical properties of the banzhaf power index. _MOR_, 4(2):99-131.
* Elkind et al. (2007) Edith Elkind, Leslie Ann Goldberg, Paul Goldberg, and Michael Wooldridge. 2007. Computational complexity of weighted threshold games. In _AAAI_.
* Ellis (2010) Paul D Ellis. 2010. _The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results_. Cambridge University Press.
* Faruqui et al. (2015) Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, and Noah Smith. 2015. Sparse overcomplete word vector representations. _arXiv preprint arXiv:1506.02004_.
* Fatima et al. (2008) Shaheen S Fatima, Michael Wooldridge, and Nicholas R Jennings. 2008. A linear approximation method for the shapley value. _AIJ_, 172(14):1673-1699.
* Filmus et al. (2019)Yuval Filmus, Joel Oren, Yair Zick, and Yoram Bachrach. 2019. Analyzing power in weighted voting games with super-increasing weights. _TCS_, 63(1):150-174.
* Friedman et al. (2010) Jerome Friedman, Trevor Hastie, and Rob Tibshirani. 2010. Regularization paths for generalized linear models via coordinate descent. _Journal of statistical software_, 33(1):1.
* Garden (1992) Ellen R Garden. 1992. _ANOVA: Repeated measures_. 84. Sage.
* Han et al. (2015) Song Han, Huizi Mao, and William J Dally. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. _arXiv preprint arXiv:1510.00149_.
* Hubara et al. (2017) Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2017. Quantized neural networks: Training neural networks with low precision weights and activations. _JMLR_, 18(1):6869-6898.
* Kamboj et al. (2011) Sachin Kamboj, Willett Kempton, and Keith S Decker. 2011. Deploying power grid-integrated electric vehicles as a multi-agent system. In _AAMAS_.
* Kim et al. (2015) Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2015. Character-aware neural language models. _arXiv preprint arXiv:1508.06615_.
* Kudo and Richardson (2018) Taku Kudo and John Richardson. 2018. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. _arXiv preprint arXiv:1808.06226_.
* Le et al. (2015) Quoc V Le, Navdeep Jaitly, and Geoffrey E Hinton. 2015. A simple way to initialize recurrent networks of rectified linear units. _arXiv preprint arXiv:1504.00941_.
* Lee et al. (2017) Jason Lee, Kyunghyun Cho, and Thomas Hofmann. 2017. Fully character-level neural machine translation without explicit segmentation. _Transactions of the Association for Computational Linguistics_, 5:365-378 

Gurvan L Hostis, David Grangier, and Michael Auli. 2016. Vocabulary selection strategies for neural machine translation. _arXiv preprint arXiv:1610.00072_.
* Ling et al. (2015) Wang Ling, Isabel Trancoso, Chris Dyer, and Alan W Black. 2015. Character-based neural machine translation. _arXiv preprint arXiv:1511.04586_.
* Lundberg et al. (2020) Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. 2020. From local explanations to global understanding with explainable ai for trees. _Nature machine intelligence_, 2(1):2522-5839.
* Lundberg and Lee (2017) Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In _NIPS_.
* Mash et al. (2017) Moshe Mash, Yoram Bachrach, and Yair Zick. 2017. How to form winning coalitions in mixed human-computer settings. In _IJCAI_.
* Matsui and Matsui (2000) Tomomi Matsui and Yasuko Matsui. 2000. A survey of algorithms for calculating power indices of weighted majority games. _Journal of the Operations Research Society of Japan_, 43(1):71-86.
* Matsui and Matsui (2001) Yasuko Matsui and Tomomi Matsui. 2001. Npcompleteness for calculating power indices of weighted majority games. _TCS_, 263(1-2):305-310.
* Meinshausen and Buhlmann (2010) Nicolai Meinshausen and Peter Buhlmann. 2010. Stability selection. _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_, 72(4):417-473.
* Molchanov et al. (2017) Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov. 2017. Variational dropout sparsifies deep neural networks. _arXiv preprint arXiv:1701.05369_.
* Neklyudov et al. (2017) Kirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha, and Dmitry P Vetrov. 2017. Structured bayesian pruning via log-normal multiplicative noise. In _NIPS_.
* Nimon and Oswald (2013) Kim F Nimon and Frederick L Oswald. 2013. Understanding the results of multiple linear regression: Beyond standardized regression coefficients. _Organizational Research Methods_, 16(4):650-674.
* Provilkov et al. (2019) Ivan Provilkov, Dmitri Emelianenko, and Elena Voita. 2019. Bpe-dropout: Simple and effective subword regularization. _arXiv preprint arXiv:1910.13267_.
* Ramos et al. (2003) Juan Ramos et al. 2003. Using tf-idf to determine word relevance in document queries. In _Instructional conference on machine learning_, pages 133-142.
* Samek et al. (2019) Wojciech Samek, Gregoire Montavon, Andrea Vedaldi, Lars Kai Hansen, and Klaus-Robert Muller. 2019. _Explainable AI: interpreting, explaining and visualizing deep learning_, volume 11700. Springer Nature.
* Sennrich et al. (2015) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. _arXiv preprint arXiv:1508.07909_.
* Shapley (1953) Lloyd S Shapley. 1953. A value for n-person games. _Contributions to the Theory of Games_, 2(28):307-317.
* Shi and Knight (2017) Xing Shi and Kevin Knight. 2017. Speeding up neural machine translation decoding by shrinking run-time vocabulary. In _ACL_.
* Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In _EMNLP_.
* Jr. (1988) Philip D Stratifin Jr. 1988. The shapley--shubik and banzhaf power indices as probabilities. _The Shapley value: essays in honor of Lloyd S. Shapley_, page 71.
* Tjandra et al. (2017) Andros Tjandra, Sakriani Sakti, and Satoshi Nakamura. 2017. Compressing recurrent neural network with tensor train. In _IJCNN_.
* Ueda et al. (2011) Suguru Ueda, Makoto Kitaki, Atsushi Iwasaki, and Makoto Yokoo. 2011. Concise characteristic function representations in coalitional games based on agent types. In _IJCAI_, volume 11, pages 393-399.
* Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. _arXiv preprint arXiv:1804.07461_.
* Warstadt et al. (2019) Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2019. Neural network acceptability judgments. _Transactions of the Association for Computational Linguistics_, 7:625-641.
* Weintraub (1992) E Roy Weintraub. 1992. _Toward a history of game theory_, volume 24. Duke University Press.
* Wen et al. (2016) Wei Wen, Chunpeng Wu, Vandan Wang, Yiran Chen, and Hai Li. 2016. Learning structured sparsity in deep neural networks. _arXiv preprint arXiv:1608.03665_.
* Yogatama et al. (2015) Dani Yogatama, Manaal Faruqui, Chris Dyer, and Noah Smith. 2015. Learning word representations with hierarchical sparse coding. In _ICML_.
* Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In _NIPS_