Enhancing Interpretability Using Human Similarity Judgements to Prune Word Embeddings (2023)

Abstract: Interpretability methods in NLP aim to provide insights into the semantics underlying specific system architectures. Focusing on word embeddings, we present a supervised-learning method that, for a given domain (e.g., sports, professions), identifies a subset of model features that strongly improve prediction of human similarity judgments. We show this method keeps only 20-40% of the original embeddings, for 8 independent semantic domains, and that it retains different feature sets across domains. We then present two approaches for interpreting the semantics of the retained features. The first obtains the scores of the domain words (co-hyponyms) on the first principal component of the retained embeddings, and extracts terms whose co-occurrence with the co-hyponyms tracks these scores’ profile. This analysis reveals that humans differentiate e.g. sports based on how gender-inclusive and international they are. The second approach uses the retained sets as variables in a probing task that predicts values along 65 semantically annotated dimensions for a dataset of 535 words. The features retained for professions are best at predicting cognitive, emotional and social dimensions, whereas features retained for fruits or vegetables best predict the gustation (taste) dimension. We discuss implications for alignment between AI systems and human knowledge.

## introduction the popularity of large language models (llms) such as chatgpt 1 or bloom @xcite has recently prompted an active area of research around the notion of 'alignment', i.e. the ability of nlp models to meet human expectations (see @xcite for a survey). while techniques such as supervised fine-tuning (sft) and reinforcement learning with human feedback (rlhf) 1 https://openai.com/chatgpt have become de facto standards to steer models towards human behaviour, the structural differences that make models in need of alignment are not fully elucidated. why is it that nlp systems organise their knowledge the way they do? and which operations might increase their similarity to human cognition? these questions remain unsolved, not only for llms but also for simple word embedding models such as glove @xcite or word2vec @xcite . alongside the question of alignment, a range of model compression techniques have recently been proposed, including pruning, distillation and quantization @xcite , to increase system efficiency at runtime. many distilled models perform on a par with their larger counterparts @xcite , prompting questions about the nature of semantic encoding in both the original and the compressed architecture. some investigations have focused on the increased (or decreased) fairness of distilled models, in particular their ability to faithfully reproduce reality, with inconclusive results so far @xcite . others have concentrated instead on the correlation between compression and a model's ability to reproduce human behaviour itself @xcite . most importantly, there is evidence that pruned networks develop different internal representations @xcite . in this paper, we bring together the question of alignment and the methodological toolbox given by pruning techniques. the general aim of the following experiments is to understand the semantics of non-contextual word embeddings (glove) by evaluating how those embeddings can be fine-tuned in a way that supports explainability and best predicts human similarity judgments (hsjs) for words in specific categories (i.e., co-hyponyms). the guiding assumption is that for co-hyponyms that belong to a basic-level category @xcite , learning a small subset of relevant features can markedly improve the prediction of hsjs as compared to the use of all features. for example, considering a set of co-hyponyms that all belong to the sports category, it is expected that improved prediction of hsjs between those words can be achieved by identifying a humanly-salient lowdimensional subspace that encodes domain-specific discriminatory properties such as ball game or played in a team. we have three aims in this study. aim 1 is to determine whether pruning improves prediction of hsjs for word-pairs in a study consisting of 8 internal replications (independent datasets). this preliminary step is a necessary prerequisite to support our two subsequent goals, which are focused solely on explainability. aim 2 focuses on quantifying the position of each co-hyponym in the pruned latent space, and then querying the entire vocabulary to identify words whose co-occurrence with cohyponyms tracks those positions. this provides a data-driven description of the latent dimensions underlying the pruned feature-set. aim 3 is to identify the semantics of the pruned sets via a probing task. specifically, we evaluate how well these (sub)sets of features predict a set of human annotations for a set of 535 pre-defined words for which annotations on interpretable features have been collected @xcite . our main contribution is in showing that pruning supervised by hsj is a transparent and effective method to study which human-relevant semantics are contained in word embeddings. ## related work our work builds upon existing research that establishes a connection between human comparison processes and image representations created by deep neural networks (dnns). several prior studies in the area of computer vision have used feature-reweighting @xcite or feature-pruning @xcite to improve alignment between human similarity judgments (hsj) and dnn-generated image representations. extending the approach to language, richie and bhatia (2020) applied a feature reweighting to optimize the prediction of hsj from word-pair embeddings. following @xcite they modeled word-pair similarity as a weighted dot product, using regression, and solving for as many weights as features. while interesting, this procedure has various issues. first, the weights are proportional to feature-products rather than to feature value. this makes the method less interpretable. second, the method operationalizes the assumption that the dnn has learned a meaningful basis set of features that is applicable to all domains, and that the features just need saliency adjustment per domain. third, the data do not lend themselves to downstream analyses as they do not select a subset of features or directly reweight them. in contrast, the view we present for adapting the features to the domain is that there exists a subspace of meaning/features in the dnn whose saliency is already properly calibrated, and what needs to be done is just to identify the relevant/irrelevant features for the domain. pruning and reweighting are therefore two different approaches to understanding latent content. prior work has also used pruning to predict hsj. @xcite showed that when applied to image embeddings extracted from the penultimate layer of vgg-19, pruning markedly improves prediction of out of sample similarity judgments while maintaining around 20% or fewer of the layer's features. however, image and word embeddings are derived in different ways and and it is unclear whether the findings generalize to the text domain. the other work relevant to the current effort is on interpretation of word embeddings. @xcite and utsumi (2020) present a probing method for studying the semantic dimensions latent in word embeddings by constructing a mapping between embedding-vectors and human-rated semantic features. we do the same, but instead of using the entire feature set, use the feature-subsets produced by a supervised-pruning procedure. this is a fundamental departure from prior work as it ultimately probes for semantic features in pruned embedding subsets and in this way highlights the latent dimensions that are important to humans. finally, our effort links up to recent work on alignment, which focuses on elucidating differences between computational systems and human behaviour: recent examples are @xcite , who compare pragmatic phenomena in humans and large language models, or @xcite , who attempt to reproduce human word acquisition by implementing conceptual attribute comparison in their model. of relevance to our approach, @xcite remove spurious correlations between network features via pruning, with the aim to re-duce machine-specific biases in the learned model. their work is however focused on images rather than text. ## conclusion we have shown in this work that supervised pruning applied to a word embedding model can improve prediction of human similarity judgements. thus from an engineering perspective, pruning joins several techniques for improving prediction of human similarity judgments, which include reweighting the embedding space keeping all features @xcite or constructing new embedding spaces from association norms to predict similarity @xcite . the method was shown to select different features across domains, demonstrating high levels of conceptual discrimination. further, the pruned feature sets were interpretable with respect to the first component of a pca analysis, allowing us to describe how humans discriminate between elements of a category -and giving the model itself a tool to justify its semantic 'beliefs' to a potential user. the probing task provided additional, fine-grained information on the semantics in the different pruned feature-sets. given its performance and inherent interpretability, supervised pruning can advance computations related to word-similarity, word-analogy and sentiment analysis, as well as domain adaptation. most importantly, this method allows an ai system to construct a model of human domain knowledge through pruned embeddings, which diverges from the ai system's own internal organization of the domain (i.e., via full embeddings). this alignment can increase the synergy between the ai system and users, establishing a more robust foundation of shared understanding.