Disentangling Linguistic Features with Dimension-Wise Analysis of Vector Embeddings (2025)

Abstract: Understanding the inner workings of neural embeddings, particularly in models such as BERT, remains a challenge because of their high-dimensional and opaque nature. This paper proposes a framework for uncovering the specific dimensions of vector embeddings that encode distinct linguistic properties (LPs). We introduce the Linguistically Distinct Sentence Pairs (LDSP-10) dataset, which isolates ten key linguistic features such as synonymy, negation, tense, and quantity. Using this dataset, we analyze BERT embeddings with various methods, including the Wilcoxon signed-rank test, mutual information, and recursive feature elimination, to identify the most influential dimensions for each LP. We introduce a new metric, the Embedding Dimension Impact (EDI) score, which quantifies the relevance of each embedding dimension to a LP. Our findings show that certain properties, such as negation and polarity, are robustly encoded in specific dimensions, while others, like synonymy, exhibit more complex patterns. This study provides insights into the interpretability of embeddings, which can guide the development of more transparent and optimized language models, with implications for model bias mitigation and the responsible deployment of AI systems.

## introduction word embeddings are central to natural language processing (nlp), enabling machines to represent and interpret text in continuous vector spaces. from early models like word2vec @xcite and glove @xcite , to advanced models like gpt-2 @xcite and bert @xcite , embeddings have evolved to capture complex linguistic nuances. bert, in particular, leverages bidirectional trans1 our code is available at https://github.com/ realnav1234/ldsp_embeddings . formers to generate contextualized word representations, enhancing syntactic and semantic understanding @xcite . despite these advancements, embeddings are often seen as "black boxes," where the highdimensional nature of the spaces they occupy makes interpretation difficult @xcite . the field of interpretable embeddings seeks to address these challenges by making the dimensions of embeddings more transparent and meaningful @xcite @xcite . however, most systems still rely on popular embedding models like gpt, bert, word2vec, and glove, which prioritize performance over interpretability @xcite . our research introduces a generalizable framework for identifying specific embedding dimensions in models like bert and gpt-2 that encode distinct lps. this work responds to the growing need for interpretable models, especially for tasks like bias mitigation @xcite , task-specific optimization @xcite , and 2) accuracy for different lps across bert, gpt-2, and mp-net. a simple logistic classifier is able to perform at these levels of accuracy on the highest edi subset of dimensions of embeddings from each of these models. this paper makes three contributions. first, is the introduction of the ldsp-10 dataset, consisting of sentence pairs that isolate nine lps. second is a generalizable framework and quantifiable metric (edi score) for identifying influential embedding dimensions, applicable to different models and linguistic features. third is a comprehensive analysis of bert, gpt-2, and mpnet embeddings, revealing key dimensions related to each lp. ## related works research on interpretable embeddings can be divided into two categories: interpretable embeddings and representation analysis. the former focuses on designing models that naturally produce interpretable representations, while the latter involves post-hoc analysis to uncover how existing embeddings encode human-interpretable features. ## linguistically distinct sentence pairs (ldsp-10) dataset we curated a dataset of 1000 ldsps for each of the 10 lps we wanted to investigate. to generate the dataset, we used google's gemini-1.5-flash model api due to its reliability and cost-efficiency, while being able to produce consistent outputs across a variety of linguistic contexts. we prompted the model with a description of the lp and a set of reference ldsps as few-shot examples to ensure high-quality outputs. these outputs were generated in batches of 100 ldsps at a time. to ensure reproducibility and transparency, the detailed prompts used to generate the dataset are provided in appendix a. during the dataset creation process, we found that the order of the sentences in each pair was not always consistent, which a key invariant central to the correctness of our methods. we add carefully crafted instructions to the prompt to explicitly enforce the correct ordering. manual validation was conducted to assess the quality of the generated data. the evaluation revealed that more than 99% of the sampled sentence pairs adhered to our expectations: (1) minimal distinctions and (2) consistent ordering. the system exhibited a low rate (<1%) of syntactic or content biases, with errors occurring primarily in cases involving more complex property sentence pair control they sound excited. the farmer has 20 sheep. ## dimension-wise embedding analysis for each sentence in the ldsp-10 dataset, we use the final hidden layer's output of three distinct models (bert, gpt-2, and mpnet) and use meanpooling over the tokens to compute sentence embeddings. the framework outlined in this section is generalizable to any model, layer, or pooling mechanism. ## results in this section, we focus on bert embeddings as a case study for applying our framework. we show visualizations for control, negation, and intensifier, but all other lps and related tables & plots can be found in appendix . the results for gpt-2 and mpnet were similar, and can be reviewed in detail in appendix e and appendix f. ## discussion the results of this study provide a clear demonstration of the ability to extract specific lps within high-dimensional embeddings. our analysis shows that certain lps are robustly encoded in distinct embedding dimensions, as evidenced by high embedding dimension importance (edi) scores and agreement across multiple analytical methods. these methods were chosen after rigorous experimentation, where principal component analysis, simple logistic regression, and other methods were rejected due to their inability to capture the nuanced, non-linear information encoded in these embeddings. negation yielded one of the the highest maximum edi scores and a significant number of dimensions with high interpretability. this supports the notion that negation is a well-structured and salient linguistic feature in bert embeddings. in contrast, some properties exhibited minimal evidence of dimension-specific encoding, which we hypothesize to be due to a lack of a binary or clear-cut way of encoding these relationships. synonymy showed low maximum edi scores and inconsistent results across our methods. synonym pairs in our dataset could be permuted without affecting the consistency of the data, and 0-1 labels for our classifiers and mutual information were meaningless; therefore, our methods are unable to extract the dimensional distribution of synonym encodings. in summary, this study underscores the heterogeneous nature of linguistic encoding in bert embeddings, with some properties exhibiting clear, interpretable patterns while others remain elusive. the proposed edi score and analytical framework provide valuable tools for advancing the interpretability of embeddings, with implications for bias mitigation, model optimization, and the broader goal of responsible ai deployment. ## limitations while our study provides insight into the interpretability of embedding dimensions, it is constrained due to data availability. generating highquality ldsps with llm-based tools is difficult, as ensuring diversity, minimal redundancy, and high linguistic quality becomes significantly more difficult with more data generated. overly simplistic, repetitive outputs are difficult to avoid, despite careful prompt engineering. additionally, we limit our experiments to small open-source models due to compute and credit constraints, but analysis on larger, newer, and more widely-used models could solidify our generalizability claim and provide valuable insights. future work may analyze edi scores across representations at different layers to understand how information about specific lps propagate through the network. while we hypothesize that our method can isolate dimensions responsible for encoding gender or other characteristics that may not be necessarily informative to the specified task and can introduce biases, more experiments and analysis are needed in order to validate this. to this end, future work may conduct evaluations using downstream task accuracy and counterfactual measures, such as mean-ablating high-edi dimensions to observe information loss or making edi-informed modifications to dimensions. guoping zeng. 2015. a unified definition of mutual information with applications in machine learning. mathematical problems in engineering, 2015. xiangyan zeng, yen-wei chen, and caixia tao. 2009. feature selection using recursive feature elimination for handwritten digit recognition. in 2009 fifth international conference on intelligent information hiding and multimedia signal processing, pages 1205-1208. henghui zhu, ioannis ch. paschalidis, and amir tahmasebi. 2018. clinical concept extraction with contextual word embedding. preprint, arxiv:1810.10566.