SCDT our: Embedding Axis Ordering and Merging for Interpretable Semantic Change Detection (2025)

Abstract: In Semantic Change Detection (SCD), it is a common problem to obtain embeddings that are both interpretable and high-performing. However, improving interpretability often leads to a loss in the SCD performance, and vice versa. To address this problem, we propose SCDTour, a method that orders and merges interpretable axes to alleviate the performance degradation of SCD. SCDTour considers both (a) semantic similarity between axes in the embedding space, as well as (b) the degree to which each axis contributes to semantic change. Experimental results show that SCDTour preserves performance in semantic change detection while maintaining high interpretability. Moreover, agglomerating the sorted axes produces a more refined set of word senses, which achieves comparable or improved performance against the original full-dimensional embeddings in the SCD task. These findings demonstrate that SCDTour effectively balances interpretability and SCD performance, enabling meaningful interpretation of semantic shifts through a small number of refined axes.

## introduction the meanings of words shift over time due to changes in culture, society, and contexts. scd is the task of detecting these changes automatically, which plays a vital role in aiding linguistic analysis @xcite . additionally, it also contributes to the efficient additional training of masked language models (mlms) by identifying words whose meanings have changed over time @xcite . a wide range of methods have been proposed using static @xcite @xcite and contextualised word embeddings @xcite @xcite dim. sorted perf. int. recently, interpretability has emerged as a key focus in scd. while earlier approaches prioritised improving accuracy using contextualised or static word embeddings, recent work has focused on transparency by generating definitions @xcite , building usage graphs @xcite , analysing embedding space structure @xcite , or leveraging external knowledge @xcite @xcite . however, a key challenge remains: improving interpretability often leads to reduced performance, and vice versa @xcite . this trade-off limits practical applications that demand both reliable predictions and interpretable explanations. 2 we address this issue by utilising interpretable embeddings whose axes are obtained via independent component analysis (ica). ica has been used to derive interpretable axes in word embeddings that encode meaning-specific information @xcite . we propose scd-tour, an interpretable axis-sorting method that ## method we propose scdtour, which introduces a changespecific weight to account for the contribution of each axis to scd. unlike wordtour (sato, 2022), which sorts words based on pairwise similarity to obtain one-dimensional embeddings, and ax-istour @xcite , which aligns ica axes and merges similar ones for better compression, our method incorporates change-specific signals to reorder and merge topic-like axes, thereby achieving both interpretability and scd performance. building on this idea, axistour @xcite proposed to sort ica-transformed axes @xmath4: to extend this approach for semantic change detection (scd), we introduce an additional criterion: the change-specific weight, which evaluates how much each axis contributes to the performance on the scd task. formally, we define it as: it quantifies the drop in performance e when the i-th axis is removed. our proposed method, scdtour, combines both criteria to produce a more informative and scd-relevant axis ordering: ## conclusion we presented scdtour, a method that orders and merges interpretable axes using meaning-and change-specific weights. it maintains interpretability while preserving scd performance, even in low-dimensional settings. in future work, we plan to extend the comparison to contextualised embeddings and multilingual scd tasks, in order to more broadly evaluate the trade-offs between interpretability and performance.