E mb L ex C hange at S em E val-2020 Task 1: Unsupervised Embedding-based Detection of Lexical Semantic Changes (2020)

Abstract: This paper describes EmbLexChange, a system introduced by the “Life-Language” team for SemEval-2020 Task 1, on unsupervised detection of lexical-semantic changes. EmbLexChange is defined as the divergence between the embedding based profiles of word w (calculated with respect to a set of reference words) in the source and the target domains (source and target domains can be simply two time frames t_1 and t_2). The underlying assumption is that the lexical-semantic change of word w would affect its co-occurring words and subsequently alters the neighborhoods in the embedding spaces. We show that using a resampling framework for the selection of reference words (with conserved senses), we can more reliably detect lexical-semantic changes in English, German, Swedish, and Latin. EmbLexChange achieved second place in the binary detection of semantic changes in the SemEval-2020.

## introduction semeval 2020 task 1 is defined on the unsupervised detection of word sense changes over time in german, english, swedish, and latin. in particular, this challenge focused on detection and quantification of the sense changes of word w in the transition from the time period t 1 to the time period t 2 in the above mentioned four languages, where the input for each language are the text corpora dating from t 1 and t 2 . this challenge involved two subtasks: in order to evaluate the two subtasks, the participating systems are evaluated against a ground truth corpus annotated by native speakers or scholars of the respective languages @xcite . human languages constantly change due to cultural, technological, and social drift. lexical semantic changes of human languages can materialize in the form of introducing/borrowing new words, or for the existing words can involve acquiring/losing some word senses @xcite . computational methods for automated detection of semantic changes can be extremely helpful in the study of historical texts or corpora spanning a very long period of time, e.g., semantic analysis of 1000 years of poetry @xcite , or in the design of the ocr algorithm for text digitization, or in designing an information retrieval system incorporating the semantic changes @xcite . applications in the study of historical texts aside, the proposed methods detect lexical-semantic drift also in the same time period for different domains. this can be useful for compiling glossaries and specific training material in certain industries where new senses are introduced for words as compared to their standard usage e.g. to facilitate a more efficient training for new employees. in the past decade, a variety of methods were introduced in the literature for automatic detection of lexical-semantic changes @xcite , where we only can refer to a subset of work, including but not limited to (i) co-occurrence-based methods @xcite , (ii) embeddingbased approaches @xcite @xcite @xcite @xcite , and (iii) topic-models-based @xcite approaches. in this paper, we extend our recently introduced domdrift embeddingbased approach for the detection of semantic changes @xcite introduced for the extension of the computational analyses on 1000+ languages @xcite . similar our earlier work on weld @xcite and similar to @xcite , domdrift works based on a comparison of relative distances of words in the embedding spaces of the source and the target domains. to increase the stability of detection in domdrift, we extend domdrift to emblexchange by the following modifications: (i) instead of creating word profiles against all common words between the source and target domain, we use only a subset of pivot words, which are frequent words with unchanged relative frequencies. (ii) we create multiple word profiles by resampling from a set of pivot words. we show that the emblexchange can reliably detect the lexical-semantic changes in english, german, swedish, and latin achieving an average accuracy of 0.686 as second best system of the competition where the first place system achieved an accuracy of 0.687. ## system overview here we detail the steps of the emblexchange system, where the overview is depicted in figure 1 . the emblexchange framework is developed based on the following assumptions: h1: frequent words change at slower rates @xcite . h2: the relative frequency of unchanged words is not dramatically different in different time periods/domains. h3: changes of the word sense are reflected in the context, which are captured by the embedding model resulting in changes of the neighbors in the embedding space. thus, the relative drift of a query word (a word which we target to investigate its lexical semantic change) with respect to unchanged words in the embedding space can characterize the lexical-semantic change. ## data the dataset used in this shared task includes corpora of english, german, latin, and swedish texts. for each language, the text corpora of two time periods are given. more details on the exact time frames and data sizes are provided in ## experiment the goal of semeval task 1 is to detect the words with a change in their semantics in the transition from the time period t 1 to the time period t 2 in english, german, swedish, and latin languages. we closely follow the steps described in §2. ## results the results of emblexchange in the detection and the quantification of changes in the lexical semantics in english, german, swedish, and latin are provided in ranking: the kendall-τ p-values for english, german, and swedish show that there is a significant correspondence between the emblexchange scores and the ground truth scores in those languages. the spearman rank-order correlation coefficient is also calculated for all languages, with an average of 0.357 over the four languages. the case of latin has been more challenging in both binary and graded prediction of lexical semantic change. ## discussion and conclusions in this paper, we proposed emblexchange, a framework for the detection of lexical semantic changes in an unsupervised manner. we defined emblexchange as the divergence between the embedding-based profiles of word w (calculated for a set of pivot words) in the source and the target domains (e.g. between two historical time-frames). with the selection of pivot words by a resampling framework, we raise the reliability of this divergence. the underlying assumption of our method is that the changes in lexical semantics of word w would affect its co-occurring words and subsequently alters the neighborhoods in the embedding spaces. we showed that emblexchange can reliably detect lexical-semantic changes in english, german, swedish, and latin achieving the second place in the binary detection of semantic changes in the semeval-2020. the detection of semantic changes in latin has been more challenging than for other languages. one reason behind this can be the imbalance of embedding training instances for latin t 1 and latin t 2 as well as the overall smaller corpora for latin in comparison to the other languages (shown in the semeval overall results show that emblexchange works better in the binary detection of semantic changes versus its performance in the ranking problem setting @xcite . however, we should note that the manual creation of ranking ground truth is a much more challenging task than the creation of binary classification ground truth. thus, we believe that the classification results might be more reliable than the ones for the ranking. the emblexchange requires only the raw texts in the time-frames/domains of interest. then the semantic changes can be detected based on the divergence between the embedding-based profiles of words of source and target domains. one advantage of using an embedding-based profile is that by increasing the window size in the embedding training we can move from syntactic changes toward semantic changes which can be investigated in more depth as a future direction of research.