The UCD -Net System at S em E val-2020 Task 1: Temporal Referencing with Semantic Network Distances (2020)

Abstract: This paper describes the UCD system entered for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection. We propose a novel method based on distance between temporally referenced nodes in a semantic network constructed from a combination of the time specific corpora. We argue for the value of semantic networks as objects for transparent exploratory analysis and visualisation of lexical semantic change, and present an implementation of a web application for the purpose of searching and visualising semantic networks. The results of the change measure used for this task were not among the best performing systems, but further calibration of the distance metric and backoff approaches may improve this method.

## introduction semeval 2020 task 1: unsupervised lexical semantic change detection addresses the problem of identifying lexical semantic change using corpus pairs from different time periods in english, swedish, german, and latin @xcite . measuring and characterising lexical semantic change is of interest to linguists as well as to researchers in the social sciences and humanities who aim to compare shifts in lexical and conceptual meaning across time periods. in common with word sense disambiguation tasks, the evaluation of systems which aim to measure lexical semantic change is complicated by the lack of obvious concrete objectives and natural sources of ground truth labels. semeval 2020 task 1 aims to address this by providing a human-labelled ground truth dataset across multiple languages, indicating the loss and/or gain of word senses, and the overall degree to which each word has changed meaning between the two time periods covered by the corpora. many previous approaches to this task have been based on constructing vector space representations of target words in each of the two corpora, and measuring the difference between these representations @xcite . the system described in this paper differs from these approaches by its use of semantic networks to represent the meanings of words in the corpora. a semantic network in this context refers to a graph, with nodes representing words connected by edges that represent association strengths between them. such networks have often been used in psychology and cognitive science as models for human lexical or conceptual knowledge @xcite . the construction of an overall semantic network to represent the structure of meaning across the corpora from both time periods provides a transparent object for exploration and visualisation of the conceptual structure that target words are located in. the system described adopts an approach inspired by temporal referencing @xcite , treating the time-specific earlier and later corpora as a single large corpus, constructing a single semantic network from co-occurrences derived from this overall corpus, but replacing target words (and only target words) with time-specific tokens. the change in meaning of the target words can then be measured using node distance metrics in the overall network. the system worked best on english data, ranking joint fifth and eighth of submitted systems for english. in both tasks, the system performed worst on the latin and german. averaged across all four languages, the system placed overall 14th out of 26 systems for which a paper was submitted at the task of ranking target words according to their degree of semantic change. in the binary task of predicting whether or not a target word had gained or lost a sense, the average performance of the system underperformed other systems due to incorrectly backing off to predicting change in many cases where the temporally referenced target nodes were not connected in the network -this was usually the result of sparsity in the network (particularly in latin) rather than actual semantic change. overall, visualisation of the constructed semantic networks indicates that qualitatively important changes in target word associations are discovered, but calibrating node distance measures is particularly difficult in the binary task. this system description paper outlines the methodology for constructing a semantic network from sentences in the dataset, the network distance metric used, and presents an analysis of the strengths and weaknesses of this approach. in addition, we present an interactive web application for exploring the semantic network and visualising the differences between the target words according to their place in the network. this application is available online and may be useful to other researchers in order to explore the different associations of target word senses across the two time periods in a qualitative manner. these subtasks are evaluated independently across the four languages involved. the ground truth labels were assigned according to the procedure described in @xcite . annotators were presented with pairs of sentences containing a usage of the target word in the earlier and the later corpus, and asked to score the relatedness of the two usages on a four point scale. changes in the sense frequency distribution are used as ground truth for the ranking task, and the changes in attestation of senses above a threshold are used for the binary task @xcite . the task is totally unsupervised: neither the ground truth labels nor the class distributions were available prior to system submission, and the number of submissions per system is limited to ten. a different method of annotation was used for latin, due to the lack of native speakers. annotators with good knowledge of latin were recruited for the task, and a set of words known to have changed meaning from pre-christian to the christian era were used alongside control words that did not change. the annotators referenced their judgements to a entries in a sense definition dictionary. ## visualisation one of the main motivations of this system is to illustrate an application of interactive semantic network visualisation as a method for exploratory and qualitative analysis of subtle lexical semantic change. the visualisation is implemented as an r shiny web application. the figures in this section show screenshots from the network representations displayed by the web app. these are neighbourhood or ego graphs of order one, that is, they show nodes within one step of the nodes entered in the search field. the colours are determined by a community detection method that can be selected by the user, but the communities were not used in the change-scoring process. in the interactive application (figure 2 ), the ppmi determining whether an edge exists between two nodes can be changed to increase or decrease the sparsity of the network drawn. a rank threshold can also be applied, whereby edges only exist between the top scoring n words associated with each other word. the network is drawn using the r visnetwork package, using a force-directed algorithm @xcite , which models the network mechanically as repelling particles connected by springs. the result is that in a graph of suitable density and degree, nodes are spaced apart enough to be distinguished, but the edges pull together nodes into clusters that share many relations. the visnetwork package implements a drag, pan, and zoom enabled central widget, and this is combined with input boxes for search terms and sliders for setting the order of the neighbourhood graph, the maximum node degree, and the association score threshold. network visualisations are often of limited use in static materials -if a network is drawn with enough nodes and density to show important overall patterns, the labelling of the nodes becomes too dense to easily visualise. interactive visualisation provides a responsive and scalable alternative. the system can be accessed online at: https:// concept-lab.lib.cam.ac.uk/shiny/viewers/viewer-langchange/ , with documentation at https://concept-lab.lib.cam.ac.uk/viewerdocs/october-docs.html ## conclusion this system implements a semantic network temporal referencing approach to measuring lexical semantic change from unsupervised data. we present methods for constructing the semantic network, and results from using the resistance distance metric to measure the distance between time-specific target words in the overall semantic network. we also advocate for interactive semantic network exploration as a method to qualitatively analyse subtle distinctions in conceptual meaning that are important for questions usually addressed by close reading, but do not consist in clear changes in word sense distributions.