Word Sense Disambiguation of F rench Lexicographical Examples Using Lexical Networks (2022)

Abstract: This paper focuses on the task of word sense disambiguation (WSD) on lexicographic examples relying on the French Lexical Network (fr-LN). For this purpose, we exploit the lexical and relational properties of the network, that we integrated in a feedforward neural WSD model on top of pretrained French BERT embeddings. We provide a comparative study with various models and further show the impact of our approach regarding polysemic units.

## introduction word sense disambiguation is a long-standing research field in nlp investigating supervised, unsupervised, knowledge-based and mixed approaches @xcite . lexical resources have always played a crucial role not only serving as sense inventories, but also as sources of information to help the disambiguation process (a.o. @xcite ). in particular, the structure and lexical content of lexical networks have been successfully exploited for this task with graph-based algorithms (a.o. @xcite ). with the deep learning revolution, supervised approaches relying on neural networks and pretrained word embeddings have quickly gained popularity. in such framework, wsd is often seen as a token classification task, where tokens are assigned a sense label among an exist set of senses. classical supervised models are built on a multilayer perceptron (mlp) for predicting a sense label for the target tokens @xcite and lately the use of pretrained contextualized word embedding has become standard (ex. @xcite ). such supervised systems are dependent on senseannotated datasets that tend to have limited coverage due to the manual annotation cost. furthermore, in these systems, rare senses are often disadvantaged towards more frequent ones. to tackle this problem, more and more research works propose approaches integrating lexical network knowl-edge to such models. several strategies have been proposed: either integrating lexical knowledgee.g. glosses @xcite -, or integrating structural properties -e.g. use of graph-based algorithms such as personalized pagerank @xcite , use of hyperonym/hyponym/synonym relations in a lexical network to compress the sense tagset and then make the labeling task easier @xcite -. other models such as ewise @xcite and ewiser @xcite ) enhance the wsd system with explicit and implicit knowledge using graph structure information from lexical knowledge networks and existing sense embeddings. in this paper, we are interested in adapting the ewiser model to specific lexical data: the data from the french lexical network (fr-ln, polguère (2014)) and its derived database of lexicographical usage examples (dble-ln-fr). in particular, we exploited the linguistic richness of its relation types, by integrating trainable weighted relations. our system gets better or comparable results than the original system. this paper is organized as follows. section 2 presents our dataset and its particularities. section 3 introduces the model and its adaptations. sections 4 and 5 are respectively devoted to introducing the experimental setup and discussing and comparing the results. lexical networks used as lexical knowledge in nlp are generally variants of wordnet @xcite . in this paper, we rely on the french lexical network fr-ln foot_0 , which is under construction. it is based on the model of lexical systems @xcite and is in line with the research projects conducted in the framework of explanatory and combinatorial lexicology @xcite . it contains among others syntagmatic, paradigmatic, copolysemic and phraseological relations. the complete fr-ln contains 29,220 word senses and 80,036 relations between them. in this paper, we focus only on the 62,641 paradigmatic and syntagmatic links, which are standardized using the system of 686 distinct meaning-text lexical functions (lfs) @xcite . ## a model integrating graph knowledge the proposed model is a variant of ewiser @xcite ) that we adapted using some specific features of fr-ln, namely the richness of its relation types, and the semantic weights associated to relations (cf. section 2). ewiser can be seen as a token classification system. it takes as input a sequence of words that feeds a bert layer. for each target word, a feedforward module is then applied to predict its sense label given the input sequence. the exact modelling is depicted by the equation 1 taken and derived directly from the original paper 5 . in the above equation, b corresponds to the sum of the last four bert hidden layer, which is given as input to a 2-layer feedforward to compute the logits h 1 . this output is encoded with graph information from the lexical network using a which is the corresponding adjacency matrix. each node corresponds to a possible word sense in the training dataset. in the original ewiser paper, matrix a encodes hypernym and hyponyms relations from wordnet, whereas in our case it encodes paradigmatic and syntagmatic relations from fr-ln. the parameters of a may be frozen or trainable @xcite . in this paper, we use two strategies to compute the elements a i,j of a relying on some features of fr-ln. every node pair (i, j) have a set s i,j of present relations between i and j. each relation r has a weight w(r), and a i,j is the sum of the weights of the relations between i and j: a i,@xmath0,j w(r). we consider two weighing schemes for every relation r: (1) w(r) = 1, the element a i,j being the cardinality of s i,j [struct]; (2) w(r) = s r + 1 where s @xmath1, 1, 2 is the semantic weight of r, a i,j determining to what extent the semantic content of i is included in the one of j [sem]. the struct strategy is taken from @xcite , whereas sem is a contribution of this paper. for each weighting scheme, we experimented three settings: (a) the element a i,j is frozen, (b) a i,j is trainable, (c) w(r) is trainable, the weight of each relation being learnt from the training dataset. the setting (c) is a proposal of this paper, whereas (a) and (b) are taken from @xcite . ## results and discussion to evaluate our models, we used the accuracy of the system predictions, i.e. the percentage of correct predictions. the system was preliminary tuned on the dev dataset. the mlp-baseline obtained better performances using the camembert embeddings, whereas the barycenter performances were better using flaubert. ## conclusion we presented a preliminary study of various word sense disambiguation systems on the french dataset, dble-ln-fr-v2. we proposed a weighted training model in order to incorporate the richness of lexical and semantic information from the fr-ln network effectively and showed comparable performance to state of the art systems. a first path of future research would be to enhance the scarcity of a matrix: e.g. adding neighbors of various pos, or including transitive clo-sures of relations. we would like to explore the incorporation of sense embeddings using various graph representation learning algorithms. furthermore, we would like to experiment tagset compression like in @xcite .