AMuSE-WSD : A n All-in-one Multilingual System for Easy W ord S ense D isambiguation (2021)

Abstract: Over the past few years, Word Sense Disambiguation (WSD) has received renewed interest: recently proposed systems have shown the remarkable effectiveness of deep learning techniques in this task, especially when aided by modern pretrained language models. Unfortunately, such systems are still not available as ready-to-use end-to-end packages, making it difficult for researchers to take advantage of their performance. The only alternative for a user interested in applying WSD to downstream tasks is to rely on currently available end-to-end WSD systems, which, however, still rely on graph-based heuristics or non-neural machine learning algorithms. In this paper, we fill this gap and propose AMuSE-WSD, the first end-to-end system to offer high-quality sense information in 40 languages through a state-of-the-art neural model for WSD. We hope that AMuSE-WSD will provide a stepping stone for the integration of meaning into real-world applications and encourage further studies in lexical semantics. AMuSE-WSD is available online at http://nlp.uniroma1.it/amuse-wsd .

## introduction word sense disambiguation (wsd) is the task of associating a word in context with its most appropriate sense from a predefined sense inventory @xcite . learning the meaning of a word in context is often considered to be a fundamental step in enabling machine understanding of text @xcite : indeed, a word can be polysemous, that is, it can refer to different meanings depending on the context. over the past few years, wsd has received growing attention and has been proven to be useful in an increasing range of applications, such as machine translation @xcite @xcite , information extraction @xcite , information retrieval @xcite ), text categorization @xcite and question answering @xcite . wsd approaches usually fall into two categories: knowledge-based @xcite @xcite , which leverage computational lexicons, and supervised @xcite @xcite @xcite , which train machine learning models on sense-annotated data. while early work mainly belongs to the former category @xcite , recent studies have shown the superiority in performance of the latter category, especially thanks to complex neural networks @xcite . however, as wsd systems become more and more reliant on increasingly complex input representations -pretrained language models are becoming the de facto representation method in several nlp tasks -and involve neural architectures, the entry requirements for end users have also become higher. indeed, the implementation of state-of-the-art wsd systems is frequently anything but straightforward. what is more, such systems are often not ready to be used off-the-shelf and require additional preprocessing and postprocessing modules for document splitting, tokenization, lemmatization and part-of-speech tagging. these complications may make the use of recent high-performing wsd systems unattractive, or even out of reach, for researchers who want to take advantage of explicit semantic information in other areas of research, but who are not experts in wsd. in order to make wsd more accessible, there have been several attempts at providing ready-touse wsd systems that can be easily integrated into other systems @xcite @xcite . nevertheless, current ready-to-use wsd systems are either english-only or based on approaches that now lag behind state-of-the-art mod-els in terms of performance. in this paper, we fill the gap and present amuse-wsd, an easy-to-use, off-the-shelf wsd package that provides sense annotations in multiple languages through a state-of-the-art neural-based model. the main features of amuse-wsd can be summarized as follows: we hope that amuse-wsd will facilitate the integration of semantic information into tasks that may benefit from high-quality sense information, enabling the exploitation of wsd in multiple languages. foot_0 ## system description amuse-wsd is the first all-in-one multilingual system for wsd based on a state-of-the-art neural model. our system encapsulates this model in a pipeline which provides word-level semantic information in an end-to-end fashion so that a user is not required to provide anything more than raw text as input. in this section, we provide a description of the various components of our model, focusing on its preprocessing pipeline (section 2.1) and our wsd system (section 2.2), which is the core of amuse-wsd. furthermore, we provide an overview of its performance on several standard benchmarks for wsd (section 2.3). ## amuse-wsd api amuse-wsd can easily be used to integrate sense information into downstream tasks. indeed, amuse-wsd is a simple solution for out-of-thebox sense disambiguation as it offers a restful api that provides access to a full end-to-end stateof-the-art pretrained model for multilingual wsd. the main advantage of our system is that it is fully self-contained as it takes care of the preprocessing information usually needed by a wsd system, namely, document splitting, tokenization, lemmatization and part-of-speech tagging (see section 2.1). this means that amuse-wsd takes just raw text as input, freeing the user from the burden of having to compose complex preprocessing pipelines. furthermore, our system is optimized to speed up inference, especially on cpu, which makes amuse-wsd accessible to a broader audience with smaller hardware budgets. finally, the api is available both as a web api, so as to obtain sense annotations without installing any software, and offline api, so that a user can host amuse-wsd locally and minimize inference times. in the following, we provide more details on the main set of functionalities offered by amuse-wsd. document-level preprocessing. the amuse-wsd api lets the user obtain disambiguated text for batches of sentences in a single query to the service, reducing the load on the network and the latency of the responses. on top of this, one common use case for end users is the disambiguation of long texts. in order to assist users in such cases, amuse-wsd supports the disambiguation of documents of arbitrary length, performing document splitting to enable faster inference by transparently batching text segments. sentence-level preprocessing. the only piece of information the end user needs to provide to amuse-wsd is raw text. nonetheless, the underlying model needs lemmas and part-of-speech tags in order to select the proper sense for a word (see section 2.1). to this end, amuse-wsd integrates a preprocessing pipeline, transparent to the user, that performs tokenization, lemmatization and part-of-speech tagging. in the search for the optimal compromise between latency and accuracy of the preprocessing, amuse-wsd employs two different preprocessing pipelines depending on the language of the input document, built around spacy and stanza. for each of the 40 supported languages, our system selects one of the preprocessing models available in either spacy or stanza, prioritizing the former for its high inference speed and falling back to stanza for lower-resource languages where the performance of the former is suboptimal. usage. the amuse-wsd api exposes an endpoint named /api/model. the endpoint accepts post requests with a json body, containing a list of documents. for each document, two parameters must be specified: each request returns a json response containing a list of objects, one for each input document. each object in the response provides the tokenization, lemmatization, pos-tagging and sense information of the corresponding input document. the amuse-wsd api returns labels that are easily usable for users interested in wordnet, babelnet and the nltk wordnet api, making it simple to switch from one resource/tool to another. for example, the response to a request that contains only one sentence will have the following structure: we refer to the online documentation for more details, including the full list of supported languages. 2 ## web interface amuse-wsd comes with a web interface which allows users to disambiguate text on the fly without having to write a single line of code. figure 1 shows the home page of amuse-wsd in which a user can type a short text (up to 3000 characters) and indicate the language of the inserted text. figure 2 shows, instead, how the web interface displays the senses for the input sentence "the quick brown fox jumps over the lazy dog", while figures 3 and 4 show a correctly disambiguated word, that is, bank, which is ambiguous both in english and italian. for each content word (nouns, adjectives, adverbs and verbs), the interface shows the corresponding sense predicted by the underlying wsd model, including its definition from wordnet and an image, if available. each meaning is linked the dropdown menu allows users to select the language of the input text from among the 10 available in the online interface. to its corresponding web page in babelnet 5, foot_4 a multilingual encyclopedic dictionary that includes wordnet, open multilingual wordnet, wikipedia, wikidata and wiktionary, inter alia. users can click on a meaning to obtain further information about it, ranging from other definitions to its translation in other languages, from its hypernyms (generalizations) to its hyponyms (specializations). we believe that this interface will be especially useful for researchers, teachers, students and curious people who may be interested in understanding how wsd can be taken advantage of in other fields. moreover, we also believe that an easy-to-use interface will attract the attention of more researchers on the task of wsd itself, encouraging future developments in this area. ## conclusion over the past few years, wsd has witnessed an increasing rate of development, especially thanks to the renewed interest in neural networks and the advent of modern pretrained language models. however, research implementations are often far from being ready-to-use as they either take as input pre-parsed documents or require setting up a preprocessing pipeline to take care, at least, of document splitting, tokenization, lemmatization and pos tagging. unfortunately, currently available ready-to-use wsd systems now lag behind the state of the art and offer solutions based on non-neural approaches. in this paper, we addressed this issue and presented amuse-wsd, an all-in-one multilingual system for easy word sense disambiguation. to the best of our knowledge, amuse-wsd is the first system for end-to-end wsd to encapsulate a state-of-the-art neural model for sense disambiguation in multiple languages. our system makes it easy to obtain and use word-level semantic information about word meanings through a restful api. this api is available both online, that is, a user can disambiguate text through http requests to our server, and offline, that is, a user can download prepackaged docker images and run them locally to minimize inference times on large bulks of documents. we provide different configurations to satisfy different needs, from images that are optimized to run on constrained hardware to high-performing images. last but not least, amuse-wsd comes with an intuitive web interface that lets users disambiguate short documents on the fly without writing a single line of code. not only does this interface showcase the capabilities of our system, but we also hope it will attract the interest of new researchers to the field of lexical semantics.