Semi-Supervised and Unsupervised Sense Annotation via Translations (2021)

Abstract: Acquisition of multilingual training data continues to be a challenge in word sense disambiguation (WSD). To address this problem, unsupervised approaches have been proposed to automatically generate sense annotations for training supervised WSD systems. We present three new methods for creating sense-annotated corpora which leverage translations, parallel bitexts, lexical resources, as well as contextual and synset embeddings. Our semi-supervised method applies machine translation to transfer existing sense annotations to other languages. Our two unsupervised methods refine sense annotations produced by a knowledge-based WSD system via lexical translations in a parallel corpus. We obtain state-of-the-art results on standard WSD benchmarks.

## introduction word sense disambiguation, the task of identifying the meaning of a word in context, is one of the central problems in natural language understanding @xcite . it is a well-studied benchmark for evaluating contextualized representations of words @xcite , and is better understood than tasks such as wic @xcite . modern wsd methods can be divided into supervised and knowledge-based approaches. the former depend on sense-annotated corpora, such as semcor @xcite , while the latter rely instead on semantic knowledge bases such as wordnet @xcite . while supervised wsd systems typically outperform knowledge-based systems @xcite , their utility is limited by the availability of sufficiently large sense-annotated corpora for training. this includes systems based on contextualized embeddings @xcite . in particular, there is a severe lack of high-quality sense-annotated corpora for languages other than english. this limitation has motivated the development of methods aimed at automatically disambiguating a large number of word tokens in a given unannotated corpus, ideally covering a wide range of word and sense types, while minimizing noise @xcite @xcite . the automatically tagged corpus can then be used to train a supervised wsd system, satisfying the dependency on training data without the need for manual annotation. following recent theoretical work @xcite on establishing the semantic equivalence of mutual translations, we introduce three translation-based methods for generating sensetagged corpora. all three methods make use of lexical knowledge bases, and semantic information obtained from word-level translations. semisupervised labelprop creates a synthetic parallel corpus (bitext) by applying machine translation to a monolingual manually-annotated corpus, and projecting annotations to the target language. similarly, unsupervised labelgen applies a knowledge-based wsd system to the english side of a bitext, and projects the resulting sense annotations across bitexts onto other languages. finally, unsupervised labelsync produces sense-annotated corpora in two languages at once by independently applying a knowledgebased wsd system to each side of a raw bitext, and then refining the initial annotations based on the confidence scores and multilingual information. our experiments on standard wsd test sets demonstrate that the new methods achieve stateof-the-art results in both semi-supervised and unsupervised sense annotation. we train two different reference supervised wsd systems on the generated data, and apply the resulting models to multilingual wsd benchmarks. our results compare favourably to models trained on data produced by the previous state-of-the-art sense annotation methods. indeed, some of the results obtained with our unsupervised methods rival those obtained by training on a manually sense-annotated corpus. our contributions are as follows: we present three novel, scalable methods that can generate annotated corpora for any language for which a suitable lexical knowledge base is available. we show that these methods achieve state-of-the-art results on multiple languages. we make our code and corpora available. 1 ## related work the sense tagging systems that we consider in this work, including our three novel methods, can be divided into four types according to two criteria (figure 1 ). the first criterion is whether the method involves supervision in the form of a sense-annotated corpus. the second criterion is whether the method operates as a traditional self-contained wsd system, or instead assigns sense tags to a subset of the words in a corpus which can then be used to train a supervised wsd system. in this section, we discuss the most relevant examples of each of the four resulting types. supervised wsd systems rely on sense annotations to train disambiguation models, which are evaluated on benchmark datasets. examples include glossbert @xcite , ewise @xcite , and ewiser @xcite . because they require labelled training data in the target language, such systems are generally impractical for languages other than english, nor are they directly comparable to our proposed methods. knowledge-based wsd systems remain important due to the limited coverage of existing annotated corpora, as well as their english bias. these include graph-based systems such as ukb @xcite , ukb enhanced with syntagnet @xcite , and systems based on multilingual bert @xcite and babelnet @xcite , such as sensembert @xcite . we compare our unsupervised results to both ukb+syntagnet and sensembert. semi-supervised corpus tagging systems depend on sense annotated corpora in one language to produce sense annotations in other languages. the current state-of-the-art method in this setting is mu-lan @xcite , which propagates sense 1 https://www.cs.ualberta.ca/ ∼kondrak annotations from semcor and wordnet gloss corpus (wng, @xcite to semantically similar contexts in wikipedia corpora using contextual word representations from mbert. our la-belprop method differs by leveraging machine translation to directly propagate sense annotations across word alignment links. unsupervised corpus tagging systems produce sense annotations "from scratch". train-o-matic @xcite annotates wikipedia in multiple languages by applying the personalized pagerank (ppr) algorithm to ba-belnet. onesec @xcite combines wikipedia categories and babelnet synset representations to produce wsd training data, and outperforms train-o-matic. however, both train-o-matic and onesec annotate nominal instances only, and hence are not applicable to all-words wsd. on the other hand, eurosense @xcite jointly disambiguates content words of all parts of speech in a parallel corpus using a knowledge-based wsd system. our label-sync and labelgen methods differ in that they explicitly leverage lexical translation information obtained from a bitext. other work on using translations for wsd: @xcite propose to distinguish senses only if a "minimum subset" of languages translate them differently. @xcite demonstrates how senses can be induced by clustering lexical translations, and proposes an unsupervised wsd system based on such induced sense inventory and translation information. @xcite frame wsd as translation selection, and propose a method based on multilingual feature vectors. finally, @xcite annotate english words with their chinese translations using manually crafted sense-totranslation mappings. these methods are not comparable with our work as they do not link their sense annotations to the wordnet sense inventory, and therefore are not applicable to modern wsd datasets. ## unsupervised asymmetric labelgen unlike labelsync, our second unsupervised method, labelgen, assumes that the source language is english, and treats the two sides of the bitext differently. the goal is to leverage available english resources to improve wsd performance on other languages, rather than to generate english sense annotations. ## evaluation following prior work, we extrinsically evaluate our corpus construction approaches by providing the generated annotations as training data for supervised wsd systems (reference systems), which are then evaluated on standard multi-lingual wsd benchmarks. while our methods could also be applied to low-resource languages, the current lack of evaluation datasets precludes such experiments in this work. ## conclusion we have introduced new methods to address the knowledge acquisition bottleneck in word sense disambiguation in both the semi-supervised and unsupervised settings. the methods leverage recent advances in machine translation, alignment, and contextual embeddings. extrinsic experiments with a variety of wsd systems demonstrate that the quality of the corpora created by our methods is substantially higher compared to those produced by prior work. our methods for automatic sense tagging can produce annotated corpora for many languages, and approach the quality of manual annotation in some cases. we make our corpora available for further research. one advantage of our unsupervised methods is that they can be applied to annotate any bitext involving any languages. we posit that our results could be further improved by annotating corpora with broader domain coverage, or by matching the domain of the source corpus to the domain of the data to be disambiguated. we leave this as a direction for future work.