Cross-Lingual and Supervised Learning Approach for I ndonesian Word Sense Disambiguation Task (2018)

Abstract: Ambiguity is a problem we frequently face in Natural Language Processing. Word Sense Disambiguation (WSD) is a task to determine the correct sense of an ambiguous word. However, research in WSD for Indonesian is still rare to find. The availability of English-Indonesian parallel corpora and WordNet for both languages can be used as training data for WSD by applying Cross-Lingual WSD method. This training data is used as an input to build a model using supervised machine learning algorithms. Our research also examines the use of Word Embedding features to build the WSD model.

## introduction one of the biggest challenges in natural language processing (nlp) is ambiguity. ambiguity exists when there are many alternatives of linguistic structures that can be composed for an input language. some words can have more than one meaning (word sense). for example the word "kali" in indonesian can posses two senses, i.e. river and frequency (as described in word sense disambiguation (wsd) is a task to determine the correct sense of a polysemous word. even though it becomes a fundamental task in nlp, research on wsd for indonesian language has not attracted many interests. to our knowledge, the only published work was @xcite . uliniansyah and ishizaki applied the corpus-based approach using naive bayes as the classifier. the training data was collected from news websites and manually annotated. the words in training data were processed using the morphological analysis to obtain lemma. the features being used were some words around the target word (including the words before and after the target word), the nearest verb from the target word, the transitive verb around the target word, and the document context. unfortunately, neither the model nor the corpus from this research is made publicly available. this paper reports our study on wsd task for indonesian using the combination of the crosslingual and supervised learning approach. training data is automatically acquired using cross-lingual wsd (clwsd) approach by utilizing wordnet and parallel corpus. then, the monolingual wsd model is built from training data and it is used to assign the correct sense to any previously unseen word in a new context. among various approaches to wsd, supervised learning approach is the most successful one to date. the supervised wsd uses machine learning techniques for inducing a classifier from senseannotated data sets. training data used to learn the classifier contains a set of examples in which each occurrence of an ambiguous word has been annotated with the correct sense according to existing sense inventory. @xcite despite of its success, the supervised learning approach has a drawback of requiring manually sense-tagged data. manually labeling data for training set is costly and time-consumptive. as an alternative, the sense labeling can be done automatically by utilizing existing resources. cross sentence in indonesian english translation meaning of "kali" saya makan dua kali pagi ini i ate twice this morning frequency rumah saya di dekat kali my house is near the river river sense intersection, an approach described in @xcite and @xcite , inspires clwsd process in our study. gliozzo et.al. proposed an unsupervised wsd technique to automatically acquire sense tagged data that exploited the polysemic differential between two languages using aligned corpora and multilingual lexical databases. an aligned multilingual lexical resource (e.g. multiwordnet) allowed them to disambiguate aligned words in both languages by simply intersecting their senses. @xcite then applied the sense intersection approach in multilingual settings. @xcite considered four languages, e.g. english, italian, romanian, and japanese in their experiment to reduce more ambiguity. for the supervised learning approach to wsd tasks, the common features are the surrounding words of target word, pos tags of the surrounding words, and local collocation. current studies @xcite examined the potential use of word embedding as a feature for the sense classification task. @xcite described four different strategies to use word embedding, e.g. concatenation, average, fractional decay, and exponential decay. ## clwsd for building indonesian wsd training data we utilize clwsd using parallel corpus and wordnet to acquire wsd training data. the model is then learned from the training data to disambiguate the word sense in testing data. our clwsd approach is illustrated in figure 1 . the input for clwsd process is english-indonesian parallel corpus. the corpus used in our experiment is identic++, which is identic corpus (larasati, 2012) that has been extended by adding the instances of english-indonesia parallel sentences from movie subtitles. in addition to the parallel corpus, we harnessed lexical database, namely princeton wordnet @xcite and wordnet bahasa @xcite . clwsd process consists of several steps: an example is given to illustrate the process. consider the following entry of the parallel corpus. ## summary in our study, clwsd has been implemented to provide the training data and then the model based on the training data is built by the classifier to perform monolingual indonesian wsd. we took advantage of existing of the parallel corpus and wordnet to obtain the sense-labeled words by a cross lingual approach. we retrieved all possible senses for the translation pairs and then found the intersection between senses from english and indonesian language. the data acquired by clwsd process is released at https://github.com/ rmahendra/indonesian-wsd. we ran several experiments on monolingual wsd task and concluded that any supervised machine learning model outperforms the baseline method. moreover, we found that the use of embedding vector can produce better f1-score of sense classification than the use of the traditional bag-of-words features. the study still has rooms for improvement. we need to test our methodology in larger corpus and involve more target words for experiment. detail evaluation of clwsd to produce indonesian training data can be more explored. on the other hand, it is interesting to check how sensitive our proposed approach works when considering the semantic difference between senses.