Word Sense Disambiguation for M arathi language using Supervised Learning (2023)

Abstract: The task of disambiguating word senses, often referred to as Word Sense Disambiguation (WSD), is a substantial difficulty in the realm of natural language processing. Marathi is widely acknowledged as a language that has a relatively restricted range of resources. Consequently, there has been a paucity of academic research undertaken on the Marathi language. There has been little research conducted on supervised learning for Marathi Word Sense Disambiguation (WSD) mostly owing to the scarcity of sense-annotated corpora. This work aims to construct a sense-annotated corpus for the Marathi language and further use supervised learning classifiers, such as Naïve Bayes, Support Vector Machine, Random Forest, and Logistic Regression, to disambiguate polysemous words in Marathi. The performance of these classifiers is evaluated.

## introduction word sense disambiguation (wsd) is the process of determining the exact interpretation of a polysemous term in a given context @xcite . many natural language processing (nlp) applications employ wsd, either directly or indirectly. sentiment analysis, machine translation, information retrieval, text summarization, etc. are some of the applications in nlp where wsd is employed. many researchers have surveyed the various methods for word sense disambiguation in various languages @xcite @xcite @xcite . the existing methods for word sense disambiguation are generally categorized into two distinct groups @xcite : marathi is an indo-aryan language mostly used in the state of maharashtra, india. nevertheless, the progress in constructing natural language processing (nlp) systems for marathi has been rather restricted when juxtaposed with prominent languages like english. the insufficient focus on marathi language processing presents difficulties and impedes the development of sophisticated natural language processing (nlp) applications for those who speak marathi @xcite . supervised methodologies for word sense disambiguation (wsd) have shown superior accuracy in comparison to other techniques. nevertheless, one drawback of these methods is their reliance on corpora that have been annotated with sense information. several commonly used supervised learning methods include naïve bayes, support vector machine, decision tree, etc. @xcite . in this paper, we present the development of a sense-annotated corpus and the use of several supervised machine learning methods, including naïve bayes, support vector machine, decision tree, random forest and logistic regression, for the purpose of disambiguating words having multiple meanings in the marathi language. the performance of these algorithms for word sense disambiguation (wsd) is assessed. this study represents the first use of these algorithms for marathi word sense disambiguation (wsd) and the subsequent evaluation of their performance. the subsequent sections of this work are organized as follows: section 2 provides an overview of the existing literature and research that is relevant to the topic at hand. in section 3, the implementation of supervised learning algorithms is shown. in the next section 4, an analysis of the acquired findings is presented. section 5 provides a conclusion to the material presented and offers recommendations for future research endeavours. ## related work kumari et al. examine several supervised machine learning algorithms used for the purpose of disambiguating the proper interpretation in hindi text. they have constructed a sense-tagged corpus by selecting 20,000 phrases from news websites like as danik bhaskar and navbharat times. three classifiers, including the naïve bayes classifier, support vector machine (svm) based classifier, and multiple layer perceptron (mlp) based classifier, have been used on the defined feature sets. upon conducting a comparison between the judgments made by the classifier and those made by human annotators, they concluded that the svm classifier had superior values for accuracy, recall, precision, and f-score @xcite . @xcite propose the use of support vector machine (svm) algorithm in conjunction with the term frequency-inverse document frequency (tf-idf) technique as the feature extraction method, and utilize wikipedia as the training data, to address the word sense disambiguation (wsd) challenge in the indonesian language. initially, the training data is sourced from wikipedia articles. the articles are then subjected to pre-processing techniques aimed at reducing word variation. subsequently, the articles are transformed into classifiable features using tfidf, followed by the utilization of a support vector machines classifier to ascertain the meaning of an unclear word inside a phrase. the suggested technique has an accuracy level of 87.7% across all classifiers @xcite . walia et al. have used a supervised methodol-ogy, namely the k-nearest neighbors (k-nn) algorithm, to resolve word ambiguities in the gurmukhi language. they have used the punjabi corpora, sourced from the evaluations and language resources distribution agency in paris, france. the corpora has been sense-tagged with 100 words. the algorithm's efficiency was evaluated by conducting tests on a collection of 120 phrases, each containing 8 confusing terms. the k-nn based method to word sense disambiguation (wsd) in gurmukhi demonstrated an average accuracy ranging from 53% to 76% @xcite . pal et al. propose to use supervised approach for the purpose of word sense disambiguation (wsd) in bengali, with appropriate adjustments. their study is conducted using four widely used supervised approaches, namely the decision tree (dt), the support vector machine (svm), the artificial neural network (ann), and the naïve bayes (nb) algorithm, for the purpose of sense categorization in the baseline experiment. the aforementioned methods are independently used on a dataset consisting of the 13 most often utilized ambiguous terms in the bengali language. the aforementioned approaches provide accuracy results of 63.84%, 76.9%, 76.23%, and 80.23% respectively @xcite . pal et al. use a supervised technique to address the challenge of word meaning disambiguation in the bangla language. the naive bayes probabilistic model is often used as a baseline approach for sense classification. when applied to a database including the 19 most frequently used ambiguous terms in the bangla language, it achieves a reasonable level of accuracy, namely 81%. here, pal et al. provide two modifications to the baseline approach. firstly, they include a lemmatization process into the system. secondly, they use a bootstrapping technique to enhance the operational process. consequently, the approach exhibits a modest improvement in accuracy, reaching a level of 84% precision @xcite . walia et al. present the implementation of three supervised approaches, namely naïve bayes, k-nn, and decision trees classifiers, for the purpose of word sense disambiguation in the punjabi language. to provide a comparative analysis of the three supervised approaches, a set of 20 ambiguous terms has been chosen. additionally, the researchers have conducted experiments using three distinct window context sizes, namely 3, 5, and 7, in order to evaluate the efficacy of each of the strategies. the findings of the study suggest that increasing the window size leads to improved accuracy. additionally, the findings indicate that among the three supervised procedures, naïve bayes exhibits superior performance compared to the other two techniques @xcite . singh and kumar have conducted an analysis of a punjabi language word sense disambiguation (wsd) system that employs supervised approaches. a manually produced corpus consisting of 150 ambiguous punjabi noun terms has been created. this study examines six supervised machine learning algorithms, namely decision list, decision tree, naive bayes, k-nearest neighbour (k-nn), random forest, and support vector machines (svm). the word embedding features have been tested on six classifiers for the punjabi word sense disambiguation (wsd) challenge. the use of word embedding features has been shown to improve the performance of supervised classifiers in the task of word sense disambiguation (wsd) for the punjabi language. the present study demonstrates that the lstm classifier, when using word embedding feature, has reached an accuracy rate of 84% @xcite . naseer and hussain employ a statistical method known as "bayesian classification" to address the issue of word sense disambiguation for certain urdu terms. this study examines four urdu words, consisting of one noun and three verbs, as its primary emphasis. their experiment involves using various window widths where n represents the number of words to the left and right of the ambiguous word, with n being equal to 3, 5, and 7, respectively. the method demonstrated superior performance when applied to a specific term that had a significantly high frequency within the corpus. however, the accuracy of the algorithm was considerably lower when dealing with words that occurred less often. the augmentation of window size yielded improved performance outcomes. the 7x7 window configuration yielded the highest accuracy (98.35%) and recall (92.17%) values @xcite . lee et al. provide a description of the participating systems in the senseval-3 english lexical sample task and multilingual lexical sample task, as presented by the authors. the word sense disambiguation (wsd) systems used support vector machine (svm) learning techniques and included many knowledge sources. the features used in this approach include the part-of-speech (pos) of neighboring words, individual words within the surrounding context, local collocations, and syntactic relations. the omission of feature selection is justified based on the findings of their earlier study @xcite , which demonstrated that svm achieves superior performance without the inclusion of feature selection. the findings of this inquiry demonstrate that the incorporation of four pre-existing knowledge sources resulted in an enhancement of the micro-averaged recall performance on the training data, increasing from 0.628 to 0.638 @xcite . abid et al. address the issue of word sense disambiguation (wsd) within the specific linguistic framework of the urdu language. in this study, the authors use machine learning (ml) techniques, including the bayes net classifier (bn), support vector machine (svm), and decision tree (dt), to perform word sense disambiguation (wsd) on urdu literature written in its original script. the findings indicate that bayesian networks (bn) exhibit a higher f-measure compared to support vector machines (svm) and decision trees (dt). the bayes net classifier achieved a maximum f-measure of 0.711 when applied to a raw urdu corpus consisting of 2.5 million words @xcite . junaida et al. present a hybrid methodology that combines a multi-class support vector machine (svm) with a corpus-based technique for the purpose of malayalam word sense disambiguation. the training set consists of a restricted number of ambiguous words that have been annotated with 16 word sense disambiguation (wsd) classes. the system undergoes evaluation using manually generated words, and its correctness is assessed by n-fold cross-validation. the findings from the 10fold cross-validation demonstrate the suitability of the suggested multi-class support vector machine (svm) for the malayalam word sense tagger. the evaluation was conducted using a one-against-one technique, using both word-only and word plus part-of-speech (pos) features. the findings indicate that the one versus one technique yields the highest performance outcomes, with an overall average accuracy, recall, and f-measure values of 63.058, 57.78, and 57.9, respectively, for the 10folds @xcite . singh et al. examine the use of the naive bayes (nb) classifier in hindi word sense disambiguation (wsd). the research focuses on the utilization of eleven distinct characteristics, including local context, collocations, unordered list of words, nouns, and vibhaktis. the evaluation process was conducted on a manually constructed sense annotated corpus in the hindi language. this corpus included 60 polysemous nouns in hindi. a accuracy of 77.52% was found while using an unordered list of words in the feature vector. the use of nouns in the feature vector, after the application of morphology, resulted in an accuracy rate of 86.11%. a accuracy of 56.49% was achieved by including vibhaktis in the feature vector @xcite . sarmah et al. aim to offer a supervised machine learning strategy, namely decision tree, for the job of word sense disambiguation for assamese language. the training and test dataset consisted of a collection of polysemous terms in assamese language, each having various genuine occurrences and manual sense annotation. the dt method yielded an average f-measure of 0.611 when a 10fold cross-validation assessment was conducted on a set of 10 assamese ambiguous phrases @xcite . parameswarappa and narayana provide a study that focuses on the use of compound words hint and syntactic properties within a limited context for the purpose of target word sense disambiguation for kannada language. the kannada shallow parser has been used for the purpose of doing syntactic analysis. the process of resolving the ambiguity of the target word is accomplished by using a naive bayes classifier. the accuracy values of the models vary between about 51% and 71% when evaluated on typical corpora. their system handles only one target word @xcite . borah et al. have devised an automated method for word sense disambiguation (wsd) in the assamese language, using a naive bayes classifier. the assamese language has a high degree of morphological complexity. the used features include unigram co-occurrences (uco), pos of target word (post), pos of next word feature, and local collocation. the system achieved optimal performance, with an f1-measure of 86%, when all four characteristics were aggregated @xcite . gopal et al. introduced a method for malayalam word sense disambiguation that employs a naive bayes classifier inside a supervised framework. this framework primarily utilizes two corpora known as the sense corpus and the ambiguous corpus. an ambiguous corpus encompasses a comprehensive collection of words that have many alternative meanings, whereas a sense corpus has synsets, synonyms, and antonyms. the findings demonstrate a 90% level of accuracy when using the naïve bayes classifier algorithm on a corpus consisting of 100,000 words @xcite . ## implementation based on the survey of work done for word sense disambiguation of various languages in the previous section 2, we propose to use supervised learning approaches for word sense disambiguation of marathi language. we have used baseline supervised learning classifiers like naïve bayes, support vector machine, decision tree, random forest and logistic regression. this is the first attempt in our knowledge to implement these supervised algorithms for marathi word sense disambiguation and comparison of their performance. the proposed system is shown in figure 1 . the input to this system is a sense-annotated dataset containing marathi sentences with ambiguous words. the sentences in the sense-annotated dataset are pre-processed by removing the special symbols, removing stopwords and performing lemmatization. five separate models are trained on the sense-annotated dataset using algorithms like naïve bayes, support vector machine, decision tree, random forest and logistic regression. the sentences in test dataset are also pre-processed. then the models are tested separately on a few test sentences to determine the most appropriate sense of the ambiguous word. a collection of 650 sense-annotated marathi sentences has been compiled, which have been derived from several marathi websites and publications, including different genres such as news, philosophy, sports, and fiction. some sentences have also been generated manually by us. these sentences were carefully selected to include 12 terms that possess multiple meanings, resulting in a total of around 42 distinct senses. the set of ambiguous terms under consideration comprises 2 verbs, 2 adverbs, 2 adjectives, and 6 nouns. the current dataset has around 15-16 sentences per sense. the training set consists of 500 sense-annotated marathi sentences. the performance of these supervised learning algorithms is evaluated on the remaining 150 test sentences. ## results and discussions in this section, we present and discuss the results of the application of the supervised algorithms for marathi word sense disambiguation. the results obtained are illustrated in the following table1 and figure 2 . it is observed that the accuracy of naive bayes and support vector machine (svm) is the same but the precision and f-1 score of naive bayes is better than that of svm. logistic regression gives average results as compared to naive bayes and svm. decision tree and random forest perform poorly for this task of marathi word sense disambiguation on the test dataset. ## conclusion our study encompasses a comprehensive analysis of the literature surveyed, which together assert that the use of supervised machine algorithms is viable for marathi word sense disambiguation (wsd). additionally, researchers have shown that the use of sense-annotated datasets might potentially enhance the accuracy of disambiguating marathi words. this study focuses on the implementation of several supervised machine learning methods, including naive bayes, support vector machines (svm), decision trees (dt), random forests (rf), and logistic regression (lr), for the purpose of marathi word sense disambiguation. our study revealed that naive bayes (nb) exhibit superior performance on our test dataset. however, the precision, recall, and f1-score metrics for all the implemented algorithms do not meet the desired standards due to the limited quantity of the training and testing datasets. supervised learning algorithms may exhibit superior performance relative to unsupervised and knowledge-based techniques when the size of the dataset is expanded.