C o LI -Kanglish: Word-Level Language Identification in Code-Mixed K annada- E nglish Texts Shared Task using the Distilka model (2022)

Abstract: Due to the intercultural demographic of online users, code-mixed language is often used by them to express themselves on social media. Language support to such users is based on the ability of a system to identify the constituent languages of the code-mixed language. Therefore, the process of language identification that helps in determining the language of individual textual entities from a code-mixed corpus is a current and relevant classification problem. Code-mixed texts are difficult to interpret and analyze from an algorithmic perspective. However, highly complex transformer- based techniques can be used to analyze and identify distinct languages of words in code-mixed texts. Kannada is one of the Dravidian languages which is spoken and written in Karnataka, India. This study aims to identify the language of individual words of texts from a corpus of code-mixed Kannada-English texts using transformer-based techniques. The proposed Distilka model was developed by fine-tuning the DistilBERT model using the code-mixed corpus. This model performed best on the official test dataset with a macro-averaged F1-score of 0.62 and weighted precision score of 0.86. The proposed solution ranked first in the shared task.

## introduction language identification is the process of determining the natural language that a document is written in. automatic language identification has been widely researched for 50 years @xcite . however, while recognizing text in different languages might come naturally to a human reader, it is still challenging for the computer. natural language processing (nlp) focuses on teaching computers to comprehend spoken and written language in a manner similar to humans. nlp research is evolving and rapidly expanding. classification tasks such as text classification, sentiment analysis, name entity recognition, and speech recognition have been solved in the past using algorithms of nlp. similarly, language identification, is a recent research problem that can be dealt with using nlp techniques @xcite focusing on language identification when it comes to a low-resource language or a mixed language, in order to identify the language could be a huge challenge. as we know, india has a rich language culture covering different geographical areas such as hindi, bengali, and kannada. kannada is spoken mainly in karnataka, which is a southern state of india @xcite . people of karnataka read, write, and speak kannada, but many find it difficult to use kannada script to post comments on social media. as a result, kannada is a low-resource language since social media users typically use roman script or a combination of kannada and roman script. in this shared task, a dataset was created using kannada youtube comments named "coli-kanglish". @xcite . the main objective of this shared task is to create a novel method for language identification in mixed languages that consists of tokens from english, kannada, mixed languages of kannada and english, name, location, and other categories. in this study, the dataset is initially prepared by task organizers. then, the data processing technique is utilized. finally, we evaluate the model using macro-averages and weighted average scores. the following points are the contribution of our study: to the best of our knowledge, this is the first study that applies a distilbert-based model to identify mixed language in kannada and english (coli-kanglish) datasets, and the fact that it presents the best performance is highlighted. ## related works language identification has been studied for half a decade, and automatic language identification has been proposed rigorously in social media data. in 2014, barman et al, presented an initial study on automatic language identification using indian language code mixed in social media communication. the dataset of bengali, hindi, and english in facebook comments. the authors conclude that character n-gram features, contextual information is also important, and information from dictionaries can be useful for language identification tasks. apart from facebook data, the studies also investigate twitter data with support vector machine (linear kernel), which contains bilingual tweets written in the most commonly used iberian languages (i.e., spanish, portuguese, catalan, basque, and galician) as well as english language. the study achieved 0.792 for macro-f1 @xcite . in the same year, 2017, transformers were introduced. the paper "attention is all you need," describing attention mechanisms, provides context for any position in the input sequence. @xcite . furthermore, if the input data is a natural language sentence, the transformer does not have to process one word at a time. this allows for more parallelization than a recurrent neural network and therefore reduces training times. in 2018, bert (bidirectional encoder representations from transformers) from transformer-based technique, was developed with large amounts of pre-trained data and the ability to capture context, so bert has become a well-known architecture since then @xcite . due to some constraints of bert, distilbert has emerged to optimize the training by reducing the size of bert and increasing the speed of bert,while trying to retain as much performance as possible. moreover, distilbert is 40% smaller than the original bert base model, is 60% faster than it, and retains 97% of its functionality. distilbert has been used in a variety of text classification tasks, including language identification. there are several papers using distilbert for text classification tasks, for example. bambroo & awasthi, 2021 proposed a fine-tuned distilbert on legal-domain specific corpora and discovered that this model outperformed other algorithms while also being faster at the task of legal document classification. another study is to carry out a word-level language identification (wlli) of malayalam-english code-mixed data from youtube. according to the study, distilbert produced the highest precision score with 91.74% in hindi and english pairs @xcite . ## conclusion in this paper, we describe our proposed method for the shared task of word-level language identification in code-mixed kannada-english dataset. on comparing with the performance metrics of other solutions based on distilbert, that were developed for this shared task, it was found out that the distilka model performed significantly better with a macro-averaged f1score of 0.62. the proposed model secured the first rank in the shared task. in future work, we will try to adjust the parameters of the new model in order to improve its performance significantly. future work will include further application of language identification tasks to several lowresource languages.