Discriminating between standard R omanian and M oldavian tweets using filtered character ngrams (2020)

Abstract: We applied word unigram models, character ngram models, and CNNs to the task of distinguishing tweets of two related dialects of Romanian (standard Romanian and Moldavian) for the VarDial 2020 RDI shared task (Gaman et al. 2020). The main challenge of the task was to perform cross-genre text classification: specifically, the models must be trained using text from news articles, and be used to predict tweets. Our best model was a Naive Bayes model trained on character ngrams, with the most common ngrams filtered out. We also applied SVMs and CNNs, but while they yielded the best performance on an evaluation dataset of news article, their accuracy significantly dropped when they were used to predict tweets. Our best model reached an F1 score of 0.715 on the evaluation dataset of tweets, and 0.667 on the held-out test dataset. The model ended up in the third place in the shared task.

## introduction language identification can be challenging for nlp techniques when languages are hardly distinguishable. one example of this challenge is the identification of moldavian, a dialect of romanian which exhibits almost no difference with standard romanian. the distinction between romanian and moldavian is only motivated by the presence of a political boundary, which corresponds to no real isogloss. in spelling, the two languages are almost identical, with a minor exception involving the distribution of the letters 'â' and 'î', although other grammatical distinctions can be found in number, gender and case morphology, and in lexical choices. in particular, the lexical divergence was the result of moldavian being under the influence of russian, in the years in which it was part of the soviet union. an additional challenge for language identification is that existing resources might belong to domains that are different from the domain on which one needs to perform a classification task. for instance, in certain cases one can find data from textbooks, encyclopedias and newspaper articles, but not from social media, even though language identification is often used to classify online messages @xcite @xcite . this raises the question of how to use out-of-domain data when performing language identification in a restricted domain. the vardial 2020 rdi shared task @xcite invited participants to perform cross-genre language identification by training a classifier on newspaper articles, and using it to distinguish standard romanian from moldavian tweets. in this paper, we present the contribution of the team phlyers to the task. ## methods previous methods used for language identification typically involve bag-of-words models @xcite , naïve bayes models applied to word and character ngrams @xcite and support vector machines @xcite . deep learning methods based on cnns and lstms this work is licensed under a creative commons attribution 4.0 international licence. licence details: http: //creativecommons.org/licenses/by/4.0/ . the code for the models employed in this work is found at https: //github.com/andreaceolin/vardial2020 . we thank monica-alexandrina irimia for comments about the project. the architectures of the cnn and the tdnn are shown in figure 1 . ## conclusion we applied word unigram models, character ngram models, and two neural network models to classify tweets of two related dialects (standard romanian and moldavian) for the vardial 2020 rdi shared task @xcite , with training data from a different domain. two of the models we proposed, a linear svm model based on tfidf word unigrams and a cnn model, reached a high accuracy on the news evaluation dataset, but failed to generalize to the tweets evaluation dataset. on the contrary,