Aligning E stonian and R ussian news industry keywords with the help of subtitle translations and an environmental thesaurus (2021)

Abstract: This paper presents the implementation of a bilingual term alignment approach developed by Repar et al. (2019) to a dataset of unaligned Estonian and Russian keywords which were manually assigned by journalists to describe the article topic. We started by separating the dataset into Estonian and Russian tags based on whether they are written in the Latin or Cyrillic script. Then we selected the available language-specific resources necessary for the alignment system to work. Despite the domains of the language-specific resources (subtitles and environment) not matching the domain of the dataset (news articles), we were able to achieve respectable results with manual evaluation indicating that almost 3/4 of the aligned keyword pairs are at least partial matches.

## introduction and related work the ability to accurately align concepts between languages can provide significant benefits in many practical applications. for example, in terminology, terms can be aligned between languages to provide bilingual terminological resources, while in the news industry, keywords can be aligned to provide better news clustering or search in another language. accurate bilingual resources can also serve as seed data for various other nlp tasks, such as multilingual vector space alignment. in this paper, we describe the experiments on an estonian-russian dataset of news tags -labels that were manually assigned to news articles by journalists and editors at ekspress meedia, one of the largest news publishers in the baltic region. the dataset contains both estonian and russian tags, but they are not aligned between the two languages. we adapted the machine learning term alignment approach described by @xcite to align the russian and estonian tags in the dataset. the alignment approach in @xcite is a reproduction and adaptation of the approach described by @xcite . @xcite managed to reach a precision of over 0.9 and therefore approach the values presented by @xcite by tweaking several parameters and developing new machine learning features. they also developed a novel cognate-based approach which could be effective in texts with a high proportion of novel terminology that cannot be detected by relying on dictionary-based features. in this work, we perform the implementation of the proposed method on a novel, estonian-russian language pair, and in a novel application of tagset alignment. section 1 lists the related work, section 2 contains a description of the tag dataset used, section 3 describes the system architecture, section 4 explains the resources used in this paper, section 5 contains the results of the experiments and section 6 provides conclusions and future work. ## dataset description the dataset of estonian and russian tags was provided by ekspress meedia as a simple list of one tag per line. the total number of tags was 65,830. the tagset consists of keywords that journalists assigne to articles to describe an article's topic, and was cut down recently by the editors from more than 210,000 tags. ## system architecture the algoritm used in this paper is based on the approach described in @xcite which is itself a replication and an adaptation of @xcite . the original approach designed by @xcite was developed to align terminology from comparable (or parallel) corpora using machine-learning techniques. they use terms from the eurovoc @xcite thesaurus and train an svm binary classifier @xcite (with a linear kernel and the trade-off between training error and margin parameter @xmath0. the task of bilingual alignment is treated as a binary classification -each term from the source language s is paired with each term from the target language t and the classifier then decides whether the aligned pair is correct or incorrect. @xcite use two types of features that express correspondences between the words (composing a term) in the target and source language: to match words with morphological differences, they do not perform direct string matching but utilize levenshtein distance. two words were considered equal if the levenshtein distance (levenshtein, 1966) was equal or higher than 0.95. for closed-compounding languages, they check whether the compound source term has an initial prefix that matches the translation of the first target word, provided that translation is at least 5 characters long. additional features are also constructed by: a subset of the features is described below (for a full list of features, see @xcite ): the system requires several language-specific resources: the constructed features are then used to train the svm classifier which can be used to predict the alignment of terms between two languages. ## resources for the estonian-russian experiment while the dgt translation memory and the eurovoc thesaurus support all official eu languages, there is no russian support since russia is not an eu member state. in order to train the classifier, we therefore had to find alternative resources. for the parallel corpus, we made experiments with the estonian open parallel corpus foot_2 and the estonian-russian opensubtitles corpus from the opus portal foot_3 . the opensubtitles corpus performed better, most likely due to its much larger size (85,449 parallel estonian-russian segments in the estonian open parallel corpus vs. 7.1 million segments in the opensubtitles corpus). while finding parallel estonian-russian corpora was trivial due the the list of available corpora on the opus portal, finding an appropriate bilingual terminological database proved to be more difficult. ideally, we would want to use a media or news-related estonian-russian terminological resource, but to the best of our knowledge, there was none available. note that the terminological resource needs to have at least several thousand entries: the eurovoc version used by @xcite contained 7,083 english-slovene term pairs. we finally settled on the environmental thesaurus gemet foot_4 , which at the time had 3,721 estonian-russian term pairs. for the transliteration rules, we used the python pip package transliterate foot_5 to generate the reverse dictionary-based features. @xcite ran a total of 10 parameter configurations. we selected three of those to test on the estonian-russian dataset. the first one is the configuration with a positive/negative ratio of 1:200 in the training set, which significantly improved recall compared to the reproduction of @xcite , the second one is the same configuration with additional term filtering, which was overall the best performing configuration in @xcite , and the third one is the cognates approach which should give greater weight to cognate words. as shown in ## conclusions and future work in this paper, we reused an existing approach to terminology alignment by @xcite to align a set of estonian and russian tags provided by the media company ekspress meedia. the approach requires several bilingual resources to work and it was difficult to obtain relevant resources for the estonian-russian language pair. given the domain of the tagset, i.e. news and media, the selected resources (subtitle translations and an environmental thesaurus) were less than ideal. nevertheless, the approach provided respectable results with 74% of the positive tag pairs evaluated to be at least a partial match. when asssessing the performance of the approach, one has to take into account the fact that the tagset is heavily unbalanced with almost 60,000 estonian tags compared to a little over 6,000 russian tags. this means that for many estonian tags, a true equivalent was simply not available in the tagset. for future work, we plan to integrate additional features into the algorithm, such as those based on novel neural network embeddings which may uncover additional hidden correlations between expressions in two different languages and may provide an alternative to large parallel corpora which are currently needed for the system for work. in terms of the estonian and russian language pair, additional improvements could be provided by taking into account the compound-like structure of many estonian words. finally, we will look into techniques that would allow us to pre-filter the initial list of tag pairs to reduce the total processing time.