Neural Translator Designed to Protect the Eastern Border of the E uropean U nion (2021)

Abstract: This paper reports on a translation engine designed for the needs of the Polish State Border Guard. The engine is a component of the AI Searcher system, whose aim is to search for Internet texts, written in Polish, Russian, Ukrainian or Belarusian, which may lead to criminal acts at the eastern border of the European Union. The system is intended for Polish users, and the translation engine should serve to assist understanding of non-Polish documents. The engine was trained on general-domain texts. The adaptation for the criminal domain consisted in the appropriate translation of criminal terms and proper names, such as forenames, surnames and geographical objects. The translation process needs to take into account the rich inflection found in all of the languages of interest. To this end, a method based on constrained decoding that incorporates an inflected lexicon into a neural translation process was applied in the engine.

## introduction the internet, even in its legal form, may be a source of criminal information. government bodies all over the world search through internet sites for potentially criminal texts, to prevent certain acts to which such texts may give rise. for example, the polish state border guard, whose function is to protect the eastern border of the european union, tracks texts that may concern criminal activities such as general smuggling, trafficking of drugs, medicines, alcohol and cigarettes, people trafficking, human organs trafficking, weapons and explosives, sex crime, document fraud, and trafficking of stolen cars and machines. two factors make this task difficult for employees of the state border guard. firstly, the texts of interest are sparse and not easy to detect. the problem of the detection of such texts is tackled in @xcite . secondly, criminal texts may appear in foreign languages, not known to a particular employee. in such cases a machine translation engine may be of significant help to the user. this paper describes a neural translator designed for the needs of the polish state border guard. the translator is a component of a system designed to search for and store criminal content. the system is being developed within a research project entitled "advanced internet analysis supporting the detection of criminal groups" 1 (the project's short name is ai searcher). the architecture of the ai searcher system is described in section 2. section 3 reports on the data that was used for the training of language pairs applied in the system. section 4 describes how the translation engine was adapted to the domain of criminal texts. details on the lexicalized 1 the project is financed by the polish national center for research and development. ## the ai searcher project the ai searcher project was launched in december 2018. this three-year program has the aim of developing a system to support the protection of the eastern border of the european union by searching the internet for criminal texts that may be of interest to employees of the polish state border guard. the user scenario is the following: the employee of the state border guard types an inquiry into an edit window. the query expansion module expands the inquiry to a set of queries that are semantically related to the inquiry. the translation module translates the set of queries into russian, ukrainian, and belarusian. the crawler searches the internet to find texts in polish, russian, ukrainian, and belarusian related to the queries. the translation module translates the foreign texts back to polish. finally, the classifier analyzes the texts to return those with potentially criminal content. ## training data the translator engines designed for the system are trained on the opus resources. 2 the sets for training, validation and testing are based on the tatoeba challenge foot_1 (tiedemann 2020). statistics on the bilingual corpora used in the project are given in ## translation of terminology and personal names the state border guard expects that the translation engine should correctly translate proper names, such as surnames, forenames, geographical locations and objects, brands of cigarettes and alcohol, etc. the lists of such names were created semi-automatically: the names underwent automatic transliteration between the cyrillic and latin alphabets, and the most frequent names were carefully verified by native speakers. it is worth noting that all verified forenames and surnames were listed and checked together with their inflected forms (there exist 6-7 grammatical cases in all of these languages). forenames and surnames in their base latin form were provided to us by employees of the state border guard, names of geographical objects were collected from the available open-streetmap resources, and criminal terminology, including brands of cigarettes, cars and alcohol, was gathered from various websites and forums. the adaptation of the translation engine also took into account a lexicon of criminal terms, which consisted of 1203 entries in each of the language pairs. ## lexical constraints the incorporation of lexicon in neural machine translation has been studied thoroughly in recent years @xcite @xcite @xcite ). the methodology used in the described experiments was based on a constrained decoding and "code-switching" approach. our approach was focused on constrained decoding, which uses the grid beam search algorithm introduced by @xcite and extended by @xcite and @xcite . we designed an algorithm based on constrained decoding in order to take into account inflected forms of proper names. to evaluate the performance of the algorithm, we carried out experiments in two different scenarios: general and domain-specific. we compared our method with baseline translation, i.e. translation without lexical constraints, in terms of translation speed and translation quality. the lexicalized method resulted in a decrease in translation quality in the general scenario, which shows that augmenting general-domain texts with a specialized lexicon may impair the performance of a neural translator. in the domain-specific scenario the translation showed significant progress, with an increase of over 3 bleu points. the cost of the algorithm is a decrease in the translation speed. the details of the experiment are reported in @xcite . there, several manual metrics for the evaluation of terminology translation were introduced: placement rate, duplication rate and inflection rate. the metrics evaluated the proportions of output sentences in which the target lexicon terms were, respectively, properly placed, not duplicated unnecessarily and correctly inflected. the manual evaluation results showed that our method ensures terminological adequacy and consistency as well as linguistic correctness when translating into a morphologically rich language in domain-specific scenarios. ## examples of lexicalized translation tables 5 and 6 show examples of sentences translated with the unadapted and adapted translation engine into russian and ukrainian, respectively. the lexicon entries consist of a term in the source language with the equivalent in the target language along with a comma-separated list of its inflectional forms. for each sentence, a manual english translation is given for clarity. ## conclusions in this case study, a translation engine is part of a system that searches for criminal content in internet documents written in the polish, russian, ukrainian and belarusian languages. the adaptation of the translation to the domain of criminal texts consists in the incorporation of lexicon into the neural machine translation engine. the criminal terminology is expected to be translated according to lexical constraints, and the lexical entries should be correctly inflected. an algorithm based on constrained decoding was designed to achieve this goal. the project described here is ending in december 2021. work in the near future will focus on further improving belarusian translation and on increasing efficiency.