Bilingual Terminology Alignment Using Contextualized Embeddings (2023)

Abstract: Terminology Alignment faces big challenges in NLP because of the dynamic nature of terms. Fortunately, over these last few years, Deep Learning models showed very good progress with several NLP tasks such as multilingual data resourcing, glossary building, terminology understanding. . . etc. In this work, we propose a new method for terminology alignment from a comparable corpus (Arabic/French languages) for the Algerian culture field. We aim to improve bilingual alignment based on contextual information of a term and to create a significant term bank i.e. a bilingual Arabic-French dictionary. We propose to create word embeddings for both Arabic and French languages using ELMO model focusing on contextual features of terms. Then, we mapp those embeddings using Seq2seq model. We use multilingual-BERT and All-MiniLM-L6 as baseline mod- els to compare terminology alignment results. Lastly we study the performance of these models by applying evaluation methods. Experimentation’s showed quite satisfying alignment results.

## introduction for many years now, humans have wanted to enhance the machine's learning and understanding capacity to reach our potential of thinking, awareness, and power of judgment. making us wonder, is it close enough for a machine to be able to recognize and realize as we do? in artificial intelligence and nlp tasks, new models are frequently created to automate and facilitate life in different areas. however, some fields have a long road to go, such as cross-lingual alignment and contextual translation. terminology alignment is a very tough task to handle in nlp since one term can have several meanings according to its position and use. aligned terms are often incorrect or misplaced especially while working with non-similar language families. for example, a sentence or a term might be translated into 3 or more different expressions and still not have the correct corresponding meaning. we can define bilingual terminology alignment as the process of mapping two terms or sentences in two different languages. alignment provides significant benefits in many nlp tasks when properly applied like machine translation, clustering, building bilingual dictionaries, multilingual data resourcing. . . etc. the primary purpose of this work is to build a bilingual term bank for arabic and french languages. bilingual alignment can be applied either to sentences or terms, in this article, we focus on bilingual terms only. according to @xcite , we have a source language sentence containing the terms: an alignment a is defined as a subset of the cartesian product of the word positions @xcite . a ⊆ (j, i) : @xmath0, ..., j; @xmath1, ..., i as shown in this example (see figure 1 ): 2 the remainder of this paper is organized as follows. in section 2, we present related works on bilingual terminology alignment. section 3 presents our methodology and system architecture. section 4 is dedicated to experiments and results of our method. we conclude the paper with a general conclusion and future perspectives. ## related work although bilingual terminology alignment (referred to as bta in the rest of the paper) task is challenging and tough, considerable efforts have been invested into this research field starting in the early 90s by ibm watson research center @xcite who introduced statistical alignment models, called ibm models using parallel corpora. basically, there are 5 basic statistical models (ibm models, 2023) ibm1,2,3,4,5. another one was added later combining ibm4 and hmm model(hidden markov model) based on assumptions such as: many existing methods use ibm models, @xcite applied ibm1 model using an unsupervised em-based hybrid model 1 to extract bilingual terminology from comparable corpora through document alignment constraints. using giza++, @xcite aligned their parallel corpus using the ibm4 model. as in @xcite the famous texsis tool for terminology extraction is based on the ibm4 model for alignment. a combination of ibm1, ibm4 and hmm models is introduced in @xcite to perform 1 expectation maximization model. ## proposed approach in this section, we describe our methods and models for bta using context-based embeddings for both arabic and french languages. we begin with a system design that briefly shows the main used models and techniques for creating bilingual term pairs. starting with creating contextual we to find the best equivalent of a given term from the source s to the target t language. ## experiments & results in this section, we examine the performance of the baseline models for french and arabic languages based on two tests. first, we start by using wes in seq2seq model with fasttext embeddings to compare wes without contextual information with elmo's embeddings for the mapping results. in the second experiment, we compare baseline models' results for the bta task. lastly, we evaluate the model's performance using evaluation metrics: precision, recall, and f1-score. ## conclusion in this paper, the new trending models in terminology alignment and machine translation are presented to improve the quality of alignment for several languages, especially arabic. we chose to focus on the contextual angle of terminology alignment, to improve alignment quality. we use the elmo model to create contextual word vectors in order to capture terms' diversity of meaning, then use the seq2seq model to align those vectors. we believe that the use of contextual word vectors might have a real impact on the alignment quality. we use mbert, seq2seq(fast-text), and all-minilm-l6 models to compare with our proposed method. although mbert outperforms all the models in our experiments, the results are very satisfying for the other models as well. therefore, we think that the model we use in bilingual mapping should depend on the data size, data quality, and model parameters. in terms of future works, we are longing to create new aligned term banks, and dictionaries for other languages. we also hope to apply new models with new features to enhance the alignment quality.