Building Machine Translation System for Software Product Descriptions Using Domain-specific Sub-corpora Extraction (2022)

Abstract: Building Machine Translation systems for a specific domain requires a sufficiently large and good quality parallel corpus in that domain. However, this is a bit challenging task due to the lack of parallel data in many domains such as economics, science and technology, sports etc. In this work, we build English-to-French translation systems for software product descriptions scraped from LinkedIn website. Moreover, we developed a first-ever test parallel data set of product descriptions. We conduct experiments by building a baseline translation system trained on general domain and then domain-adapted systems using sentence-embedding based corpus filtering and domain-specific sub-corpora extraction. All the systems are tested on our newly developed data set mentioned earlier. Our experimental evaluation reveals that the domain-adapted model based on our proposed approaches outperforms the baseline.

## introduction the development of machine translation (mt) systems for a specific domain (e.g., science, politics, economics) is often a challenging task because of the lack of parallel corpus in these domains. it is impractical to develop large corpora in every domain as it requires a huge amount of time and cost even for a single domain. there can be following methods to build mt systems in this scenario: (i) training an mt system on the available data set from other domains while tuning the model parameters on in-domain development set, or (ii) extracting in-domain parallel texts from one or more corpora and then building an mt system on the concatenation of these extracted text pairs. the first method, although tuned on an in-domain development data, is not much useful because the training is done only on an out-of-domain data set. in contrast, the second method is better because it is aimed to extract the in-domain data which are then used for training. however, producing a sufficiently large in-domain data set is a difficult task. in this work, we mainly focus on the second method, i.e, we extract parallel texts similar to indomain in order to build an mt system and also tune the parameters on the in-domain data set. this approach is useful for building mt system in a specific scenario, which is the domain of software product descriptions from linkedin foot_0 web pages in our case. linkedin is an american business and employment-oriented online service that operates via websites and mobile apps. it is primarily used for professional networking and career development, and allows job seekers to post their curricula vitae (cvs) and employers to post jobs. it also contains product pages for brands to promote their products and grow their businesses, for product users to share their experiences and be recognised for their expertise, and for buyers to make confident decisions about products in a trusted environment. this work involves an initial analysis of domainspecific mt for public taxonomies on software product descriptions available in linkedin web pages using a novel approach of sub-corpora extraction. our contributions in this work are as follows: (i) we develop a first ever parallel development and test corpus of software product descriptions which are originally written in english and then manually translated into french; we used this corpus to tune the system parameters and to test our mt systems, and (ii) we investigate methods for filtering a parallel corpus; first, we use laser @xcite , the state-of-the-art tool for bitext mining with the help of measuring bilingual sentence similarity and then we use keybert @xcite to extract key phrases from the indomain data set developed by us, which is further used to extract relevant parallel texts from several corpora. more precisely, we exploit our development data set to extract parallel data which is similar to the domain of software production. we refer to this extracted data as subcorpus. afterwards, we build mt systems using these sub-corpora for training and the same development data set for tuning parameters. finally, we evaluate all the systems on a separately held-out test data set. our experimental results show that our approach of corpus filtering and keyphrase-based sub-corpus extraction improves the performance of the mt system even when trained on a much smaller data set. ## related work nmt has undergone huge evolution during the last few years. for example, in the shared tasks on news and biomedical translation in wmt 2019, it is found that several nmt systems perform at the same level of a human translator for some high-resource language pairs according to human judgement @xcite . however, for many language pairs and for many domains, there is still no (sufficient) data available in order to build highquality mt systems. domain adaptation is a well-explored research area in mt. the main objective is to facilitate adaptation of the mt system to a specific domain. for example, @xcite propose an approach of lexicon induction to extract an in-domain lexicon and then build a pseudo-parallel in-domain corpus with word-for-word back-translation of monolingual in-domain target sentences. @xcite conduct a survey of the state-of-the-art domain adaptation techniques for neural machine translation (nmt). the work of @xcite demonstrates the usefulness of infrequent n-gram recovery (inr) and feature decay algorithms (fda) for domain adaptation. back-translation (or forward translation) is often used for domain adaptation, too @xcite . the vast majority of investigated mt systems covers only a limited set of domains, predominantly news @xcite @xcite . there is also work on biomedical domain, spoken language @xcite and some types of user-generated content @xcite . however, to the best of our knowledge, there is no previous work that involves the development of mt systems for software product descriptions. moreover, no parallel corpus in this domain has been published so far. ## data development we develop the first ever corpus of software product descriptions in english and their manual translations into french. the corpus is suitable for development and testing of mt systems in this domain. the data set is collected from the linkedin webpage of software product de-scriptions. 2 we scrape the contents of webpages and collect 1, 395 text segments in english on the description of software products. these texts are then manually translated by native french speakers who are also proficient in english. product descriptions are usually different from natural texts and should be translated with special considerations. bearing this in mind, the translators used the following guidelines to perform the translation task: the whole translation process was a challenging task and took a significant amount of time. one of the reasons was the presence of a large number of software or technical terms, some of which are not easy or straightforward to translate into french. people often use them as is, i.e, they keep them in english instead of translating into french. for example, the phrase "stacking-plans" was found to be very difficult to translate into french as its literal translation does not make much sense and therefore it should be left untranslated. in addition, the translators have to remember which terms should be translated and which should not, as they encounter many such terms. for example, the term "cerebra digital assistants" should not be translated as it is the name of a software. once the translation of 1, 395 segments is complete, 696 were held out as tuning data set and the rest 699 as test data set. therefore, all the mt models are tuned on these 696 segment pairs and evaluated on the 699 segment pairs. we refer to this data set as swp corpus foot_4 which is now available online for free access. ## corpus filtering for domain adaptation in this section we describe our proposed approaches of corpus filtering and sub-corpora extraction for domain adaptation using laser and keybert. here we use laser for filtering and keybert is used for extracting domain-specific sub-corpora, respectively. ## experiments • data set: there are a number of different parallel corpora available but not all of them are suitable for building a decent quality mt system. we explore a wide range of corpora available on the opus website. 5 we use europarl corpus for tuning as mentioned earlier in section 4. the optimal value of key terms is then applied to extract sub-corpora from a set of other corpora. although we use several corpora in the initial stage of experiments, we consider using 12 specific corpora in our later stage of experiments and filter them using our proposed approach as discussed earlier. following is a short description of the corpora we used in our experiments. although we could have used more corpora, we decided to use the above 12 corpora because of the following reasons: (i) after manually inspecting several corpora in a random manner, we found the above corpora to be good quality 10 (ii) many of them contain texts from multiple domains and thus are better to be used for domain adaptation using subcorpora extraction, and (iii) some of them are from software domain which is useful for our experiments. we also built a separate mt model using only those corpora that belong to software domain but we obtain low bleu score of 10.41 as they are small in size. due to this reason, we decided to combine other corpora as well. initially we use laser and keybert (discussed in section 4) for corpus filtering and sub-corpus extraction, respectively. to build mt models, we use opennmt 11 @xcite with transformer architecture @xcite . subword nmt 12 @xcite is used to apply byte-pair enconding (bpe) during the preprocessing. we use sacrebleu @xcite for automatic evaluation of mt outputs. we perform preprocessing in the following steps: we can notice from the table that the domain-adapted system outperforms the baseline system by 1.4 bleu points which is 4.3% relative improvement. another important observation is that both systems are trained on the same corpora but the domain-adapted system is the filtered version of it. our proposed approach significantly reduces the corpus size by more than three times and at the same time produces the higher bleu score. ## output analysis in this section we show some of the translation outputs produced by our mt system and compare them with the human translated references. ## conclusions and future work in this work we proposed an approach of corpus filtering and sub-corpora extraction by using laser and keybert. in addition, we developed the first ever parallel corpus of software proceedings of the 15th biennial conference of the association for machine translation in the americas orlando, usa, september 12-16, 2022. volume 1: research track english text french translation reference translation instant analysis of millions l'analyse de millions de analyse instantanée de of real-time conversations conversations en temps réel millions de conversations happening online. se déroule en ligne. en ligne en temps réel. find out what your trouvez ce que vos découvrez ce que vos customers are doing, clients font, ressentent clients font, ressentent feeling, and thinking et pensent quand il et pensent de votre when it comes to s'agit de votre marque. marque. your brand. no hidden fees or pas de frais cachés ou pas de frais cachés ni long-term commitments. d'engagements à long terme. d'engagements à long terme. know what matters, savoir ce qui compte, sachez ce qui est important when it matters. quand c'est important. et quand c'est important. meet, call, chat, share meet, appel, chat, réunions, appels, files, manage tasks. partager des fichiers, discussions, partage de gérer des tâches. fichiers, gestion des tâches. learn more about fico apprenez-en plus sur en savoir plus sur analytics workbench, a fico analytics workbench, fico analytics workbench, next-generation analytics un outil d'analyse de un outil d'analyse de tool that empowers nouvelle génération qui dernière génération qui companies to improve permet aux entreprises permet aux entreprises business decisions across d'améliorer les décisions d'améliorer leurs décisions the customer lifecycle. although it is a time consuming task, it is better to manually evaluate at least a small part of the translation outputs which will provide the clearer picture of translation quality to some extent.