SMT for restricted sublanguage in CAT tool context at the European Parliament

This paper shows that it is possible to efficiently develop Statistical Machine Translation (SMT) systems that are useful for a specific type of sublanguage in real context of use even when excluding the exact Translation Memory (TM) matches from the test set in order to be integrated in CAT "Computer Aided Translation" tools. It means that the included part is quite different from the existing translations and consequently harder to translate even for an SMT system trained on the same translation data. Because we believe on the proximity of sublanguages even though it is still hard to practically define the sublanguage notion, we are proposing on the framework of the MT@EP project at the Directorate General for Translation (DG TRAD) of the European Parliament (EP) to develop SMT systems specific for each EP Parliamentary Committee optimised for restricted sublanguages and constrained by the EP's particular translation requirements. Sublanguage-specific systems provide better results than generic systems for EP domains showing a very significant quality improvement (5-25% of BLEU score), mainly due to the EP context specificity and to the proximity of sublanguages. This approach is also satisfactory for pairs of under-resourced languages, such as the Slavic families and German. 1. Previous work In general, a sublanguage is a subset of the language (Harris, 1970) identified with a particular semantic domain or a linked family of domains (Kittredge, 1978), (Kittredge, 1982). In our previous research (Hajlaoui, 2008), we showed that, despite the great distance between mother languages (e.g. Arabic and French) (Hajlaoui, Daoud, & Boitet, 2008), the two correspondent sublanguages are very near one to another as shown in Figure 1. It was a new illustration of Kittredge's analysis (Kittredge, 1993), (Kittredge & Lehrberger, 1982). Translating and The Computer 36 229 Figure 1: Sublanguages are very near one to another We also showed that SMT system works very well for small sublanguages with a very small training corpus (less than 10 000 words) (Hajlaoui & Boitet, 2008). This proves that, in the case of very small sublanguages, SMT may be of sufficient quality, starting from a corpus 100 to 500 smaller than for the general language. We are proving in this work the validity of this approach in real context of use clarifying and answering some related questions. We describe also the type of resources we need, mainly Thematic Translation Memory (TTM) presenting some promising results. The issue consists to do further research on the type of sublanguages for which it is possible to develop efficient (useful) SMT systems in the context of CAT tools. In the following section, we are testing our conjuncture which consists that SMT systems work very well for domain sublanguages using small training corpus. 2. SMT applied on sublanguages Assuming that a sublanguage is a subset of the language identified with a particular semantic (family of) domain, in our EP context, health, environment, economy, etc. seems to constitute the restricted sublanguages we are looking for. In the context of existing applications developed in the DG TRAD, one of the constraints to take into account concerning the use of MT in the EP workflows is to take the ad-hoc vocabulary to translate EP documents (amendments, laws, etc.). Our objective is to help EP translators by reusing an existing base of Translation Memory data to better translate unmatched sentences. Our technical choice involves automatic selection of data to resolve problems of context and quality. The corpus must obviously reach a critical size to allow reliable statistical treatment. SMT approach works very well for restricted domains with little or no human revision, for example the rules-based TAUM-METEO system is purposely developed for the weather service in Canada to provide weather forecasts in French and English. Based on some statistical information, we know that environment (ENVI), economy (ECON), and Control of budget (CONT) are ones of the main domains in terms of number of documents treated at the EP. Consequently, we built SMT systems for those domains using the Moses decoder (Koehn, et al., 2007) with the phrase-based factored translation models (Koehn & Hieu, 2007) to mainly translate from English to French. The language models for French were 3-gram ones over each training domain data using the IRSTLM toolkit (Federico, 2008). We used Translating and The Computer 36 230 Minimum Error Rate Training (MERT) (Och, 2003) to optimize the systems. The domain data is randomly split to three sets (training, tuning, and testing). The following tables (Table 1, Table 2 and Table 3) show BLEU, METEOR and TER scores of our English-to-French SMT systems build specifically for three domains announced above comparing them to the existing "Generic" systems and to "Google" MT systems. The scores are computed on tokenized, truecased text, using the MultEval tool version 0.5. 1 (Clark, 2011). The BLEU score show that the improvement can reach 25% of a "Specific" system over the "Generic" system and much more (about 29%) over "Google" system depending on the domain. MT systems Training set (Nb. Sent) Tuning set (Nb. Sent) Testing set (Nb. Sent) BLEU METEOR TER Specific 49003 102