SMART : Research Directions

More than half of the EU citizens are not able to hold a conversation in a language other than their mother tongue, let alone to conduct a negotiation, or interpret a law. In a time of wide availability of communication technologies, language barriers are a serious bottleneck to European integration and to economic and cultural exchanges in general. More effective tools to overcome such barriers, in the form of software for machine translation and other cross-lingual textual information access tasks, are in strong demand. Statistical methods are a promising approach, in that they achieve performances equivalent or superior to those of rule-based systems, at a fraction of the development effort. There are, however, some identified shortcomings in these methods, preventing their broad diffusion. SMART 1 is an attempt to address these shortcomings by deploying the methods of modern Statistical Learning. The scientific focus is on developing new and more effective statistical approaches while ensuring that existing know-how is duly taken into account. Field evaluation on three user scenarios will ensure that advances make their way out of the laboratories, in the form of both improvements over existing technologies and of new applications. I. EXTENDED ABSTRACT The multiplication of communication means and their worldwide diffusion make language barriers more critical than ever and create the need for more effective tools, be they in the form of translation aids or of engines for searching multilanguage document collections. Traditionally, Machine Translation (MT) is based on large and complex sets of handwritten rules. Starting from the late 1980s, though, methods for automatically acquiring knowledge from large amounts of manually translated documents have been described and tested [1], [2]. As such knowledge is in the form of parameters of a probabilistic model, these methods are collectively referred to as Statistical Machine Translation (SMT). In most initial probabilistic models the building blocks were pairs of words of the two languages, but recent work [3], [4], 1Statistical Multilingual Analysis for Retrieval and Translation. The SMART Consortium is composed of: • Xerox S.A. S. • Amebis d.o. o, Kamnik • Celer Soluciones S.L. • Jozef Stefan Institute • National research Council Canada • University of Bristol • University of Helsinki • Universit̀a degli Studi di Milano • University of Southampton • University College London. [5], [6] suggests that better results can be obtained if larger building blocks (bi-phrases) are used instead. Currently, the Machine Translation systems performing best in comparative evaluations 2 are phrase-based log-linear probabilistic models. Despite its success, though, the situation of SMT is still only partially satisfactory: 1) All proposals so far involve training model parameters by maximising some function which either cannot be computed exactly or is characterised by many local maxima. Moreover, all approaches assume that at decoding time a complex heuristic search through a combinatorially large set of alternatives is performed. This in turn requires relying on identifying suitable heuristics with little or no performance guarantees. 2) Current SMT systems are trained in batch mode on a training set, and the corresponding translation model is frozen and used at operation time. This is quite unsatisfactory when Computer-Aided Translation tools are envisaged to improve the productivity of professional human translators, a scenario which seems one of the most realistic for actual deployment of MT technologies. In this context, the human translator is rapidly annoyed by a system that will stubbornly repeat the same incorrect translation for repeated or similar source language material. 3) Probabilistic models strongly depend on the collection of manually translated documents they are trained on. While some large collections of documents with their translations (parallel corpora) are indeed easily available, (Europarl [7], ACQUIS [8], MULTEXT-East, Canadian Hansard, Hong Kong Hansard), it is often the case that no large parallel corpus is available for a given language pair in the domain and genre of interest. In such cases, the system designer is left with the uncomfortable choice of training on a small corpus of appropriate documents or on a larger but less appropriate corpus. 4) Another major obstacle for current SMT systems, and a disadvantage with respect to rule-based systems, consists in the difficulty of producing fluent output. While all SMT systems do incorporate some form of language modelling to ensure that the readability of the target sentence is somehow taken into account, most of them rely on very simple n-gram based models. Such models 2http://www.nist. gov/speech/tests/mt/. are indeed easy to train and fast to apply at decoding time, but they only consider local phenomena, and sometimes lead to grossly ungrammatical and hardly intelligible output. 5) Cross-Lingual Textual Information Access (CLTIA) is a generic term which covers Cross-Language Information Retrieval (CLIR), Categorization/Clustering (CLC/C) and terminology extraction. In current CLTIA systems, queries and document fragments are generally translated word by word, even though multi-word expressions and other contextual dependencies are often crucial to solve translation ambiguities. This raises the important issue of designing or adapting bilingual lexicon extraction methods, language models and tools used in CLTIA to deal with general, non-contiguous multi-word units. 6) Some recent approaches to CLTIA rely on kernel methods in latent, language-independent concept spaces. Such methods do not aim at precisely identifying word translations, but rather at establishing the most important interlingual concepts in a collection of documents, thus capturing different aspects of the translation process. What is the most appropriate combination of latent, language independent analysis and advanced “surface” translation methods is still an open question. The focus of SMART will be on addressing these issues. Research directions that will be considered can be stated as follows: 1) Recent advances in Machine Learning show that complex problems such as natural language parsing, that used to be considered highly combinatorial and to require heuristic search, can be formalised as global optimisation problems over convex functions of possibly very many variables [9]. We will follow the same strategy to approach the problem of phrase-based machine translation. A first possibility consists in retaining a model in which relatively complex features are extracted from pairs of a source sentence and a candidate translation, but the relative weight of such features is estimated by maximising margin instead of likelihood or smoothed error. A second, more radical solution, consists in departing completely from the complex features currently used in SMT and adopting a kernel-based approach to implicitly consider as features, and weight independently, all possible co-occurrences of word sequences in the source and in the target language. 2) Not much attention (see [10] for an exception) has been devoted to the development of adaptive SMT models. Within the project we will define and implement a theoretical framework for modelling situations in which the user provides feedback by either accepting the proposed translation or by correcting it. This will form the basis for a new class of algorithms for online training of translation models, be they loglinear phrase-based models or large-margin models such as those considered in the previous point. In this new setting, the model is initialised by means of a training set, but undergoes constant evolution as the human translator provides feedback. If this update can be done successfully, then this would lead to a class of computerassisted translation tools that would be better accepted by actual end users. 3) Domain-specific translation model adaptation is in a similar seminal stage. Hildebrand and co-workers [11] propose IR techniques to select from the training set sentence-pairs whose source is similar to a given testset, and train only on those. Munteanu and co-workers [12] go further, and propose a classifier for identifying sentences which are translation of one another in a comparable corpus. In the course of the project we will investigate these and other domain-adaptation mechanisms, both for the case in which the original training set is still available and the case in which only the trained model is available, but not the data it was trained on. 4) The main approach followed so far for improving translation fluency consists in taking the syntax of the source sentence (as automatically determined by a parser) into account, in a tree-to-string [13] or even a tree-to-tree [14], [15] mapping. While this can be effective, its success depends on the availability of a syntactic parser, which is a complex resource to create. As an alternative, we propose to investigate methods that do not require explicit syntax modelling. String kernels [16] can for instance leverage long-distance information not available to standard n-gram based language models. Rational kernels [17] can be used to take morphological information into account. 5) We will explore what combinations or variations of latent, language-independent analysis methods are most appropriate for CLTIA tasks. We will also explore to what extent some techniques from phrase-based SMT and multilingual lexicon extraction can be used to support latent methods for CLTIA and, reciprocally, how phrase-based SMT could usefully exploit results from latent, language-independent analysis. 6) The main approach to CLIR used so far consists in using available bilingual resources to translate queries prior to a standard monolingual search [18]. However, most such resources provide word-to-word translation, without any contextual clues, an