Fast, Scalable Phrase-Based SMT Decoding (2016)

Abstract: The utilization of statistical machine translation (SMT) has grown enormously over the last decade, many using open-source software developed by the NLP community. As commercial use has increased, there is need for software that is optimized for commercial requirements, in particular, fast phrase-based decoding and more efficient utilization of modern multicore servers. In this paper we re-examine the major components of phrase-based decoding and decoder implementation with particular emphasis on speed and scalability on multicore machines. The result is a drop-in replacement for the Moses decoder which is up to fifteen times faster and scales monotonically with the number of cores.

## introduction smt has steadily progressed from a research discipline to commercial viability during the past decade as can be seen from services such as the google and microsoft translation services. as well as general purpose services such as these, there is a large number of companies that offer customized translation systems, as well as companies and organization that implement in-house solutions. many of these customized solutions use moses as their smt engine. for many users, decoding is the most time-critical part of the translation process. making use of the multiple cores that are now ubiquitous in todays servers is a common strategy to ameliorate this issue. however, it has been noticed that the moses decoder, amongst others, is unable to efficiently use multiple cores @xcite . that is, decoding speed does not substantially increase when more cores are used, in fact, it may actually decrease when using more cores. there have been speculation on the causes of the inefficiency as well as potential remedies. this paper is the first we know of that focuses on improving decoding speed on multicore servers. we take a holistic approach to solving this issue, creating a decoder that is optimized for multi-core processing speed by concentrating on four main areas: the result is a decoder that is significantly faster than the moses baseline for singlethreaded operation, and scales with the number of cores. we will maintain the moses decoder's embarrassingly parallel, one sentence-per-thread decoding framework. as far as possible, model scores and functionality are compatible with moses to aid comparison and ease transition for existing users. the source code is available in the existing moses repository foot_0 . the rest of the paper will be broken up into the following sections. the rest of this section will discuss prior work and an outline of the phrase-based model. section 2 will then describe the modifications to improve decoding speed. we describe the experiment setup in section 3 and present results section 4. we conclude in the last section and discuss possible future work. ## proposed improvements we will also concentrate on four main areas for optimization. ## experimental setup we trained a phrase-based system using the moses toolkit with standard settings. the training data consisted of most of the publicly available arabic-english data from opus (tiedemann, 2012) containing over 69 million parallel sentences, and tuned on a held out set. the phrase-table was then pruned, keeping only the top 100 entries per source phrase, according to p(t|s). all model files were then binarized; the language models were binarized using kenlm @xcite , the phrase table using the probing phrase-table, lexicalized reordering model using the compact data structure @xcite . these binary formats were chosen for their best-in-class multithreaded performance. standard moses phrase-based configurations are used, except that we use the cube-pruning algorithm @xcite with a pop-limit of 400 foot_3 , rather than the basic phrase-based algorithm. the cube-pruning algorithm is often employed by users who require fast decoding as it gives them the ability to trade speed with translation quality via a simple pop-limit parameter. as a baseline, we use a recent foot_4 version of the moses decoder taken from the github repos- for all experiments, we used a dell poweredge r620 server with 16 cores, 32 hyperthreads, split over 2 physical processors (intel xeon e5-2650 @ 2.00ghz). the server has 380gb ram. the operating system was ubuntu 14.04, the code was compiled with gcc 4.8.4 and boost 1.59 foot_5 and the tcmalloc library.