Modelling Source- and Target- Language Syntactic Information as Conditional Context in Interactive Neural Machine Translation (2020)

Abstract: In interactive machine translation (MT), human translators correct errors in automatic translations in collaboration with the MT systems, which is seen as an effective way to improve the productivity gain in translation. In this study, we model source-language syntactic constituency parse and target-language syntactic descriptions in the form of supertags as conditional context for interactive prediction in neural MT (NMT). We found that the supertags significantly improve productivity gain in translation in interactive-predictive NMT (INMT), while syntactic parsing somewhat found to be effective in reducing human effort in translation. Furthermore, when we model this source- and target-language syntactic information together as the conditional context, both types complement each other and our fully syntax-informed INMT model statistically significantly reduces human efforts in a French–to–English translation task, achieving 4.30 points absolute (corresponding to 9.18% relative) improvement in terms of word prediction accuracy (WPA) and 4.84 points absolute (corresponding to 9.01% relative) reduction in terms of word stroke ratio (WSR) over the baseline.

## introduction interactive mt (imt) is viewed as an effective mean to increase productivity in the translation industry. in principle, imt aims to reduce human c 2020 the authors. this article is licensed under a creative commons 3.0 licence, no derivative works, attribution, cc-by-nd. effort in automatic translation workflows by employing an iterative collaborative strategy with its two most important components, the human agent and the mt engine. figure 1 represents the interactive protocol. ref: we decide therefore, citizens, to take control of things. as of today, nmt @xcite represents the state-of-theart in mt research. this has led researchers to test interactive-predictive protocol on nmt too, and papers @xcite that pursued this line of research suggest that nmt is superior than phrase-based statistical mt @xcite as far as interactive-predictive translation is concerned. in a different mt research context, @xcite have successfully integrated ccg (combinatory categorical grammar) syntactic categories @xcite into the target-side of the then state-of-the-art recurrent neural network (rnn) mt models @xcite . in this work, we investigate the possibility of modelling the target-language syntax in the form of supertags @xcite as a conditional context in an interactivepredictive protocol on transformer @xcite , the current state-of-the-art nmt model. in a reference-simulated setting, we found that our target-language syntax-informed interactive setup can significantly reduce human effort in a frenchto-english translation task. we also extract syntactic features from constituency-based parse trees of the source french sentences following @xcite , and use them as the conditional context in the interactive-predictive transformer framework. experiments show that this contextual information can reduce human efforts in translation to some extent. in addition, we apply the above strategies together, and model supertags and constituency parse tree-based features collectively as the conditional context for interactive prediction in nmt. our experimental results indicate that these syntactic feature types are complementary. as a result, this collaborative strategy turns out to be the bestperforming in the french-to-english task while significantly outperforming those setups that include either feature type on wpa and wsr. to the best of our knowledge, this is the very first study that investigates the possibility of integrating syntactic knowledge into an interactive mt model. the strategy of exploiting syntactic knowledge from the source and/or target languages for im-proving the translation quality is not new in mt research. it was successfully applied in the era of classical mt @xcite , and is continually being applied to improve the current state-of-the-art nmt models, e.g. @xcite . ## fully syntactified interactive nmt this section presents our fully syntactified interactive nmt model. in nmt, at time step i, the conditional probability of predicting output token y i given a source sentence x j in our supertag-based interactive-predictive scenario, we first predict the ccg supertag (ŝ i ) of the word (y i ) to be predicted next. as a result, the length of the conditional context becomes twice the number of words in context plus one. as far as the target-syntactified interactive nmt is concerned, the conditional probability of predicting the output token y i is modelled as p(y i |{ŝ 1 , ŷ1 , ..., ŝi-1 , ŷi-1 , ŝi }, x j 1 ), where ŝi-1 1 is the ccg sequence of the validated prefix ŷi-1 1 and ŝi is the supertag of the word (y i ) to be predicted next. as for the modelling of source-side syntax, we extract a chunk sequence from the constituency parse tree of a source sentence by setting random a maximum chunk size ({1...6}) for every sentence (cf. section 4.2). let us define a chunk sequence c m 1 extracted from the input source sentence x j in our fully syntactified interactive nmt model, the conditional probability of predicting the output token y i is modelled as p(y i |{ŝ 1 , ŷ1 , ..., ŝi-1 , ŷi-1 , ŝi }, l j+m 1 ), where ŝi-1 1 is the ccg sequence of the validated prefix ŷi-1 1 , ŝi is the supertag of the word (y i ) to be predicted next, and l j+m 1 is the input sequence constituting j and m numbers of words and chunk identifiers, respectively. ## results and discussion in this section, first we explain the strategy that we adopted for evaluating the interactive-predictive mt systems. then, we present our evaluation results along with some discussions and analysis. ## conclusion in this paper, we have integrated a rich and complex syntactic knowledge in the form of supertags and/or syntactic constituency parse into the current state-of-the-art neural mt model, transformer. furthermore, we tested whether integration of such knowledge sources into transformer could indeed reduce human efforts in translation in an interactive-predictive scenario. we carried out our experiments on french-to-english, a high resource widely-used translation-pair in industry. we compared our syntax-informed and baseline transformer models on an interactive-predictive platform. the use of syntactic constituency parse as conditional context has minor impact on reducing human effort in translation. we modelled target-language supertags as conditional context in interactive nmt in two different ways, and both of these significantly positively impact productivity in translation. interestingly, supertags (target) and constituency parse (source) together as a context turns out to be the best-performing setup with significant gains over either feature type. in this sense, we can say that source and target-side syntactic contextual features complement as far as neural interactive prediction is concerned. in fact, the conditional context in this setup includes both source-language constituency parse and target-language ccg, which essentially provides the inmt model with better syntactic agreement between the source and target sentences. we conjecture that this could be the reason why this collaborative strategy turned out to be best-performing. our analysis shows that the onflyccg setup (where ccg assigned by on the fly ccg supertagger) significantly outperformed predccg (where ccg predicted by transformer) in terms of assigning correct ccg to the words of user's choice by large margins (17.06% to 17.12%). in fact, our proposed setup (onflyccg), to a certain extent, provides a way to inject correct context into the interactive model. this could be the reason why onflyccg turned out to be best-performing. our analysis unraveled many sides of our syntax-aware models in an interactive-predictive environment. for an example, we particularly found that our syntax-informed interactivepredictive models have positively impacted more for the translation of longer sentences. given the importance of interactive mt in translation industry, the findings of this work can be crucial for their production as our methods can positively impact their productivity gain in translation. given the fact that linguistic tools such as supertaggers and constituency parsers are only readily available for a handful of languages, in future, we will continue to pursue this line of investigation with exploring integration of languageindependent contextual knowledge in interactivepredictive nmt. in future, we plan to evaluate our interactive mt systems with human agents.