Venue: NAACL 2016

# A Recurrent Neural Networks Approach

for Estimating the Quality of Machine Translation Output

Hyun Kim

Creative IT Engineering,

Pohang University of Science and

Technology (POSTECH),

Polhang, Republic of Korea

hkim.postech@gmail.com&Jong-Hyeok Lee

Computer Science and Engineering,

Pohang University of Science and

Technology (POSTECH),

Polhang, Republic of Korea

jhlee@postech.ac.kr

###### Abstract

This paper presents a novel approach using recurrent neural networks for estimating the quality of machine translation output. A sequence of vectors made by the prediction method is used as the input of the final recurrent neural network. The prediction method uses bi-directional recurrent neural network architecture both on source and target sentence to fully utilize the bi-directional quality information from source and target sentence. Our experiments show that the proposed recurrent neural networks approach achieves a performance comparable to the existing state-of-the-art models for estimating the sentence-level quality of English-to-Spanish translation.

## 1 Introduction

Estimating the quality of machine translation output, called _quality estimation_ (QE) [20]; Blatz et al., 2004], is to predict quality scores/categories for unseen machine-translated sentences without reference translations at various granularity levels (sentence-level/word-level/document-level). Quality estimation is of growing importance in the field of machine translation (MT) since MT systems are widely used and the quality of each machine-translated sentence is able to vary considerably.

Previous research on QE, addressed as a regression/classification problem to compute quality scores/categories, has mainly focused on feature extraction and feature selection. Feature extraction is to find the relevant features, such as baseline features [3] and latent semantic indexing (LSI) based features [5], capturing various aspects of quality from source and target sentences1 and external resources. Feature selection is to select the best features by using selection algorithms, such as Gaussian processes [7] and heuristic [8], among already extracted features. Finding desirable features has played a key role in the QE research.

Footnote 1: In this paper, a ‘target sentence’ means the machine-translated sentence from a source sentence.

In this paper we present a recurrent neural networks approach for estimating the quality of machine translation output at sentence level, which does not require manual effort for finding the best relevant features. The remainder of this paper is organized as follows. In Section 2, we propose a recurrent neural networks approach using a sequence of vectors made by the prediction method as input for quality estimation. And we describe the prediction method using bi-directional recurrent neural networks architecture in Section 3. In Section 4, we report evaluation results, and conclude our paper in Section 5.

## 2 Recurrent Neural Networks Approach for Estimating Quality Score

Because recurrent neural networks (RNNs) have the strength for handling sequential data [6], we apply RNNs to estimate the quality score of translation.

The input of the final RNN is a sequence of vectors that have quality information about whether tar

get words in a target sentence are properly translated from a source sentence. We will refer to this sequence of vectors as _quality vectors_ (\(q_{y_{1}},...~{},q_{y_{T_{y}}}\)). Each quality vector \(q_{y_{j}}\)2 has the quality information about how well a target word \(y_{j}\) in a target sentence \(\mathbf{y}=(y_{1},\,...\,,y_{{}_{T_{y}}})\) is translated from a source sentence3\(\mathbf{x}=(x_{1},\,...\,,x_{{}_{T_{x}}})\). Quality vectors are generated from the prediction method (of Section 3).

Footnote 2: I \(\leqq j\leq T_{y}\) where \(T_{y}\) is the length of target sentence.

To predict a quality estimation score (QE score) as an HTER score [5] in [0,1] for each target sentence, a logistic sigmoid function is used such that

\[\begin{split}&\mathrm{QE\,score}(\mathbf{y},\mathbf{x})\\ &=\mathrm{QE\,score^{\prime}}(q_{y_{1}},\,...\,,q_{y_{T_{y}}})\\ &=\sigma(W_{{}_{QE}}^{\top}\,\mathbf{s})\end{split} \tag{1}\]

where \(\mathbf{s}\) is a summary unit of the whole quality vectors and \(W_{{}_{QE}}\in\mathbb{R}^{r}\). \(r\) is the dimensionality of summary unit.

To get the summary unit \(\mathbf{s}\), the hidden state \(v_{j}\) employing \(p\) gated hidden units for the target word \(y_{j}\) is computed by

\[v_{j}=f(q_{y_{j}},v_{j-1})\,. \tag{2}\]

The gated hidden unit [7] for the activation function \(f\) is used to learn long-term dependencies of translation qualities for target words. We consider the QE score as the integrated/condensed value reflecting the sequential quality information of sequential target words. Because the last hidden state \(v_{{}_{T_{y}}}\) is a summary of the sequential quality vectors, we fix the summary unit \(\mathbf{s}\) to the last hidden state \(v_{{}_{T_{y}}}\).

## 3 Prediction method using Bi-directional RNN Architecture to Make Quality Vectors

In this section, we detail the ways to get the quality vectors (\(q_{y_{1}},...\,,q_{y_{T_{y}}}\)) for computing QE score.

Since the training data for QE4 are not enough to use a neural networks approach for making quality vectors, we use an alternative based on large-scale parallel corpora such as Europarl. We modify the word prediction method of RNN Encoder-Decoder [6] using parallel corpora to make the quality vectors.

Footnote 4: These data, provided in WMT Quality Estimation Shared Task, consist of source sentences, target sentences, and quality scores.

In subsection 3.1, we describe the underlying word prediction method of RNN Encoder-Decoder. We i) extend the prediction method to use the additional backward RNN architecture on target sentence in subsection 3.2 and ii) modify to get the quality vectors (\(q_{y_{1}},...\,,q_{y_{T_{y}}}\)) in subsection 3.3.

Figure 1: An illustration of the proposed recurrent neural networks model for quality estimation



Figure 1 is the graphical illustration of the proposed RNNs approach.

### Word Prediction Method of RNN Encoder-Decoder

RNN Encoder-Decoder proposed by Cho et al. (2014) is able to predict the target word given a source sentence and all preceding target words by using a softmax function. And it is extended by Bahdanau et al. (2015) to use information of relevant source words for predicting the target word such that

\[p(y_{j}| \left\{y_{1},...,y_{j-1}\right\},\mathbf{x}) \tag{3}\] \[=g(y_{j-1},\vec{s}_{j-1},c_{j})\,.\]

\(g\) is a nonlinear function predicting the probability of \(y_{j}\). \(\vec{s}_{j-1}\) is the hidden state of the forward RNN on target sentence and contains information of preceding target words \(\{y_{1},...~{},y_{j-1}~{}\}\). \(c_{j}\) is the context vector which means relevant parts of source sentence associated with the target word \(y_{j}\). \(\vec{s}_{j-1}\) and \(y_{j-1}\) are related to all preceding target words \(\{y_{1},...,y_{j-1}\}\), and \(c_{j}\) is related to \(\mathbf{x}\) in the word prediction function of (3).

### Additional Backward RNN Architecture on Target Sentence

Bahdanau et al. (2015) introduce bi-directional RNN architecture only on source sentence to extend RNN Encoder-Decoder. In our proposed QE model, bi-directional RNN architecture is used both on source and target sentence. By applying bi-directional RNN architecture both on source and target sentence, we can fully and bi-directionally utilize source and target sentence for predicting target words, such that

\[p( y_{j}| \mathbf{y}_{\hat{\boldsymbol{y}}y_{j}},\mathbf{x}) \tag{4}\] \[=g([y_{j-1};y_{j+1}],[\vec{s}_{j-1};\vec{s}_{j+1}],c_{j})\] \[=\frac{\exp(y^{\top}_{j}W_{o_{1}}W_{o_{2}}t_{j})}{\sum\nolimits_{ k=1}^{K_{y}}\exp(y^{\top}_{k}W_{o_{1}}W_{o_{2}}t_{j})}\,,\]

which is the extended version of (3) using the additional backward RNN architecture.5

Footnote 5: The additional backward RNN on target sentence use the context vectors shared by the forward RNN on target sentence.

To reflect further all following target words \(\{y_{j+1},...~{},y_{T_{Y}}\}\) when predicting the target word \(y_{j}\), the hidden state \(\widetilde{s}_{j+1}\) of the backward RNN and the next target word \(y_{j+1}\) are added. \([\vec{s}_{j-1};\overline{s}_{j+1}]\) and \([y_{j-1};y_{j+1}]\) are related to \(\mathbf{y}_{\hat{\boldsymbol{y}}y_{j}}\)6, and \(c_{j}\) is related to \(\mathbf{x}\) in the word prediction function of (4).

Footnote 7: Original \(\tilde{t}_{j}\)Bahdanau et al. (2015) is

\(W_{o_{1}}\in\mathbb{R}^{K_{y}\times q}\) and \(W_{o_{2}}\in\mathbb{R}^{q\times l}\) are weight matrices of softmax function. \(K_{y}\) is the vocabulary sizes of target language and \(q\) is the dimensionality of quality vectors. \(l\) is the dimensionality of maxout units such that

\[t_{j}=[max\{\tilde{t}_{j,2k-1},\tilde{t}_{j,2k}\}]_{k=1,...,l}^{\top}\,, \tag{5}\]

where \(\tilde{t}_{j,k}\) is the \(k\)-th element of a vector \(\tilde{t}_{j}\). And

\[\tilde{t}_{j}=S^{\prime}_{o}[\vec{s}_{j-1};\bar{s}_{j+1}]+V^{\prime}_{o}[E_{ \mathbf{y}}y_{j-1};E_{y}y_{j+1}]+C_{o}c_{j}, \tag{6}\]

where \(S^{\prime}_{o}\in\mathbb{R}^{2l\times 2n}\), \(V^{\prime}_{o}\in\mathbb{R}^{2l\times 2m}\), and \(C_{o}\in\mathbb{R}^{2l\times 2n}\). \(E_{\mathbf{y}}\in\mathbb{R}^{m\times K_{y}}\) is the word embedding matrix on target sentence. \(m\) and \(n\) are the dimensionality of word embedding and hidden states of forward and backward RNNs. The hidden state \(\bar{s}_{j+1}\) of the backward RNN and next target word \(y_{j+1}\) are used in (6).8

Footnote 9: This is softmax function, the bias term is not used for the simplicity of deriving the quality vectors. Generally, bias terms are visually omitted in other equations to make the equations unculttered.

From the extended prediction method of (4), the probability of the target word \(y_{j}\) is computed by using information of relevant source words in source sentence \(\mathbf{x}\) and all target words surrounding the target word \(y_{j}\) in target sentence.

### Quality Vectors on Target Sentence

Word prediction method predicts the probability of target words as a number between 0 and 1. But we want to get quality vectors of \(q\)-dimensionality which have the more intrinsic quality information for target words.

To make quality vectors, we regard that the probability of the target word involves the quality information about whether the target word \(y_{j}\) in target sentence is properly translated from source sentence. Thus, by decomposing the sof 

the quality vector \(q_{y_{j}}\) for the target word \(y_{j}\) is computed by

\[q_{y_{j}}=\left[~{}row_{y_{j}}(W_{o_{1}})~{}\circ~{}[W_{o_{2}}t_{j}]^{\top}~{} \right]^{\top}, \tag{7}\]

where \(\circ\) is an element-wise multiplication. All of quality information about possible \(K_{y}\) target words at position \(j\) of target sentence is encoded in \(t_{j}\). Thus, by decoding \(t_{j}\), we are able to get quality vector \(q_{y_{j}}\) for the target word \(y_{j}\in\mathbb{R}^{K_{y}}\) at position \(j\) of target sentence. Figure 2 and 3 show the ways to compute the quality vector \(q_{y_{j}}\).

## 4 Experiments

The proposed RNNs approach was evaluated on the WMT15 Quality Estimation Shared Task9 at sentence level of English-Spanish.

Footnote 9: [http://www.statmt.org/wmt15/quality-estimation-task.html](http://www.statmt.org/wmt15/quality-estimation-task.html)

We trained10 the proposed model through a two-step process. First, by using English-Spanish parallel corpus of Europarl v7 Koehn (2005), we trained bi-directional RNNs having 1000 hidden units on source and target sentence to make quality vectors. Next, by using the training set of WMT15 QE task, to predict QE scores we trained the final RNN that use the quality vectors generated in previous step as the input and have 100 hidden units.

Footnote 10: Stochastic gradient descent (SGD) algorithm with adaptive learning rate Adadelta Zeiler (2012) is used to train the proposed model.

Table 1 and 2 present the results of the proposed approach (Bi-RNN) and the official results for the scoring and ranking11 variants of the WMT15 Quality Estimation Shared Task at sentence level. At both variants of the task, the proposed RNNs approach achieved the performance over the baseline performance. Also our experiments showed that the performance of the proposed RNNs approach is included to the best performance group (at the scoring variant of Table 1) or is close to the best performance group (at the ranking variant of Table 2).

Footnote 11: The ranking variant of the QE task measures how close a proposed ranking of target translations from best to worst is to the true ranking.

## 5 Conclusion

This paper proposed a recurrent neural networks approach using quality vectors for estimating the quality of machine translation output at sentence level.

Figure 3: The ways of computing the quality vector \(q_{y_{j}}\) ( \(\circ\) is an element-wise multiplication)

\begin{table}
\begin{tabular}{|c|c c|} \hline System ID & **DetaAvg**\(\uparrow\) & Spearman’s \(\rho\)\(\uparrow\) \\ \hline \(\bullet\) IORIA/17+LSI+MT+FILTRE & 6.51 & 0.36 \\ \(\bullet\) LORIA/17+LSI+MT & 6.34 & 0.37 \\ \(\bullet\) RTM-DCU/RTM-FS+PLS-SVR & 6.34 & 0.37 \\ \(\bullet\) RTM-DCU/RTM-FS-SVR & 6.09 & 0.35 \\ \hline
**Bi-RNN** & **6.08** & **0.33** \\ Baseline SVM & 2.16 & **0.13** \\ \hline \end{tabular}
\end{table}
Table 2: Proposed approach (Bi-RNN) results and official results for the **ranking variant** of WMT15 Quality Estimation Shared Task at sentence level. A total of 4 tied official winning systems are indicated by a \(\bullet\). DeltaAvg metric is used as a primary metric Bojar et al. (2015).

 

This approach does not require manual effort for finding the best relevant features which the previous QE research has mainly focused on.

To make quality vectors we used an alternative prediction method based on large-scale parallel corpora, because the QE training data were not enough. By extending the prediction method to use bi-directional RNN architecture both on source and target sentence, we were able to fully utilize the bi-directional quality information from source and target sentence for quality estimation.

The proposed RNNs approach achieved a performance comparable to the existing state-of-the-art models at sentence-level QE. Our experiments have showed that RNNs approach is a meaningful step for QE research. Applying RNNs approach to word-level QE and studying other ways to make quality vectors better are remained for the future study.

## Acknowledgments

This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the "ICT Consilience Creative Program" (IITP-2015-R0346-15-1007) supervised by the IITP (Institute for Information & communications Technology Promotion)

## References

* [Bahdanau et al.2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In _ICLR 2015_.
* Blatz et al. [2004] John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto Sanchis, and Nicola Ueffing. 2004. Confidence estimation for machine translation. In _Proceedings of the 20th international conference on Computational Linguistics_, page 315. Association for Computational Linguistics.
* Bojar et al. [2015] Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scanton, Lucia Specia, and Marco Turchi. 2015. Findings of the 2015 workshop on statistical machine translation. In _Proceedings of the Tenth Workshop on Statistical Machine Translation_, pages 1-46, Lisbon, Portugal, September. Association for Computational Linguistics.
* Cho et al. [2014] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In _Conference on Empirical Methods in Natural Language Processing (EMNLP 2014)_.
* Gonzalez-Rubio et al. [2013] Jesus Gonzalez-Rubio, J Ramon Navarro-Cerdan, and Francisco Casacuberta. 2013. Dimensionality reduction methods for machine translation quality estimation. _Machine translation_, 27(3-4):281-301.
* Goodfellow et al. [2015] Ian Goodfellow, Aaron Courville, and Yoshua Bengio. 2015. Deep learning. Book in preparation for MIT Press.
* Koehn [2005] Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In _MT summit_, volume 5, pages 79-86. Citeseer.
* Langlois [2015] David Langlois. 2015. Loria system for the wmt15 quality estimation shared task. In _Proceedings of the Tenth Workshop on Statistical Machine Translation_, pages 323-329, Lisbon, Portugal, September. Association for Computational Linguistics.
* Shah et al. [2015] Kashif Shah, Trevor Cohn, and Lucia Specia. 2015. A bayesian non-linear method for feature selection in machine translation quality estimation. _Machine Translation_, pages 1-25.
* Snover et al. [2006] Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In _Proceedings of association for machine translation in the Americas_, pages 223-231.
* Specia et al. [2009] Lucia Specia, Marco Turchi, Nicola Cancedda, Marc Dymetman, and Nello Cristianini. 2009. Estimating the sentence-level quality of machine translation systems. In _13th Conference of the European Association for Machine Translation_, pages 28-37.
* Specia et al. [2013] Lucia Specia, Kashif Shah, Jose GC De Souza, and Trevor Cohn. 2013. Quest-a translation quality estimation framework. In _ACL (Conference System Demonstrations)_, pages 79-84. Citeseer.
* Zeiler [2012] Matthew D Zeiler. 2012. Adadelta: An adaptive learning rate method. _arXiv preprint arXiv:1212.5701_.