Venue: NAACL 2016

# Interpretese vs. Translationese:

The Uniqueness of Human Strategies in Simultaneous Interpretation

 He He Computer Science University of Maryland hhe@cs.umd.edu &Jordan Boyd-Graber Computer Science University of Colorado Jordan.Boyd.Graber @colorado.edu &Hal Daume III Computer Science and umiacs University of Maryland hal@cs.umd.edu

###### Abstract

Computational approaches to simultaneous interpretation are stymied by how little we know about the tactics human interpreters use. We produce a parallel corpus of translated and simultaneously interpreted text and study differences between them through a computational approach. Our analysis reveals that human interpreters regularly apply several effective tactics to reduce translation latency, including sentence segmentation and passivization. In addition to these unique, clever strategies, we show that limited human memory also causes other idiosyncratic properties of human interpretation such as generalization and omission of source content.

## 1 Human Simultaneous Interpretation

Although simultaneous interpretation has a key role in today's international community,1 it remains under explored within machine translation (mt). One key challenge is to achieve a good quality/speed trade-off: deciding when, what, and how to translate. In this study, **we take a data-driven, comparative approach and examine:** (i) What distinguishes simultaneously interpreted text (Interpretsee2) from batch-translated text (Translationes)? (ii) What strategies do human interpreters use?

Footnote 1: Unlike consecutive interpretation (speakers stop after a complete thought and wait for the interpreter), simultaneous interpretation has the interpreter to translate _while_ listening to speakers.

Footnote 2: Language produced in the process of translation is often considered a dialect of the target language: “Translationsee” Baker (1993). Thus, “Interpretese” refers to interpreted language.

Most previous work focuses on qualitative analysis Bendazzoli and Sandrelli (2005); Camayd-Freixas (2011); Shimizu et al. (2014) or pattern counting Tohyama and Matsubara (2006); Sridhar et al. (2013). In contrast, we use a more systematic approach based on feature selection and statistical tests. In addition, most work ignores _translated_ text, making it hard to isolate strategies applied by interpreters as opposed to general strategies needed for any translation. Shimizu et al. Shimizu et al. (2014) are the first to take a comparative approach; however, they directly train mt systems on the interpretation corpus without explicitly examining interpretation tactics. While some techniques can be learned implicitly, the model may also learn undesirable behavior such as omission and simplification: byproducts of limited human working memory (Section 4).

Prior work studies simultaneous interpretation of Japanese\(\leftrightarrow\)English Tohyama and Matsubara (2006); Shimizu et al. (2014) and Spanish\(\leftrightarrow\)English Sridhar et al. (2013). We focus on Japanese\(\leftrightarrow\)English interpretation. Since information required by the target English sentence often comes late in the source Japanese sentence (e.g., the verb, the noun being modified), we expect it to reveal a richer set of tactics.3 Our contributions are three-fold. First, we collect new human translations for an existing simultaneous interpretation corpus, which can benefit future comparative research.4 Second, we use classification and feature selection methods to examine linguistic characteris

tics comparatively. Third, we categorize human interpretation strategies, including word reordering tactics and summarization tactics. Our results help linguists understand simultaneous interpretation and help computer scientists build better automatic interpretation systems.

## 2 Distinguishing Translationese and Interpretese

In this section, we discuss strategies used in Interpretese, which we detect automatically in the next section. Our hypothesis is that tactics used by interpreters roughly fall in two non-exclusive categories: (i) _delay minimization_, to enable prompt translation by arranging target words in an order similar to the source; (ii) _memory footprint minimization_, to avoid overloading working memory by reducing communicated information.

SegmentationInterpreters often break source sentences into multiple smaller sentences Camayd-Freixas (2011); Shimizu et al. (2013), a process we call segmentation. This is different from what is commonly used in speech translation systems Fujita et al. (2013); Oda et al. (2014), where translations of segments are directly concatenated. Instead, humans try to incorporate new information into the precedent partial translation, e.g., using "which is" to put it in a clause (Table 1, Example 3), or creating a new sentence joined by conjunctions Table 1, Example 5).

PassivizationPassivization is useful for interpreting from head-final languages (e.g., Japanese, German) to head-initial languages (e.g., English, French) He et al. (2015). Because the verb is needed early in the target sentence but only appears at the end of the source sentence, an obvious strategy is to wait for the final verb. However, if the interpreter uses passive voice, they can start translating immediately and append the verb at the end (Table 1, Examples 4-5). During passivivization, the subject is often omitted when obvious from context.

GeneralizationCamayd-Freixas (2011) and AlKhanji et al. (2000) observe that interpreters focus on delivering the gist of a sentence rather than duplicating the nuanced meaning of each word. More frequent words are chosen as their retrieval time is faster Dell and O'Seaghdha (1992); Cuetos et al. (2006) (e.g., "honorific" versus "polite" in Table 1, Example 1). Although Volansky et al. (2013) show that generalization happens in translation too, it is likely more frequent in Interpretese given the severe time constraints.

SummarizationFaced with overwhelming information, interpreters need efficient ways to encode meaning. Less important words, or even a whole sentence can drop, especially when the interpreter falls behind the speaker. In Table 1, Example 2, the literal translation "as much as possible" is reduced to "very", and the adjective "Japanese" is omitted.

Before we study these characteristics quantitatively in the next section, we visualize Interpretese and Translationales by a word cloud in Figure 1. The size of each word is proportional to the difference between its frequencies in Interpretese and Translationese (Section 3). The word color indicates whether it is more frequent in Interpretese (black) or Translationese (gold). "the" is over-represented in Interpretese, a phenomenon also occurs in Translationese vs. the original text Etemadi and Toutanova (2014). More conjunction words (e.g., "and", "so", "or", "then") are used in Interpretese, likely for segmentation, whereas "that" is more frequent in Translationese--a sign of clauses. In addition, the pronoun "I" occurs more often in Translationese while "be" and "is" occur more often in Interpretese, which is consistent with our passivization hypothesi.

Figure 1: A word cloud visualization of Interpretese (black) and Translationese (gold).

 



## 3 Classification of Translationese and Interpretees

We investigate the difference between Translationese and Interpretese by creating a text classifier to distinguish between them and then examining the most useful features. We train our classifier on a bilingual Japanese-English corpus of spoken monologues and their simultaneous interpretations (Matsubara et al., 2002). To obtain a three-way parallel corpus of aligned translation, interpretation, and their shared source text, we first align the interpreted sentences to source sentences by dynamic programming following Ma (2006).5 This step results in 1684 pairs of text chunks, with 33 tokens per chunk on average. We then collect human translations from Gengo6 for each source text chunk (one translator per monologue). The original corpus has four interpretors per monologue. We use all available interpretation by copying the translation of a text chunk for its additional interpretation.

Footnote 6: [http://gengo.com](http://gengo.com) (“standard” quality).

### Discriminative Features

We use logistic regression as our classifier. Its job is to tell, given a chunk of English text, which translation produced it. We add \(\ell_{1}\) regularization to select the non-zero features that best distinguish Interpretese from Translationese. We experiment with three dif

\begin{table}
\begin{tabular}{c|l|l} \hline  & Source (S), translation (T) and interpretation (I) text & Tactic \\ \hline
1 & (S) \(\Sigma\cap\mathcal{D}\) \#\(\mathcal{D}\)

ferent sets of features: (1) **POS:**\(n\)-gram features of POS tags (up to trigram);7 (2) **LEX:** word unigrams;

Footnote 6: The number of words in the source or target is not a common feature, but it is not a common feature.

(3) **LING:** features reflecting linguistic hypothese (Section 2), most of which are counts of indicator functions normalized by length of the chunk (Appendix A).

The top linguistic features listed in Table 3 are consistent with our hypotheses. The most prominent ones--also revealed by pos and lex--are the segmentation features, including counts of conjunction words (CC), content words (nouns, verbs, adjectives, and adverbs) that appear more than once (repeated), demonstratives (demo) such as _this, that, these, those_, segmented sentences (sent), and proper nouns (NNP). More conjunction words and more sentences in a text chunk are signs of segmentation. Repeated words and the frequent use of demonstratives come from transforming clauses to independent sentences. Next are the passivization features, indicating more passivized verbs (passive) and fewer pronouns (pronoun) in Interpretese. The lack of pronouns may be results of either subject omission during passivization or general omission. The last group are the vocabulary features, showing fewer numbers of stem types, token types, and content words in Interpretese, evidence of word generalization. In addition, a smaller number of content words suggests that interpreters may use more function words to manipulate the sentence structure.

### Classification Results

Recall that our goal is to understand Interpretese, not to classify Interpretese and Translationese; however, the ten-fold cross validation accuracy of LING, POS, LEX are 0.66, 0.85, and 0.94. LEX and POS yield high accuracy as some features are overfitting, e.g., in this dataset, most interpreters used "parsing" for " while the translator used "syntactic analysis". Therefore, they do not reveal much about the characteristics of Interpretese except for frequent use of "and" and CC, which indicates segmentation. Similarly, Volansky et al. (2013) and Eetemadi and Toutanova (2014) also find lexical features very effective but not generalizable for detecting Translationese and exclude them from analysis. One reason for the relatively low accuracy of ling may be inconsistent use of strategies among humans (Section 4).

## 4 Strategy Analysis

To better understand under what situations these tactics are used, we apply two-sample \(t\)-tests to compare the following quantities between Interpretese and Translationese: (1) number of inversions (nonmonotonic translations) on all source tokens (inv-all), verbs (inv-verb) and nouns (inv-noun); (2) number of segmented sentences; (3) number of natural passivization (pass-st), meaning copying a passive construction in the source sentence into the target sentence, and intentional passivization (pass-t), meaning introducing passivization into the target sentence when the source sentence is in active voice; (4) number of omitted words on the source side and inserted words on the target side;8 (5) average word frequency given by Microsoft Web \(n\)-gram--higher means more common.9 For all pairs of samples, the null hypothesis \(H_{0}\) is that the means on Interpretse and Translationese are equal; the alternative hypotheses and results are in Table 2.

Footnote 8: The number of unaligned words in the source or target.

Footnote 9: [http://weblm.research.microsoft.com/](http://weblm.research.microsoft.com/)

As expected, segmentation and intentional passivization happen more often during interpretation. Interpretese has fewer inversions, especially for verbs; reducing word order difference is important for delay minimization. Since there are two to four different interpretations for each lecture, we further analyze how consistent humans are on these decisions. All interpreters agree on segmentation 73.7% of the time, while the agreemen

\begin{table}
\begin{tabular}{l c|c|c c} \hline \hline Ling & & POS & lex \\ \hline CC & + & \textbackslash{S}\) CC & + & And & + \\ repeated & + & \textbackslash{CC} & + & parsing & + \\ demo & + & \textbackslash{C}\) CC IN & + & gradual & – \\ sent & + & NN CC PR & syntax & – \\ passive & + & \textbackslash{CS}\) CC DT & + & keyboard & + \\ pronoun & – & CC RB DT & ++ & attitudinal & – \\ NNP & + & , RB DT & + & text & – \\ stern type & - & CC DT & + & adhoc & + \\ tok type & – & NN FN WN & + & construction & – \\ content & – & NN CC RB & – & Furthermore & – \\ \hline \end{tabular}
\end{table}
Table 3: Top 10 highest-weighted features in each model. The sign shows whether it is indicative of Interpretese (+) or Translationsee (–).

 

only 57.1%--passivization is an acquired skill; not all interpreters use it when it can speed interpretation.

The tests also confirm our hypotheses on generalization and omission. However, these tactics are not inherent to the task of simultaneous interpretation. Instead, they are a byproduct of humans' limited working memory. Computers can load much larger resources into memory and weigh quality of different translations in an instant, thus potentially rendering the speaker's message more accurately. Therefore, directly learning from corpus of human interpretation may lead to suboptimal results Shimizu et al. (2014).

## 5 Conclusion

While we describe how Translationese and Interpretee are different and characterize _how_ they differ, the contribution of our work is not just examining an interesting, important dialect. Our work provides opportunities to improve conventional simultaneous MT systems by exploiting and modeling human tactics. He et al. (2015) use hand-crafted rules to decrease latency; our data-driven approach could yield additional strategies for improving mt systems. Another strategy--given the scarcity and artifacts of interpretation corpus--is to select references that present delay-minimizing features of Interpretese from translation corpus Axelrod et al. (2011). Another future direction is to investigate cognitive inference Chernov (2004), which is useful for semantic/syntactic prediction during interpretation Grissom II et al. (2014); Oda et al. (2015).

## Appendix A Feature Extraction

We use the Berkeley aligner Liang et al. (2006) for word alignment, the Stanford pos tagger Toutanova et al. (2003) to tag English sentences, and Kuromoji 10 to tokenize, lemmatize and tag Japanese sentences. Below we describe the features in detail.

Footnote 10: [http://www.atilika.org/](http://www.atilika.org/)

**Inversion:** Let \(\{A_{i}\}\) be the set of indexes of target words to which each source word \(w_{i}\) is aligned. We count \(A_{i}\) and \(A_{j}\) (\(i<j\)) as an inverted pair if \(\max(A_{i})>\min(A_{j})\). This means that we have to wait until the \(j\)th word to translate the \(i\)th word.

**Segmentation:** We use the punkt sentence segmenter Kiss and Strunk (2006) from NLTK to detect sentences in a text chunk.

**Passivization:** We compute the number of passive verbs normalized by the total number of verbs. We detect passive voice in English by matching the following regular expression: a _be_ verb (be, are, is, was, were etc.) followed by zero to four non-verb words and one verb in its past participle form. We detect passive voice in Japanese by checking that the dictionary form of a verb has the suffix "\(\mathcal{U}\mathcal{G}\)".

**Vocabulary** To measure variety, we use \(V_{t}/N\) and \(V_{s}/N\), where \(V_{t}\) and \(V_{s}\) are counts of distinct tokens and stems, and \(N\) is the total number of tokens. To measure complexity, we use word length, number of syllables per word, approximated by vowel sequences; and unigram and bigram frequency from Microsoft Web \(N\)-gram.

**Summarization** We use the sentence compression ratio, sentence length, number of omitted source words, approximated by counts of unaligned words, and number of content words.

## Acknowledgments

We thank CIAIR Nagoya University, Japan) for providing the interpretation data which formed the foundation of this research. We also thank Alvin Grisson II, Naho Orita and the reviewers for their insightful comments. This work was supported by NSF grant his-1320538. Boyd-Graber is also partially supported by NSF grants CCF-1409287 and nCSE-1422492. Any opinions, findings, conclusions, or recommendations expressed here are those of the authors and do not necessarily reflect the view of the sponsor.

\begin{table}
\begin{tabular}{c|r c r|r|r c|r c|r} \hline Sample & inv-all & inv-verb & inv-noun & segment & pass-t & pass-st & omit & insert & word freq \\ \hline \(H_{a}\) & \multicolumn{3}{c|}{\(\mu_{I}<\mu_{T}\)} & \multicolumn{3}{c|}{\(\mu_{I}>\mu_{T}\)} & \multicolumn{3}{c|}{\(\mu_{I}>\mu_{T}\)} & \multicolumn{3}{c|}{\(\mu_{I}>\mu_{T}\)} & \multicolumn{1}{r|}{\(\mu_{I}>\mu_{T}\)} \\ \(t\)-stat & -1.55 & **-3.81** & **-2.13** & **4.21** & **5.67** & 1.41 & **16.16** & **10.66** & **7.88** \\ \(p\)-value & .12 & \textless{}.001 & .03 & \textless{}.001 & \textless{}.001 & .16 & \textless{}.001 & \textless{}.001 & \textless{}.001 \\ \hline \end{tabular}
\end{table}
Table 2: Two-sample \(t\)-tests for Interprestee and Translationese. The test statistics are bolded when we reject \(H_{0}\) at the 0.05 significance level (two-tailed).

 



## References

* [Al-Khanji et al.2000] Raja Al-Khanji, Said El-Shiyab, and Riyadh Hussein. 2000. On the use of compensatory strategies in simultaneous interpretation. _Journal des Traducteurs_, 45(3):548-577.
* [Axelrod et al.2011] Amittai Axelrod, Xiaodong He, and Jianfeng Gao. 2011. Domain adaptation via pseudo in-domain data selection. In _Proceedings of Empirical Methods in Natural Language Processing (EMNLP)_.
* [Baker1993] Mona Baker. 1993. Corpus linguistics and translation studies: Implications and applications. In Mona Baker, Gill Francis, and Elena Tognini-Bonelli, editors, _Text and Technology: In Honour of John Sinclair_, pages 233-250.
* [Bendazzoli and Sandrelli2005] Claudio Bendazzoli and Annalisa Sandrelli. 2005. An approach to corpus-based interpreting studies: Developing EPIC (european parliament interpreting corpus). In _Proceedings of Challenges of Multidimensional Translation_.
* [Camayd-Freixas2011] Erik Camayd-Freixas. 2011. Cognitive theory of simultaneous interpreting and training. In _Proceedings of the 52nd Conference of the American Translators Association_.
* [Chernov2004] Ghely V Chernov. 2004. _Inference and Anticipation in Simultaneous Interpreting. A Probability-prediction Model_. Amsterdam: John Benjamins Publishing Company.
* [Cuetos et al.2006] F. Cuetos, B. Alvarez B, M. Gonzalez-Nosti, A. Meot, and P. Bonin. 2006. Determinants of lexical access in speech production: role of word frequency and age of acquisition. _Mem Cognit_, 34.
* [Dell and O'Seaghdha1992] G.S. Dell and P.G. O'Seaghdha. 1992. Stages of lexical access in language production. _Cognition_.
* [Eetemadi and Toutanova2014] Sauleh Eetemadi and Kristina Toutanova. 2014. Asymmetric features of human generated translation. In _Proceedings of Empirical Methods in Natural Language Processing (EMNLP)_.
* [Fujita et al.2013] Tomoki Fujita, Graham Neubig, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2013. Simple, lexicalized choice of translation timing for simultaneous speech translation. In _Proceedings of Interspeech_.
* [Grissom et al.2014] Alvin C. Grissom II, He He, Jordan Boyd-Graber, John Morgan, and Hal Daume III. 2014. Don't until the final verb wait: Reinforcement learning for simultaneous machine translation. In _Proceedings of Empirical Methods in Natural Language Processing (EMNLP)_.
* [He et al.2015] He He, Alvin Grissom II, Jordan Boyd-Graber, John Morgan, and Hal Daume III. 2015. Syntax-based rewriting for simultaneous machine translation. In _Proceedings of Empirical Methods in Natural Language Processing (EMNLP)_.
* [Kiss and Strunk2006] Tibor Kiss and Jan Strunk. 2006. Unsupervised multilingual sentence boundary detection. _Computational Linguistics_, 32:485-525.
* [Liang et al.2006] Percy Liang, Ben Taskar, and Dan Klein. 2006. Alignment by agreement. In _Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)_.
* [Ma2006] Xiaoyi Ma. 2006. Champollion: A robust parallel text sentence aligner. In _Proceedings of the Language Resources and Evaluation Conference (LREC)_.
* [Matsubara et al.2002] Shigeki Matsubara, Akira Takagi, Nobuo Kawaguchi, and Yasuyoshi Inagaki. 2002. Bilingual spoken monologue corpus for simultaneous machine interpretation research. In _Proceedings of the Language Resources and Evaluation Conference (LREC)_.
* [Oda et al.2014] Yusuke Oda, Graham Neubig, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2014. Optimizing segmentation strategies for simultaneous speech translation. In _Proceedings of the annual meeting of the Association for Computational Linguistics (ACL)_.
* [Oda et al.2015] Yusuke Oda, Graham Neubig, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2015. Syntax-based simultaneous translation through prediction of unseen syntactic constituents. In _The 53rd Annual Meeting of the Association for Computational Linguistics (ACL)_, Beijing, China, July.
* [Shimizu et al.2013] Hiroaki Shimizu, Graham Neubig, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2013. Constructing a speech translation system using simultaneous interpretation data. In _Proceedings of International Workshop on Spoken Language Translation (IWSLT)_.
* [Shimizu et al.2014] Hiroaki Shimizu, Graham Neubig, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2014. Collection of a simultaneous translation corpus for comparative analysis. In _Proceedings of the Language Resources and Evaluation Conference (LREC)_.
* [Rangarajan Sridhar et al.2013] Vivek Kumar Rangarajan Sridhar, John Chen, and Srinivas Bangalore. 2013. Corpus analysis of simultaneous interpretation data for improving real time speech translation. In _Proceedings of Interspeech_.
* [Tohyama and Matsubara2006] Hitomi Tohyama and Shigeki Matsubara. 2006. Collection of simultaneous interpreting patterns by using bilingual spoken monologue corpus. In _Proceedings of the Language Resources and Evaluation Conference (LREC)_.
* [Toutanova et al.2003] Kristina Toutanova, Dan Klein, Christopher Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In _Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)_.
* [Volansky et al.2013] Vered Volansky, Noam Ordan, and Shuly Wintner. 2013. On the features of translatoinese. _Literary and Linguistic Computing_, pages 98-118.