T RANSLATIONESEB ETWEEN H UMAN AND M ACHINE T RANSLATION
Shuly Wintner
Department of Computer Science
University of Haifa
Haifa, Israel
shuly@cs.haifa.ac.il

Introduction

O RIGINAL OR T RANSLATION ?
E XAMPLE (O OR T?)
We want to see countries that can produce the bestproduct for the best price in that particular business . I haveto agree with the member that free trade agreements bydefinition do not mean that we have to be less vigilant all of asudden .

E XAMPLE (T OR O?)
I would like as my final point to say that we support free
trade , but we must learn from past mistakes . Let us hopethat negotiations for free trade agreements with the fourCentral American countries introduce a number of otherdimensions absent from these first generation agreements .
© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

2 / 112

Introduction

U NDERLYING A SSUMPTIONS

Research in Translation Studies can inform Natural Language
Processing, and in particular improve the quality of machinetranslation
Computational methodologies can shed light on pertinent questionsin Translation Studies© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

3 / 112

IntroductionO UTLINE
1I NTRODUCTION

2I DENTIFICATION OF T RANSLATIONESE

3

T RANSLATION S TUDIES H YPOTHESES

4S UPERVISED C LASSIFICATION

5U NSUPERVISED C LASSIFICATION6A PPLICATIONS FOR M ACHINE T RANSLATION

7T HE P OWER OF I NTERFERENCE

8

C ONCLUSION© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

4 / 112

Introduction

Translationese

T RANSLATIONESET HE LANGUAGE OF TRANSLATED TEXTSTranslated texts differ from original ones
The differences do not indicate poor translation but rather astatistical phenomenon, translationese (Gellerstam, 1986)
Toury (1980, 1995) defines two laws of translation:
T HE LAW OF INTERFERENCE Fingerprints of the source text thatare left in the translation product
T HE LAW OF GROWING STANDARDIZATION Effort to standardize
the translation product according to existing norms in the target
language and culture© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

5 / 112

Introduction

Translationese

T RANSLATIONESET HE LANGUAGE OF TRANSLATED TEXTST RANSLATION UNIVERSALS (Baker, 1993)“features which typically occur in translated text rather
than original utterances and which are not the result ofinterference from specific linguistic systems”
SIMPLIFICATION (Blum-Kulka and Levenston, 1978, 1983)
EXPLICITATION (Blum-Kulka, 1986)

N ORMALIZATION (Chesterman, 2004)© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

6 / 112

Introduction

Translationese

T RANSLATIONESEW HY DOES IT MATTER ?

Language models for statistical machine translation(Lembersky et al., 2011, 2012b)
Translation models for statistical machine translation(Kurokawa et al., 2009; Lembersky et al., 2012a, 2013)
Cleaning parallel corpora crawled from the Web(Eetemadi and Toutanova, 2014; Aharoni et al., 2014)
Understanding the properties of human translation (Ilisei et al.,2010; Ilisei and Inkpen, 2011; Ilisei, 2013; Volansky et al., 2015;
Avner et al., 2016)© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

7 / 112

Introduction

Corpus-based Translation Studies

C ORPUS - BASED T RANSLATION S TUDIESE XAMPLE (S UN AND S HREVE (2013))© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

8 / 112

Introduction

Corpus-based Translation Studies

C OMPUTATIONAL I NVESTIGATION OF T RANSLATIONESE

Translated texts exhibit lower lexical variety (type-to-token ratio)
than originals (Al-Shabab, 1996)
Their mean sentence length and lexical density (ratio of content tonon-content words) are lower (Laviosa, 1998)
Corpus-based evidence for the simplification hypothesis (Laviosa,2002)© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

9 / 112

Introduction

Corpus-based Translation Studies

C OMPUTATIONAL I NVESTIGATION OF T RANSLATIONESEE XAMPLE (P UNCTUATION MARKS ACROSS O AND T)
Mark
,
(
’
)
/
[
]
”
!
.
:
;
?
© Shuly Wintner (University of Haifa)

Frequency
O
T

Ratio

LL

Weight

37.83
0.42
1.94
0.40
0.30
0.01
0.01
0.33
0.22
38.20
1.20
0.84
1.57
2.68

0.76
0.58
0.77
0.56
1.00
0.45
0.44
1.46
1.25
1.10
1.03
1.01
1.41
1.19

T
T
T
T
—
T
T
O
O
O
—
—
O
O

T1
T2
T3
T4
—
—
—
O7
O6
O5
O4
O3
O2
O1

49.79
0.72
2.53
0.72
0.30
0.02
0.02
0.22
0.17
34.60
1.17
0.83
1.11
2.25

Translationese

12 December, 201610 / 112Identification of TranslationeseO UTLINE
1I NTRODUCTION

2I DENTIFICATION OF T RANSLATIONESE

3

T RANSLATION S TUDIES H YPOTHESES

4S UPERVISED C LASSIFICATION

5U NSUPERVISED C LASSIFICATION6A PPLICATIONS FOR M ACHINE T RANSLATION

7T HE P OWER OF I NTERFERENCE

8

C ONCLUSION© Shuly Wintner (University of Haifa)

Translationese

12 December, 201611 / 112Identification of Translationese

M ETHODOLOGY

Corpus-based approach
Text classification with (supervised) machine-learning techniques
Feature design
Evaluation: ten-fold cross-validation
Unsupervised classification© Shuly Wintner (University of Haifa)

Translationese

12 December, 201612 / 112Identification of Translationese

T EXT C LASSIFICATION WITH M ACHINE L EARNINGE XAMPLE (F ROM H E ET AL . (2012))© Shuly Wintner (University of Haifa)

Translationese

12 December, 201613 / 112Identification of Translationese

T EXT C LASSIFICATION WITH M ACHINE L EARNING

Supervised machine learning: a training corpus lists instancesof both classes
Each instance in the two classes is represented by a set of numericfeatures that are extracted from the instances
A generic machine-learning algorithm is trained to distinguishbetween feature vectors representative of one class and those
representative of the other
The trained classifier is tested on an ‘unseen’ text, the test set
Classifiers assign weights to the features© Shuly Wintner (University of Haifa)

Translationese

12 December, 201614 / 112Identification of Translationese

T EXT C LASSIFICATION (AUTHORSHIP ATTRIBUTION )A PPLICATIONS

Determine the gender/age of the author (Koppel et al., 2003)
Tell Shakespeare from Marlowe (Juola, 2006)
Identify suicide letters
Spot plagiarism
Filter out spam
Identify the native language of non-native authors (Tetreault et al.,2013)

© Shuly Wintner (University of Haifa)

Translationese

12 December, 201615 / 112Identification of TranslationeseI DENTIFYING T RANSLATIONESEU SING TEXT CLASSIFICATION

Baroni and Bernardini (2006)
van Halteren (2008)
Kurokawa et al. (2009)
Ilisei et al. (2010); Ilisei and Inkpen (2011); Ilisei (2013)
Koppel and Ordan (2011)
Popescu (2011)
Volansky et al. (2015); Avner et al. (2016)© Shuly Wintner (University of Haifa)

Translationese

12 December, 201616 / 112Identification of Translationese

E XPERIMENTAL S ETUP

Corpus: EUROPARL (Koehn, 2005)
4 million tokens in English (O)400,000 tokens translated from each of the ten source languages(T): Danish, Dutch, Finnish, French, German, Greek, Italian,Portuguese, Spanish, and Swedish
The corpus is tokenized and then partitioned into chunks ofapproximately 2000 tokens (ending on a sentence boundary)Part-of-speech tagging
Classification with Weka (Hall et al., 2009), using SVM with adefault linear kernel© Shuly Wintner (University of Haifa)

Translationese

12 December, 201617 / 112

Translation Studies Hypotheses

O UTLINE
1I NTRODUCTION

2I DENTIFICATION OF T RANSLATIONESE

3

T RANSLATION S TUDIES H YPOTHESES

4S UPERVISED C LASSIFICATION

5U NSUPERVISED C LASSIFICATION6A PPLICATIONS FOR M ACHINE T RANSLATION

7T HE P OWER OF I NTERFERENCE

8

C ONCLUSION© Shuly Wintner (University of Haifa)

Translationese

12 December, 201618 / 112

Translation Studies Hypotheses

H YPOTHESESS IMPLIFICATION Rendering complex linguistic features in the source
text into simpler features in the target (Blum-Kulka and Levenston,
1983; Vanderauwerea, 1985; Baker, 1993)
E XPLICITATION The tendency to spell out in the target text utterancesthat are more implicit in the source (Blum-Kulka, 1986; Øverås, 1998;
Baker, 1993)
N ORMALIZATION Efforts to standardize texts (Toury, 1995), “a strong
preference for conventional grammaticality” (Baker, 1993)
I NTERFERENCE The fingerprints of the source language on thetranslation output (Toury, 1979)
M ISCELLANEOUS© Shuly Wintner (University of Haifa)

Translationese

12 December, 201619 / 112

Translation Studies Hypotheses

F EATURES S HOULD ...

1

Reflect frequent linguistic characteristics we would expect to bepresent in the two types of text

2

Be content-independent, indicating formal and stylistic differences
between the texts that are not derived from differences in contents,
domain, genre, etc.

3

Be easy to interpret, yielding insights regarding the differences
between original and translated texts© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

20 / 112

Translation Studies Hypotheses

Simplification

S IMPLIFICATIONL EXICAL VARIETY Three different type-token ratio (TTR) measures,
where V is the number of types and N is the number of tokens per
chunk, V1 is the number of types occurring only once in the chunk
V /Nlog(V )/log(N )100 × log(N )/(1 − V1/V )M EAN WORD LENGTH In characters
S YLLABLE RATIO the number of vowel-sequences that are delimited byconsonants or space in a word
M EAN SENTENCE LENGTH In words
L EXICAL DENSITY The frequency of tokens that are not nouns,
adjectives, adverbs or verbs
M EAN WORD RANK The average rank of words in a frequency-ordered
list of 6000 English most frequent words
M OST FREQUENT WORDS The frequencies of the N most frequentwords, N = 5, 10, 50© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

21 / 112

Translation Studies Hypotheses

Explicitation

E XPLICITATIONE XAMPLE (E XPLICITATION )T
israelische Ministerpräsident Benjamin Netanjahu

O
MerkelE XPLICIT NAMING The ratio of personal pronouns to proper nouns;
this models the tendency to spell out pronouns
S INGLE NAMING The frequency of proper nouns consisting of a single
token, not having an additional proper noun as a neighbor
M EAN MULTIPLE NAMING The average length (in tokens) of propernouns
C OHESIVE MARKERS The frequencies of several cohesive markers(because, but, hence, in fact, therefore, ...)
© Shuly Wintner (University of Haifa)

Translationese

12 December, 201622 / 112

Translation Studies Hypotheses

Normalization

N ORMALIZATIONR EPETITIONS The frequency of content words (words tagged as nouns,
verbs, adjectives or adverbs) that occur more than once in a chunk

E XAMPLE (R EPETITIONS , B EN -A RI (1998))
O Manchmal denke ich: Haben Sie vielleicht ein kaltes Herz? ... Haben
Sie wirklich ein kaltes Herz?
T Je pense quelquefois: aurait-elle un cœur insensible? ... Avez-vous
vraiment un cœur de glace?
C ONTRACTIONS Ratio of contracted forms to their counterpart fullform
AVERAGE PMI The average PMI (Church and Hanks, 1990) of all
bigrams in the chunk
T HRESHOLD PMI Number of bigrams in the chunk whose PMI isabove 0© Shuly Wintner (University of Haifa)

Translationese

12 December, 201623 / 112

Translation Studies Hypotheses

Interference

I NTERFERENCE

Fingerprints of the source text, “source language shining through”(Teich, 2003)
Not necessarily a mark of bad translation! Rather, a different
distribution of elements in T and O
Positive vs. negative interference

E XAMPLE (I NTERFERENCE )P OSITIVE doch is under-represented in translation to German
N EGATIVE One is is over-represented in translations from German toEnglish (triggered by Man ist)

© Shuly Wintner (University of Haifa)

Translationese

12 December, 201624 / 112

Translation Studies Hypotheses

Interference

I NTERFERENCE
POS n- GRAMS Unigrams, bigrams and trigrams of POS tags, modelingvariations in syntactic structure
C HARACTER n- GRAMS Unigrams, bigrams and trigrams of characters,
modeling shallow morphology
P REFIXES AND SUFFIXES The number of words in a chunk that begin
or end with each member of a list of prefixes/suffixes, respectively
C ONTEXTUAL FUNCTION WORDS The frequencies in the chunk of
triplets hw1 , w2 , w3 i, where at least two of the elements are function
words, and at most one is a POS tag
P OSITIONAL TOKEN FREQUENCY The frequency of tokens appearing in
the first, second, antepenultimate, penultimate and last positions in a
sentence© Shuly Wintner (University of Haifa)

Translationese

12 December, 201625 / 112

Translation Studies Hypotheses

Miscellaneous

M ISCELLANEOUS

F UNCTION WORDS The list of Koppel and Ordan (2011)
P RONOUNS Frequency of the occurrences of pronouns in the chunk
P UNCTUATION ? ! : ; - ( ) [ ] ‘ ’ “ ” / , .
1 The normalized frequency of each punctuation mark in the chunk
2 A non-normalized notion of frequency: n/tokens
3 n⁄p, where p is the total number of punctuations in the chunk
R ATIO OF PASSIVE FORMS TO ALL VERBSS ANITY CHECK Token unigrams and bigrams© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

26 / 112

Supervised ClassificationO UTLINE
1I NTRODUCTION

2I DENTIFICATION OF T RANSLATIONESE

3

T RANSLATION S TUDIES H YPOTHESES

4S UPERVISED C LASSIFICATION

5U NSUPERVISED C LASSIFICATION6A PPLICATIONS FOR M ACHINE T RANSLATION

7T HE P OWER OF I NTERFERENCE

8

C ONCLUSION© Shuly Wintner (University of Haifa)

Translationese

12 December, 201627 / 112

Supervised Classification

R ESULTS : S ANITY C HECK

Category
Sanity© Shuly Wintner (University of Haifa)

FeatureToken unigrams
Token bigrams

Translationese

Accuracy (%)
100
100

12 December, 201628 / 112

Supervised Classification

Simplification

R ESULTS : S IMPLIFICATION

Category

Simplification© Shuly Wintner (University of Haifa)

FeatureTTR (1)
TTR (2)
TTR (3)
Mean word rank (1)Mean word rank (2)N most frequent words
Mean word length
Syllable ratio
Lexical density
Mean sentence length

Translationese

Accuracy (%)
72
72
76
69
77
64
66
61
53
65

12 December, 201629 / 112

Supervised Classification

Simplification

A NALYSIS : S IMPLIFICATION
Lexical density fails altogether to predict the status of a text, beingnearly on chance level (53% accuracy)
The first two TTR measures perform relatively well (72%), and theindirect measures of lexical variety (mean word length, 66% andsyllable ratio, 61%) are above chance level
Mean word rank (77%) is closely related to the feature studied by
Laviosa (1998) (n top words) with two differences: our list of
frequent items is much larger, and we generate the frequency list
not from the corpora under study but rather from an externalmuch larger reference corpus
In contrast, the design that follows Laviosa (1998) more strictly (nmost frequent words) has a lower predictive power (64%)
While mean sentence length is much above chance level (65%), theresults are contrary to common assumptions in Translation Studies© Shuly Wintner (University of Haifa)

Translationese

12 December, 201630 / 112

Supervised Classification

Simplification

S IMPLIFICATIONE XAMPLE (M EAN SENTENCE LENGTH PER ‘ LANGUAGE ’)© Shuly Wintner (University of Haifa)

Translationese

12 December, 201631 / 112

Supervised Classification

Explicitation

R ESULTS : E XPLICITATION

Category
Explicitation© Shuly Wintner (University of Haifa)

FeatureCohesive Markers
Explicit naming
Single naming
Mean multiple naming

Translationese

Accuracy (%)
81
58
56
54

12 December, 201632 / 112

Supervised Classification

Explicitation

A NALYSIS : E XPLICITATION

Explicit naming, single naming and mean multiple naming do not
exceed 58% accuracy
On the other hand, the classifier that uses 40 cohesive markers(Blum-Kulka, 1986; Koppel and Ordan, 2011) achieves 81%
accuracy
Such cohesive markers are far more frequent in T than in O
For example, moreover is used 17.5 times more frequently in T
than in O; thus 4 times more frequently; and besides 3.8 times© Shuly Wintner (University of Haifa)

Translationese

12 December, 201633 / 112

Supervised Classification

Normalization

R ESULTS : N ORMALIZATION

Category
Normalization© Shuly Wintner (University of Haifa)

FeatureRepetitions
Contractions
Average PMI
Threshold PMI

Translationese

Accuracy (%)
55
50
52
66

12 December, 201634 / 112

Supervised Classification

Normalization

A NALYSIS : N ORMALIZATION

None of these features perform very wellRepetitions and contractions are rare in EUROPARL; the corpus
may not be suited for studying these phenomena
Threshold PMI, designed to pick on highly associated words andtherefore attesting to many fixed expressions, performs
considerably better, at 66% accuracy, but contrary to the
hypothesis (Kenny, 2001)
English has far more highly associated bigrams than translations:
O has stand idly , stand firm, stand trial, etc.; conversely, T
includes poorly associated bigrams such as stand unamended© Shuly Wintner (University of Haifa)

Translationese

12 December, 201635 / 112

Supervised Classification

Normalization

N ORMALIZATIONE XAMPLE (N UMBER OF BIGRAMS WHOSE PMI IS ABOVETHRESHOLD ACCORDING TO ‘ LANGUAGE ’)© Shuly Wintner (University of Haifa)

Translationese

12 December, 201636 / 112

Supervised Classification

Normalization

N ORMALIZATIONE XAMPLE (T RANSLATED LANGUAGE IS LESS PREDICTABLE )© Shuly Wintner (University of Haifa)

Translationese

12 December, 201637 / 112

Supervised Classification

Interference

R ESULTS : I NTERFERENCE

Category

Interference

FeaturePOS unigrams
POS bigrams
POS trigrams
Character unigrams
Character bigrams
Character trigrams
Prefixes and suffixes
Contextual function words
Positional token frequency© Shuly Wintner (University of Haifa)

Translationese

Accuracy (%)
90
97
98
85
98
100
80
100
97

12 December, 201638 / 112

Supervised Classification

Interference

A NALYSIS : I NTERFERENCE

The interference-based classifiers are the best performing ones
Interference is the most robust phenomenon typifying translations
The character n-gram findings are consistent with Popescu (2011):
they catch on both affixes and function words
For example, typical trigrams in O are -ion and all whereas typicalto T are -ble and thePart-of-speech trigrams is an extremely cheap and efficient classifier
For example, MD+VB+VBN, e.g., must be taken, should begiven, can be used

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

39 / 112

Supervised Classification

Interference

I NTERFERENCEE XAMPLE (T HE AVERAGE NUMBER OF THE POS TRIGRAM
MODAL + VERB BASE FORM + PARTICIPLE IN O AND TEN T S )© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

40 / 112

Supervised Classification

Interference

A NALYSIS : I NTERFERENCE

Positional token frequency yields 97% accuracy
The second most prominent feature typifying O is sentences
opening with the word ‘But’; there are 2.25 times more cases ofsuch sentences in O
A long prescriptive tradition in English forbids writers to open asentence with ‘But’, and translators are conservative in their lexical
choices (Kenny, 2001)
This is in fact standardization rather than interference© Shuly Wintner (University of Haifa)

Translationese

12 December, 201641 / 112

Supervised Classification

Interference

A NALYSIS : I NTERFERENCE

Some interference features are coarse and may reflect
corpus-dependent characteristics
For example, some of the top character n-gram features in O
include sequences that are ‘illegal’ in English and obviously stemfrom foreign names: Haarder and Maat or Gazpron
To offset this problem we use only the top 300 features in severalof the classifiers, with a minor effect on the results© Shuly Wintner (University of Haifa)

Translationese

12 December, 201642 / 112

Supervised Classification

Interference

R ESULTS : R EDUCED PARAMETER S PACE
(300 MOST FREQUENT FEATURES )Category

Interference© Shuly Wintner (University of Haifa)

FeaturePOS bigrams
POS trigrams
Character bigrams
Character trigrams
Positional token frequency

Translationese

Accuracy
96
96
95
96
93

12 December, 201643 / 112

Supervised Classification

Miscellaneous

R ESULTS : M ISCELLANEOUS

Category

Miscellaneous

FeatureFunction words
Punctuation (1)Punctuation (2)Punctuation (3)Pronouns
Ratio of passive forms to all verbs© Shuly Wintner (University of Haifa)

Translationese

Accuracy (%)
96
81
85
80
77
65

12 December, 201644 / 112

Supervised Classification

Miscellaneous

A NALYSIS : M ISCELLANEOUS
The function words classifier replicates Koppel and Ordan (2011);
despite the good performance (96% accuracy) it is not very
meaningful theoretically
Subject pronouns, like I , he and she are prominent indicators of O,
whereas virtually all reflexive pronouns (such as itself , himself ,yourself ) typify T
T has about 1.15 times more passive verbs, but it is highly
dependent on the source language
Punctuation marks are good predictors
The mark ‘.’ (actually indicating sentence length) is a strong
feature of O; ‘,’ is a strong marker of T
In fact, these two features alone yield 79% accuracy
Parentheses are very typical to T, indicating explicitation
Translations from German use many more exclamation marks, 2.76
times more than original English!!!
© Shuly Wintner (University of Haifa)

Translationese

12 December, 201645 / 112

Supervised Classification

Miscellaneous

A NALYSIS : P UNCTUATIONE XAMPLE (P UNCTUATION MARKS ACROSS O AND T)
Mark
,
(
’
)
/
[
]
”
!
.
:
;
?
© Shuly Wintner (University of Haifa)

Frequency
O
T

Ratio

LL

Weight

37.83
0.42
1.94
0.40
0.30
0.01
0.01
0.33
0.22
38.20
1.20
0.84
1.57
2.68

0.76
0.58
0.77
0.56
1.00
0.45
0.44
1.46
1.25
1.10
1.03
1.01
1.41
1.19

T
T
T
T
—
T
T
O
O
O
—
—
O
O

T1
T2
T3
T4
—
—
—
O7
O6
O5
O4
O3
O2
O1

49.79
0.72
2.53
0.72
0.30
0.02
0.02
0.22
0.17
34.60
1.17
0.83
1.11
2.25

Translationese

12 December, 201646 / 112

Unsupervised ClassificationO UTLINE
1I NTRODUCTION

2I DENTIFICATION OF T RANSLATIONESE

3

T RANSLATION S TUDIES H YPOTHESES

4S UPERVISED C LASSIFICATION

5U NSUPERVISED C LASSIFICATION6A PPLICATIONS FOR M ACHINE T RANSLATION

7T HE P OWER OF I NTERFERENCE

8

C ONCLUSION© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

47 / 112

Unsupervised Classification

Motivation

S UPERVISED C LASSIFICATION

Inherently dependent on data annotated with the translation
direction
May not be generalized to unseen (related or unrelated) domains:
modality (written vs. spoken), register, genre, date, etc.

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

48 / 112

Unsupervised Classification

Motivation

DATASETS

Europarl, the proceedings of the European Parliament, between theyears 2001-2006
the Canadian Hansard, transcripts of the Canadian Parliament,
spanning years 2001-2009
literary classics written (or translated) mainly in the 19th century
transcripts of TED and TEDx talks© Shuly Wintner (University of Haifa)

Translationese

12 December, 201649 / 112

Unsupervised Classification

Motivation

S UPERVISED C LASSIFICATION

feature / corpusFWchar-trigrams
POS-trigrams
contextual FW
cohesive markers© Shuly Wintner (University of Haifa)

EUR
96.3
98.8
98.5
95.2
83.6

HAN
98.1
97.1
97.2
96.8
86.9

Translationese

LIT
97.3
99.5
98.7
94.1
78.6

TED
97.7
100.0
92.0
86.3
81.8

12 December, 2016

50 / 112

Unsupervised Classification

Motivation

PAIRWISE C ROSS - DOMAIN C LASSIFICATIONU SING FUNCTION WORDS

train / test
EUR
HAN
LIT

© Shuly Wintner (University of Haifa)

EUR
59.7
64.3

HAN
60.8
61.5

Translationese

LIT
56.2
58.7

X-validation
96.3
98.1
97.3

12 December, 2016

51 / 112

Unsupervised Classification

Motivation

L EAVE - ONE - OUT C ROSS - DOMAIN C LASSIFICATIONU SING FUNCTION WORDS

train / test
EUR + HAN
EUR + LIT
HAN + LIT

© Shuly Wintner (University of Haifa)

EUR

HAN
64.1

59.8

Translationese

LIT
63.8

X-validation
94.0
92.9
96.0

12 December, 201652 / 112

Unsupervised Classification

Unsupervised Classification

C LUSTERINGA SSUMING GOLD LABELS

feature / corpusFWchar-trigrams
POS-trigrams
contextual FW
cohesive markers© Shuly Wintner (University of Haifa)

EUR
88.6
72.1
96.9
92.9
63.1

HAN
88.9
63.8
76.0
93.2
81.2

Translationese

LIT
78.8
70.3
70.7
68.2
67.1

TED
87.5
78.6
76.1
67.0
63.0

12 December, 2016

53 / 112

Unsupervised Classification

Cluster Labeling

C LUSTER L ABELINGLabeling determines which of the clusters is O and which is T
Clustering can divide observations into classes but cannot label
those classes
Let Om (O-markers) denote a set of function words that tend to be
associated with O; Tm (T-markers) is a set of words typical of T
Create unigram language models of O and T:
p(w | Om ) =p(w | Tm ) =tf (w )+ε
|Om |+ε×|V |tf (w )+ε
|Tm |+ε×|V |The similarity between a class X (either O or T) and a cluster Ci isdetermined using the Jensen-Shannon divergence (JSD) (Lin, 1991)q
DJS (X , Ci ) = 2 JSD(PX ||PCi )© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

54 / 112

Unsupervised Classification

Cluster Labeling

C LUSTER L ABELINGThe assignment of the label X to the cluster C1 is then supported
by both C1 ’s proximity to the class X and C2 ’s proximity to theother class:
“O” if DJS (O, C1 )×DJS (T , C2 ) <label(C1 ) =DJS (O, C2 )×DJS (T , C1 )“T” otherwise
C2 is then assigned the complementary label
We select O- and T-markers from a random sample of Europarland Hansard texts, using 600 chunks from each corpus
Labeling precision is 100% in all clustering experiments
This facilitates majority voting of several feature sets
© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

55 / 112

Unsupervised Classification

Clustering Consensus among Feature Sets

C LUSTERING C ONSENSUS
method / corpusFWFW
char-trigrams
POS-trigrams
FW
POS-trigrams
contextual FW
FW
char-trigrams
POS-trigrams
contextual FW
cohesive markers© Shuly Wintner (University of Haifa)

EUR
88.6

HAN
88.9

LIT
78.8

TED
87.5

91.1

86.2

78.2

90.9

95.8

89.8

72.3

86.3

94.1

91.0

79.2

88.6

Translationese

12 December, 201656 / 112

Unsupervised Classification

Sensitivity Analysis

S ENSITIVITY A NALYSIS© Shuly Wintner (University of Haifa)

Translationese

12 December, 201657 / 112

Unsupervised Classification

Sensitivity Analysis

S ENSITIVITY A NALYSIS© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

58 / 112

Unsupervised Classification

Mixed-domain Classification

M IXED - DOMAIN C LASSIFICATION

The in-domain discriminative features of translated texts cannot beeasily generalized to other, even related, domains
Hypothesis: domain-specific features overshadow the features oftranslationese
We mix 800 chunks each from Europral and Hansard,
yielding 1,600 chunks, half of them O and half T
Running the clustering algorithm on this dataset yields perfect
separation by domain (and chance-level identification oftranslationese)
Adding the literature corpus and clustering to three clusters yields
the same results© Shuly Wintner (University of Haifa)

Translationese

12 December, 201659 / 112

Unsupervised Classification

Mixed-domain Classification

M IXED - DOMAIN C LASSIFICATION

method / corpus
KMeansaccuracy by domain
XMeansgenerated # of clusters
accuracy by domain© Shuly Wintner (University of Haifa)

EUR
HAN

EUR
LIT

HAN
LIT

EUR
HAN
LIT

93.7

99.5

99.8

92.2

2
93.6

2
99.5

3
–

3
92.2

Translationese

12 December, 201660 / 112

Unsupervised Classification

Mixed-domain Classification

C LUSTERING IN A M IXED - DOMAIN S ETUP

Given a set of text chunks that come from multiple domains, suchthat some chunks are O and some are T, the task is to classify the
texts to O vs. T, independently of their domain
A two-phase approach
A flat approach© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

61 / 112

Unsupervised Classification

Mixed-domain Classification

C LUSTERING IN A M IXED - DOMAIN S ETUP

method / corpus
Flat
Two-phase© Shuly Wintner (University of Haifa)

EUR
HAN
92.5
91.3

EUR
LIT
60.7
79.4

Translationese

HAN
LIT
77.5
85.3

EUR
HAN
LIT
66.8
67.5

12 December, 201662 / 112

Applications for Machine TranslationO UTLINE
1I NTRODUCTION

2I DENTIFICATION OF T RANSLATIONESE

3

T RANSLATION S TUDIES H YPOTHESES

4S UPERVISED C LASSIFICATION

5U NSUPERVISED C LASSIFICATION6A PPLICATIONS FOR M ACHINE T RANSLATION

7T HE P OWER OF I NTERFERENCE

8

C ONCLUSION© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

63 / 112

Applications for Machine Translation

A PPLICATIONS FOR M ACHINE T RANSLATION

Fundamentals of statistical machine translation (SMT)
Language models
Translation models© Shuly Wintner (University of Haifa)

Translationese

12 December, 201664 / 112

Applications for Machine Translation

Fundamentals of SMT

F UNDAMENTALS OF SMT

Motivation
When I look at an article in Russian, I say, “This is really
written in English, but it has been coded in some strange
symbols. I shall now proceed to decode.”
Warren Weaver, 1955
The noisy channel model
The best translation T̂ of a source sentence S is the target
sentence T that maximizes some function combining thefaithfulness of (T , S) and the fluency of T

© Shuly Wintner (University of Haifa)

Translationese

12 December, 201665 / 112

Applications for Machine Translation

Fundamentals of SMT

F UNDAMENTALS OF SMT
Standard notation: Translating a foreign sentence F = f1 , · · · , fm
into an English sentence E = e1 , · · · , el
The best translation:Ê= arg maxE P(E | F ))×P(E )= arg maxE P(F |EP(F )= arg maxE P(F | E ) × P(E )The noisy channel thus requires two components: a translation
model and a language modelÊ = arg max
E ∈English

P(F | E )| {z }

Translation model© Shuly Wintner (University of Haifa)

Translationese

×

P(E )
| {z }

Language model

12 December, 2016

66 / 112

Applications for Machine Translation

Fundamentals of SMT

F UNDAMENTALS OF SMT

A language model to estimate P(E ) (estimated from amonolingual E corpus)
A translation model to estimate P(F | E ) (estimated from abilingual parallel corpus)
A decoder that given F can produce the most probable E
Evaluation: BLEU scores (Papineni et al., 2002)© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

67 / 112

Applications for Machine Translation

Language Models

L ANGUAGE M ODELSR ESEARCH QUESTIONSTest the fitness of language models compiled from translated texts
vs. the fitness of LMs compiled from original texts
Test the fitness of language models compiled from texts translated
from other languages
Test if language models compiled from translated texts are better
for MT than LMs compiled from original texts© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

68 / 112

Applications for Machine Translation

Language Models

L ANGUAGE M ODELSM ETHODOLOGY

The fitness of a language model to a reference corpus is evaluatedusing perplexitys
N
1
PP(LM, w1 w2 . . . wN ) = N ∏P(w|
w
1 . . . wi−1 )i
i=1 LM
Train SMT systems (Koehn et al., 2007) using different LMs andevaluate their quality on a reference set
Quality is measured in terms of BLEU scores (Papineni et al., 2002)© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

69 / 112

Applications for Machine Translation

Language Models

P ERPLEXITY R ESULTS

German to English translations
Orig. Lang. 1-gram 2-gram 3-gram 4-gram
Mix
451.50
93.00
69.36
66.47
O-EN
468.09103.74
79.57
76.79
T-DE
443.14
88.48
64.99
62.07
T-FR
460.98
99.90
76.23
73.38
T-IT
465.89102.31
78.50
75.67
T-NL
457.02
97.34
73.54
70.56

© Shuly Wintner (University of Haifa)

Translationese

12 December, 201670 / 112

Applications for Machine Translation

Language Models

P ERPLEXITY R ESULTS

French to English translations
Orig. Lang. 1-gram 2-gram 3-gram
Mix
472.05
99.04
75.60
O-EN
500.56115.4891.14
T-DE
486.78108.5084.39
T-FR
463.58 94.59
71.24
T-IT
476.05102.6979.23
T-NL
490.09110.67
86.61© Shuly Wintner (University of Haifa)

Translationese

4-gram
72.68
88.31
81.41
68.37
76.36
83.55

12 December, 2016

71 / 112

Applications for Machine Translation

Language Models

P ERPLEXITY R ESULTS

Italian to English translations
Orig. Lang. 1-gram 2-gram 3-gram
Mix
395.99
88.46
67.35
O-EN
415.47
99.92
79.27
T-DE
404.64
95.22
73.73
T-FR
395.99
89.44
68.38
T-IT
384.55
81.90
60.85
T-NL
411.5898.78
76.98© Shuly Wintner (University of Haifa)

Translationese

4-gram
64.40
76.34
70.85
65.54
57.91
73.94

12 December, 2016

72 / 112

Applications for Machine Translation

Language Models

P ERPLEXITY R ESULTS

Dutch to English translations
Orig. Lang. 1-gram 2-gram 3-gram
Mix
434.89
90.73
69.05
O-EN
448.11
100.1778.23
T-DE
437.68
93.67
71.54
T-FR
445.00
97.32
75.59
T-IT
448.11 100.19
78.06
T-NL
423.1383.99
62.17© Shuly Wintner (University of Haifa)

Translationese

4-gram
66.08
75.46
68.57
72.55
75.19
59.09

12 December, 2016

73 / 112

Applications for Machine Translation

Language Models

L ANGUAGE M ODELSA BSTRACTION

No punctuation
No named entities
No nouns
No words: only parts of speech© Shuly Wintner (University of Haifa)

Translationese

12 December, 201674 / 112

Applications for Machine Translation

Language Models

A BSTRACTION R ESULTS

Orig. Lang.
MIX
O-EN
T-DE
T-FR
T-IT
T-NL

© Shuly Wintner (University of Haifa)

No Punctuation
Perplexity Improvement (%)
105.9119.73
131.94122.50
7.16
99.52
24.58
112.71
14.58
126.44
4.17

Translationese

12 December, 201675 / 112

Applications for Machine Translation

Language Models

A BSTRACTION R ESULTS

Named Entity Abstraction
Orig. Lang. Perplexity Improvement (%)
MIX
93.88
18.51
O-EN
115.20T-DE
107.48
6.70
T-FR
88.96
22.77
T-IT
99.17
13.91
T-NL
110.72
3.89© Shuly Wintner (University of Haifa)

Translationese

12 December, 201676 / 112

Applications for Machine Translation

Language Models

A BSTRACTION R ESULTS

Orig. Lang.
MIX
O-EN
T-DE
T-FR
T-IT
T-NL

© Shuly Wintner (University of Haifa)

Noun Abstraction
Perplexity Improvement (%)
36.02
11.34
40.62
38.67
4.81
34.75
14.46
36.85
9.30
39.44
2.91

Translationese

12 December, 2016

77 / 112

Applications for Machine Translation

Language Models

A BSTRACTION R ESULTS

Part-of-speech Abstraction
Orig. Lang. Perplexity Improvement (%)
MIX
7.99
2.66
O-EN
8.20
T-DE
8.08
1.47
T-FR
7.89
3.77
T-IT
8.00
2.47
T-NL
8.11
1.11

© Shuly Wintner (University of Haifa)

Translationese

12 December, 201678 / 112

Applications for Machine Translation

Language Models

M ACHINE T RANSLATION R ESULTS

DE to ENLM
BLEU

FR to ENLM
BLEU

IT to ENLM
BLEU

NL to EN
LM
BLEU

MIX
O-EN
T-DE
T-FR
T-IT
T-NL

MIX
O-EN
T-DE
T-FR
T-IT
T-NL

MIX
O-EN
T-DE
T-FR
T-IT
T-NL

MIX
O-EN
T-DE
T-FR
T-IT
T-NL

21.43
21.10
21.90
21.16
21.29
21.20

© Shuly Wintner (University of Haifa)

28.67
27.98
28.01
29.14
28.75
28.11

Translationese

25.41
24.69
24.62
25.37
25.96
24.77

24.20
23.40
24.26
23.56
23.87
24.52

12 December, 201679 / 112

Applications for Machine Translation

Language Models

D OES S IZE M ATTER ?

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

80 / 112

Applications for Machine Translation

Language Models

D OES S IZE M ATTER ?

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

81 / 112

Applications for Machine Translation

Language Models

T RANSLATION E XAMPLESC OHESIVE MARKERSS OURCE Enfin, ce qui est grave dans le rapport de M. Olivier, c’est
qu’il propose une constitution tripotage.
O Finally, which is serious in the report of Mr Olivier, is that it
proposes a constitution tripotage.
T Finally, and this is serious in the report by Mr olivier, it is because itproposes a constitution tripotage.
S OURCE C’est quand même quelque chose de précieux qui a étésouligné par tous les membres du conseil européen.
O Even when it is something of valuable which has been pointed out by
all the members of the European Council.
T It is nevertheless something of a valuable which has been pointed
out by all the members of the European Council.
© Shuly Wintner (University of Haifa)

Translationese

12 December, 201682 / 112

Applications for Machine Translation

Language Models

T RANSLATION E XAMPLESV ERBSS OURCE Une telle Europe serait un gage de paix et marquerait le refus
de tout nationalisme ethnique.
O Such a Europe would be a show of peace and would the rejection ofany ethnic nationalism.
T Such a Europe would be a show of peace and would mark therefusal of all ethnic nationalism.
S OURCE Votre rapport, madame Sudre, met l’accent, à juste titre, surla nécessité d’agir dans la durée.
O Your report, Mrs Sudre, its emphasis, quite rightly, on the need toact in the long term.
T Your report, Mrs Sudre, places the emphasis, quite rightly, on theneed to act in the long term.
© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

83 / 112

Applications for Machine Translation

Language Models

T RANSLATION E XAMPLESI NTERFERENCES OURCE On ne dit rien non plus sur la responsabilité des fabricants,
notamment en grande-bretagne, qui ont été les premiers responsables.
O We do not say nothing more on the responsibility of the
manufacturers, particularly in Britain, which were the first responsible.
T We do not say anything either on the responsibility of the
manufacturers, particularly in great Britain, who were the first
responsible.

© Shuly Wintner (University of Haifa)

Translationese

12 December, 201684 / 112

Applications for Machine Translation

Translation Models

T RANSLATION M ODELSR ESEARCH QUESTIONS

Are parallel corpora (manually) translated in the same direction of
the MT task better than ones directed in the other direction?
If corpora consisting of texts (manually) translated in both
directions are available, how to build a translation model adapted
to the unique properties of the translated text?

© Shuly Wintner (University of Haifa)

Translationese

12 December, 201685 / 112

Applications for Machine Translation

Translation Models

T RANSLATION M ODELSD IRECTION MATTERS !

Task
FR-EN
EN-FR
DE-EN
EN-DE
IT-EN
EN-IT

© Shuly Wintner (University of Haifa)

S →T
33.64
32.11
26.53
16.96
28.70
23.81

Translationese

T →S
30.88
30.35
23.67
16.17
26.84
21.28

12 December, 201686 / 112

Applications for Machine Translation

Translation Models

T RANSLATION M ODELSD IRECTION MATTERS !

Task: French-to-English
Corpus subset S → T T → S
250K
34.35
31.33
500K
35.21
32.38
750K
36.12
32.90
1M
35.73
33.07
1.25M
36.24
33.23
1.5M
36.43
33.73

© Shuly Wintner (University of Haifa)

Translationese

12 December, 201687 / 112

Applications for Machine Translation

Translation Models

T RANSLATION M ODELSD IRECTION MATTERS !

Task: English-to-French
Corpus subset S → T T → S
250K
27.74
26.58
500K
29.15
27.19
750K
29.43
27.63
1M
29.94
27.88
1.25M
30.63
27.84
1.5M
29.89
27.83

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

88 / 112

Applications for Machine Translation

Translation Models

T RANSLATION M ODELSD OMAIN ADAPTATIONG OAL : Given any bi-text consisting of both S → T and T → S subsets,
improve translation quality by taking advantage of information
pertaining to the direction of translation
T ECHNIQUES :Union: simple concatenation of corpora
Two phrase-tables: train a phrase table for each subset and pass
both to MOSES
Phrase table interpolation: using perplexity minimization(Sennrich, 2012)
Add a feature in the phrase table indicating the direction oftranslation© Shuly Wintner (University of Haifa)

Translationese

12 December, 201689 / 112

Applications for Machine Translation

Translation Models

T RANSLATION M ODELSA DAPTATION RESULTS

Task: French-to-English
SystemMIX MIX-EO MIX-FO
S →T
35.21
35.21
35.73
UNION
35.27
35.36
35.94
PPLMIN-1 35.46
35.59
36.26
PPLMIN-2 35.75
35.65
36.20
CrEnt
35.54
35.45
36.75
PplRatio
35.59
35.78
36.22

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

90 / 112

Applications for Machine Translation

Translation Models

T RANSLATION M ODELSA DAPTATION RESULTS

Task: English-to-French
SystemMIX MIX-FO MIX-EO
S →T
29.15
29.15
29.94
UNION
29.27
29.44
30.01
PPLMIN-1 29.64
29.94
29.65
PPLMIN-2 29.50
30.45
30.12
CrEnt
29.47
29.45
30.44
PplRatio
29.65
29.62
30.34

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

91 / 112

The Power of InterferenceO UTLINE
1I NTRODUCTION

2I DENTIFICATION OF T RANSLATIONESE

3

T RANSLATION S TUDIES H YPOTHESES

4S UPERVISED C LASSIFICATION

5U NSUPERVISED C LASSIFICATION6A PPLICATIONS FOR M ACHINE T RANSLATION

7T HE P OWER OF I NTERFERENCE

8

C ONCLUSION© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

92 / 112

The Power of Interference

T HE P OWER OF I NTERFERENCE

Translations to English from L1 differ from translations to English
from L2
Translations from two languages are more similar to each otherwhen the two source languages are closer
The native language of ESL speakers can be accurately identified
by looking at their English texts© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

93 / 112

The Power of Interference

Cross-classification

C ROSS - CLASSIFICATION

A classifier is trained to distinguish between O and T translated
from L1 ; it is then used to distinguish O from T translated from L2(Koppel and Ordan, 2011)

E XAMPLE (T RAIN ON L1, TEST ON L2)
IT
FR
ES
DE
FI

© Shuly Wintner (University of Haifa)

IT
99
90
85
73
58

FR
93
98
90
74
70

ES
91
90
98
74
64

Translationese

DE
75
83
82
98
81

FI
63
70
69
70
99

12 December, 2016

94 / 112

The Power of Interference

Clustering of Translations

C LUSTERING OF T RANSLATIONS FROM S EVERAL
L ANGUAGESFirst, translations to English from 14 European languages wereused for cross-classification using POS trigrams as features
The accuracy of classification was 76% (compare to a baseline of
100/14=7%)
Then, an unsupervised hierarchical clustering algorithm grouped
together English (and French) texts translated from 16 European
languages
Texts were represented using feature vectors, similarly to thesupervised experiments© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

95 / 112

The Power of Interference

Clustering of Translations

C ROSS - CLASSIFICATIONE XAMPLE (C ONFUSION MATRIX OF 14- WAY CLASSIFICATION OFE NGLISH ( LEFT ) AND F RENCH ( RIGHT ) TRANSLATIONESE )© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

96 / 112

The Power of Interference

Clustering of Translations

T HE E UROPEAN T REE OF L ANGUAGESE XAMPLE (“G OLD ” TREE )© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

97 / 112

The Power of Interference

Clustering of Translations

T HE E UROPEAN T REE OF L ANGUAGESE XAMPLE (T REES OBTAINED WITH POS- TRIGRAMS , E NGLISH( LEFT ) AND F RENCH ( RIGHT ))© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

98 / 112

The Power of Interference

Clustering of Translations

T HE E UROPEAN T REE OF L ANGUAGESE XAMPLE (E VALUATION OF TYPOLOGY TREES AGAINST GOLDTREE )
FeaturePOS-trigrams
positional tokens
function words
cohesive markers
feature combination
random tree

© Shuly Wintner (University of Haifa)

Dist
EN T FR T
1.02
1.16
1.10
1.15
1.32
1.82
1.95
1.95
1.00
1.46
2.00

Translationese

Distw
EN T FR T
1.07
1.34
1.30
1.37
1.43
1.90
2.15
2.03
1.13
1.58
1.95

12 December, 2016

99 / 112

The Power of Interference

Native Language Identification

NATIVE L ANGUAGE I DENTIFICATION

Given a set of essays composed by ESL students, identify theauthors’ native language (out of 11 languages)
The dataset consists of TOEFL essays, contributed by the ETS
Each text is annotated by the prompt, the proficiency level and thenative language
The task is to identify the native language
Features
Results (Tsvetkov et al., 2013)© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

100 / 112

The Power of Interference

Native Language Identification

NATIVE L ANGUAGE I DENTIFICATIONARA CHI FRE GER HIN ITA JPN KOR SPA TEL TUR
ARA
CHI
FRE
GER
HIN
ITA
JPN
KOR
SPA
TEL
TUR

80
3
2
1
2
2
2
1
2
0
4

0
80
2
1
0
0
1
5
0
1
0

2
0
81
1
0
3
1
2
2
0
2

© Shuly Wintner (University of Haifa)

1
1
5
93
1
1
2
0
0
0
2

3
4
1
0
1
2
0
0
77
1
1 87
0
1
1
0
1
8
18
1
0
2

1
6
1
0
0
1
87
9
2
2
2

0
7
0
1
1
0
5
81
1
1
4

Translationese

4
1
3
1
5
3
0
1
78
1
4

2
0
0
0
9
0
0
0
1
73
0

3
1
3
2
4
2
1
0
5
3
80

P (%)

R (%)

F1

80.8
88.9
86.2
87.7
74.8
82.1
78.4
80.2
77.2
85.9
76.9

80.0
80.0
81.0
93.0
77.0
87.0
87.0
81.0
78.0
73.0
80.0

80.4
84.2
83.5
90.3
75.9
84.5
82.5
80.6
77.6
78.9
78.4

12 December, 2016

101 / 112

ConclusionO UTLINE
1I NTRODUCTION

2I DENTIFICATION OF T RANSLATIONESE

3

T RANSLATION S TUDIES H YPOTHESES

4S UPERVISED C LASSIFICATION

5U NSUPERVISED C LASSIFICATION6A PPLICATIONS FOR M ACHINE T RANSLATION

7T HE P OWER OF I NTERFERENCE

8

C ONCLUSION© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

102 / 112

Conclusion

C ONCLUSION

Machines can accurately identify translated texts
Translation “universals” should be reconsidered. Not only are they
dependent on genre and register, they also vary greatly acrossdifferent pairs of languages
The best performing features are those that attest to the‘fingerprints’ of the source on the target
Interference, a pair-specific phenomenon, dominates othermanifestations of translationese
Translationese features are overshadowed by more salient featuresof the text, including genre, register, domain, etc.

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

103 / 112

Conclusion

C ONCLUSION

Language models compiled from translated texts are better for
SMT than ones compiled from original texts
Translation models translated in the same direction as that of the
SMT task are better than ones translated in the reverse direction
Translation models can be adapted to translationese, thereby
improving the quality of SMT© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

104 / 112

ConclusionF UTURE D IRECTIONS

The features of Hebrew translationese: morphological markers(Avner et al., 2016)
The features of machine translationese
Robust identification of machine translation output
More generally, the similarities and differences among various
interference phenomena:
Translation
Non-native speakers
Learners’ productions© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

105 / 112

Conclusion

ACKNOWLEDGEMENTS
Gennadi Lembersky, Noam Ordan, Ella Rabinovich, Vered Volansky
Israel Ministry of Science and Technology© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

106 / 112

Conclusion

M AIN P UBLICATIONSElla Rabinovich and Shuly Wintner. Unsupervised Identification ofTranslationese. Transactions of the Association for Computational
Linguistics 3:419-432, 2015
Vered Volansky, Noam Ordan and Shuly Wintner. On the features oftranslationese. Digital Scholarship in the Humanities 30(1):98-118,
April 2015
Gennadi Lembersky, Noam Ordan and Shuly Wintner. Improving
Statistical Machine Translation by Adapting Translation Models toTranslationese. Computational Linguistics 39(4):999-1023, December
2013
Gennadi Lembersky, Noam Ordan and Shuly Wintner. Language
Models for Machine Translation: Original vs. Translated Texts.
Computational Linguistics 38(4):799-825, December 2012

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

107 / 112

Conclusion

B IBLIOGRAPHY I
Roee Aharoni, Moshe Koppel, and Yoav Goldberg. Automatic detection of machine translated text and translation quality
estimation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages
289–295, Baltimore, Maryland, June 2014. Association for Computational Linguistics. URL
http://www.aclweb.org/anthology/P14-2048.
Omar S. Al-Shabab. Interpretation and the language of translation: creativity and conventions in translation. Janus,
Edinburgh, 1996.
Ehud Alexander Avner, Noam Ordan, and Shuly Wintner. Identifying translationese at the word and sub-word level. Digital
Scholarship in the Humanities, 31(1):30–54, April 2016. doi: http://dx.doi.org/10.1093/llc/fqu047. URL
http://dx.doi.org/10.1093/llc/fqu047.
Mona Baker. Corpus linguistics and translation studies: Implications and applications. In Mona Baker, Gill Francis, andElena Tognini-Bonelli, editors, Text and technology: in honour of John Sinclair, pages 233–252. John Benjamins,
Amsterdam, 1993.
Marco Baroni and Silvia Bernardini. A new approach to the study of Translationese: Machine-learning the difference
between original and translated text. Literary and Linguistic Computing, 21(3):259–274, September 2006. URL
http://llc.oxfordjournals.org/cgi/content/short/21/3/259?rss=1.
Nitza Ben-Ari. The ambivalent case of repetitions in literary translation. Avoiding repetitions: A “universal” of translation?
Meta, 43(1):68–78, 1998.
Shoshana Blum-Kulka. Shifts of cohesion and coherence in translation. In Juliane House and Shoshana Blum-Kulka,
editors, Interlingual and intercultural communication Discourse and cognition in translation and second language
acquisition studies, volume 35, pages 17–35. Gunter Narr Verlag, 1986.
Shoshana Blum-Kulka and Eddie A. Levenston. Universals of lexical simplification. Language Learning, 28(2):399–416,
December 1978.
Shoshana Blum-Kulka and Eddie A. Levenston. Universals of lexical simplification. In Claus Faerch and Gabriele Kasper,
editors, Strategies in Interlanguage Communication, pages 119–139. Longman, 1983.

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

108 / 112

Conclusion

B IBLIOGRAPHY II
Andrew Chesterman. Beyond the particular. In Anna Mauranen and Pekka Kujamäki, editors, Translation universals: Do
they exist?, pages 33–50. John Benjamins, 2004.
Kenneth Ward Church and Patrick Hanks. Word association norms, mutual information, and lexicography. Computational
Linguistics, 16(1):22–29, 1990. ISSN 0891-2017.
Sauleh Eetemadi and Kristina Toutanova. Asymmetric features of human generated translation. In Proceedings of the2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 159–164. Association for
Computational Linguistics, October 2014. URL http://www.aclweb.org/anthology/D14-1018.
Martin Gellerstam. Translationese in Swedish novels translated from English. In Lars Wollin and Hans Lindquist, editors,
Translation Studies in Scandinavia, pages 88–95. CWK Gleerup, Lund, 1986.
Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. The WEKA data
mining software: an update. SIGKDD Explorations, 11(1):10–18, 2009. ISSN 1931-0145. doi:
10.1145/1656274.1656278. URL http://dx.doi.org/10.1145/1656274.1656278.
Qiwei He, Bernard P. Veldkamp, and Theo de Vries. Screening for posttraumatic stress disorder using verbal features in self
narratives: A text mining approach. Psychiatry research, 198(3):441–447, 08 2012. URL
http://linkinghub.elsevier.com/retrieve/pii/S0165178112000625?showall=true.
Iustina Ilisei. A Machine Learning Approach to the Identification of Translational Language: An Inquiry into
Translationese Learning Models. PhD thesis, University of Wolverhampton, Wolverhampton, UK, February 2013. URL
http://clg.wlv.ac.uk/papers/ilisei-thesis.pdf.
Iustina Ilisei and Diana Inkpen. Translationese traits in Romanian newspapers: A machine learning approach. International
Journal of Computational Linguistics and Applications, 2(1-2), 2011.
Iustina Ilisei, Diana Inkpen, Gloria Corpas Pastor, and Ruslan Mitkov. Identification of translationese: A machine learning
approach. In Alexander F. Gelbukh, editor, Proceedings of CICLing-2010: 11th International Conference on
Computational Linguistics and Intelligent Text Processing, volume 6008 of Lecture Notes in Computer Science,
pages 503–511. Springer, 2010. ISBN 978-3-642-12115-9. URL http://dx.doi.org/10.1007/978-3-642-12116-6.

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

109 / 112

Conclusion

B IBLIOGRAPHY III
Patrick Juola. Authorship attribution. Foundations and Trends in Information Retrieval, 1(3):233–334, 2006. URL
http:/dx.doi.org/10.1561/1500000005.
Dorothy Kenny. Lexis and creativity in translation: a corpus-based study. St. Jerome, 2001. ISBN 9781900650397.
Philipp Koehn. Europarl: A parallel corpus for statistical machine translation. In Proceedings of the tenth MachineTranslation Summit, pages 79–86. AAMT, 2005. URL http://mt-archive.info/MTS-2005-Koehn.pdf.
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan,
Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. Moses:
Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association
for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pages 177–180,
Prague, Czech Republic, June 2007. Association for Computational Linguistics. URL
http://www.aclweb.org/anthology/P07-2045.
Moshe Koppel and Noam Ordan. Translationese and its dialects. In Proceedings of the 49th Annual Meeting of the
Association for Computational Linguistics: Human Language Technologies, pages 1318–1326, Portland, Oregon,
USA, June 2011. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P11-1132.
Moshe Koppel, Shlomo Argamon, and Anat Rachel Shimoni. Automatically categorizing written texts by author gender.
Literary and Linguistic Computing, 14(3), 2003.
David Kurokawa, Cyril Goutte, and Pierre Isabelle. Automatic detection of translated text and its impact on machinetranslation. In Proceedings of MT-Summit XII, pages 81–88, 2009.
Sara Laviosa. Core patterns of lexical use in a comparable corpus of English lexical prose. Meta, 43(4):557–570, December
1998.
Sara Laviosa. Corpus-based translation studies: theory, findings, applications. Approaches to translation studies. Rodopi,
2002. ISBN 9789042014879.
Gennadi Lembersky, Noam Ordan, and Shuly Wintner. Language models for machine translation: Original vs. translated
texts. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages
363–374, Edinburgh, Scotland, UK, July 2011. Association for Computational Linguistics. URL
http://www.aclweb.org/anthology/D11-1034.
© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

110 / 112

Conclusion

B IBLIOGRAPHY IV
Gennadi Lembersky, Noam Ordan, and Shuly Wintner. Adapting translation models to translationese improves SMT. In
Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics,
pages 255–265, Avignon, France, April 2012a. Association for Computational Linguistics. URL
http://www.aclweb.org/anthology/E12-1026.
Gennadi Lembersky, Noam Ordan, and Shuly Wintner. Language models for machine translation: Original vs. translated
texts. Computational Linguistics, 38(4):799–825, December 2012b. URL
http://dx.doi.org/10.1162/COLI_a_00111.
Gennadi Lembersky, Noam Ordan, and Shuly Wintner. Improving statistical machine translation by adapting translation
models to translationese. Computational Linguistics, 39(4):999–1023, December 2013. URL
http://dx.doi.org/10.1162/COLI_a_00159.
Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory, 37(1):145–151, January 1991. ISSN 0018-9448. doi: 10.1109/18.61115. URL http://dx.doi.org/10.1109/18.61115.
Lin Øverås. In search of the third code: An investigation of norms in literary translation. Meta, 43(4):557–570, 1998.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: A method for automatic evaluation of machinetranslation. In ACL ’02: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics,
pages 311–318, Morristown, NJ, USA, 2002. Association for Computational Linguistics. doi:
http://dx.doi.org/10.3115/1073083.1073135.
Marius Popescu. Studying translationese at the character level. In Galia Angelova, Kalina Bontcheva, Ruslan Mitkov, andNicolas Nicolov, editors, Proceedings of RANLP-2011, pages 634–639, 2011.
Rico Sennrich. Perplexity minimization for translation model domain adaptation in statistical machine translation. In
Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics,
pages 539–549, Avignon, France, April 2012. Association for Computational Linguistics. URL
http://www.aclweb.org/anthology/E12-1055.
Sanjun Sun and Gregory M. Shreve. Reconfiguring translation studies. Unpublished manuscript, 2013. URL
http://sanjun.org/ReconfiguringTS.html.
© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

111 / 112

Conclusion

B IBLIOGRAPHY V
Elke Teich. Cross-Linguistic Variation in System and Text: A Methodology for the Investigation of Translations andComparable Texts. Mouton de Gruyter, 2003.
Joel Tetreault, Daniel Blanchard, and Aoife Cahill. A report on the first native language identification shared task. In
Proceedings of the Eighth Workshop on Building Educational Applications Using NLP. Association for
Computational Linguistics, June 2013.
Gideon Toury. Interlanguage and its manifestations in translation. Meta, 24(2):223–231, 1979.
Gideon Toury. In Search of a Theory of Translation. The Porter Institute for Poetics and Semiotics, Tel Aviv University,
Tel Aviv, 1980.
Gideon Toury. Descriptive Translation Studies and beyond. John Benjamins, Amsterdam / Philadelphia, 1995.
Yulia Tsvetkov, Naama Twitto, Nathan Schneider, Noam Ordan, Manaal Faruqui, Victor Chahuneau, Shuly Wintner, andChris Dyer. Identifying the L1 of non-native writers: the CMU-Haifa system. In Proceedings of the Eighth Workshop
on Innovative Use of NLP for Building Educational Applications, pages 279–287. Association for Computational
Linguistics, June 2013. URL http://www.aclweb.org/anthology/W13-1736.
Hans van Halteren. Source language markers in EUROPARL translations. In Donia Scott and Hans Uszkoreit, editors,
COLING 2008, 22nd International Conference on Computational Linguistics, Proceedings of the Conference, 18-22
August 2008, Manchester, UK, pages 937–944, 2008. ISBN 978-1-905593-44-6. URL
http://www.aclweb.org/anthology/C08-1118.
Ria Vanderauwerea. Dutch novels translated into English: the transformation of a ‘minority’ literature. Rodopi,
Amsterdam, 1985.
Vered Volansky, Noam Ordan, and Shuly Wintner. On the features of translationese. Digital Scholarship in the
Humanities, 30(1):98–118, April 2015.

© Shuly Wintner (University of Haifa)

Translationese

12 December, 2016

112 / 112