Translation in Transition:
Human and Machine Intelligence
October 15–17, 2020

Book of Abstracts

Contents
TT5 Program Committee and Organizers .......................................................................................... iii
Foreword .................................................................................................................................................iv
Abdullah A. Hasan & Husam Qasim
Aspire: an MT-based Bilingual Corpus Aligner Utilizing Triangulated Parameters to
Achieve Higher Accuracy ....................................................................................................................... 1
Alina Karakanta
Subtitling in Transition: The case of TED talks ................................................................................... 4
Ana Guerberof Arenas & Antonio Toral
CREAMT: creative shifts, readers engagement and translation reception in literarytexts translated in three modalities ........................................................................................................ 7
Deniz Özkan, Ena Hodzik, & Ebru Diriker
Predictive eye-movements in student vs. professional Turkish (A) - English (B)simultaneous interpreters: between-group and individual differences ........................................... 10
Devin Gilbert
Using Commercially Available Customizable NMT to Study Translator Style ............................. 14
Haruka Ogawa
Production Duration and Translator Characteristics ...................................................................... 19
Jacob T. Høgh
Testing of Perceived Sound Quality on Interprefy App ................................................................... 23
Jia Feng, Shirong Chen, Michael Carl, Jinjin Chen, & Yueqi Zhu
Eye-Voice Span in Sight Interpreting: Evidence from Both Process and Product ........................ 25
Kamal Deep Garg, Ajit Kumar, & Vishal Goyal
Addressing the Rare Word Problem in Punjabi to English Neural MachineTranslation ............................................................................................................................................ 28iKamal Kumar Gupta, Rejwanul Haque, Asif Ekbal, & Pushpak Bhattacharyya
Augmenting Dependency Tags in Interactive Neural Machine Translation .................................. 32
Kara Warburton
Supporting Translators Through Keyword Mining ......................................................................... 34
Katarzyna Stachowiak-Szymczak
Psychosocial awareness in community translation: A report on human and machinetranslation and post-editing................................................................................................................. 39
M. Cristina Toledo-Báez & Michael Carl
Assessing low and high translation variation in post-editing ........................................................... 41
Masaru Yamada, Mayuka Yamamoto, Nanami Onish, Atsushi Fujita, Rei Miyata,& Kyo Kageura
Metalanguage for the translation process .......................................................................................... 46
Michael Carl
Explorations in Empirical Translation Process Research: A Book Presentation .......................... 52
Miguel A. Jiménez-Crespo & Joseph Casillas
Are literal translations always easier to process?: a process-based study of effort
based on comparable corpus data....................................................................................................... 56
Oliver Czulo
FSEM: Current developments and perspectives ............................................................................... 60
Valentina Ragni & Lucas Nunes Vieira
Going off track? Activity tracking in translation. Translators perceptions, ethicalissues and positive uses. ....................................................................................................................... 64
Wenchao Su & Xiaoxing Zhao
Researching Automatic Speech Recognition on the Process and Quality of Written
Translation: Discussions of Indicators ............................................................................................... 66ii

TT5 Program Committeex

Chantal Gagnon

xKoen PlevoetsxDaniel Gallego-Hernández

xMarie-Aude Leferx

Dorothy Kennyx

Oliver Czulo

x

Ekaterina Lapshinova-Koltunskix

Ralph KrugerxFederico Gasparix

Silvia Hansen-SchirraxIsabelle Delaere

x

Sonia González Cruzx

Jennifer Festx

Stella Neumannx

Joke Daemsx

Éric André Poirier

TT5 Organizers
x Michael Carlx Isabel Lacruz
x Kairong Xiao
x Devin Gilbertiii

Foreword
Since its inauguration in Copenhagen (2014), Translation in Transition has becomea central meeting point for empirical translation studies in Europe throughsuccessive editions of the conference in Germersheim (2015), Ghent (2017), andBarcelona (2019). The fifth iteration of this conference (TT5) was held outside
Europe, at Kent State University (USA). It was, as before, a forum of discussion
focused on empirical research in the fields of translation and interpreting.
As in the previous iterations, TT5 provides a forum for discussion to learn more
about how human translators exercise their skill cognitively and also how computer
programs can be designed to help human translators, by automatically translating
written text, by recognizing and translating spoken utterances, or—more indirectly—
by logging translation events and analyzing recorded process data.
The special focus of TT5 is on human and machine intelligence. In times ofincreasing machine intelligence, translation aides and translation technologies arechanging at a rapid pace, fundamentally transforming the status of translation and thetranslation profession. TT5 aims at discussing related questions, including: How canhumans cope with machine intelligence that is being developed? Also, how dohumans cope with machine intelligence, and how can machines adapt to the humancondition? What are the fundamental mechanisms that underlie human translation
performance? What is the effect of technology on the translation process, translation
performance, job satisfaction, the translation product, and society?
The following extended abstracts represent this discussion in translation process
research. They were submitted and accepted by poster presenters at the fifth
iteration of Translation in Transition.

iv

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Aspire: an MT-based Bilingual Corpus Aligner Utilizing Triangulated
Parameters to Achieve Higher Accuracy

Abdullah A. Hasan

Husam Qasim

Mayo Clinic
912 Newcastle Dr. Champaign, IL 61822

Kent State University
475 Janik Dr, Kent OH 44240

hasan.abdullah@mayo.edu

hqasim@kent.edu

1

Introduction

Aligning bilingual corpus is essential formachine translation (MT) engine training, corpus
linguistics studies, term extraction, concordance
search, and the creation of translation memories
for use in computer- assisted translation (CAT)tools.
While segment alignment based on semantic
features has recently offered more promising
results than length-based algorithms, there is stillroom for improvement. For example, the MTbased tool Bleualign (Sennrich & Volk, 2010)
offers little customizability, and does not perform
adequately when handling corpora containing
clutter and noise, which is typical of content
harvested from the web, or when there areversion differences between the source and thetarget. These shortcomings can also be observedin algorithms that do not use MT, like the lengthbased approach (Gale & Church, 1991) andH nalign dic i na - based approach (Varga etal., 2007).
On the other hand, proprietary tools that rely ingreat part on manual segment alignment or usercorrection using a graphical editor requiresignificant effort and time to produce highquality translation memories. Without manual
correction, alignments created using thesegraphical editors may produce poor results.

In light of the previous, a new MT-basedalignment tool, Aspire, was created in Python totolerate higher levels of noise in data and toachieve higher accuracy in alignment. Aspire
addresses the problem of misalignments andnoise by using a semantic scoring algorithm that
relies on scikit-lea n im lemen a i n of Tf-idf(term frequency-inverse document frequency),
which is a term weighting model that is usefulfor measuring semantic similarity. In addition to
Tf-idf, Aspire uses dynamic programming andtriangulated parameters such as positionalfeatures, anchors, length correspondence, as wellas other strong indicators, such as numbers,
punctuation marks, and acronyms to achieve
highly accurate alignment. Even without humanreview, Aspire can achieve high levels of
accuracy. When tested on content obtained fromthe web, it correctly aligned 96% of the possible
pairs, and its error rate was less than 1%. The
remaining pairs were false negatives.

2

Methodology

Forming a probability model as whole, thetriangulated, interdependent parameters take into
account semantic, positional, and length
correspondence to align segments even when twosegments that should align are not positioned
similarly in the source and target

Page 1

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

texts or when they achieve a low semantic score.
The main factors taken into consideration andmeasured by the tool to achieve higher alignment
accuracy are:a. A lexical score based on machinetranslation of the target text, which iscompared to the source text. Semantic
comparison is performed using Tf-idf cosine
similarity after the text is tokenized andstemmed.
b. A length score. A pair with both highsemantic and length scores is used for setting
anchors, which are highly reliable points ofreference that neighboring pairs can use as
hints for calculating positional scores.
c. A positional score that represents theegmen
di ance f m he la known
anchor.
d. A special tokens score based onacronyms, numbers, and other uniquesymbols. This score is used for elevating thescore of low-accuracy pairs surrounded byhigh-confidence pairs.
Each factor is assigned a level of importance, i.e.
weight, which can be customized. Semantic
correspondence is typically given the highest
weight. Eventually, scores and weights are used
to produce an average score for each segment
pair. The more two segments correspond in termsof semantic equivalence, position, and length, thehigher the average score is.

Weights and minimums for semantic and length
scores are adjustable. Accordingly, advanced
users can make them stricter if higher accuracy isdesirable at the expense of coverage1. This
flexibility may come in handy depending on how
a user wants to employ the tool.

3

Manual alignments were conducted on bilingual
texts obtained from the websites of Human
Rights Watch and The World HealthOrganization.
The
samplesize
wasapproximately 20,000 words divided across thefollowing language pairs: English-Arabic,
English-Spanish, and English-French. The same
sample was then automatically aligned using
three tools: Aspire, Hunalign, and Bleualign. The
outputs of the three tools were compared with themanual alignments in terms of coverage rate anderror rate2. The same segmentation was used forboth manual and automatic alignments, and thesame MT was used for Aspire and Bleualign.
The following table offers a comparison of
accuracy tests:

Algorithm

Weights are customizable. The following
may serve as a typical weighting scheme:

Coverage rate

Error rate

Aspire96%

0.30%

Hunalign

91%

10%Bleualign

43%

56%

Table 1: Aspire's coverage rate and error rate

a. Semantic: 70% importance.
b. Length: 10% importance.
c. Positional: 10% importance.
d. Special tokens: 10% importance.
e.

compared to Hunalign and Bleualign.

4

After aligning the most likely pairs, the algorithm
looks for any unpaired fragments and joins themwith their most likely matches using the samescoring parameters, thus forming one-to-many andmany-to-one pairs even when the source and thetarget do not employ similar sentence boundaries.

InterfaceAspire can be run with or without user editing, but
a visual interface is available if manual correction
is needed. The user can also customize the weightassigned to each parameter as needed. The user
can preview alignment results and scores, filteraligned segments by weighted scores, reject false
positives, and override false negatives asnecessary before exporting the final alignments.

1 Good alignments that are captured by the tool.

2

Evaluation method and results

Rate of bad alignments and noise.

Page 2

Book of Abstracts: Translation in Transition (TT5)

Figure 1: A

e ie

fA ie

October 15 - 17, 2020c ing e linterface5

Future Implications

References

Tests so far show that Aspire captures a greaternumber of alignments with a smaller risk for
false positives. This helps creating translation
memories that are both larger and cleaner. By
extracting highly accurate alignments, Aspire
could potentially be used to offer better MT
training data. The tool could also be used torecycle legacy translations to build bettertranslation memories for use in CAT tools, term
extraction, or concordance search.
With the rise of affordable and self-hosted
machinetranslation engines, MT-basedalignment will likely become a standardalignment method.

6

Gale, W. A., & Church, K. W. 1991. A program foraligning
sentencesinbilingualcorpora.
Proceedings of the 29th Annual Meeting on
Association for Computational Linguistics.
Sennrich, Rico, & Volk, Martin. 2010. MT- based
Sentence Alignment for OCR- generated Parallel
Texts. Proceedings of AMTA 2010.
Varga, D., Halácsy, P., Kornai, A., Viktor, N., László,
N., & Viktor, T. 2007. Parallel corpora for medium
density languages.

Limitations

The tool was tested on three language pairs
only. More testing on a larger sample and on
more language pairs should prove helpful indemonstrating alignment quality for othercontent types.

Page 3

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Subtitling in Transition: The case of TED talks

Alina Karakanta
Fondazione Bruno Kessler
University of Trento
akarakanta@fbk.eu

1

2

Introduction

Subtitles are a vital tool for facilitating
accessibility and the spread of information toaudiences with different linguistic needs andpreferences. With the advances in digitaltechnologies, subtitling, as a form of audiovisualtranslation, is no longer a task practiced
exclusively by professionals, but it has become
extremely popular among volunteers and fans(Bogucki, 2009; Díaz-Cintas, J. & Muñoz
Sánchez, 2005; Ferrer Simó, 2005). One of thelargest projects of volunteer subtitling is the TED
Talk Translators programme, with ~3000 talks invarious topics subtitled in 116 languages bymore than 33,000 volunteers. The programme
has been constantly growing and adapting to
respond to the increasing global interest in TED
talks. Over the years, subtitling guidelines havebeen drafted, new tools have been introduced anda thorough quality check process has been set up.
In this work, we pose the question of whether
this constant transition and refinement of theworkflows of the programme is mirrored in thequality of the produced subtitles, in terms of theirform and conformity to traditional subtitling
constraints (Díaz-Cintas and Ramael, 2007). We
conduct a corpus analysis based on TED talk
subtitles, focusing here on their conformity to thesubtitling constraint of length (maximum numberof characters per subtitle).

Methodology

Data: We use MuST-Cinema (Karakanta et al.,2020), a multilingual speech-to-subtitles corpus
compiled from the subtitle files (.srt) of TED
alk in 7 lang age ai (Engli h Ge man,Spanish, Italian, French, Portuguese, Dutch,
Romanian). The corpus contains (audio, source
language subtitles, target language subtitles)
triplets.
Metric: We investigate the conformity to thesubtitling constraint of length. Due to the limited
space on screen, a subtitle cannot exceed a
specific length. According to the TED translation
guidelines, conformity is measured with amaximum subtitle length of <=84 characters(maximum 2 lines of up to 42 characters each).
3

Results

We compute the percentage of non-conforming
subtitles per year for each language. The year
corresponds to the date of release of the talk, but
due to the popularity of the programme, talks arenormally translated in a short time after theirrelease. Figure 1 shows the percentage of nonconforming subtitles per year for French. We
observe a consistent decrease in the percentage
of subtitles not respecting the length constraint,
especially after 2012. The same tendency isobserved for all languages, even though somePage 4

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Figure 1. Percentage of non-conforming subtitles per year (French)languages have consistently higher nonconformity percentages than others (higher for
Spanish and French, lowest for Portuguese). For
all languages, there is a sharp reduction in thenumber of non-conforming subtitles in year
2009, gradually reaching an almost full
conformity around 2012. This suggests a shift
f m a
lain
an la i n f hecetranscript, which does not consider subtitling
constraints, to well-formed subtitles.
This shift can be attributed to several factors: as
the pool of volunteers grew, multiple roles could
be allocated to review and quality-check thetranslation. Subtitling guidelines were drafted
and training material was released for thevolunteers. Technology and tools seem to haveplayed an important role. To ensure better
management, the translation project wastransferred under the Amara collaborative
platform. The task of subtitling is performed
through the Amara subtitle editor, which allows
for automatically controlling length and reading
speed constraints. This facilitates the work of
subtitlers by notifying them when subtitles
exceed the maximum length, leading them torephrase, condense and omit information in order
to avoid delivering a translation with error
messages. Therefore, it is possible that the
adoption of the subtitling tool increased thesubtitle quality in terms of length conformity. By
conducting a qualitative evaluation of several
talks from different years, we observed that themain reasons for the non-conformity to thelength constraint are the lack of compression
and/or improper subtitle segmentation. The
following examples in French and Italian comef m Al G e 2006 alk A e ing he clima e
c i i 1. The number of characters is shown inparentheses. The source language subtitles(English) here are verbatim transcriptions of the
spoken utterances and have a frequent
segmentation, which leads to one-liners andkeeps the number of characters low:
116
00:06:03,464 --> 00:06:07,281
What can you do about the climate crisis? (41)117
00:06:08,137 --> 00:06:11,476
I want to start with a couple of -- (35)118
00:06:11,500 --> 00:06:14,069
I'm going to show some new images, (34)119
00:06:14,093 --> 00:06:19,216

1

https://www.ted.com/talks/al_gore_averting_the_cli
mate_crisis#t-353391

Page 5

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020and I'm going to recapitulate just four or five.
(48)

4

French:71
00:06:02,820 --> 00:06:10,820
Qu'est-ce que vous pouvez faire contre leréchauffement
climatique?
J'aimerais
commencer avec -- (96)72
00:06:10,820 --> 00:06:19,820
Je vais vous montrer de nouvelles diapositives et
je vais juste en détailler quatre ou cinq. (92)Conclusion

Our findings show that there is a chronologicalshift towards higher subtitling quality (in termsof subtitling constraints) in TED Talks, as aresult of a thorough quality assurance process,well-defined subtitling guidelines, training
material and the use of a subtitling tool. Theseelements could be adopted as best practices forfurther projects of volunteer subtitling to ensure
the efficient diffusion of knowledge throughaudiovisual content.

References
Italian:71
00:06:02,820 --> 00:06:10,820
Cosa si può fare per la crisi del clima? (40)72
00:06:10,820 --> 00:06:19,820
Voglio iniziare mostrando alcune nuove
immagini, solo 4 o 5. (60)
While both languages use the same timestamps,
the French subtitles exceed by a large margin the
84-character limit. The false start J'aimerais
commencer avec is reproduced from the English
captions and included in subtitle 71 for French,but it is omitted in Italian. In the French subtitle
72 there is an unnecessary repetition of thephrase je vais, leading to a subtitle of 92
characters. In addition, the French subtitle
follows the phrasing of the English one. The
Italian subtitle 72, however, is much morecompact; the subtitler has merged the two mainclauses in one and rephrased them in order toprovide a shorter solution. In general, the Italianversion has a larger degree of compression andomission than the French. This example showsthat, despite a transition point towards higher
conformity being present in 2009, there is somevariation among the languages, which should be
investigated in other factors, such as degree ofe an i n f he a ge lang age,l n eeexperience, and properties of the talks (speechrate, disfluencies etc).

Amara - Award-winning Subtitle Editor andEnterprise Offerings. https://amara.org/en/ Last
accessed on 3 June 2020.
B g cki . (2009) Ama e S b i ling n he In e ne .
In: Cintas J.D., Anderman G. (eds) AudiovisualTranslation. Palgrave Macmillan, London
Díaz-Cintas, J. and Aline Remael. 2007.AudiovisualTranslation: Subtitling. Translation practices
explained. Routledge.
Díaz-Cintas, J. & Muñoz Sánchez, P. 2006. Fansubs:
AudiovisualTranslation
in an
Amateur
Environment. The Journal of Specialised
Translation 6, 37-52.
Karakanta, A.; Negri M. & Turchi M. 2020. MuSTCinema: a Speech-to-Subtitle corpus. In
Proceedings of the 12th International Conferenceon Language Resources and Evaluation (LREC2020), Marseille, France, May 13-15 2020.
TED
Ourorganization.
https://www.ted.com/about/our-organization. Last
accessed on 3 June 2020.
TED
Translators
Cheat-sheet.
https://translations.ted.com/TED_Translators_Cheat-sheet. Last accessed on 3 June 2020.
Fe e Sim , M. R. 2005. Fan b
canla i n : lainfluencia del aficionado en los criterios
fe i nale .
Puentes
6:27-43.

Page 6

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

CREAMT: creati e hif , eadee gage e

and translation

reception in literary texts translated in three modalities

1

Ana Guerberof Arenas

Antonio ToralCentre for Translation Studies

Center for Language and Cognition

University of Surrey/MSC IF at
University of Groningen

University of Groningena.guerberof.arenas@rug.nl

a.toral@rug.nl

Introduction

Technology in general and machine translation(MT) in particular are intrinsically included in thetranslation process in the language industry. In
fact, the 2020 Language Industry Survey by theEuropean
AssociationofTranslation
1
Companies shows machine translation postediting (MTPE) to be the most popular service
and MT the strongest technology trend.
Automation has focused primarily on howtechnical or scientific translations are produced
and processed because of their repetitive natureand the priority given to accuracy. In recent
years, however, there has been an interest to see
how MT can benefit or hinder the translation of
more creative texts, i.e. literary works ormarketing.
2

there are few studies in machine-assistedtranslation that focus on the ultimate user of thetranslation (e.g. Castilho and Guerberof 2018;
G e be f, M ken and O B ien 2019).
2.1

The study presented here seeks to re-address thisfocus by applying MT to literary texts andfocusing on the final user, to answer thesequestions:
RQ1: Can we quantify the creativity in texts
translated by humans as opposed to those
produced with the aid of machines?
RQ2: Do users reading translated material
produced with and without the aid of machines
have different reading experiences?
2.2

Related Word

Some studies show that MT might help
professional literary translators to be moreproductive (Toral et al. 2018). However,an la
e ce i n i ha he m e c ea i e
the literary text, the less useful MT is (Moorkens
et al. 2018). On the other hand,

Objectives

Pilot projectIn the beginning of 2019, we carried out a pilot
experiment (Guerberof Arenas & Toral, 2019) in
one language direction (English-to- Catalan) that
included a questionnaire to assess narrative
engagement (Mangen and Kuiken 2014), using ascale created for thishttps://ec.europa.eu/info/sites/info/files/2020_language_industry_survey_report.pdf
1Page 7

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

purpose (Busselle and Bilandzic 2009), and togauge the translation reception of a fictional
piece translated in three conditions: from scratch(fully human translation by professionaltranslators), machine translated, and a post-edited
version (machine translation output corrected byprofessional translators). We found out thatreaders were more engaged with humantranslation, as expected, followedby,surprisingly, the machine translated text, andlastly by the post-edited text.
2.3

Current Project

We are presenting here the results of a follow- up
experiment that seeks to explore further thesequestions by looking at the three modalities:
Machine-translated (MT), Post- edited (MTPE)
and translated without any aid (HT) applied to ashort story.
We analyzed, on the one hand, the two main
aspects of creativity acceptability and novelty,
by looking at errors and creative shifts
respectively (Bayer-Hohenwarter, 2012), and,on the other, the narrative engagement andreception of these translated texts in threemodalities with a larger cohort of Catalan readers(n > 80).
The results2 show that, in response to RQ1,
creativity could be quantified when comparing
translated texts in different modalities. We foundthat the HT texts score higher in creativity than
MTPE, and MTPE scores higher than MT,
especially when analyzing novelty in thetranslated texts.
In response to RQ2, the clear answer is yes,readers engage differently depending on themodality. We saw a pattern where HT always

show higher scores in narrative engagement andtranslation reception, where MTPE scores slightly
higher in enjoyment. However, there were
statistically significant differences only between
HT and MT, and MTPE and MT, but not between
HT and MTPE.
Our objective is ultimately to understand how theuse of MT technology might or might notconstrain creativity in the an laprocess, andhow readers engage and receive texts that havebeen translated assisted by this technology. Our
next steps are to use this methodology to analyze
English literary texts translated into Catalan andDutch in the three aforementioned modalities3.
Funding information
This project has received funding from theE
ean Uni n H i n 2020 e ea ch andinnovation programme under the MarieSk dka-Curie grant agreement No. 890697
and f m CLCG 2020 b dge fe ea chparticipants and has been partially funded by the
Expanding Excellence in England Programme
funded by Research England.

References
Bayer-H hen a e , G. (2012). C ea i e Shif a a
Means of Measuring and Promoting Translational
Creativity.
Meta,
56(3),
663 692.
https://doi.org/10.7202/1008339arBusselle R., & Bilandzic, H. (2009). Mea ing
Na a i e Engagemen . Media Psychology, 12:4,
321-347. Routledge, Taylor and Francis Group.
Castilho, S., & Guerberof Arenas, A. (2018). Reading
Comprehension of Machine Translation Output:
Wha make f a be e ead? . In Proceedings of
21st Annual Conference EAMT, Alicante, Spain.
G e be f, A., M ken , J., and O B ien, S. (2019).
Wha is the impact of raw MT on Japanese users of

2

The full results of this experiment will beavailable in Guerberof-Arenas, A. & Toral, A.
(f hc ming). The Im ac f P -editing andMachine Translation on Creativity and ReadingE e ience . Translation Spaces. John Benjamins
Publishing.
3
https://cordis.europa.eu/project/id/890697

Page 8

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Word preliminary results of a usability study using
eye-tracking . In Proceedings of the Seventh
Machine Translation Summit (MT Summit 2019),
Dublin, Ireland.
Guerberof Arenas, Ana, and Antonio Toral.
Mea ing Reade
Engagemen in Li e a
Texts: A Study Comparing Human Translation to
Machine Assisted Translation. In Book of
Abstracts EST Congress 2019, 239. Stellenbosch:
EST, 2019.

Mangen, A., & K iken, D. (2014). L
in an iPad:Na a i e engagemen
n a e and able .
Scientific Study of Literature, 4(2), 150-177.
Moorkens, J., Toral, A., Castilho S., & Way, A. (2018).
Pe ce i n
f Li e a
P -editing using
S a i ical and Ne al Machine T an la i n .
Translation Spaces 7:2.
T al, A., Wieling, M., & Wa , A. (2018). P editing effort of a novel with statistical and neuralmachinean la i n . Frontiers in Digital
Humanities, 5, 9.

Page 9Predictive eye-movements in student vs. professional
Turkish (A) - English (B) simultaneous interpreters:
between-group and individual differences

Deniz Özkan

Ena Hodzik

Ebru Diriker

Koç University
Istanbul, Turkey

Bo a i i Uni er i
Istanbul, Turkey

Bo a i i University
Istanbul, Turkey

denizozkan@ku.edu.tr

ena.hodzik@boun.edu.tr diriker@boun.edu.tr

1. Introduction
Studies on predictive processing of spoken
language have found that language users can form
predictions on different levels as an integral partof language comprehension. They can preactivate the semantic/conceptual (Federmeier &Kutas, 1999; Federmeier & Kutas, 2001) and thestructural features (Van Berkum et al., 2005;
Wicha et al., 2004) of an upcoming argument
based on various linguistic elements in the spoken
input, such as thesemantic
and/ormorphosyntactic cues (Altmann & Kamide, 1999;
Kamide, Altmann, & Haywood, 2003). Such preactivation has been demonstrated as anticipatory
looks to the referents of upcoming arguments invisual scenes (e.g. Altmann & Kamide, 1999;
Huettig & Altmann, 2005) using the visual world
paradigm (VWP; Cooper, 1974; Tanenhaus,
Spivey-Knowlton, Eberhard, & Sedivy, 1995).
In a fairly under-investigated language in thiscontext, Turkish, Brouwer, Özkan, & Küntay
(2018) found that Turkish-speaking adults,
similar to Dutch-speaking adults, could predictthe upcoming argument based on the verb
semantics in verb-medial sentences. Moreover,Özge, Küntay, & Snedeker (2019) recently
showed that Turkish-speaking adults and 4-yearold children can predict the upcoming argument

based on morphosyntactic cues (i.e., casemarkers) in both verb-medial and verb-final
sentences. This finding implies that the casemarkers are interpreted incrementally and thethematic roles of the upcoming nouns can be
assigned based on those case marker cues.
Recent psycholinguistic
research started
investigating individual differences in predictive
processing. In an investigation of spoken
language processing using the VWP, Huettig &Janse (2016) found that the higher an indi id alworking memory capacity (WMC), the more they
engaged in prediction. Huettig (2015) argues that
language-based predictions are mediated byworking memory, where information in thesensory and linguistic input (in visuospatial
memory) is connected to information in long-term
memory (including phonological and semantic
representation ). The higher an indi id al
WMC, the faster these connections will be made.
As such, WMC appears as one important correlateof predictive processing.
Prediction has also been found in the interpretingstudies literature, where professional interpreters
have shown more instances of prediction thanstudent interpreters (Jörg 1997; Riccardi 1996). In
addition, expertise related advantages have also
been observed in tasks measuring working

Page 10

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

memory capacity (WMC) (Padilla et al., 2005;
Yudes et al., 2013). Nevertheless, while
individual variations in WMC have been related
to SI performance in untrained bilinguals(Christoffels et al., 2003) and trainee interpreters(Zhang and Yu, 2019, Lin et al., 2018), studies
with professional interpreters have found either
non-significant or weaker associations of WMCwith SI performance (Timarová et al., 2015).
Similarly, Liu et al. (2004) found differences in SI
performance between professional and novice
interpreters despite the two groups being matched
for WMC.
To our knowledge, predictive processing based onverb semantic cues has not been previously
studied in the interpreter population using an
online measure, such as eye tracking. What ismore, even though WMC has been associatedwith prediction skill and found to be enhanced ininterpreters, the relationship between the two has
not been previously investigated in the context ofinterpreting studies. As such, this study aims toiden if
he he hefe i nal in e e esemantic prediction skills during spoken language
processing differ from that of students andwhether WMC is associated with prediction skillsin those two groups. Based on previous literature,
we expected (1) the professional interpreters to
exhibit better prediction skills compared to thestudent interpreters (i.e., earlier initiation ofprediction or a stronger effect); (2) the
professional interpreters to have larger WMC
than the student interpreters; and (3) WMC skills
to be involved in predictive processing differently
for the two groups (i.e., larger involvement in thestudent interpreters than the professional
interpreters).

2. Method
Our study investigated eye movements based onverb semantic cues in student (N = 21) andprofessional (N = 20) interpreters with Turkish as
their A and English as their B language in a VWP
eye tracking experiment. Participants completed averb-based semantic prediction task and an
operation span task. All participants had normal
or corrected-to-normal vision.

The eye tracking experiment was the same as theone that was used with Turkish-speaking
participants in Brouwer et al. (2018). The
experiment had 24 visual displays with twocolored line drawings, corresponding to a target(e.g., a cake) and a competitor item (e.g., a tree).
The visual displays were paired with prerecorded
sentences that were either semanticallyconstraining (1) or neutral (2), corresponding tosemantic and neutral conditions, respectively.
(1) Çocuk yiyor bu b y k keki [semantic]
The b ea he big cake(2) Çocuk gör yor bu b y k keki [neutral]The b
ee he big cakeA female native adult speaker of Turkish recorded
the sentences, which were minimally manipulated
in PRAAT to adjust the prediction window (fromverb onset to noun onset) to 2400 ms (seeBrouwer et al., 2018 for the reasoning). Because
our participants were adults, we did not use themotivational filler items, as was done in Brouwer
e al. s original experiment with children. The
onset of the prerecorded sentences wassynchronized with the onset of the presentation of
the visual display, so that the participants did nothave a preview of the visual display.
Experimental stimuli were presented by E-Prime
software, and a Tobii T120 eye tracker recorded
he a ici an ga e beha ii h a am lingrate of 60 Hz. The procedure lasted about 15
minutes.
We used an automated version of the operation
an a k f a e ing a ici an
kingmemory span (Mizrak & Oztekin, 2016;
Unsworth, Heitz, Schrock, & Engle, 2005). The
participants were required to decide whether theanswer for an arithmetic operation was correct ornot, as they tried to remember a set of unrelated
letters (F, H, J, K, L, N, P, Q, R, S, T, Y) in thesame order as presented. The participants firstcompleted three practice sessions before theexperimental procedure began. The experiment
involved a letter presented on the screen for 1000
ms after each arithmetic operation. The list length
for the letters varied between 3 to 7. The operation
span score presented the number of correct items

Page 11

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

recalled in the correct position. The procedure
lasted about 10 minutes. This task constitutes acomplex span task as it involves both a processing
and storage component, but what sets it apart fromother complex span tasks like reading or listening
span, which only involve processing of verbal
information, is the fact that it also engages moregeneral (math) problem solving skills (Turner &Engle, 1989).

3. Results and discussion
We found a prediction effect (i.e., greater
likelihood of looks to the target object in thesemantic versus the neutral condition) for bothgroups in the verb-based semantic prediction task,
as indicated by previous research with Turkishspeaking adult participants (Brouwer et al., 2019).
However, this effect seemed to be stronger for the
professionals than the students. Based on
preliminary interpretation of the data, the twogroups exhibited different gaze behavior: theprofessional interpreters were earlier than thestudents in initiating predictive looks to the target
objects by about 300 ms.
Unlike previous literature, there were no
significant group differences in the WMCs of ourparticipants. However, when the two groups weredivided into high versus low WMC sub-groups
based on the median WMC scores, the high WMC
student interpreter group was found to outperformthe low WMC student interpreter group in theprediction task, whereas no such performancedifference was observed for the high versus low
WMC professional groups.

4. Conclusions
In line with our hypothesis, these findings
suggested that the professional interpreters
exploited the semantic prediction cue faster andmore competently than the students did. In
addition, it is the only type of cue that the
professional interpreters made use of, and perhaps
needed, unlike the students, who also relied on
their working memory span. It seems that withexperience, working memory is exploited less,not more, for the purpose of predictive
processing. This in turn may imply more selective
processing of information in the auditory input by
virtue of expertise (Liu et al., 2004).
Given the strategic importance of prediction
during interpreting, our results may help identify
the experience-related effects on predictive
processing of spoken language. Thus, the current
exploratory study may inform both conference
interpreting and psycholinguistics literatures.

References
Altmann, G. T., & Kamide, Y. (1999).
Incremental interpretation at verbs:
Restricting the domain of subsequent
reference. Cognition, 73(3), 247-264.
Brouwer, S., Özkan, D., & Küntay, A. C. (2018).
Verb-based prediction during language
processing: the case of Dutch andTurkish. Journal of child language, 46(1), 80- 97.
Christoffels, I. K., De Groot, A. M. B. & Waldorp,
L. J. (2003). Basic skills in a complex task: Agraphical model relating memory and lexical
retrieval to simultaneous interpreting.
Bilingualism: Language and Cognition, 6, 201211.
Cooper, R. M. (1974). The control of eye fixation by
the meaning of spoken language: A new
methodology for the real-time investigation of
speech perception, memory, and language
processing. Cognitive Psychology, 6(1), 84 107.
Federmeier, K. D., & Kutas, M. (1999). A rose by
any other name: Long-term memory structureand sentence processing. Journal of memory andLanguage, 41(4), 469-495.

Page 12Federmeier, K. D., & Kutas, M. (2001). Meaning andmodality: Influences of context, semantic memoryorganization, and perceptual predictability on picture
processing. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 27(1), 202.

Book of Abstracts: Translation in Transition (TT5)

Huettig, F., & Altmann, G. T. (2005). Word
meaning and the control of eye fixation:
Semantic competitor effects and the visual
world paradigm. Cognition, 96(1), B23-B32.
Huettig, F. (2015). Four central questions about
prediction in language processing. Brain
research, 1626, 118-135.
Huettig, F., & Janse, E. (2016). Individual
differences in working memory and processing
speed predict anticipatory spoken language
processing in the visual
world. Language, Cognition and
Neuroscience, 31(1), 80-93.
Jörg, U. (1997). Bridging the gap: Verb anticipationin German-English simultaneous interpreting. In
M. Snell-Hornby, Z. Jettmarova & K. Kaindl(eds.), Translation as Intercultural
Communication: Selected Papers from the EST
Congress, Prague 1995. Amsterdam: John
Benjamins Publishing.

October 15 - 17, 2020

Tanenhaus, M. K., Spivey-Knowlton, M. J.,
Eberhard, K. M., & Sedivy, J. C. (1995).
Integration of visual and linguistic informationin spoken language comprehension. Science,
268(5217), 16321634.
Tima, ., e k , I. & Me lae , R. (2015).
Simultaneous interpreting and working memory
capacity, In A. Ferreira & J. W. Schwieter (eds.)
Psycholinguistic and cognitive inquiries intotranslation and interpreting. Amsterdam,
Netherlands: Benjamins.
Turner, M. L., & Engle, R. W. (1989). Is working
memory capacity task dependent? Journal ofmemory and language, 28(2), 127- 154.
Unsworth, N., Heitz, R. P., Schrock, J. C., & Engle,
R. W. (2005). An automated version of theoperation span task. Behavior research methods,
37(3), 498-505.

Kamide, Y., Scheepers, C., & Altmann, G. T.
(2003). Integration of syntactic and semantic
information in predictive processing: Crosslinguistic evidence from German and English.
Journal of psycholinguistic research, 32(1), 3755.

Van Berkum, J. J. A., Brown, C. M., Zwitserlood, P.,
Kooijman, V., & Hagoort, P. (2005). AnticipatingUpcoming Words in Discourse: Evidence From
ERPs and Reading Times. Journal of
Experimental Psychology: Learning, Memory,
and Cognition, 31(3), 443-467.

Lin, Y., Lv, Q., & Liang, J. (2018). Predicting
Fluency With Language Proficiency, Working
Memory, and Directionality in SimultaneousInterpreting. Frontiers in Psychology, 9, 1543.

Wicha, N. Y., Moreno, E. M., & Kutas, M. (2004).
Anticipating words and their gender: An eventrelated brain potential study of semantic
integration, gender expectancy, and genderagreement in Spanish sentence reading. Journalof cognitive neuroscience, 16(7), 1272-1288.

Liu, M., Schallert, D. L., & Carroll, P. J. (2004).
Working memory and expertise in simultaneousinterpreting. Interpreting, 6(1), 19-42.
Mak, E., &ekin, I. (2016). W king mem
capacity and controlled serial memory search.
Cognition, 153, 52-62.

Özge, D., Küntay, A. and Snedeker, J. (2019). Why
wait for the verb? Turkish speaking children use
case markers for incremental language
comprehension. Cognition 183, 152-180.

Yudes, C., Macizo, P., Morales, L. & Bajo M. (2013).
Comprehension and error monitoring in simultaneousinterpreters. Applied Psycholinguistics, 34, 1039-1057.
Zhang, W. & Yu, D. W. (2019). A duet and/or aconcerto? Simultaneous interpreters' working
memory and interpreting expertise. Babel- Revue
Internationale De La Traduction- International
Journal of Translation, 65, 519537.

Padilla, F., Bajo, M. T., & Macizo, P. (2005).
Articulatory suppression in language
interpretation: Working memory capacity, dual
tasking and word knowledge. Bilingualism:
Language and Cognition, 8(3), 207-219.

Page 13

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Using Commercially Available Customizable NMT
to Study Translator Style

Devin Gilbert
Center for Research and Innovation in Translation and Translation Technology (CRITT)
Kent State University, Kent, Ohio, USA
dgilbe10@kent.edu

1

Introduction

Winters and Kenny remarked during last yea
Translation in Transition conference that there has
been a shift in the last 10 20 years of howtranslation scholars consider the use of machinetranslation (MT) with literary texts, arguing that
researchers are now considering MT to aid literaryan lain he c ea i n f fl en
whereas the field of translation studies used toonly look at applying MT to literature as a sourceof inspiration to create texts that violated targetlanguage norms (2019: 58). The focus of thiscurrent research trend is whether or not prose canbe translated by MT to the point that it could beconsidered usable for post-editing (Besacier,2014; Besacier & Schwartz, 2015; Toral & Way,
2015a; Toral & Way, 2015b; Moorkens et al.,2018).
Neural MT (NMT) systems have performed
relatively well on literary texts that are similar tothe data used to train these systems (Toral & Way,2018: 277), but no research on the application ofcommercially-available, customizable NMT fortranslating niche literary texts has yet been
conducted. This study adds to previous researchby training a commercially available,
customizable NMT engine with data from a single
author-translator combination (Angolan

Portuguese (Ondjaki 2009) to American English(Gilbert, 2020)) in order to use the customized
enginean la i n f li e a e f m he ame
author to analyze translator style.
While the customized engine did yield significant
improvements in TER1 score (base engine
average TER score: 53.8; customized engine
average TER score: 48.2; t(57) = - 2.2, p = 0.035),hidaach f cmi ing a NMT
engine with very little data to translate literarytexts would not be recommended for applicationsin the language industry. Instead, this study looks
beyond post-editing to a novel application of anovel technology: using a customized NMT
engine as a tool for studying translator style.
Specifically, what will the differences between
he c
mi ed engine
and he ba eenginee eal ab t a particularan lale? Al ,hae ea chmethodologies can be employed to leverage acustomized NMT engine as a productive tool for
textual analysis of literary translations?
2

Procedure

G gle base NMT engine was customized using
just 1,367 sentence pairs (1,095 for training, 136
for validation, and 136 for testing; altogether
19,921 source words). The cmi ed engine

1

Snover et al. (2006). Note that these TER scores were calculated on segments from the held-out test set describedin Section 2.

Page 14

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

output on a held-out test set (58 segments, 942
source words) taken from the same work that its
training data was taken from was compared with
G gle ba e NMT engine, and b h
were compared to the human translation.
These three translated versions of the held-out test
set were analyzed to identify textual features thathe cmi ed engine lea ned f m he h mantranslator. A textual feature was considered toha e been lea ned f m he h man an la if
the output of the customized engine was different
from the base engine and this difference could betied to something in the human-translated training
da a. The e lea ned e al fea ee e hencategorized according to their commonalities.
Therefore, everything in Table 1 represents
something that the customized engine diddifferently than the base engine and that could betied to a discrete textual feature in the trainingdata.
S b e en l , a c n i enc
e cen ageacalculated for each textual feature category (Table
1, % column under Te Se ). This consistency
percentage was calculated by dividing the numberof observed occurrences of each category in theheld-out test set (this number is shown in Table 1
in he N . c l mn nde Te Se ) b he al
number of times each corresponding textual
feature could have been employed as a translation
option by the customized engine.
Finally, similar calculations were made for theaining da a b
ing find f nc i n n he
TMX file containing all the sentence pairs used totrain the customized engine (calculation results
h n in c l mn labeled T aining Da a ). F
categories that were not amenable to a simple
find f nc i n, a am le f 30 en ence ai was
randomly selected from the training data and wasthen analyzed. These categories will be marked in2

See he b ec i n i led I he T an la i n API
A ML T an la i n he ighl f me? nde heec i n A ML T an la i n beginne g ide n
https://cloud.google.com/translate/automl/docs/begin
n ers-guide

Table 1 i h a da h in he N . c l mn nde
T aining Da a .
3Results and Discussion

Analysis of textual features in this d parallel
corpus revealed discrete categories that thecustomized engine learned from its training data,hich in n can be ied
he an lale.
Google states that the main advantage ofcustomizable NMT is accuracy with domainspecific vocabulary.2 Rows 5 10 of Table 1
correspond to what is meant by this. Rows 1 4
correspond to textual features that we could
consider as dealing with more than mere lexical
choice since they deal with patterns that are more
complex than what are essentially terminologicalone-to-one correspondences, as in rows 5 10.
Rows 11 13 deal with punctuation. Comparing
the consistency percentages found in thecmi ed engine
n he held-out test
set and the percentages found for the training data,
there appears to be a general correlation between
the two.
The customized engine was quite consistent when
it came to what we can consider domain-specificvocabulary. We can see this with examples suchas bocadinho [little bit], which is Angolan slang;cacimbo[mi /de ],ame e l gicalphenomenon in Angola where mornings andnights, during a certain part of the year, come withhick f g/mi ; and camarada professor(a)[comrade teacher], which is peculiar to AngolanPg ee d e
he c n-communist
exit from colonialism. The lack in consistency
i h camarada professor iling, h e e ,given the consistency that the training data
enjoyed. Relating to this gamut of textual features,
one interesting example where the customized
engine avoided mistranslation comes from theh a e, ou as plantas a darem ares duma
primeira respiração na frescura da manhã, entresilêncios e cacimbos molhados [humantranslation: or like plants stretching out as if

Page 15

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

he e aking hei fib ea h n a chillmorning, in the middle of quietness and damp
cacimb .]. The cmi ed engine lea ned
translate cacimbo via borrowing even though theterm occurs only four times in the training data,
but the base engine translated the last part of theh a e a amid ilence and e a e i e .

data). Also, it translated the imperfect verb tense
ing he a iliaebld (i.e., I
ldlook at thing in he en e f I f en edl k a hing ). While i did n em lhistrategy in the held-out test set in all of the casesthat the human translator did, the overall
percentage is comparable to results from thetraining data. A cursory reading of the parallel
corpus shows this textual feature is a salientcha ac e i ic f he an laal i ing leand is rather frequent because the narrator in Os
da minha rua often reminisces of recurrent past
events using the imperfect tense.

Table 1 shows that the customized engine
cceed in lea ning m e han jd mainspecific vocabulary. For example, it omitted
opti nal ha and ed c n ac i n m e f enthan the base engine (although, it did not use
contractions as often as was found in the training

Category

Test Set
No.
%

Training DataNo.
%

contractions instead of full form

3

21.4%

-

78.6%

i nal ha

4

66.7%

-

75.0%

ge in ead f highe -register verb

3

75.0%

-

54.5%

5

33.3%

-

20.0%

bocadinho

1

100%

11

84.6%

ing cacimbo in ead f an la ing i

1

100%

4

100%

2*

100%

2

100%

1
2

eli i n f

34

ld a 9n f

5imperfect tensebi a 9n f

6

b

7

b

ing f chl name J en de em L a

8c m ade eache a 9n f

camarada professor(a)

8

38.1%

29

100%

9

n g ne dsol (ter) ido embora

1

100%

3

100%

despedida

3

75.0%

4

100%

11

capitalizing all words in title of a work

1

100%

1

100%

12

standard English quotation marks

3

75.0%

284

99.3%

a i n ma k ( ) in ead f ( ' )

14

66.7%

788

99.2%

10

13

g

n a 9n f

db e in ead f fa e ell a 9n fe f igh ingle
* One

f he

ime , he cmi ed engine a iall bed he chl name a Juventude in Luta

Table 1 Textual Feat res of Translator s St le

In line with the above categories, there were otherexamples of the customized engine following ageneral trend of using a lower register than thebase engine. For example, the base engine
an la e he h a e um frasco grande e bonitoaa bea if l and la ge ja ,hile hecustomized engine translated it as a nice big ja .

Thi
a
imila
he h man an lal i n: a big e ja . An he e am le f
the customized engine lowering the register canbe observed when the customized engine
an la ed A camarada professora tá muito
bonita a C m ade eache l k ealle ,he ea he ba e engine ende ed C m ade
eache i e bea if l. S ch e am le a e n

Page 16

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

represented in Table 1 because it is difficult toplace them into any one, discrete category.
N ne hele , he cmi ed engine abili
pick up on patterns of lower register is remarkablegiven the very low number of sentence pairs thatwere used in training.
As for the textual feature categories relating topunctuation, it was remarkable to see thecustomized engine capitalize all of the words inthe title of a book that was cited in the held-out
test set, whereas the base engine did not. Only one
such example that the customized engine could
ha e lea ned f m a f nd in he aining
data. However, as with other categories that weretreated 100% consistently by the customized
engine but that also only occurred once wemust take the consistency with a grain of saltbecause we are essentially only looking at a single
data point that happened to be translated in a waythat was consistent with the human training data.
This becomes even more apparent when we seethat the other two punctuation categories, whichhave multiple occurrences in the held-out test set,
were much less consistent than in the humantraining data. In the case of quotation marks,
however, this could be due to the fact that
Portuguese orthography uses two different typesof quotation marks (and « »), and the dashes
can also be used for purposes other than quotation.
Despite little training data, the consistency withwhich the customized engine exhibited the textual
features listed in Table 1 is very similar to thatobserved in the human-derived training datawith the exception of contractions and how it
handled camarada professor(a). I i c ithat this fairly heavily represented term was nothandled very consistently at all while lexical itemswith very few instances in the training data suchas cacimbo and despedidae e lea ned e
consistently by the customized engine. Perhaps
multi-word terms experience more interference
than single-word terms.
4Conclusion

el cida e ce ain a ec f a an lale,it also sheds light on how this technology canmake considerable gains with very few examples
in the training data. While the customized engine
sometimes applied textual features learned fromits human training data inconsistently, this isnonetheless a useful tool for scholars studying
translator style. The research methodology used inthis study could be used as a starting point forf he
ali a i e anal i f a an lak,and its inclusion in research on translator style
could lend such studies more objectivity, or atleast more systematicity.
It would be especially interesting to use thismethodology multiple times for the sametranslator, training different customized engineswith data from different source authors. For
example, one engine could be customized withda a f m G eg
Raba aan la i n f J li
Ca
Rayuela [Hopscotch] while anotherengine could be trained with Raba a translation
f Gab iel Ga c a M
eCien años desoledad [One Hundred Years of Solitude]. One
could even use the Rayuela engine to translate atest set from Cien años de soledad to see whichaspects of translator style are specific to thetranslator and which are specific to when they aretranslating a certain author or a certain work.
This study also brings up some questions aboutcommercially available, customizable NMT
systems. There were multiple textual features thathe cmi ed engine lea ned f m hetranslator despite there only being one instance of
it in the training data. How consistently will acmi ed NMT engine ca e al fea e
ha i ha nl
een nce? C n e el , ince
there were textual features that were notconsistently applied by the customized engine
despite there being many instances in the training
da a, h m ch d e
ch a cmi ed engine
consistency depend on the consistency of the
training data? Or what other factors impinge on acmi ed NMT engine abili
a l textual
features present in its training data?

This study not only illustrates how commerciallyavailable, customizable NMT engines can be used

Page 17

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

References
La en Be acie . 2014. T ad c i n a ma i ed nee li ai e: Une de il e ,Traitement Automatique du Langage Naturel(TALN), Marseille, France.
Laurent Besacier and Lane Schwartz. 2015.
A ma ed T an la i n f a Li e a W k: A
Pil S d , Proceedings of the Fourth

Antonio Toral and Andy Way. 2015a. Machinea i ed an la i n f li e a e : A ca e d ,Translation Spaces, 4(2), 240 267.
An ni T al and And Wa . 2015b. T an la ing
literary text between related languages using
SMT , Proceedings of the Fourth Workshop

on Computational Linguistics for Literature,
123 132. https://www.aclweb.org/anthology/W150714

Workshop on Computational Linguistics for
Literature, 114 122.
Devin Gilbert. 2020. The People On My Street,
unpublished manuscript.
Joss Moorkens, Antonio Toral, Sheila Castilho, andAnd Wa . 2018. T an lae ce i n fliterary post-editing using statistical and neuralmachine an la i n , Translation Spaces, 7(2),240 262.
Ondjaki. 2009. Os da minha rua. Livro de Bolso,
BIS, Barcelona.

An ni T al and And Wa . 2018. Wha le el fquality can neural machine translation attain onli e a e ? , Translation Quality Assessment,
263 87.
Ma i n Win e and D
h Kenn . 2019. T a d
the study of computer-aided literary translation inrealld e ing , Translation in Transition 4
Book of Abstracts, 58 59.
https://eventum.upf.edu/_files/_event/_23119/_edi
torFiles/file/book-abs.pdf

Matthew Snover, Bonnie Dorr, Richard Schwartz,
Linnea Micci lla, and J hn Makh l. 2006. A
study of translation edit rate with targeted human
ann a i n , Proceedings of association formachine translation in the Americas.

Page 18

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Production Duration and Translator Characteristics

Haruka Ogawa
Kent State University
475 Janik Drive, Kent, Ohio 44242, USA
hogawa@kent.edu

1

Introduction

It is commonly acknowledged that more difficult
tasks take more time. In translation process
research, however, it has been found thattranslation production duration is not always
associated with text difficulty (Hvelplund, 2011).
Hence, it should be analyzed in consideration ofan laindi id al diffe ence (S n, 2015;
Akbari & Segers, 2017). This study aims to better
understand production duration in relation toan lacha ac e i ic . The e ea ch e i nasks whether sentence level production duration isa cia ed i h an la
backg nd, elfevaluation of their own translation, and translationstyles.
2

Data

The data was extracted from a study called
ENJA15 (Carl et al., 2016) in CRITT TPR-DB.1
The .sg tables for the from-scratch translation task
were used, where the textual and process data
were organized at the sentence level. In thisdataset, 39 participants translated two English
source texts (STs) into Japanese without any
external resources. Production duration was
divided by the number of ST words per sentence(henceforth nDur), which was then analyzed interms of the translator characteristics using mixed
effect models. All the data was handled on
RStudio (1.3.1073) and the lme4 package (Bateset al. 2015) was used for the analysis, and effects
package (Fox & Weisberg, 2019) for
visualization.
The translator characteristics examined here
were: years of training, years of experience, L1,
satisfaction rate on the task, initial orientation, andend revision. The first four features were extracted
from the metadata of ENJA15, which is asummary of the questionnaires that theparticipants filled out after the experiment. Their
years of training range from 0 to 7 and those ofexperience from 0 to 24. Out of 39 participants,
36 use Japanese and 3 English as their L1. The
satisfac i n n he a k i he a ici ane nehe e i n H
a i fied a e
with the translation you have produced through1

I e e en Cen e f Re ea ch and Inn a i n inTranslation and Translation Technology Translation
P ce Re ea ch Da aba e , a ailable a t

https://sites.google.com/site/centretranslationinnovation/tprdb/public-studies?authuser=0

Page 19

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

from- c a ch an la i n? The ch e ne f hefollowing: HD (highly dissatisfied), SD(somewhat dissatisfied), N (neutral), SS(somewhat satisfied), and HS (highly satisfied).
The last two features were added to the dataset atthe session level2 following the classification oftranslation styles in Dragsted and Carl (2013). For
the initial orientation (i.e., the phase of translation
before the participant starts typing theirtranslation), each session was annotated as a headstarter (who immediately started typing), a quickplanner (who read the first few ST sentences
before typing), a scanner (who quickly scanned
the ST), or a systemic planner (who read the entire
ST). For the end revision, sessions were classified
into three groups: Long (where the participant
spent more than 25% of the session duration forrevision after producing all the translation), Short(where the participant spent some time but shorter
than 25% of the session duration for end revision),and None (where the participant did not spend any
time for end revision).

3

Results

Figure 1 shows the result of the mixed effectmodel analysis. Although we can observe someinteresting tendencies in the figure, the years ofexperience and the satisfaction rate were the only
characteristics that produced statisticallysignificant results.
The top middle plot indicates that the longer
years of experience the participants had, the lesstime they spent on translation. This can be
explained by more automated processes as a resultof their experience (Jääskeläinen & TirkkonenCondit 1991). The bottom left plot reveals a trendthat the participants who were satisfied with theirown work (SS and HS) spent less time than those
who were dissatisfied (HD and SD). If nDur istaken as a measure of translation difficulty, thisresult supports some previous studies
demonstrating that pre-task subjective assessment

Figure 1. Translator characteristics and nDur

Figure 1. Averaged Production Duration and Translator Characteristics
2

Although we are interested in the characteristics ofthe translators, I decided to annotate the sessions, notthe participants, because some translators useddifferent approaches in two different sessions.

Page 20

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Figure 2. Interaction Effectsof their own performance correlates withtranslation difficulty (Sun & Shreve, 2014; Liu etal., 2019).
Further examination using mixed effectmodeling revealed that the years of training has
statistically significant interaction effects with theinitial orientation, end revision, and years ofexperience, as shown in Figure 2. Let us firstcompare the most experienced group (the orangeline) and the least experienced group (the blueline) in the bottom left plot, which visualizes theinteraction effect between the years of training
and experience. In the most experienced group,
the years of training positively correlates with
nDur, whereas the correlation is negative in theleast experienced group. Yildiz (2020) suggeststhat more advanced translation students are moresensitive to potential problems and therefore
spend more time than students at lower levels.
Although the top left plot in Figure 1 seems tocorroborate his finding as a general tendency, theresult was not statistically significant. Moreover,the plot in Figure 2 suggests that the students withno or little experience has the opposite tendency
in the present study.

The top left plot, which shows the interaction
effect between the years of training and the initialorientation, suggests that i) the head-starters spent
about the same amount of time for translation
regardless of the years of training, ii) the moretraining the participants had, the more time theyspent on translation in the group of quickplanners, but iii) this tendency overturned in thegroup of scanners and systemic-planners. It isinteresting to note that the most trained groupspent a considerably different amount of timedepending on the initial orientation, while there is
not much difference in the group with zero
training.
The top right plot suggests that the moretraining the participants had, the less time theyspent on the task when they did not have the end
revision phase. This tendency again gets reversed
when they spent some time for end revision,
although the differences among the five groups
are not so evident as when they had no end
revision time. It is to be investigated why two of
the least trained groups spent more time when
they did not have the end revision phase.

Page 21

Book of Abstracts: Translation in Transition (TT5)4October 15 - 17, 2020

Concluding Remarks

This study has shed light on some of therelationships between production duration and thecharacteristics of individual translators. Although
there were only two characteristics that were
statistically significant, one may be able to findbetter results if more granular units of analysis(i.e., word or phrase level) are adopted. At thesame time, the visualization of interaction effects
have revealed that different participant groups
have contradicting tendencies in terms of
production duration, which demands careful
examination of translation process data for future
research.

References
Akbari, A., & Segers, W. 2017. Translation Difficulty:
How to Measure and What to Measure. Lebende
Sprachen, 62(1), 3 29.
Bates, D., Mächler, M., Bolker, B., and Walker, S.
2015. Fitting linear mixed-effects models using
lme4. Journal of Statistical Software, 67(1):1 48.
Carl, M., Aizawa, A., & Yamada, M. 2016. ENJA15: af ee cf Engli h
Ja ane e T an la i n
Process Data. 22nd Annual Meeting of the
Association for Natural Language Processing.
Dragsted, B., & Carl, M. 2013. Towards aClassification of Translation Styles Based on Eyetracking and Key-logging Data. Journal of Writing
Research, 5(1), 133 157.

Fox, J., & Weisberg, S. 2019. An R Companion to
Applied Regression, 3rd edition. Sage, Thousand
Oaks
CA. https://socialsciences.mcmaster.ca/jfox/Books/
Companion/index.html.
Hvelplund, K. T. 2011. Allocation of Cognitive
Resources in Translation. An Eye-tracking and Keylogging Study. PhD thesis. Copenhagen:Copenhagen Business School.
Jääskeläinen, R., Tirkkonen-Condit, S. 1991.
Automated processes in professional vs. nonprofessional translation: a think-aloud protocol
study. In S. Tirkkonen-Condit (ed.). Empirical
Research in Translation and Intercultural Studies.
Tübingen: Gunter Narr. 89-109.
Liu, Y., Zheng, B., & Zhou, H. 2019. Measuring thedifficulty of text translation: The combination of
text-focused and translator-oriented approaches.
Target: International Journal of Translation
Studies, 31(1), 125 149.
Sun, S. 2015. Measuring translation difficulty:
Theoretical and methodological considerations.
Across Languages and Cultures, 16(1), 29 54.
Sun, S., & Shreve, G. M. 2014. Measuring translationdifficulty: An empirical study. Target International
Journal of Translation Studies, 26(1), 98 127.
Yildiz, M. 2020. How Do Transla i n S den
Cognitive Efforts Vary? - An Answer inConsideration of Pauses. Journal of Education andPractice, 11(2), 48 55.

Page 22

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Testing of Perceived Sound Quality on Interprefy App

Jacob T. Høgh
Co-founder Interprefy Ltd.
Bellerivestrasse 11
CH-8004 ZURICH
Jacob.hoegh@interprefy.com

1

Introduction

The 2015 global market for interpreting isestimated to be worth US $8 billion; the Swiss
market is valued at CHF 180 million, plus CHF
115 million for the rental equipment, making a
total of CHF 300 million.
Interprefy Ltd. has developed a cloud-based
platform for Remote Simultaneous Interpreting(RSI) at online meetings, conferences and multilingual events. Current audio/visual technology is
f m he 60 andda ed in man c nfe encehalls and the equipment used is both expensive
and cumbersome to transport.

2

Speaker Microphone testing

Goal was testing of perceived sound quality on
the Interprefy App using the method Mean
Opinion Score with a jury of as many
people/Interpreters possible on earphones,
headsets, or USB microphones via smartphone or
laptop.
2.1

Proposal

The purpose of this proposal is to tell how wetested the sound quality of Interprefy RSI
platform. It is our ambition to be known as a fullrange (uncompressed) HD audio-quality SaaSprovider. We want 90% of all interpreters to say,that they would recommend and use Interprefy
App again. This allows clients not only to source
and use specialized interpreters in many, also

marginal languages from anywhere in the world,
but it also enables them to eliminate the travel andaccommodation expenses of the interpreters.
2.2

LayoutIn close collaboration with more than 50
interpreters or so called wild-testers and ourstrategic partner in Chicago, InterpreNet, Ltd.
We e b il a la f m f
em e im l ane
interpretation. We identified the trend setting andmost influential key actors in the industry andoffered them to use Interprefy App in theirteaching master classes for linguistic students
free-of-charge in exchange for valuable input. A
Swiss patent application was submitted in June
2015. In the same month, Interprefy Ltd. won theprestigious Language Technology InnovateAward 2015 in Brussels.
Over time, a variety of additional services andback- products such as text-into-speech, CAI(computer aided interpreting), and a moderator
UI, where the moderator can reduce the resolutionof the video from the conference/floor while
always maintaining full-range, uncompressed HD
audio built-in the coding webRTC. Five top
short-term actions:●
We are co-operating with professor Franz
Pöchhacker and University of Vienna, andProfessor Sabine Braun at University of Surrey(UK), getting help from master classes with

Page 23

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

feedback about the Interprefy platform itself andtested the sound-quality●
We tested the perceived sound quality on
Interprefy App with a jury of as many
people/Interpreters possible using earphones,
headsets, or USB microphones via smartphone or
laptop (with an Interpreter-token).
●
We tested three scenarios: 1. P/A-system(Hybrid-In), 2. Remote speaker (microphone
Infront of the mouth), 3. Court room / RDSI
(Remote Dialogue Simul Interpretation)-setting(more than one speaker sitting 1 m frommicrophoneColumn width: 7.7 cm●

Reference speech level was set to 60dB●
We used the method of Mean Opinion
Score (rating the sound quality or the proportionof speech transmission from 1-5.) according to thecategories:
1.
Ambiance/Back-ground noise
2.
Reverberation/(Echo)
3.
Over-all Impression/Timbre
4.
Speech transmission Index (%words
intelligible.)
Footnote:

Higher bitrate gives better sound, but packets arebigger, so packet loss and audio drops might
increase in poor internet conditions. Bitratesetting has been available for Opus codec longtime ago, but finally implemented for open source
webRTC in January 2017.
2.3

Conclusion

Connection between frequency range and bitrate
is not direct but chance of getting 20kHz range
on 64kbps connection is much higher than on 32.
N ice ha in e e e
ISO e i e 20kH
being transmitted to interpreter. Until now thesesound quality tests have not been documented,
and might never be released to public, but I would
like to share the results with you after we've
correlated the jury's responses carefully with ourInternal test results.

2.4

Final statement

I would like to invite you to a demo of Interprefy
platform and to test the full-range anduncompressed audio quality. The Interprefy RSI
solution is easily scalable and in cooperation withinterpreters as ambassadors, we can reduce themarketing expenditures. The main benefits arethese:
i.
old-fashioned, heavy hardware is nolonger required; andii. interpreters in numerous languages can besourced world-wide, with their travel andaccommodation expenses eliminated.
iii. Interprefy RSI offers low latency and lip-sync.
We only broadcast the audio/interpretation to theend-users without video of the interpreters, andthe audio-files are 1/30 of a video file. Numerous
use-cases with clients have validated thetechnology. The present COVID19-situation has
made Zoom.us and Interprefy evolve into one

stop-boutiques for online webmeetings andremote interpreting. In addition, interpreters arenelc ming
k em el n In e ef
game-changing technology and not be limited totheir metropole area during and after the lockdown.
3

Acknowledgement

Thank you for taking your time to read this
proposal and for inviting me to join you
virtually during a remote demonstration of theInterpreter UI at the Translation in Transition(TT5)-conference in October.

References
Please confer the following articles:

https://wiki.xiph.org/Opus_Recommended_Set
tings
http://wiki.hydrogenaud.io/index.php?title=Opus

Page 24

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Eye-Voice Span in Sight Interpreting:
Evidence from Both Process and Product

Jia Feng

Shirong Chen

Renmin University of China

Renmin University of China

fengaijia@hotmail.com

1966567062@qq.com

Michael Carl

Jinjin Chen

Kent State University

University of Macau

mcarl6@kent.edu

chenjj0601@163.com

Yueqi Zhu
Renmin University of China
419352489@qq.com

1

Introduction

Sight interpreting, as one of the basic modes ofin e e ing, i a h b id f m, in ha a i ensource text is turned into an oral-or signeda ge e in an he lang age in eal ime(Cenkova, 2015: 374). The eye-voice span
(EVS) in sight interpreting measures the timelag, or the temporal delay between reading
source text and producing target text. Like the
other
two
time
lag
metrics
ininterpreting/translation, namely ear-voice span insimultaneous interpreting and eye-key span in
written translation, eye-voice span in sightinterpreting is seriously under-researched,although time lag metrics are long regarded as
ha ing heen ialbec me eal ablemeasures in translation/interpreting processien ed e ea ch (Tima, D ag ed, &Hansen, 2011: 121) as they could provide

insights into the temporal characteristic of
simultaneity in interpreting, the temporaldevelopment of the target speech in relation to
the source speech, the speed of translation, as
well as into the cognitive load and cognitive
processing involved in interpreting/translation
process. This dearth of research so far can belargely attributed to the methodologicalchallenges involved, for example, recording eye
movements during reading in sight interpreting,as well as synchronizing eye movements datawith spoken production data, which was astrenuous and heavily time-consuming task thatwas previously done manually.
Thanks to the innovative breakthrough made byCarl and Yamada (2017), spoken production
process during sight interpreting was recorded
by Audacity and then later transcribed by
Automatic Speech Recognition (ASR), andPage 25

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

further synchronized with the eye movements
data. On top of that, product data are integrated
with the process data as the ST and TT arealigned at word level and the process data related
to ST and TT at word level are related andmapped to the product data. This innovation
enables a more accurate and finer-grained
analysis of eye-voice-span.
2

Research Design

The experiments were carried out in an eyetracking lab with sound-proof walls and doors,
as well as stable illumination. Eye calibration
was done every time before the interpreter
started sight interpreting. Immediately after theinterpreting task, participants filled the modified
NASA Task Load Index (NASA- TLX) (Sun2012) in order to measure their translationdifficulty in sight interpreting each text. A
practice exercise was done in the beginning andinterpreters sight interpret a short text to getfamiliarized with the procedure. They took a fiveminute break after each text.
2.1

Participants

We recruited 9 professional interpreters (7females and 2 males) whose L1 is Chinese andL2 English. All of them are graduates fromworld-renowned interpreting institute after two
years of formal training in interpreting during
their graduate study. The average length ofworking as a professional interpreter was 6.4
years.
2.2

interpreter during sight interpreting.
The
spoken production of target text (TT) wasrecorded by an audio recorder (namely
Audacity) simultaneously. With the help ofautomatic speech recognition technique, thespoken translation output of TT was later
automatically transcribed into texts. It was thencorrected and synchronized using ELAN, anannotation tool that allows annotations for audio
data. In this way, the audio output in sightinterpreting was successfully transcribed into
written product data with timestamps, which
was then merged with Translog II data anduploaded to
CRITT TPRDB platform(https://sites.google.com/site/centretranslationi
nnovation/tpr-db). After that, based on the
BOLT Chinese-English Word Alignment
Guidelines(https://www.ldc.upenn.edu/sites/www.ldc.upe
nn.edu/files/bolt-chinese-alignment-guidelinesv2.pdf), the English source texts and thetranscribed Chinese target texts was manuallyaligned, providing a meaning linkage betweenaligned words.

We tried to explore the following researchquestions:1)

How long is the average EVS in
English-Chinese sight interpreting?

2)

Does ST text difficulty and sightinterpreting task difficulty influence
EVS?

3)

Is EVS influenced by the source text
features such as part-of-speech?

4)

Is EVS related to metrics from process
data, e.g. reading measures, keylogging measures?

Materials

Each of the participants sight interpreted 6 texts
from English into Chinese. The average lengthof the texts are about 130 words (with the range
from 100 words to 150 words). Four texts arenews articles (Texts 1-4) and 2 texts are excerpts
from articles of an encyclopedia (Texts 5-6).
2.3

Research Questions

3

Procedures

Preliminary Findings

4

What follows are our preliminary findings:

We used Translog II (Carl 2012) to display thesource text (ST) and an eye-tracker (Tobii
TX300) to record the eye movements of the

Page 26

1)

In this study, the word-based EVS ofprofessional interpreters when theysight interpreted English source texts

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

References

to Chinese was about 2.25s (M=2.25,
Mean=4.86, SD=10.79).
2)

3)

4)

Carl, M. 2012. Translog-II: A Program for

Recording User Activity Data for Empirical
Reading and Writing Research. Presented at

The EVS performance of professional
interpreters were quite consistentacross all six source texts. In otherwords, their EVS were not influenced
significantly by ST text difficulty andthe sight interpreting task difficulty.
This consistency may be considered as
an indicator of their expertise ininterpreting.

The Eighth International Conference on
Language Resources and Evaluation. Istanbul.
Carl, M & Yamada, M. 2017. Tutorial on

Multimodal Integration of Written andSpoken Translation Production. Presented atthe 4th International Conference on Cognitive
Research on Translation and Interpreting.
Beijing.

Comparatively speaking, part-ofspeeches like verbs, numbers, WHd ( ch aha , and h,adjectives, and determiners led tolonger EVS, while that of nouns,
adverbs, conjunctions, prepositions,and haa shorter.

Cenkova, I. 2015. Sight Interpreting/Translation. In

The Routledge Encyclopedia of Interpreting
Studies (pp. 374-375). London and New York:
Routledge.
Sun, S. 2012. Measuring Difficulty in English-

Chinese Translation: Towards a General
Model of Translation Difficulty [D]. Ph.D.

P fe i nal
in e e e
EVSsignificantly correlated with some eyetracking measurements, such as
fixation durations, total number offixations. It is suggested that the EVS
performance was closely related tointer e eeading behaviors.

Dissertation. Kent: Kent State University.
Team, A. D. 2008. Audacity (version 1.2. 6)[computer software]. Available: audacity.
sourceforge. net/download.
Timarová, S., Dragsted, B., & Hansen, I. G. 2011.
Time lag in translation and interpreting: A
methodological exploration. Methods andstrategies in process research: Integrative
approaches in translation studies, 121-146.

Page 27

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Addressing the Rare Word Problem in Punjabi
to English Neural Machine Translation

Kamal Deep Garg1

Ajit Kumar2

Vishal Goyal3

Chitkara University Institute ofEngineering and Technology

Department ofComputer Science

Department ofComputer Science

Chitkara University, Punjab,
India

Multani Mal Modi
College, Punjab, India

Punjabi University,
Punjab, India

1kamaldeep.garg@chitkara.edu.in, 2ajit8671@gmail.com,
3vishal.pup@gmail.com

Abstract
Neural Machine Translation (NMT) is anapproach using these days to develop the machinetranslation system. This approach shows the
improvement over the results that are comparable
to rule-based translation and Statistical MachineTranslation. NMT system uses fixed vocabulary
due to the computation complexity, but machinetranslation is an open vocabulary problem. The
previous work handles the out-of-vocabulary
words by using the bilingual dictionary at the
postprocessing step. In this paper, a more
effective and simple approach is being used tohandle out-of-vocabulary words in the NMT
system by encoding the unknown words. We
propose a Punjabi to English NMT using BytePair-Encoding (BPE) compression algorithm
along with word-embedding that overcomes theout-of-vocabulary problem for the languages that
do not have much translation available online. The
proposed system is evaluated by using the BLEU
score and Word Error Rate(WER).
Keywords:
NMT, OOV, BPE, BLEU, LSTM

1Introduction

India is a big country having several languages
spoken in different regions. The language changes
by the regions in India. Machine Translation (MT)system can be used to translate one regional
language to another regional language as well as
regional language to English and vice-versa. MT
is a very challenging task for Indian languages.
Morphological richness, word order difference,and size of parallel corpora of the Indian
languages make MT a complicated task. As in ourcase of Punjabi to English NMT system, Punjabi
has Subject-Verb-Object (SVO), whereas English
has Subject-Verb-Object (SOV). Moreover,Punjabi is an agglutinative language, whereas
English is an inflected language. The huge parallel
corpus is not available for the Punjabi-English
pair.
In this paper, the baseline NMT system isdeveloped by using the Punjabi-English parallel
corpus. To improve the baseline model, two morevariants: one by using BPE,and second by using aword embedding and BPE combined, isdeveloped. To evaluate all models, BLEU and
WER are used.

Page 28

Book of Abstracts: Translation in Transition (TT5)2October 15 - 17, 2020

Neural Machine Translation

The neural machine translation architecture isbased on the neural networks and conditional
probability of translated sentences from the source
sentence to target sentences(Revanuru, Turlapaty,
and Rao 2017)(Chaudhary and Patel 2018). The
NMT is based on the sequence to sequencearchitecture. This architecture is used to map thesource language text to target language
text(Sutskever, Vinyals, and Le 2014). Thisconsists of the two parts, encoder, and decoder.
The encoder takes the source sentence andconverts it to the encoded vector. The decoder
generates one word at a time based on theencoding vector and previously generated words.
The encoder converts the source sentence
into the vector of the fixed
1, 2, 3, … … … .
dimensions. The decoder produces the output
word
by
word
using
the conditional
probability(Chaudhary and Patel 2018)(Garg and
Agarwal 2018).
There are multiple ways to handle the out-ofvocabulary or unknown words in the MT. These
are the use of bilingual dictionary, character
embedding, and Subword. There are different
Subword algorithms: Byte Pair Encoding(BPE)(Sennrich, Haddow, and Birch 2016),
WordPiece(Schuster and Nakajima 2012), andSentencePiece(Kudo and Richardson 2018). Fromthese three, we have used the BPE to build a
Subword dictionary. By using the BPE algorithm,
we learned the independent encodings on oursource and target training data with 30000 words
and then applied it on train, validation, and test set
for both source and target data.
Word embedding is a feature learning technique
where words having the same meaning have thesame representation in the continuous vector
space(Stahlberg 2019)(Liu, Lu, and Neubig2018). The most commonly used methods to learn
word embedding are Word2vec, Glove, andfastText. fastText is a neural network-based
library to learn the word representations andsentence classification created by Facebook's AI
Research (FAIR) lab(Joulin et al. 2016)(Facbookn.d.). From fastText, we downloaded pre-trained
word embedding for Punjabi and English. After
downloading, we convert the Punjabi and English
vocabulary into a 300-dimensional vector and areused in the training process.

3

Experiments

The Punjabi-English dataset has been created bycollecting from various sources. To conduct theexperiments in NMT, first task is to divide thedataset into three sets: training set, validation set,
and test set. A python script has been written to
randomly divide the dataset into three sets. Table
1 shows the division of the corpus into a different
set. After this, various pre-processing steps havebeen performed on the training and validation set.
This includes the tokenization of the Punjabi andEnglish text, a true casing of English sentence,
removal of sentences having more than 40 tokens,and removal of contractions.

Corpus Set

Sentences (Parallel)

Training Set

155678

Validation Set

1212

Testing Set

1212
Table 1: Division of Dataset into three sets

3.1

Training Details

OpenNMT toolkit(Klein et al. 2017) has beenused to train the NMT model. Three models havebeen developed by using this toolkit. For all threemodels, the batch size of 64, and 20 epochs, fortraining is fixed. The Baseline model consists ofthe four layers of Bi-directional LSTM encoderand four layers of LSTM decoder of 500

Page 29

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

dimensions with a vocabulary size of 49345 and
51003 words for source and target. For the second
model, BPE is used with the baseline model. For
the third model, Word Embedding with BPE isused to train the baseline model. Each model took
10 to 12 hours for training by using NVIDIAGTX1050Ti 4GB GPU.

3.3

Prediction Analysis

From Figure 1, it is clear that by integrating the
BPE and WE to NMT model, BLEU score increase
BLEU AND WER SCORE OF ALLPROPOSED MODELS
60

3.2

Results

48.9

50

All three proposed models are evaluated by using
the BLEU score(Papineni et al. 2002) and the
WER score. The BLEU score and WER scoreobtained from the three models are shown in
Table 2 and Table 3.

40

31.56

32.31

47.8

46.1

33.04

30
20
10

Model

BLEU Score

Baseline Model

31.56

Baseline Model+BPE

32.31

Baseline

33.04

0
BLEU Score
Baseline Model

WER SCORE
Baseline Model+BPE

Baseline Model+WE+ BPE

Model+WE+ BPE

Figure 1: BLEU and WER score of all proposed
ModelsTable 2: BLEU Score of all models
Model

WER Score

Baseline Model

48.90%

Baseline Model+BPE

47.80%

Baseline Model+WE+

46.10%and WER decrease. By taking the one predicationfrom all three models, it is shown in Table 4 thatਆਪ( pa) is correctly translated to "aap" by theBPEthird model, whereas the first two models translate
it to "one" that is an incorrect translation.

Table 3: WER Score of all models

Input Punjabi Text: ਇਹ ਕ ਗਰਸ ਪਾਰਟ ਦਾ ਵ ਡਾ ਝਟਕਾ ਹ , ਇਕ ਆਪ ਨ ਤਾ ਨ ਕਕਹਾ ।Iha kgarasa p ra da

jha ak hai, ika pa n t n kih .

Baseline Model: "it is a big setback of congress party ," said one leader .
Baseline+BPE: "it is a big setback of congress party , "; said one .
Baseline+WE+ BPE: "it is a big blow of the congress party ," one aap leader said .
Table 4: Prediction given by all three proposed model

4

Conclusion

In this research, we developed the NMT system
for less-resource language i.e. Punjabi to English.
We found that by including the word embedding
and byte pair encoding to baseline NMT model,

performances increase of the overall system. We
got a BLEU score of 33.04 by adding wordembedding and byte pair encoding to baseline
NMT model. In Future, we would like to create

Page 30

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

large corpus to increase the effectiness of thesystem.

Sennrich, Rico, Barry Haddow, and Alexandra Birch.
2016. Ne al Machine T an la i n f Ra e
W d
ih S bd Uni . 54th Annual
Meeting of the Association for Computational
Linguistics, ACL 2016 - Long Papers 3: 1715
25.

References
Chaudhary, Janhavi R, and Ankit C Patel. 2018.
Machine T an la i n U ing Dee Lea ning : A
S e . International Journal of Scientific
Research in Science, Engineering andTechnology 4(2): 145 50.
Facbook.
Fa Te .
https://en.wikipedia.org/wiki/FastText (June 14,2020).
Ga g, Ank h, and Ma ank Aga al. 2018. Machine
T an la i n:A
Li e a e
Re ie .
http://arxiv.org/abs/1901.01122.

S ahlbe g, Feli . 2019. Ne al Machine T an la i n:A
Re ie .
:
1 88.
http://arxiv.org/abs/1912.02047.
Sutskever, Ilya, Oriol Vinyals, and Quoc V. Le. 2014.
Se ence
Se ence Lea ning i h Ne al
Ne
k . Advances in Neural Information
Processing Systems 4(January): 3104 12.
http://arxiv.org/abs/1409.3215.

Joulin, Armand, Edouard Grave, Piotr Bojanowski,
and T ma Mik l . 2016. Bag f T ick f
Efficien Te Cla ifica i n. 15th Conferenceof the European Chapter of the Association for
Computational Linguistics, EACL 2017 Proceedings of Conference 2: 427 31.
Klein, G illa me e al. 2017. O enNMT: O enSource Toolkit for Neural Machine T an la i n.
ACL 2017 - 55th Annual Meeting of the
Association for Computational Linguistics,
Proceedings of System Demonstrations: 67 72.
http://arxiv.org/abs/1709.03815.
Kudo,

Taku, and John Richardson. 2018.
Sen encePiece: A Sim le and Lang age
Independent
Subword
Tokenizer
andDe keni e f
Ne al TeP ce ing.
EMNLP 2018 - Conference on Empirical
Methods in Natural Language Processing:
System Demonstrations, Proceedings: 66 71.

Liu, Frederick, Han Lu, and Graham Neubig. 2018.
Handling H m g a h in Ne al Machine
T an la i n. : 1336 45.
Papineni, Kishore, Salim Roukos, Todd Ward, andWei-jing Zh . 2002. B LEU : A Me h d f
A ma ic E al a i n f Machine T an la i n.
(July): 311 18.
Revanuru, Karthik, Kaushik Turlapaty, and Shrisha
Ra . 2017. Ne al Machine T an la i n f
Indian Lang age .
ACM International
Conference Proceeding Series: 11 20.
Schuster, Mike, and Kaisuke Nakajima. 2012.
Ja ane e and K ean V ice Sea ch. ICASSP,IEEE International Conference on Acoustics,
Speech and Signal Processing - Proceedings 1:
5149 52.

Page 31

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Augmenting Dependency Tags in Interactive Neural MachineTranslation

Kamal Kumar Gupta1, Rejwanul Haque2, Asif Ekbal1, Pushpak Bhattacharyya1
1

Department of Computer Science and Engineering,
Indian Institute of Technology Patna, Patna, India

2

ADAPT Centre, School of Computing, Dublin City University, Dublin, Ireland
1{kamal.pcs17, asif, pb}@iitp.ac.in

2

firstname.lastname@adaptcentre.ie

Abstract
In recent times, Neural Machine Translation(NMT) has attracted attention to the machinetranslation researchers and developers due to its
end-to-end sequence learning flexibility. It has
been showing great success in machinetranslation for many language pairs and domains.
Despite the impressive performance by NMT, it
is not completely error free, and its applicability
is largely limited by the amount of data available
for training the models. In a real time translation
environment, where different users have different
aspects of translation and vocabulary selection, it
becomes difficult for a trained model to generate
an output which is acceptable by everyone. Inthat case, output generated by the model iscorrected by humans in the post-editing phase.
Interactive neural machine translation (INMT)
provides a human-machine collaboration
platform where an interactive translation
framework is backed with a neural machinetranslation model. INMT provides users a highly
productive environment to insert their choice ofwords in the hypothesis generated by model.

Model regenerates a new hypothesis which
ee e
he
e
ch ice fd andcontextually depends on the inserted token.
Multiple attempts of token replacements may berequired by a user to get the desired output.
In this work, we devise a mechanism tointroduce syntactic information in the form ofdependency tags (to the decoder side) to increase
the prediction accuracy of the model, and to
reduce the efforts, i.e. the number of attempts oftoken replacement by the user to obtain thedesired output. We augment the dependency tags
as external syntax information at the target side
sentences in training data. We try to model thepredicted output token on the previously
generated token as well as the dependency tags.
We perform experiments for English to Hindi, a
low-resource language pair and English toGerman, a high resource language pair. We use
the state-of-the-art Transformer (Vaswani et al.
2017) model as our baseline. We use a reference
simulated environment where the reference
sentence is considered as a user choice sequence.

Page 32

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Each time the newly generated hypothesis ismatched with the reference from left to right intoken to token manner. Each mismatch isconsidered as an error and corresponding token
value from the reference is taken as the user's
choice word. To measure the performance of ourtechnique in INMT environment, we use two
evaluation technique i.e. WPA (Knowles et al.
2016) and WSR (Peris et al. 2017). WPA stands
for word prediction accuracy which is the ratio ofcorrectly predicted tokens to total sequencelength. WSR stands for word stroke ratio whichis the ratio of the total tokens replaced to the total
sequence length. Our objective is to increase theWPA (prediction accuracy) and reduce the WSR(replacement effort). By using dependency tags
as syntax information at the target side, we
achieve an absolute improvement of 2.24% and3.48% in WPA and reduction of 4.75% and3.03% in WSR over baseline for English to Hindi
and English to German translation respectively.

References
Vaswani A, Shazeer N, Parmar N, Uszkoreit J,
J ne L, G me AN, Kai e , P l khin I.
Attention is all you need. InAdvances in neural
information processing systems 2017 (pp.
5998-6008).
Peris, Á., Domingo, M. and Casacuberta, F.,
2017. Interactive neural machine translation.
Computer Speech & Language, 45, pp.201220.
Knowles, R. and Koehn, P., 2016, October.
Neural interactive translation prediction. In
Proceedings of the Association for MachineTranslation in the Americas (pp. 107-120).

Page 33

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Supporting Translators Through Keyword Mining

Dr. Kara Warburton
Masters Program in Translation and Interpreting,
University of Illinois at Urbana-Champaign
karacw@illinois.edu

Abstract
Termbases need to include the terms thattranslators actually need in order toeffectively serve this user group. The
methodology described herein involves
using concordancing software to determine
a term's relevancy to the organization's
corpus. Salient unigrams are identified andconfirmed to be statistically interesting.
They are subsequently used as search pivots
for identifying productive bigrams andtrigrams. If adopted the method can increase
the effectiveness of termbases for translatorsand other producers of corporate content.

1

Introduction and backgroundManaging terminology to support translators
implies that the terms that translators actuallyneed should be the primary focus and therefore
these terms should be predominant intermbases that are used by translators.
This raises some key questions. What terms totranslators actually need? Do termbases thatserve translators include enough of thoseterms? And if not, how can this gap be
addressed?
This paper summarizes portions of the PhD
research completed by the author at the City
University of Hong Kong (Warburton, 2014)
which attempted to address those questions
and propose solutions.

2

A translation-oriented view oftermhood

The types of terms that translators need contributeto a definition of termhood -- or what constitutes
a term -- that is oriented towards the translationuse-case. According to traditional terminology
theory, i.e. the Wusterian General Theory ofTerminology (GTT), a term is the designation ofa concept belonging to a language for special
purposes (LSP), and further, these concepts areuniversal (language-independent) and can beclassified systematically. In recent years, thispurely semantic referential view has beenchallenged by scholars and researchers who,
having access to large-scale corpora andcomputing technologies, observe terms in their
linguistic and communicative environment.
Further, what constitutes an LSP, the core GTT
criterion, is subject to ongoing debate(Condamines 1995: 227), with some scholars
claiming that business genres constitute a form of
LSP (Rey 1995: 144). Previous research has also
shown that so-called semi-technical vocabulary isoften more challenging for translators than strictly
technical terms (Warburton 2014: 73).
It has also been declared that the end-use and endusers of a terminological resource have a bearing
on termhood (L'Homme 2005: 1130).

Page 34

Book of Abstracts: Translation in Transition (TT5)

Considering the larger body of scholarly
literature and research, the notion of termhood
is shifting towards recognition of syntagmatic,
cognitive, communicative, lexico-semantic,
and textual features, and we maintain that therelative importance of these features forestablishingtermhood in a given production environment
ultimately depends on the target application of
the terminological resource (see L'Homme
2020: 59
and Condamines 2010).
From a purely pragmatic standpoint, based on
the author's experience in commercial
enterprises, when considering translators as theprimary end user termhood is shaped by theneed for:1. designations of concepts from LSPs (theWusterian view) - the meaning of such
units may not be known to the translator;
2. multi-word units (MWU) that designate aconcept (specialized or not) and which, due
to variations in TL (target language)
syntax or morphology, could have multiple
TL equivalents (which should be avoided
if consistency is a goal);
3. any designation (specialized or not) that
occurs frequently, and is also subject topotential inconsistency in the TL;
4. any designation (specialized or not) that is
highly visible, such as on packaging or
user interfaces.
A pragmatic definition of termhood, one that
addresses the production-oriented processes oflarge organizations, thus needs to extend
beyond the purely semantic criteria of the GTT
to include aspects such as frequency ofoccurrence, visibility, and potential forinconsistency. This view is shared by other
scholars (Schreve 2001: 785; Martin 2011;
Van Campenhoudt 2006: 4, to name a few).

October 15 - 17, 2020

3

The failures of term extraction

A recognized best practice in translation,
particularly for large projects distributed among
multiple translators, is to identify and pretranslate
the key terms in a given project. Doing so ensuresa greater level of terminological accuracy andconsistency. Given the large size of these projects,
manual term identification is impossible.
Automatic term extraction (ATE) has attracted
much interest among researchers in Natural
Language Processing, and a number of effective
solutions have been developed at various research
institutes. Few of these innovative solutions,however, have been productized. The few term
extraction tools that are available on the market
underperform in comparison, and mostterminologists find them totally ineffective.
Furthermore, the algorithms used to identify andextract terms do not align with the specifictermhood criteria that serve the organization's
needs for term selection. At most, they are basedon (a) frequency of occurrence, (b) a few syntactic
patterns for multi-word terms, and (c) in some of
the more advanced ATE tools, a level of semantic
domain specificity by comparison with a generalreference corpus. Term extraction -- a key enablerof high quality translation -- is therefore rarely
carried out.

4

Statement of the problem and aim of
the research

Given the failures of ATE and the somewhatloosely defined termhood criteria for translationoriented termbases, one has to consider how wellthese termbases serve their users with respect tothe terms that they contain. Our first assumption
is that terms in corporate termbases should align,
relatively speaking, with the terms in thecompany's corpus, since those are the terms thatare contained in content that is translated.

Page 35

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Another assumption is that the terms in thetermbase should reflect translation-oriented
termhood criteria. Our study sought to (a)establish that institutional (specifically,
corporate) termbases are under-optimized
with respect to providing translators with theterms that they need, (b) explain why they are
under-optimized, and (c) determine ways toaddress this weakness.

One of the ways to address the un-documented
terms is to use ATE on the company's corpus.
However, as stated earlier, commerciallyavailable ATE tools are ineffective. The "noise"produced by term extraction tools refers to theterm candidates that are uninteresting for the
purposes of providing translation support andtherefore need to be removed, or "cleaned" fromthe raw output. These tools produce so much noise(60% or more of the output, according to trials that
we have conducted), that the cleaning effort
renders them impractical.

5

Methodology

Four case studies were completed using data
(termbases and corpora) from commercial
companies. Using empirical statisticalmethods, the termbases were compared to thecorresponding corpora, considering criteriasuch as normalized frequency (of occurrence
in both the termbase and the corpus), part ofspeechprevalence, term length (n-grams), and more.
We used a range of text processing tools andone
terminology management system,
TermWeb (from Interverbum Technologies) tomanipulate and analyze the data.
The comparisons of the termbases and thecorpora enabled us to confirm that with respect
to term correspondence, there is a large gap
between the corpus and the termbase. We
identified two types of gap, which became thefocus for our research:(1) un-documented terms, and (2) underoptimized terms. Un-documented terms areterms that do not exist in the termbase but areprevalent in the corpus. This pool of terms istherefore a promising source of terms that may
be needed by translators. Under-optimized
terms are terms in the termbase that do notoccur at all in the corpus, or are extremely rare,
and therefore, their value in the termbase isquestionable. While both these situations areproblematic and reduce the effectiveness of thetermbase, the former is of greater concern,
since it has a direct impact on the efficiency
and quality of overall content production in
multiple languages.

Our approach therefore turned to concordancing
software. Previous research has shown thatmultiword terms (MWT) are abundant interminological resources (Meyer and Mackintosh
1996: 259; Nagao 1994: 406; Maynard andAnaniadou 2001: 265; Daille et al 1996: 207, tomention a few). We further demonstrated thatbigrams and trigrams are of particular interest andvalue for terminology work (2014: 138). We
therefore considered that bigrams and trigramsmight offer potential for closing thetermbase/corpus gap. With this in mind, wedeveloped and tested a method that involves
identifying salient unigrams, and then using thoseunigrams as search keywords to identify multiword terms of which the unigrams are acomponent. We chose Wordsmith Tools since itallows batch concordancing with an input list of
search keywords.
To identify salient unigrams we used the Keyword
function of WordSmith tools. This function
creates a word list from the corpus, and comparesit to a word list created from a general reference
corpus. Using a normalized keyness formula, it
identifies unigrams that may be salient(statistically relevant). From the candidates
offered, we determined through statisticalanalysis which of the candidates were underrepresented in the termbase. We also identified
cases where there were few MWT in the termbasethat contained the keyword.

Page 36

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

These strategies enabled us to focus ourconcordance searches on the keywords that
had the most potential. We then used those
keywords in a batch concordance to identify
bigrams and trigrams formed with those
keywords.

For example, the simple removal of a non- essential
premodifier can have dramatic effects on theleveragability of the terminological unit. The term
"automatic incremental backup," fully documented
with multiple translations in one company's
termbase, did not occur a single time in its corpus.
But the term "incremental backup" occurs over 500
times. Likewise, "remote notification server,"found in the termbase, does not exist in the corpus,
but "notification server" occurs over 1,000 times.
These latter two more productive terms were
missing from the termbase.
Another example is the term "data." We identified
six un-documented bigrams containing the word
data (e.g. "data set", "response data," etc.) which
together occur 5,500 times, exceeding by far the
total number of times that the 53 existing termbase
terms containing "data" occur in the corpus.

Figure 1 Sample set of keywords
WordSmith offers various formulae for
identifying collocates of a search word, whichisthe procedure we used to find the productive
bigrams and trigrams. We tested all formulae
against our corpora and determined that the
DICE method provided the best results.

6

Observations

Our research confirmed that bigrams andtrigrams are highly effective in representing
terms of interest in the corpus and serve a key
translation need. Longer terms (4 words, 5
words etc., in length) tend to have lower
repurposing potential, whereas unigrams areproblematic because they are highly
polysemic. Our concordancing methodology
using sailient unigrams as search pivots
enabled us to identify both un-documented
terms and under-optimized terms and further,
it revealed how to adjust the boundaries of
those terms in order to make them fully
optimized.

The number of under-optimized terms in thetermbases was alarming. In the companies westudied, they range from 35 percent to 73 percentof the terms in the termbase. This is a significant
source of redundancy and wasted resources.

7

Conclusions

The main motivation for managing terminology in
a commercial setting is to reduce costs for content
authoring and translation. To meet this goal, thetermbase must reflect the company's corpus.
Documenting terms in the termbase that do notoccur in the corpus, while failing to document
those that occur frequently, significantly
decreases the value of the termbase and wastes
company resources. When selecting terms for atermbase, it is crucial to determine the mostproductive boundaries of MWTs for two reasons:(1) to avoid redundancy in the termbase caused byunder-optimized terms not to mention the wasted
resources incurred for documenting andtranslating them, and (b) to make the termbase
more representative of the corpus by selecting
statistically and semantically relevant terms. Bothrequire a corpus-based approach to term
identification and research.

Page 37

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

References
Condamines, Anne. 1995. "Terminology:
New needs, new perspectives."
Terminology.
Amsterdam. John Benjamins, V.2, No. 2, p.
219- 238.
Condamines, Anne. 2010. "Variations interminology. Application to the
management of risks related to language use
in the workplace." Terminology.
Amsterdam. John Benjamins, V16, No. 1, p.
30-50.
Daille, Beatrice, Benoit Habert, Christian
Jacquemin and Jean Royaute. 1996.
"Empirical observation of term variations
and principles for their description."
Terminology. Amsterdam.
John Benjamins, V. 3, No.2, p.197-257.
L'Homme, Marie-Claude. 2005. "Sur la notion
de terme." Meta: Translators' Journal.
Montreal.
Les Presses de l'Universite de Montreal, V.
50, No. 4, p.1112-1132.

Meyer, Ingrid and Kristen Mackintosh. 1996. "The
Corpus from a Terminographer`s Viewpoint."
International Journal of Corpus Linguistics.
Amsterdam. John Benjamins, V. 1, no 2, p.257285.
Nagao, Makoto. 1994. "A Methodology for theConstruction of a Terminology Dictionary."
Computational Approaches to the Lexicon.
B.T.S. Atkins and A. Zampolli, Eds. Oxford.
Oxford University Press, p.397-411.
Rey, Alain. 1995. Essays on Terminology.
Amsterdam. John Benjamins.
Van Campenhoudt, Marc. 2006. Que nous restet-il d'Eugen Wuster? Intervention dans lecadre du colloque international Eugen Wuster
et la terminologie de l'Ecole de Vienne. Paris.
Universite de Paris 7.
Warburton, Kara. 2014. Narrowing the gap
between termbases and corpora in commercial
environments. PhD Dissertation. City
University of Hong Kong.

L'Homme, Marie-Claude. 2020. Lexical
Semantics for Terminology - An
introduction. Amsterdam. John
Benjamins.
Maynard, Diana and Sophia Ananiadou.
2001. "Term extraction using asimilarity-based approach." Recent
Advances in Computational Terminology.
Didier Bourigault, Christian
Jacquemin, Marie-Claude L'Homme, Eds.
Amsterdam. John Benjamins, p.261278.

Page 38

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Psychosocial awareness in community translation:
A report on human and machine translation and post-editing

Katarzyna Stachowiak-Szymczak
Department of Interpreting Studies and Audiovisual Translation Institute of Applied Linguistics
University of Warsaw, Poland
km.stachowiak@uw.edu.pl

Abstract
Communitytranslators
enablewrittencommunication between parties who do not(fluently) speak/read/write the same language, orthe communication of messages, alerts,
documents, etc. to individuals and groups whose
dominant language differs from the one of thecountry or region they live/work in (Taibi 2011).
In many cases translation is affected bypsychosocial factors or calls for psychosocial
knowledge and awareness (Assis Rosa 2006;
Taibi and Ozolins 2016), e.g. when conducted
for healthcare professionals and allophonepatients, where knowledge and power imbalance
makes it difficult to come with one uniform
nomenclature for each target client. In turn, thementioned knowledge and awareness may ensure
successful information transfer, as well as thea ien digni , a n m and healiflife and dying. At the same time, building
psychosocial and cultural knowledge requires
time, effort and resources in the process of the
an la
ed ca i n andkilling. In man
countries of relatively homogeneous societies
and little experience in training translators inunderstanding and responding to psychosocial
needs and differences, resources can be limited.
At the same time, there is a strong need to ensure
proper standards of psychosocially adjusted

translation by means of machine work and postediting.
This presentation will report on a pilot study
conducted remotely on two groups ofparticipants differing in translation expertise. The
participants (N = 40, M = 18, F = 22) were
engaged in two tasks: 1) translating two medical
texts from Polish into English for two target
clients: doctors and patients (lay persons) as wellas 2) post-editing two texts from Polish into
English for two target clients: doctors andpatients. Briefings did not order the participants
to act in a specific way, i.e. e.g. by adjustingregister to the target audience, while they clearlyspecified the type of audience and the purpose of
the texts subject to translation/post-editing. The
order of the tasks, as well as the order of the texts
was counterbalanced across the participants.
Next, the results of these translations and posteditings were compared with machine translationof the experimental texts. Variables included:
lexical accuracy (assessed by independent judges
by means of a self-designed accuracy score
sheet), word frequency, word class, wordemotional load, as well as sentence structure andcomplexity. The results indicate that translatorsof less expertise tend to lexically andgrammatically over-complicate texts for lay
persons, i.e. patients. The outcomes also points

Page 39

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

out to those types and complexities of lexical andgrammatical items that are best translated bymachines vs. humans. The presentation can pointout to possible lines of training in post-editing
and translation per se, as well as indicate
potential risks of machine mistranslation
depending on the target client.

References
A i R a, Ale and a. 2006. Defining Ta ge Te
Reade . T an la i n S die and Li e a The .

In Translation Studies at the Interface ofDisciplines, edited by J.F. Duarte, Alexandra Assis
Rosa and T. Seruya. John Benjamins, Amsterdam,
The Netherlands. 99-109.
Taibi, M. 2011. P blic e ice an la i n. In The
Oxford Handbook of Translation Studies edited by
K. Malmkjær and K. Windle. Oxford University
press, Oxford, UK.
Taibi M. and Ozolins, U. 2016. Community
Translation. Bloomsbury Advances in Translation,
Sydney, Australia.

Page 40

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Assessing low and high translation variation in post-editing1

M. Cristina Toledo-Báez

Michael Carl

University of Málaga
Department of Translation andInterpreting
Málaga (Spain)

Kent State University
Modern and Classical Language Studies
Kent, OH, USA

toledo@uma.es

mcarl6@kent.edu

1. Introduction and backgroundPrevious research (Carl and Schaeffer, 2017, Carl,
2020/in press) introduce the word translation
entropy (HTra) to quantify the observed variationof word (or phrase) translation choices and theword distortion entropy (HCross), whichquantifies the amount of translation reordering in
the target language. Several interesting findings
have been reported:xx

x

There is a strong correlation between
HTra and HCross, indicating that themore translators choose different words
or word forms, the more they will also
produce the translations in a different
word order (Schaeffer et al, 2017).
HTra and HCross values correlate acrosslanguages. For instance, English words
with high HTra values when translated
into Spanish tend to have high HTra also
when translated into German or Japanese(Carl et al, 2019).
Similar correlations of HTra can be foundboth between human and machinetranslation and also across languages.
These correlations of HTra may indicatethat features of source text seem toinfluence both human and machinetranslation (Ogawa et al, 2020/in press).
2. Aim, corpus of study and research questions
The aim of this paper is to analyze some Englishto-Spanish post-edited translations with low andhigh HTra and with low and high HCross values.
Given that, according to Ogawa et al. (2020/inpress), correlations of HTra might be caused bylinguistic features of the source text, we will focus
on showing whether the linguistic features of the
English source texts in our corpus explain HTraand HCross values.
Specifically, we will try to answer the following
research questions:

1 The research presented in this study has been (partially)

carried out in the framework of research projects VIP(FFI2016-75831-P), TRIAJE (UMA18-FEDERJA-067),
MI4ALL (CEI-Andalucía Tech) and PROFETA (PIE 1933).

Page 41

1. What are the linguistic features behindhigh translation variation (HTra)? Are
there common linguistic patterns that
explain high translation variation?
2. What are the linguistic features behindhigh or low translation distortion(HCross)? Are there common linguistic
patterns that explain high or lowtranslation reordering?

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

3. What is the correlation between
translation
variation(HTra)
andtranslation distortion (HCross) for our
corpus of study?

After analyzing our corpus, we have found thathigh HTra values can be due to the following
linguistic features:
1. Choice of different phrasings. An
example f hi i c gh in ha e
c gh
. I e en a high HT a al e
(3.43) and a low HCross value (0.76). As
can be seen in Table 1, the high HTravalue can be due to the fact that 15
different translations have been used. In
c n a , HCfc gh
i only0.56 fc gh and 0.76 f
d e tothe fact that, despite being a phrasal verb
without literal or direct translation into
Spanish, there is a high syntactic
similarity in all the translations as all ofthem are infinitive verbs after
deben/ ienen ( e) .

Our analysis is based on a corpus belonging to themultiLing data set available through theTranslation Process Research Database (TPRDB) housed by the Center for Research andInnovation in Translation and Translation
Technology (CRITT). Our corpus consists of 30
alternative post-editions of six short English texts
each of which has between 110 and 160 words.
Each eaan la ed i h he G gle
Phrase-based Machine Translation in 2012. Eachtext was then post-edited by approximately 30
different translators whose mother tongue was
Spanish2.
Regarding the corpus, it is worth mentioning that,as Carl (Carl, 2020/in press) details, translationsin the multiLing corpus were manually aligned.
Given that manual alignment may be inconsistentor incoherent, it should be mentioned that ourfindings may be influenced by this manualalignment.

Source
have
to
HTra
1,38
1,45
HCross0,65
1,09
30 Translations:#Occ Cross
Cross
1
3
deben
1
hacer
21
3
tienen1que
54tienen_que

3. Findings and analysis
This section is organized in the following
subsections: 1) high HTra values; 2) high and low
HCross values; 3) high HTra and low HCross
values and 4) low HTra and high HCross values.
They correspond to the four main groups that wehave found in our analysis.

1
1
1

3
3
3

han

deben
3

cough
3,37
0,56

up
3,44
0,77

Cross
Cross
1
un
1
gasto
frente_a
1
2
hacer
1
frente1
gastar1pagar
1soltar1
desembolsar
1afrontar
1
pagar_un4
asumir1
lidiar_con_un_gasto
1
gastarse
1
1soportartenido_que_gastar
gastan

#Occ
1
1
1
7
5
4
3
1
1
1
1
1
1
1
1

Table 1. HT a and HC
al e fha e
coughalong with the 30 translations and theiroccurrences

3.1. High HTra values
Regarding HTra values, it should be noted that wehave only focused on high HTra values because
low HTra values imply that there is a great overlap
between source text and target text representationsin terms of lexico-semantics. Low HTra values
have only been analyzed when they appear withhigh HCross values, i.e., when no effect size wasgiven.

2

The data of our study is available on the CRITT website(https://sites.google.com/site/centretranslationinnovation/tpr-db) under the BML12 study.

Page 42

2. Morphological variation of the same
lexeme. An example is the past participle
nc l i a ed in lef
nc l i a ed ,whose HTra value is 2.9 and theirtranslations are variations of thec l i a le eme: in c l i a , in e
c l i ada ,e n e c l i an ,ee manece in c l i a , n e c l i a ,inc l i ada , n c l i ada , a laen e a lican l c l i
and c l i a .
3. Differences in phrasal translations. An
e am le i he h a al e b fla e(inBook of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020in he ake f fla ing, h e HT avalue is very high (4.32) and HCross
value is also high (2.56). The difficulty oftranslating this phrasal verb into Spanish
explains the high number of translations(see Table 2). It is worth stressing thatmost translations are wrong or without an
accurate meaning. All these different
translations and the very different
distribution of these translations explain
the high level of HTra and HCross forfla ing.

translation is generally avoided and,
therefore, other constructions such asactive voice, reflexive passive or
im e nal c n c i n
ih e aepreferred. However, as explained below(see section 3.3.), instances of passive
voice also present low HCross values.
2.

3.2. High and low HCross values
We have also investigated the linguistic reasons
behind the high and low HCross values for somewords.
Sourcefighting
flaring
up
HTra
4.19
4.32
4.32
HCross
2.62
2.56
2.56
32 Translations:#Occ Cross
Cross
1
4
la_lucha_que_se3lleva_a_cabo
11
combates3
que_han_surgido
13
la_lucha_contra
2
la_quema62
la_lucha3
contra_la_quema
12
armada-1intensificación12
las_luchas
2
continúan_sin
1-2
la_lucha
5
quema
1
5
de_las_rebeliones 2
que_surgieron
12
la_lucha5contra_la_nueva_guerra_prevista
1
1
que
2
el_conflicto
13
lucha_que_surge
1
3
contra_los_asesinatos
1
7
de_su_lucha_contra_lo_que_ocurre
1
3
la_lucha_emergente
16de_la_lucha_contra_la_lucha
2
5
lucha_contra_quema
1
3
violencia
2
4
lucha_contra_la_quema
1
2
que_resurja
1
3
quema
1
2
la_quema12situación_actual
1
7
la_lucha_contra_la_quema_de_nuevo
1
1
conflictos
1
4
esta_más_oprimida

Syntactic shift as a consequence of adifferent target language Pos tag. An
e am le i he aa ici legh ,whose HCross value is low (1.09). The
reason might be the fact that its
translations are VB (verb, base form,
iene , in en a ), VBD ( e b, pasten e, neg , in en ), IN ( e i i nb dina ing c nj nc i n,nega i a a m e la ) and VBG ( e b,
gerund or present participle, a and ).

SourceHCross32 Translations:#Occ Cross
1
2
1
1
1
1
1
1
1
2
4
2
2
1
2
2

Table 2. HT a and HC
al e ffigh ing
flaringalong with the 32 translations and theiroccurrences
1. Choice of different target structures. An
e am le ie in eemba a ,with a high HCross (2.24). This passive
construction presents different syntactic
re-orderings with very different syntactic
constructions as shown in Table 3. As aconsequence, the HCross level is quite
high (2.24).
This high value of HCross can be
explained by the difficulty of translating
passive voice into Spanish. Literal

set
2.24

to
2.08

lo_hizo
poner
sirve
previstoprevistoprevisto_que
previstoestá_previsto

Cross
1
1
1
1
2
2
1
1

embarrass
1.91

para
en
para
para
para
a
que
que

Cross
2
2
1
1
2
-1
2
2

1

3

significará

5

1
1
1
1
1

6
3
4
3
-2

tuvo_lugar_con_el_objetivo_de
ha_planeado
por_eso_ha_querido
que_va_asu_intención

2
2
2
2
3

1
1
1
1

4
1
1
1

su_objetivo_es_queque
para
prevee

21
2
2
1

1
1
1

8
1
1

1
1

4
1

1

3

1
2

3
1

con_el_objetivohabuscabatiene_toda_la_int
enciónintentatiene_como_obje
tivo
su_objetivo_es
pretende

1
4
2
3
2

avergonzar
evidencia_aavergonzar
avergonzar
avergonzar_aavergüence_aavergüence_aavergüence_a
sin_duda_para_un
a_mella_pública
avergonzar_a
para_avergonzar_aavergonzar_aavergonzar_aavergonzar
se_avergüence_de_elloavergüenza_aavergonzar_aavergonzarde5
poner_en_evidencia
hecho_para_avergonzar_aavergonzar_ade_avergonzar_aavergonzar_a2

avergonzar_a42

dejar_en_evidencia_aavergonzar_a

Table 3. HC
al e feemba aalong with the 32 translations and theiroccurrences
3.3. High HTra and low HCross values
We have found that there is a high effect size
between HTra and HCross (R = 0.698). However,there are instances of words with high HTra andlow HCross as well as words with low HTra andhigh HCross values:

Page 43

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Regarding instances with the highest HTra and thelowest HCross values, four groups of words canbe found:a. Verbs as past participles in passive voice.
Te am le a e e led (in i idiffic lbe e led ; e high HTra:
4.67 and low HCross: 1.93) and gi en(in he a gi en flife en ence ;high HTra: 3.93 and low HCross: 1.62).
b. Highly polysemous verbs without literaltranslation into Spanish. An example ise e in he ill ha ee e (highHTra: 3.84 and low HCross: 1.97). Again,as explained above in ha e to cough,the low HCross value can be due to thefact that there is a high syntactic similarity
in all the translations as all of them are
infinitive
verbs
after
debe / end ( e) .
c. Abstract nouns which tend to betranslated as verbs in Spanish. Twoe am le a e a a ene(in nl hea a enef he h i al aff ; highHTra: 3.84 and low HCross: 1.97) andb eakd n (inhe b eakd n fcial lida i ; high HT a: 3.45 andlow HCross: 0.99).
d. Adverbs which tend to be omitted in
Spanish. An example is nde andabl(in al h gh de el ing c n ie a eunderstandably reluctant to c m mi e ;high HTra: 3.72 and low HCross: 1.72).
This adverb has been omitted by 23 out of
30 translators.

Professionals
professionals)

4. Concluding remarks
In this paper we have analyzed translation entropy
and translation distortion in our TPR-DB EnglishSpanish corpus of post-edited translations. We
have found that:a. Nouns belonging to a compound noun
(noun + noun):
HCrossSalary (salary increases)

2.53

2.87

Nurse (hospital nurse)

2.19

2.58

Systems(agricultural
subsistence system)

0.20

1.22

1.17

The higher HCross can be explained by
the necessary inversion of order whentranslating into Spanish.
b. Verbs having an auxiliary function:a (ina c ined ; HT a: 0.85,
HC: 1.07), need (in needada , HT a: 1.22 HC: 1.30), can(in can; HT a: 0.20, HC:0.74).

As to words with low HTra and high HCross
values, two groups of instances can be found:

HTra

0.83

Table 4. Compound nouns with low HTra andhigh HCross values

3.4. Low HTra and high HCross values

Compound noun

(healthcare

Page 44

1. High translation variation can be due tothe choice of different phrasings,
morphological variation of the same
lexeme and differences in phrasaltranslations.
2. High and low translation distortion can be
explained by choice of different target
structures as well as syntactic shift as aconsequence of a different target
language.
3. There is a high effect size between
translation variation and translation
distortion (R = 0.698).
4. Four groups of words present instances ofwords with high translation variation andlow translation distortion: a. Verbs as past
participles in passive voice; b. Highly
polysemous verbs without literaltranslation into Spanish; c. Abstract
nouns which tend to be translated as verbs
in Spanish; and d. Adverbs which tend tobe omitted in Spanish.
5. Two groups of words present instances ofwords with low translation variation: a.
Nouns belonging to a noun + noun
compound noun and b. Verbs having an
auxiliary function.

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Given HCross and HTra values correlate acrosslanguages (cf. Ogawa et al., 2020/in press), it
would be interesting to carry out a cross-lingual
study to check whether similar linguistic
phenomena of translation variation are also foundfor other language pairs.

References
Carl, Michael, Andrew Tonge and Isabel Lacruz(2019). A systems theory perspective on thetranslation process. Translation, Cognition &Behavior, 2(2): 211- 232.
Carl, Michael and Moritz Schaeffer (2017). Why
Translation Is Difficult: A Corpus-based Study ofNon-literality in Post-editing and From-scratch
Translation. Hermes 56, 43-57.

Carl, Michael (2020/in press). Information and
Entropy Measures of Rendered Literal Translation.
In Michael Carl (Ed.). Explorations in EmpiricalTranslation Process Research. New York: Springer,
128-149.
Ogawa, Haruka., Devin Gilbert and Samar Almazroei(2020/in press). redBird: Rendering Entropy Dataand ST-based Information into a Rich Discourse on
Translation. Investigating relationships between
MT output and human translation. In Carl, Michael(Ed.). Explorations in Empirical Translation Process
Research. New York: Springer, 150-173.
Schaeffer, Moritz, Kevin Paterson, Victoria A.
McGowan, Sarah J. White and Kirsten Malmkjær
(2017). Reading f
an la i n . In A n L kke
Jakobsen and Bartolome Mesa-Lao (Eds.),
Translation in transition. Amsterdam: Benjamins,
18-54.

Page 45

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Metalanguage for the translation process
Masaru Yamada
Kansai University

Mayuka Yamamoto
Kansai University

Nanami Onish
Kansai University

yamada@apple-eye.com

mayuka626@gmail.com

nnm.co5vcu.krm@gmail.com

Atsushi Fujita
NICT

Rei Miyata
Nagoya University

Kyo Kageura
University of Tokyo

atsushi.fujita@nict.go.jp

miyata@nuee.nagoyau.ac.jp

kyo@p.u-tokyo.ac.jp

Keywords: metalanguage, ISO 17100, translation process, translation strategy, decision list

1

Introduction

The e e i n an la i n
ce
ha
maj
usages. In translation process research (TPR), whichki hin a beha i al-cognitive experimental
me h d l gicala adigm(Jak b en,2017),an la i nceba ically refers to cognitive andrelated process of individual translators. TPR
contributed significantly to our understanding ofhitherto underaddressed aspects of translation, i.e., thecognitive process in translation act and factors
affecting the process, such as the relation between
cognitive effort and specific linguistic features (e.g.,
Lacruz et al., 2018). On the other hand, in translationindustries, an la i n
ce refers to the e e nal
process consisting of such modules as a client-TSP
contract, brief definition, project management,
translation, revision, reviewing, quality assurance, etc.
In this presentation, we address the translation process
in the latter meaning.
Descriptions of translation processes in the latter sense,
as such those most typically given in ISO 17100 (ISO,2015), generally seem to assume alreadyknowledgeable actors as their target readers and remain
coarse. As translation industries are growing andincorporating rapidly changing technologies, weobserve a serious issue here, i.e., lack of common
understanding of translation processes among different
actors. MT developers and other natural language
processing (NLP) researchers and engineers deal witha very limited part of translation processes without so
noticing consciously. Translation learners need to gain
competences and norms through practice without being
given a scaffold of explicit and detailed knowledge on
the processes. We observe a dearth of concrete andsystematized terms and expressions in talking abouttranslation processes as one of the main causes of thissituation.

Against this backdrop, we set out a project ofde el ing a e f me alang age f facili a ing
and promoting the level of shared understanding oftranslation in general and concrete translation
processes going on in translation industries. The
overview of our metalanguages is given in thispresentation. We begin by explaining ISO 17100 as abasis of describing our translation process, and thenelucidate issues with these descriptions of operations
pertaining to each subprocess. In the following sections,
we provide brief descriptions of our proposed
metalanguages tailored for talking about translation
issues (Section 4), translation process (Section 5), andtranslation strategies (Section 6). We also touch on
how such metalanguage can be used to annotate TPR
databases (Section 7).

2

ISO 17100 as a basis of translation
process

This section describes translation process andsubprocesses for which we are targeting to create
metalanguages. We utilize ISO 17100 as a basis fordescribing details of the translation process. As
illustrated in Figure 1, the process is divided into threephases: pre-translation, translation, and posttranslation. Translatorial actions (Holz-Mänttäri, 1984),
meta-actions taken by translators and actors involvedin the processes before and after the translational action
may correspond to the pre/post-translation processes in
Fig e 1. F in ance, in Clien -TSP agreement(4.4) , ecifica i n for the translation task is decided
by the project manager through negotiation with theclient prior to the beginning of the actual job. The
content of the specification or so-called translation
brief plays an important role, as it influences the core
process of translation or translational action wheretranslation is produced by a translator throughPage 46

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

manipulations of semiotics such as the source and thetarget texts.
T an la i n
ce e ( Sec i n 5 ) in Fig e 1 hafollow the pre- an la i n ( Sec i n 4 ) i-calledTEP (translation, editing, and proofreading), which
consists of translation, check, revision, review, andproofreading in the ISO 17100. Actors to be involvedin the translation phase are: 1) the translator, who is incha ge f T an la i n (5.3.1) and Check (5.3.2)
or self-revision, 2) the reviewer, who is responsible for
Re i i n (5.3.3) and Re ie (5.3.4) , and 3) hef eade , h e f m P f eading (5.3.5) .

3

Description of operations

Given the overall flow of processes and subprocesses
and operations described above, an issue arises as tothe description of the operations for each subprocess interms of its granularity as to whether the actor cantake actions according to the given description
especially for translation training and establishing ashared understanding of each operation in detail. Let us
examine examples of descriptions excerpted from ISO
17100 (ISO, 2015) ega ding T an la i n 5.3.1 andRe i i n 5.3.3 , as follows:
5.3.1 Translation: The translator shall translate inaccordance with the purpose of the translation
project, including the linguistic conventions of thetarget language and relevant project specifications.
(p. 10)5.3.3 Revision: The reviser shall examine thetarget language content against the source language
content for any errors and other issues, and its
suitability for purpose. (pp. 10-11)

Although the description contains some important
ke
d
ch a
jec
ecifica i n
ande ,i i n
fficien l c nc e e be ed ftranslator training or for establishing a common
understanding of translation by specifying actions
necessary to achieve the translation for people who are
not experienced in translation. Once again, ISO 17100
reads, The translator shall translate in accordance withhee . H e e , he de c i i n i h hi le elof granularity is the same as saying that the translator
can translate, unless any further explanations areprovided as to what one should do for achieving it.
Otherwise, one must acquire or infer the tacit skills
through practice or just by guessing, without knowing
exactly what to do.
Thede i alag e in he en e ha n
explanation of the components of purpose is provided
in he ISO anda d. In he ha e f Clien -TSPag eemen 4.4 , he TSP i nl in c ede ain arecord of the agreement in writing. After all,an la ing in acc dance i h he
e i j
an he
a
f a ingan la ing a he clien
an ,f and inevitably most parts of the translation job
m de end n he e n
e ce i n
aci kill
if he or she happens to have them.
While fully recognizing that ISO 17100 is for TSPs andnot for learners or NLP engineers, it is used as animportant point of reference for translation practices intranslation industries, the aspect that has not been fully
explored in translation studies. Enriching thedescriptions of translation processes based on ISO
17100 would thus benefit wider range of actors
including learners and engineers. Therefore, one aim of
this study is to increase the granularity of descriptions
pertaining to translation processes in the form of

Figure 1. ISO 17100 Translation Process

Page 47

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

metalanguages through which learners and different
actors will be able to operationalize the tacit knowledgeof translation for better practice and communication.

4

Metalanguage of translation issues

Our benchmark for achieving a higher degree of detailis translation quality assurance (QA) scheme, a set ofissue categories used by reviewers to check fortranslation errors in the industries and translation
training. The QA scheme can be the criteria for the
granularity of descriptions in our metalanguage. One
well-known example is MQM, multidimensional
quality metrics (Burchardt and Lommel, 2014), which
is now the de-facto QA scheme.
The use of such a typology makes the coarse
descriptions more operationalizable. For instance, let
us see the aforementioned ISO description about
Re i i n 5.3.3 . The c nce
f e
and i eare vague; they can be operationalized in a much higher
granularity with consistency in detection when thequality metrics are employed. For these reasons, theerror categories are now utilized even by researchers
who annotate errors of both human and machinetranslation for research assessment purposes (e.g.,Specia and Shah, 2014).
We have also developed our own issue typology called
MNH-TT issue typology specialized for universitylevel training, based on MeLLANGE issue categories(Castagnoli et al., 2006; Seca , 2005). Issue categories
and their structure were systematically fine-tuned to
improve usability with learners/instructors. For that wehave compiled a list of issue categories in the form ofa decision tree, a step-by-step guide to help learners
classify issues, as provided in Figure 2 (Fujita et al.,2017).
In order to identify an issue in accordance with theissue typology, a learner first finds the issue andcategorizes it to the appropriate category. For instance,
if a part of the source text is found to be untranslated
and the part that corresponds to the source does notappear in the target text, the learner asks a question
about the issue according to the decision tree byanswering Yes or No, starting with the Q1a. When sheor he finds an answer to be Yes to the question, thenthe issue will be classified as instructed herein. If not,the leaner proceeds with the following question until
she or he finds the answer. In this example, Q1a is
answered No, Q1b is answered No, Q2a is answered
No, and Q2b is answered No. As a result, the issue isclassified into X1 omission. In this way, the issue

Figure 2. Decision tree for classifying a given issue(Fujita et al., 2017)

typology will help equip learners with common
knowledge to talk about translation issues with peers
and the instructor.
Of greater importance is that the set of issue categories
will serve as a metalanguage for the learners (Piao etal., 2019). Learners who could not distinctly recognize
errors will be able to identify 16 types of errors after
having learnt with the issue typology. That is to say,meta-recognition is made possible with metalanguage.
The expected acquired skill is not only the improved
ability to check errors during revision but also theimproved translation competence of students having
become conscious of making such errors duringtranslation.
A benefit in consistency of classification is also
expected in terms of both intra-reviser agreement(within the learner revising multiple issues) and interreviser agreement (among multiple learners revising
the same issue). The advantage is expected not only fortraining but also for enhancing shared communication

Page 48

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

among actors in interdisciplinary fields. For instance,
when MT developers and translation practitioners talkabout translation issues produced by an MT engine, thele el f each a ici an
nde anding abi e
categories should improve with the use of shared
metalanguage. These show that the level ofconcreteness of descriptions represented by QA
scheme and MNH-TT issue typology facilitates
understanding and communications among different
actors with different backgrounds or with different
degree of experiences.

5

Metalanguages of translation process

Here we introduce modules of translation processes foreach of which a subset of metalanguages is developed.
Based on the framework of translation processprovided in ISO 17100, we divided the translation
process into the five sub-phases as shown in Figure 3,
then design a set of metalanguages each of whichcorresponds to a specific sub-phase. A brief
explanation of the metalanguage for each sub-phase is
as follows:(1) Translation project management process: Based on
descriptions provided in ISO 17100 illustrating
subprocesses, we break them down into finer-grained
and operationalizable items which a person in training
can complete as answering a questionnaire. For
instance, it is not clear as to what kind of information
one needs to obtain in order to determine the purposeof translation during the Client-TSP agreement phase.
More specifically, if information such as the target
audience is required for determining the purpose, theitem should be broken down into the variables thatdefine the target audience, such as gender, age,interests, and community membership, which areeasily identified even by learners of translation. In
order to accomplish this degree of detail, we consulted
the literature on translation project management, andalso obtained authentic documents used in the actual
industry with interview data from two of the ISOcertified TSPs in Japan.
(2) Source document property and element:
T an la i n
ce(5.3.1)i di ided innde andingce d c men
andemi ican fe ing . Gi en ha , i i im
an to understandthe source document (SD) before semiotic transferring,but operations required for SD understanding have notsufficiently been clarified. For this purpose, we divided
the task of SD understanding into two phases: SD
profiling to identify the properties of the SD that areimportant for translation and SD analysis to identify
the elements within the SD to be properly translated.
As metalanguages, we are developing fine-grained
comprehensive typologies of SD properties andelements.
(3) Translation strategies: Given that translation or
textual transformation requires semiotic manipulationsof source and target languages, good translation results
f m he an lakilled mani la i n. In dedescribe the manipulations, we have drawn ontranslation strategy categories proposed by Chesterman(2016). Detailed explanations are given in the next
section.
(4) Effec f efinemen : Check (5.3.2) i a elfrevision, carried out by the translator before submitting

Figure 3. Five sets of metalanguages in the translation process

Page 49

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

the translation to the reviser. Metalanguage wedesigned to talk about this phase is a set of categories
to describe the effect of refinements added by thetranslator over the course of self-revising.

Our rearranged list of translation categories will serve
as a decision list to help translation learners to identify
an appropriate strategy from the top of the list, as
provided in Figure 4.

(5) Issue typology: As discussed, MNH-TT issue
l g i a ailable f
Re i i n (5.3.2) and
Re ie (5.3.4) ha e he e a e ie e ill checkthe translation for any issues.
Each type of metalanguage is designed so that it can beused in both classroom settings and translationindustries to describe and explain in detail thetranslation subprocesses for better practice andcommunication. For training purposes, we are planningto implement these metalanguages as an independent
module in MNH-TT, a browser-based learning
platform for translation training, so that learners using
this platform can operationalize the explicit knowledge
for every process by either filling out the template
format or annotating texts according to themetalanguages.

6

Metalanguage
strategiesoftranslation

This section gives explanations in more detail about thedescription of translation strategy by providing a
specific example to explain what it can provide andhow it is operationalized.
Amongst translation strategies proposed in previous
literature (Lörscher, 1991; Newmark, 1988; Vinay andDarbelnet, 1958/2000), we have selected the list oftranslation strategies presented by Chesterman (2016)as a point of departure. Most proposals consist of one
group of categories which makes it difficult fortranslation learners to distinguish, for example,
between semantic and pragmatic shifts, both of which
involve message-related manipulations of target text.
Che e man e f an la i n a egie (2016) helps
to disambiguate the aspect with granular sets of
categories which comprise three pillars
syntax,semantics, and pragmatics each of which branches
into approximately ten subcategories. This would be
advantageous for learners and researchers to dissect
translation shifts or strategies.
The categories were originally designed for translation
between English and German. We recompiled them sothat they are applicable to English-Japanese translation.
Nearly 300 translation samples extracted fromtranslation instruction books (Okada, 2013; Tanabe
and Mitsufuji, 2008) were used to verify the categories.

Figure 4: Decision list of translation strategies

Referring to one category for each of the three pillars
in the decision list, we can describe a given translation
shift. An example is below.
Source text: CNN reported on (a)Thursday that a giant
tornado and hailstorm had killed 51 people in China.
Target text: CNN が(a)23
⽇、中国で巨⼤ 巻と で
51
⼈ が 死
し た と
じ た 。
[Back translation: CNN reported on (a) the 23rd that 51
people had been killed in a giant tornado and hailstorm
in China.]
In hi e am le, he nde lined ni (a) Th da inhece e i an la ed in 23 ⽇ ( he 23rd) inthe target text. This translation shift is annotated withthree categories given from each pillar: G1, S1, Pr12 in
Figure 4. This means that no specific strategies aretaken in terms of syntactic and semantic strategies: G1(Literal Translation) and S1 (Semantically Equivalent),
but a pragmatic strategy, Pr12 (the domain adaption),
applies to this case.

7

Application to TPR and the future

Our metalanguage of translation strategies could
potentially be used for deeper analysis in TPR.
Translation strategy is concerned with a process
normally attributed to individual translators and thus
can correspond to the translation process as defined in
TPR. In the existing CRITT TPR-DB (Carl, 2012), therelationship between the source and target texts isrepresented only by annotations on alignments on aformal linguistic unit (at the word or phrase level). If
we also annotate the process data with our strategies, it
may increase granularity of data analysis as well. For
example, we can investigate the relationship between
each an la i n a eg and he an lac gni i eeffort required for the manipulation.

Page 50

Book of Abstracts: Translation in Transition (TT5)8October 15 - 17, 2020

Conclusion

Fine-grained metalanguages are expected to make itpossible to externalize the tacit knowledge of skilledtranslators for use in translator training and incommunication among different actors. By
operationalizing
thisknowledge
through
metalanguages, translation learners can consciously
practice the behaviors of professionals and different
actors can accurately talk about translation.

Acknowledgement
This work is partly supported by JSPS KAKENHI
Grant Number 19H05660.

References
Burchardt, B. and A. Lommel (2014) QT-LaunchPad
supplement 1: Practical guidelines for the Use of
MQM in Scientific Research on Translation Quality.
http://www.qt21.eu/downloads/MQM-usageguidelines.pdf.
Ca l, M. (2012) The CRITT TPR-DB 1.0: A databasef em i ical h man an la i ncee ea ch ,Proceedings of the AMTA 2012 Workshop on PostEditing Technology and Practice (WPTP 2012)
Association for Machine Translation in theAmericas (AMTA), pp. 9-18.
Castagnoli, S., D. Ciobanu, N. Kübler, K. Kunz and A.
V lan chi (2006) De igning a lea ne an lacfaininge , Teaching andLanguage Corpora Conference (TALC) 2006.
Chesterman, A. (2016) Memes of translation: The
spread of ideas in translation theory.
Amsterdam: John Benjamins.
Fujita, A., K. Tanabe, C. Toyoshima, M. Yamamoto,
K. Kageura and A. Ha le (2017) C n i enclassification of translation revisions: A case studyof
English-Ja ane eden
an la i n ,Proceedings of the 11th Linguistic Annotation
Workshop (LAW), pp. 57-66.
Holz-Mänttäri, J (1984) Translatorisches handeln:Theorie und methode. Helsinki: Suomalainen
Tiedeakatemia.

International Standard Organization (ISO). (2015).
ISO 17100. 2015. Translation services
Requirements for translation services. First edition.
Jak b en, A. (2017) T an la i n
ce e ea ch , in
J. Schwieter and A. Ferreira (eds) Handbook oftranslation and cognition (pp. 386 401). Blackwell
Handbooks in Linguistics. Malden, MA: John Wiley& Sons.
Lac , I., M. Ca l and M. Yamada (2018) Li e ali
and c gni i e eff : Ja ane e and S ani h ,Proceedings from 11th Edition of the Language
Resources and Evaluation Conference (LREC2018).
Lörscher, W. (1991) Translation performance,translation process and translation strategies: A
psycholinguistics investigation. Tübingen: Gunter
Narr.
Newmark, P. (1988) A textbook of translation. New
York: Prentice Hall.
Okada, N. (2013) Honyaku no fuseki to jouseki[Strategy and tactics of translation]. Sanseido.
Piao, H., S. Han and K. Kageura (2019) The e fmeta-lang age inan la i n e i i n , 2019
International
ConferenceonTranslation
Education: Computer-Aided Translator Training(CATT) of Machines and Man.
Seca , A. (2005) T an la i n e al a i n: A a e fheae ,ProceedingsoftheeCoLoRe/MeLLANGE Workshop, pp. 39-44.
S ecia, L. and K. Shah (2014) P edic ing h man
an la i nali , Proceedings of the 11th
Conference of the Association for MachineTranslation in the Americas (AMTA), pp. 288 300.
Tanabe, K. and K. Mitsufuji (2008) Puro ga oshieru
kiso karano honyaku sukiru [Buidling translation
skills]. Tokyo: Sanshusha.
Vinay, J.-P. and J. Da belne (1958/2000) A
me h d l g f
an la i n , in L. Ven i (ed.) The
Translation Studies Reader (pp. 84-95). London:
Routledge.

Page 51

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Explorations in Empirical Translation Process Research:
A Book Presentation

Michael Carl
Kent State University, Ohio, USA
mcarl6@kent.edu

1

Introduction

The b k Explorations in Empirical Translation
Process Research ,a ea in S inge ,MachineTranslation:
Technologies
andApplications, assembles fifteen original,
interdisciplinary research chapters that explore
methodological and conceptual considerations aswell as user and usage studies to elucidate therelation between the translation product andtranslation/post-editing processes. It introduces
numerous
innovative
empirical/data-driven
measures as well as novel classification schemes
and taxonomies to investigate and quantify the
relation between translation quality andtranslation effort in from-scratch translation,
machine translation post-editing and computerassisted audiovisual translation. Translation
experiments are conducted for several language
pairs in different translation modes using eyetracking and/or key-logging technology, tocompare different types of translator expertise,different types of texts, and various types of
linguistic expressions. The research addresses
questions in the translation of cognates,
neologism, metaphors, idioms, figurative andcultural specific expressions, it re-assesses thenotion of translation universals and translation
literality, elaborates on the definition oftranslation units and syntactic equivalence,
investigates the impact of translation ambiguity
and translation entropy, suggests alternativeinterpretations of the human translation edit rate,and explores the possibilities of computer-assistedtranslation via a pivot languages. The results andfindings are interpreted in the context of psycholinguistic models of bilingualism and re-frame
empirical translation process research within thecontext of modern dynamic cognitive theories ofthe mind. The book aims at bridging the gap
between translation process research and machinetranslation research.

2

Structure of the Volume

The 15 chapters in the volume are structured in fourparts. The first part, Translation Technology,
Quality, and Effort, starts with chapters that havean applied orientation, investigating the(psychological) reality of translation edit rate(TER) in post-editing. The second part, Translationand Entropy, presents four contributions that
address various aspects of word translation
entropy. The third part, Translation Segmentation
and Translation Difficulty. deploys qualitative andquantitative methods to address topics intranslation segmentation and translation difficulty.
Part four, Translation Process Research and Postcognitivism, provides conceptual, methodological,
and theoretical support for a post-cognitivist
perspective in TPR.
2.1

Translation Technology, Quality, andEffortSince its beginnings, TPR has been concerned withinvestigating the impact of translation technology
on the human translation process. The aim of using
computer assistance in translation has been to
support translators at work, to offer possibilities tolay off memory and cognitive load into the

Page 52

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

environment, to provide them with customized
collocation and retrieval tools, and to suggest
targeted translation solutions at the right time.
Specialized editing interfaces are being produced
and tested under many different conditions tofacilitate the MT post-editing process. Two factors
are of crucial importance in this endeavor: 1) the
possibilities to assess, in an objective manner, thetranslation quality and 2) to model, measure, andexplain the hoped-for reduction of translation
effort.
1) With the wide-spread deployment of datadriven MT systems at the beginning of thiscentury, automatic evaluation metrics wereneeded to compare and fine-tune the systems
towards a reference or g ld anda d . The
Transition Edit Rate (TER) is such a measure
which assumes four edit operation in MT postediting: deleting, inserting, replacing, andmoving words and groups of words. It
computes the minimum amount of assumed
edit operations to match the MT output to areference translation for instance, the postedited version. The number of assumed edit
operations is then taken as an indicator for MT
quality, where fewer edit operations indicate
better MT output.
2) Numerous models have been proposed to
explain and understand the reported andobserved translation behavior and to relatetranslation behavior with translation quality.
Countless publications refer to K ing (1986)
categories of temporal, technical, andcognitive effort: while temporal effort is often
used as a proxy for cognitive effort, gaze data
provides a more direct insight into the mental
activities though, often not without muchnoise. However, more fine-grained models arebeing increasingly used to explain thesefindings, and the experimental validation of themodels themselves becomes a matter of
research.
The first three chapters in this section relate TER
scores to human post-editing activities and assess
to what extent TER is suited to describe the actual
post-editing process. Do Carmo proposes in
Chapter 1 to reverse the view: instead of looking at
the Translation Edit Rate, he suggests taking a viewon the Human Edit Rate (HER), which may close

the gap between MT research and translation
studies. Based on Do Ca m proposal, Huang andCarl introduce, in Chapter 2, a new measure, theword-based HER (WHER) score which they showcorrelates well with measures of translation effort.
In Chapter 3, Cumbreño and Aranberri also
correlate various measures of cognitive effort with
TER scores, but they come to different conclusions.
The last chapter in this part reports a study usingtranslation technologies with different goals.
Similar to Huang and Carl, also Tardel (Chapter 4)
studies aspects of cognitive effort in computerassisted subtitling. However, she compares
different settings to find a best informationenvironment for computer-assisted subtitling.
2.2

Translation and Entropy

Entropy is a basic physical measure that quantifies
the interaction between two entities. The entropy ofan entity (e.g., an ST expression) with regards toanother entity (e.g., the set of its possible
translations) counts the number in-distinguishable
configurations (i.e., different translation for an ST
expression). It is the only physical measure that isirreversible and directional (i.e., non-symmetrical),and thus tightly linked to the notion of time, whichis also directional and irreversible. The wordtranslation entropy (HTra) represents one of threecriteria to measure transition literality (cf. e.g. Carland Schaeffer 2016). HTra has since then been used
as a powerful predictor for several translation
measures. It has been shown to correlate withvarious behavioral observations of the translation
process, such as translation production duration,
gazing time, the number of revisions, but also withproperties of the translation product, includingtranslation errors of humans and machinetranslation systems. HTra is an informationtheoretic measure that quantifies whether there arestrong translation preferences. Stronger entrenchedtranslation solutions are less translationambiguous, they carry less translation information,
they are easier to retrieve, and their production
requires less cognitive effort as compared to more
ambiguous and less entrenched translations. Moreentrenched translations are also thought to besemantically closer to their ST equivalent.
In this section, Carl (chapter 5) describes theimplementation of a ende ed li e alimea ewhich extends the three previous literality criteria:
monotonicity, compositionality, and entrenchment,

Page 53

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

with the additional constraint of TL compliance. In
Chapter 6, Ogawa et al. observe HTra correlations
for human and machine translation, across different
languages and investigate in detail to what extent
this is also the case for different word classes andsets of collocations. Wei investigates, in Chapter 7,
with great detail scan paths that are triggered
through a high HTra word and discusses patterns ofvisual search to find disambiguating clues in lowentropy context. Heilmann and Llorca-Bofí detect,in Chapter 8, an effect of cognateness, as computedwith a Levenshtein distance, on HTra and suggest
adding a formal similarity criterion to the list ofliterality criteria.
2.3

Translation
Segmentation
Translation Difficulty

andSegmentation during translation production has
been a topic of research for many years and has
been in some ways at the very core of TPR. Since
its beginning, TPR has produced many models to
describe, explain, and conceptualize the basic unitsof translation, but has up to date - not reached agenerally accepted conclusion about its nature.
Two fundamentally different approaches can be
distinguished to describe translation units: bylooking into the translation product, one can try tofind linguistic and/or cross-linguistic clues thatindicate coherent translation segments, such as
sequences of monotonous or isomorphic
translational correspondence. Incoherent andsmaller segments of translational correspondence,larger amount of translation re-ordering, and lessisomorphic ST - TT representations are taken toengender more translation difficulties and potential
increased translation effort.
Another approach investigates behavioral data
directly mainly logs of fixations and/or keystroke
data - to determine the assumed mental processesof text segmentation and integration duringtranslation production. Less fluent typing, longer
keystroke pauses, and more dispersed visual
attention and search are taken as indicators oftranslation difficulty and extended effort. The
assumption is that both approaches, the view fromthe process or the product, would converge andallow us to come to the same conclusions abouttranslation difficulty and translation effort.
Post-cognitivists reject the assumption that the mind
performs computations on objects that are faithfulrepresentations of an outside world, which is1This part of the volume addresses translation
segmentation and translation difficulty from thosedifferent angles. In Chapter 9, Carl provides adefinition of micro (translation) unit that integrates
properties of the translation process and productand allows to investigate the relation between thefirst translational response and the final translation
product. Vanroy et al. examine in Chapter 10 thetranslation product from a computationallinguistics view and base their notion of translationdifficulty on various definitions of cross-lingual
syntactic equivalence. In Chapter 11, Lacruz et al.
explore novel types of segmentation, based on the
Ja ane e
b n e , to assess translationdifficulties of culturally and contextuallydependent expressions, based on the variation ofHTra values. Chen takes a process view ontranslation difficulty in Chapter 12 when assessing
the success of several translation strategies
depending on whether the meaning andbackground knowledge of neologisms wereavailable.
2.4

Translation Process Research and Postcognitivism1

Translation process research has primarily been
concerned
withtechnologicallyheavymethodologies to collect and analyze translation
process data that help elucidate the humantranslation processes. Various explanatory models
have been deployed that were borrowed - among
others - from cognitive sciences, psycholinguistics,
and bilingualism research so as to interpret the TPR
findings in a coherent theoretical framework. With
the development of those disciplines in the past 20
years utilizing more sophisticated data acquisition
tools, new translation devices, and theirtechnological possibilities combined with thecollection of big data sets and more rigorous
analysis methods, the explanatory models in TPR
have also changed and adapted to the new situation.
As pointed out by several scholars in the field, (e.g.,
Sun and Wen 2018, Shreve and Angelone, 2010),
new process models have to be developed that areable to accommodate those novel developments
and research findings. A trend towards postcognitivist theories can be noticed in recent TPR,
and also in this volume where several chapters referusually associated with the computational theory ofmind and their protagonists.

Page 54

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

to connectionist models as the explanatory
framework.
The last part of this volume underpins this postcognitivist perspective of TPR. Chapter 13
postulates that TPR has mainly developed and beena methodology that suggests a mechanistic view on
computation. It
la e ha e e en a i n andc m a i n are independent concepts, thatcomputational devices are useful for developing
and verifying theories of the mind, and that thestatus of a statement as methodological andontological has perhaps not always been clearlymarked. Chapter 14 introduces a new triangulation
method based on artificial neural networks. It
integrates findings from bilingualism andtranslation research and assesses to what extent
results from single word translations may carryover to translation in context. Chapter 15 develops
a radical embodied post-cognitivist perspective on
the translation process. The chapter extends
translation affordances with a probabilistic
recursive layer and maps this framework onto a
dynamic systems approach, capable to explaine e en a i n h ng
c gni i n as covariation
between the model and the world.
References
Albir, A.H., Alves, F. Dimitrova, B.D., Lacruz, I.
(2015). A retrospective and prospective viewof translation research from an empirical,
experimental, and cognitive perspective: theTREC network. Translation & Interpreting Vol
7 No 1.
Carl, M., Bangalore, S, & Schaeffer, M. (2016).
New Directions in Empirical Translation
Process Research. Springer. ISBN 978-3-31920357-7
Ehrensberger-Dow, M. Hunziker Heeb, A. Jud,P., Angelone, E. (2017) Insights fromtranslation process research in the workplace.
In: Doing Applied Linguistics, De Gruyter,(116 123) DOI: 10.1515/9783110496604-014
Jakobsen, Arnt Lykke. (2017). Translation
Process Research. Book Editor(s): John W.
Schwieter Aline Ferreira The Handbook ofTranslation and Cognition Wiley,
https://doi.org/10.1002/9781119241485.ch2

Page 55

Book of Abstracts: Translation in Transition (TT5)A e li e al

October 15 - 17, 2020a la i

al a

ea ie

ce ?:

a process-based study of effort based on comparable corpus data

Miguel A. Jiménez-Crespo

Joseph Casillas

Department of Spanish and Portuguese

Department of Spanish and Portuguese

Rutgers University, 15 Seminary Place

Rutgers University, 15 Seminary Place
New Brunswick, NJ 08901

New Brunswick, NJ 08901

1

Jimenez.miguel@rutgers.edu

Joseph.casillas@rutgers.edu

Introduction

The en n-li e alan la i n candida e
involved either (1) lexical or syntacticaldiffe ence (e.g.
gaaan la ed a
gl c en n in STe e rendered througha verbal structure in TTs), as well as (2)c m le el diffe en ende ing (e.g.
hali ibad b ea ha ma ched in the Spanish TT
i h a ende ing ch a hali i
f l mellcoming out of the m h ).

Recen l , he in e e in he n i n f li e alan la i n ha
ima il e l ed a nd heempirical inquiry into the relation of post editing
MT (PEMT) output and cognitive effort (i.e.
Schaeffer and Carl 2014; Lacruz and Shreve
2014; Carl and Schaeffer 2017a, 2017b; Lacruz
2017; Lacruz, Carl and Yamada 2018). In thiscontext, it has been found that non-literaltran la i naem e diﬃcult and timec n ming [...]
d ce han li e al ne(Schaeffer and Carl 2014: 55) in terms of
cognitive or temporal effort. The present study
contributes to this area using data from aprevious product-based comparable corpus study(Jiménez-Crespo and Tercedor 2018). The
tertium comparationis, what the effort incurredin
ce ing li e al an la i n i c m a ed
to, is that of post editing the most frequent nonliteral renderings in recurring conventionalized
units that express a specific communicative
purpose in non-translational or naturally
produced texts.

2

Research Question

RQ1 - are non-li e al an la i n al am eeffortful to process than literal ones? And more
specifically, are recurring conventionalized units
to express a specific communicative purpose
found in natural or non-translational corpora
though frequency analysis, regardless of theirli e ali
ela i nhece e , m etaxing to process in terms of time and cognitiveeffort?
Hypothesis 1 (H1). We hypothesize that literaltranslations (adequate and inadequate) from thetranslational corpus imply less cognitive effort
with regard to non-literal translations that
represent the most frequent renderings to express
recurring communicative purposes in nontranslated texts, and, therefore, should be
associated with faster processing times.

Page 56

Book of Abstracts: Translation in Transition (TT5)1

October 15 - 17, 2020

Methodology

Ten professional translators took part in theexperiment. They post edited two texts using amock CAT tool setup in a PC using the
keylogging software Inputlog. The task waspresented as a regular PE process with a fullcohesive text presented segment pair bysegment pair. Time was used as a proxy forcognitive effort in tune with other previous
studies (Lacruz and Shreve 2014; Lacruz, Carland Yamada 2018). The experimental task
included an instrument that included ST-TT
pairs with randomized literal translations fromthe translational section of the corpus, as well as
the most frequent renderings from the nontranslational section of the corpus. The
independent variables were whether the ST-TT
pair was edited (edit_yes, edit_no), the timefrom the moment the translation pair waspresented to the moment the subjects typed any
key (time_to_type), as well as when the postediting of the translation was accepted(time_completion). This last variable is anindication of he c m le i n c m le e edi ing
e en , ha i , c he en g
f edi ing
ac i ni e imilad c i n ni
(Lacruz 2016: 392). This allowed to investigate
he effa cia edheien a i n
(Jak b en 2002),
an la i n n e ime(Vandepitte, Hartsuiker and Van Assche 2015)
when segments are edited or just simply
accepted, as well as the overall time effort whensegments are edited in both conditions.
2and 0.95 for time to completion around a pointnull value of 0. We consider a posterior
di ib i n f a a ame ein hich 95% f
the HDI falls outside the ROPE and a high MPE(i.e., values close to 1) as compelling evidence
for a given effect. Finally, we use Bayes Factors(BF) to compare null and alternative hypotheses
in order to determine under which model theobserved data are more probable.

3

Results

In order to assess whether non-literal translations
are more effortful to process than literal ones
(RQ1)
we
analyzed
time_to_type
andtime_to_completion as a function type oftranslation rendering (Original, Translation) in
separate models. Figure 1 plots time_to_type andtime_to_completion response times for original
and translational types of translation TT
renderings. The models included a by-subject
and by-item random intercepts with a random
slope for type of translation rendering for eachsubject. Table 1 provides summaries of bothmodels. Time_to_type response times did notvary as a function of type of translation rendering( = 0.01, HDI = [ 0.12, 0.14], ROPE = 100,
MPE = 0.58), n did ime
c m le i n ( =0.07, HDI = [ 0.24, 0.09], ROPE = 100, MPE= 0.82). That is, neither model provided
compelling
evidencesuggesting
literaltranslations from the translational corpus weremore effortful to process than non-literal
renderings from the non-translational corpus.

Statistical analysis

We fitted a series of Bayesian linear mixed
effects regression models. These models
examined how reaction time (time_to_type,
time_to_completion) varied as a function of aseries of predictors (see below). For statisticalinferences we report mean posterior point
estimates for each parameter of interest, alongwith the 95% highest density interval (HDI), thepercent of the region of the HDI contained withina region of practical equivalence (ROPE), andthe maximum probability of effect (MPE). We
established a ROPE of ± 0.87 for time_to_type

Page 57

Figure 1: Response times for time to type andtime to completion as a function of type oftranslation rendering (Original, Translation).
Small colored points represent raw data. Large
points summarize the posterior distributions (ingrey) with posterior means ± 95% and 66%
credible intervals.

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

On the contrary, we find the effect of origin oftranslation renderings has a probability of 0.58
(time to type) and 0.82 (time to completion) of
being negative and would not be considereda i icall
ignifican
nde a f e en iframework. The time to type and time tocompletion models both provide strong evidence infavor of an absence of effect of origin of translation
rendering (BF = 0.013; BF= 0.027).

Parameter Estimate
TT Intercept
9.1
Type
0.01

HDI ROPE[8.9, 9.3]
0
[ 0.1, 0.1]
100

MPE
1
0.58

BF
3.2e+101
0.013

TC Intercept
Type[9.4, 9.9]
[ 0.2, 0.1]1
0.82

6.9e+74
0.027

9.69
0.07

0
100

Table 1: Model summary of the posterior
distribution modeling response times (TT: time totype, TC: time to completion) as a function oftype of translation rendering (Type). The table
includes posterior means (Estimate), the 95%
HDI, the percentage of the HDI within the
ROPE, the maximum probability of effect (MPE),
and the Bayes Factor for each parameter.

4

Conclusions

This study started with the question of whether
non-li e al an la i n a e m e diffic l andime c n ming [...]
d ce han li e al ne(Schaeffer & Carl 2014: 55). The specificcontribution of this paper was to use the mostfrequent renderings to express a communicative
purpose from a comparable corpus as tertiumcomparationis. Results suggest that processing
effort of the most frequent ST-TT literaltranslation candidates from the translational
corpus were indistinguishable from the default
translation ones from the non-translational corpus
for both time_to_type and time_to_completion.
The results also show that when subjects decide
not to post edit a proposed translation candidate,
ST-TT li e al an la i n ai inc in highean la i n n e
ime ha a e a i icall
significant.

This study therefore highlights need toincaen i nch a defa l an la i n(i.e. Halverson 2015, 2019) from 4EA
approaches that combine process and productbased perspectives (Halverson 2019: 207).

References
Carl, Michael and Schaeffer, Moritz. (2017a).
Wh T an la i n i Diffic l : A C-basedStudy of Non-literality in Post-editing andFrom- c a ch T an la i n . Hermes 56: 43 57.
Carl, Michael and Schaeffer, Moritz. (2017b).
Mea ing T an la i n Li e ali . In A. L.
Jakobsen and B. Mesa (eds.). Translation inTransition.
Between Cognition, Computing, and Technology,
pp. 81 105. Amsterdam: John Benjamins.
Hal e n, Sand a. (2015). C gni i e
Translation Studies and the merging of
em i ical a adigm : he ca e f li e alan la i n. Translation Spaces 4(2): 310340.
Hal e n, Sand a. 2019. Defa l an la i n: a
construct for cognitive translation st die .
Translation, Cognition & Behavior 2 (2):187 210.
Jiménez-Crespo, Miguel A. and Maribel
Te ced . (2018). Le ical a ia i n, egi e
and explicitation in medical translation: acomparable corpus study of medical
terminology in US websites translated into
S ani h . TIS: Translation and Interpreting
Studies 12 (3): 405 426.
Jak b en, A n L kke. (2002). T an la i ndrafting by professional translators and bytranslation students. In G. Hansen (ed.),Empirical translation studies: Process andproduct, pp. 191 204. Copenhagen:Samfundslitteratur.
Lacruz, Isabel and Gregory Shreve. (2014).
Pa e and c gni i e eff in-editing. In
post-editing of machine translation: Processes
and a lica i n . In

Page 58

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

S. O B ien, M. Sima d, L. S ecia, M. Ca l, & L.
W. Balling (eds.). Expertise in post-editing:
Processes, technology and applications, pp.
246 274. Cambridge: Scholars Publishing.

Schaeffer, Moritz and Michael Carl. (2014).
Mea ing he C gni i e Eff
f Li e al
T an la i n P ce e . Workshop on Humans
and Computer-assisted Translation, pp. 29
37.
Gothenburg, Sweden: Association for
Computational Linguistics.

Lac , I abel. (2017). C gni i e eff intranslation, editing and post-edi ing. In J hn
Schwieter and Aline Ferreira (eds.). Handbookof translation and cognition, pp. 386-401.
Blackwell in Linguistics. Malden, MA: John
Wiley & Sons.
Lacruz, Isabel, Carl, Michael and Masaru Yamada.
(2018). Li e ali and C gni i e Eff : Japaneseand S ani h. In N. Cal la i e al. (ed .), The
LREC 2018 Proceedings: Eleventh International
Conference on Language Resources and
Evaluation, pp. 3818-3821. Paris: European
Language Resources Association.

Vandepitte, Sonia, Hartsuiker, Robert J. and Eva
Van A che. (2015). P ce and e
die fa an la i n
blem . In Aline Fe ei a, andJohn W. Schwieter (eds.), Psycholinguistic andCognitive Inquiries into Translation andInterpreting, pp. 127
143. Amsterdam/Philadelphia: John Benjamins.
Viei a, L ca N ne . (2019). P -Editing of
Machine T an la i n . In Minak O'Hagan (ed.),The Routledge Handbook of Translation andTechnology, pp. 319-335. New York-London:
Routledge.

Page 59

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

FSEM: Current developments and perspectives

Oliver Czulo
Institute for Applied Linguistics and Translatology
University of Leipzig, Leipzig, Germany
czulo@uni-leipzig.de

The Frame-Semantic Evaluation Metric
The Frame-Semantic Evaluation Measure FSEM
(Czulo et al. 2019) is designed to be a fully
automated, referenceless machine translation
evaluation measure for semantic similarity oforiginals and their machine translations. This
contribution addresses some of the currently moreprevalent research questions around FSEM.
In terms of linguistic theory, FSEM is based on
Frame semantics (Fillmore 1982; 1985), a
cognitive-linguistic, ethnographically oriented
semantics of understanding. A frame is a systemof concepts which structures knowledge about
cognitive entities: events, states, objects,
attributes etc. These mental categorization
schemata are linked to linguistic expressions bywhich they are evoked. The word Christmas, forexample, will activate extensive knowledge suchas typical events (e. g. exchanging gifts) andcontexts (e. g. winter) in many speakers ofEnglish. Frames are shaped by backgrounds,
beliefs, experiences etc., where different levels of
abstraction will have group- or individuumspecific configurations. As an example: For someusers of the lexeme Christmas, gifts areexchanged on the evening of December 24th, forothers on the morning of the 25th. Even thoughmany speakers of English may not celebrate
Christmas, the more abstract concept of holiday,

including the fact that the often come with rituals,
will make it possible for them to have a basic
understanding of the concept.
The Berkeley FrameNet (Ruppenhofer et al.
2016) is a lexical-semantic project using empiricalda ade i e a le ic n f f ame , f en n avery general level of abstraction, for English. It
has served as blueprint for various framenets inother languages (e. g. Subirats Rüggeberg and
Petruck 2003; Ohara et al. 2004; Burchardt et al.
2006; Torrent and Ellsworth 2013).
The primacy of frame model of translation(Czulo 2017) provides the translatologicalbackground for FSEM. The basic hypothesis, ormore precisely the idealized model is that for eachframe evoked in the original, there should be a
maximally comparable counterpart in thetranslation that is linguistically realized. Thec n ain ma imall c m a able acc n f
differences in how frames are structured across(sub-)cultures, such as the Family frame, where
the understanding of what types of families exist
and what the relations between family members
are, may differ, but can be compared as there is anunderlying abstract notion of Kinship. There
are, however, a number of factors which may leadto overrides of this principle, such as (potentially)
grammatically induced shifts in perspective:

Page 60

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Tray 1 [...] holds up to 125
sheets [...]

Some systematic patterns of semantic
divergences have already been described in theliterature, such as differences in how motion
events are framed in various types of languages(Talmy 2000; Slobin 2004), and this knowledge
has been applied to the analysis of translation(Ellsworth et al. 2006). The FSEM algorithm canrelate different types of motion framings and thusstate similarity for such cases. It is not intended,however, to test on typicality of framing for alanguage, which is where language models shouldcome into play and tasks could be shared between
different components.

In Fach 1 können bis zu 125
Blatt Papier eingelegt
werden [...]
Into tra 1, p to 125 sheets of paper canbe inserted
German has a considerably lower tendency toagentivize inanimate objects which may have led
to the above-seen shift of the subject Tray 1 to the
prepositional object In Fach 1. The two frames
evoked by the main verbs, Fullness forEnglish and Filling for German, can, howeverbe easily related to one another by means of
exploiting the frame hierarchy in Berkeley
FrameNet: Fillingis inchoative of
Fullness.
FSEM makes use of the Frame hierarchy to
calculate a similarity measure between an original
and a translated sentence. In one of the currently
existing pilot implementations (see Czulo et al.
2019), a spread-activation algorithm makes use ofactivated frames and their subsumed frames tocalculate a similarity measure on the basis of (sofar) manually annotated frames. A penalty isgiven for the distance of frames in the translation
which are linked to, but not identical with theoriginal frames.

Systematic description of shifts
We can think of the frame level as a level ofabstraction which allows us to describe shifts in(lexically and grammatically realized) meaning ina more systematic way than e. g. by means of anindividual lexical comparison. On top of this, asemantic level of description allows us toinvestigate reasons for shifts from various
perspectives, either on the level of semantics itself(e. g. culture-specific differences) or whencombined with other levels of description (such as
typological or pragmatic factors, or contrastive
differences as in the case of the above sentencepair).

It is conceivable that the level of abstractionthat frames provide could help assess other cases
which are not as easily covered by a lexeme-based
approach, such as metaphors. This could covercases where two metaphorical expressions in two
languages are very different in their structure, orwhere a metaphor is translated nonmetaphorically or vice versa (those are indeed
frequent cases, as shown e. g. by Samaniego
Fernández 2013). An evaluation of two
expressions such as pull a leg vs. German
veralbern ( ghl : make me ne l k
id )does not yield a match on a lexical level. If,
however, both metaphorical and nonmetaphorical expression are annotated with thetargeted meaning in terms of frames, a match iscreated where a purely lexical mapping would
fail. This may be true for other types of
expressions such as raining cats and dogs vs.
German schütten
.
Another case in which frame-semantic
annotation can be of help is that of differences in
information distribution. Padó and Erk (2005)
introduced the notions of frame groups,
describing a case where combined causation andscale information in English were split into twolexemes in German. A frame group can then be
defined as an equivalent to a single frame inanother language, covering cases of different
lexicalization strategies.

Page 61Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Challenges for FSEMWith the basic principles laid out, a number ofquestions opens up, of which I can here only
describe some of the currently most prominent.
When applying FrameNet frames, we canassume that the more abstract ones are better
applicable across languages, while the more
specific ones will be more tied to a (sub-)culture.
An automatic method of how to compare frames
is proposed by (Sikos and Padó 2018) using wordembeddings, relying on existing annotations. But
the number and granularity of frames in different
framenets can still vary, and automatic induction
methods are needed to increase coverage.
The evaluation of machine translated texts
cannot, of course, be done by FSEM alone, as thehigh level of abstraction may generate false
positives. Both of the following two sentences(and their literal translations) will generate thesame frame annotation, namely People,
Taking and Containers.
The woman grabbed the cup.
The man took the glass.
This also raises questions on how granular frame
definitions should be, which would require a
deeper discussion than this short contribution canoffer. Assuming that a certain level of abstraction
will always be desirable, FSEM evaluation needs
to be complemented with other methods ofevaluation tackling more fine-grained lexicalinformation.

translation, where more creative translation
strategies may be applied. One phenomenon inhuman translation, which seems unlikely inmachine translations, is that of differences insentence segmentation resulting in such patterns
such as A B. C.
A. B C. By means offrame semantic analysis, such differences in
information distribution might be tackled as well,elevating the metric towards text level rather thanlimiting it to the sentence level. This is a future
goal of evolving FSEM.

References
Burchardt, Aljoscha, Katrin Erk, Anette Frank, Andrea
Kowalski, Sebastian Pado, and Manfred Pinkal.
2006. The SALSA C: A Ge man CRe
ce f Le ical Seman ic . In Proceedingsof LREC 2006, 969 74. Genoa, Italy.
C l , Oli e . 2017. A ec
f a Primacy of Frame
M del f T an la i n . In Empirical Modelling ofTranslation and Interpreting, edited by S. HansenSchirra, Oliver Czulo, and Sascha Hofmann, 465
90. Translation and Multilingual Natural Language
Processing 6. Berlin: Language Science Press.
Czulo, Oliver, Tiago Timponi Torrent, Ely Matos,
Alexandre Diniz da Costa, and Debanjana Kar.
2019. De igning a F ame-Semantic Machine
T an la i n E al a i n Me ic . In Proceedings of
The Second Workshop on Human-Informed
Translation and Interpreting Technology (HiT-IT
2019). Varna, Bulgaria.
Ellsworth, Michael, Kyoko Ohara, Carlos Subirats,
and Th ma Schmid . 2006. F ame-Semantic
Analysis of Motion Scenarios in English, German,
S ani h, and Ja ane e . P e en ed a he F
hInternational
ConferenceonConstruction
Grammar, Tokyo, Japan.
Fillm e, Cha le J. 1982. F ame Seman ic . In
Linguistics in the Morning Calm, edited by Charles
J Fillmore, 111 137. Hanshin.
. 1985. F ame and he Seman ic
fUnde anding . Quaderni Di Semantica 6: 222
254.

Conclusion
The list of benefits of and challenges fordeveloping a frame-semantic machine translation
evaluation measure discussed in this contribution
is certainly incomplete, but it should give a
glimpse into the current state of conceptual
development of FSEM. Various questions stillneed to be answered in order to understand how
informative FSEM can be for the evaluation ofmachine translation, and potentially human

Ohara, Kyoko, Seiko Fuji, Toshio Ohori, Ryoko
Suzuki, Hiroaki Saito, and Shun Ishizaki. 2004.
The Ja ane e F ameNe P jec : An In d c i n .
In Proceedings of the Satellite Workshop B ilding
Lexical Resources from Semantically AnnotatedCorpora , 9 11. European Language Resources
Association.

Page 62

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Pad , Seba ian, and Ka in E k. 2005. T Cause or
Not to Cause: Cross-Lingual Semantic Matchingf Pa a h a e M delling . In Proceedings of the
Cross-Language Knowledge Induction Workshop.
Cluj- Napoca, Romania.
Ruppenhofer, Josef, Michael Ellsworth, Miriam
R. L. Petruck, Christopher R. Johnson, Collin F. Baker,
and Jan Scheffc k. 2016. F ameNe : Theand
P ac ice . h ://f amene 2.ic i.be kele .ed /d c /r1.7/book.pdf.
Samanieg Fe n nde , E a. 2013. The Im ac of
Cognitive Linguistics on Descriptive Translation
Studies: Novel Metaphors in English-SpanishNe
a e T an la i n a a Ca e in P in . In
Cognitive Linguistics and Translation: Advances inSome Theoretical Models and Applications, editedby Ana Rojo and Iraide Ibarretxe-Antuñano, 159
98.
Applications of Cognitive Linguistics 23. Berlin: De
Gruyter Mouton.

Sl bin, Dan I. 2004. The Man Wa
Sea ch f a
Frog: Linguistic Typology and the Expression of
Motion E en . In Relating Events in Narrative:
Typological Perspectives, edited by S. Strömqvist
and L. Verhoeven, 219 57. Mahwah, N.J.:
Lawrence Erlbaum Associates.
http://ihd.berkeley.edu/linguistictypolog
yofmotionevents.pdf.
Subirats Rüggeberg, Carlos, and Miriam Petruck.
2003.
S
i e: S ani h F ameNe !
In
Proceedings of the Workshop on Frame Semantics,
edi ed b E a Haji
, Anna K
c , and
Ji Mi
k . P ag e: Ma fe .
Talmy, Leonard. 2000. Toward a Cognitive Semantics.
2. Typology and Process in Concept Structuring.
Camb idge, Ma .: Camb idge, Ma . [ .a.] : MIT
Press.
Torrent, Tiago Timponi, and Michael Ellsworth. 2013.
Behind he Label : C i e ia f
Defining
Anal ical Ca eg ie
in F ameNe B a il .
Veredas 17: 44 65.

Sikos, Jennifer, and Sebastian Padó. 2018. U ing
Embeddings to Compare Framenet Frames acrossLang age . In Proceedings of the First Workshop
on Linguistic Resources for Natural Language
Processing,
91 101.

Page 63

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Going off track? Activity tracking in translation.
T a lae ce i , e hical i e a dii e e.

Dr Valentina Ragni

Dr Lucas Nunes Vieira

University of Bristol

University of Bristol

School of Modern Languages

School of Modern Languages

19, Woodland Road, BS8 1TE, UK

19, Woodland Road, BS8 1TE, UK

v.ragni@bristol.ac.uk

l.nunesvieria@bristol.ac.uk

Abstract

be misused as instruments of surveillance, whichraises questions and warrants a wider discussionof the echn l g use implications. In this paper,
we report data from a longitudinal study thate aminean lae and e ce i nfactivity-tracking.

In an increasingly automated translation industry
where recent machine translation (MT)developments are producing higher quality output(Way, 2018) with higher potential to speed up thetranslation process productivity is often deemed
imperative by clients and translators alike (e.g.
Marg, 2016; Tabor, 2010; Vallianatou, 2020).
Activity-tracking tools are used to measureproductivity by LSPs, translation vendors, as well
as freelance translators (see Moran, 2018). These
tools can collect a wealth of information about thetranslation process, from average speed or total
words translated for a client, to fine-grained byegmen da a abhe an laing andediting activity. However, the practice iscontroversial,andcurrently
completely
unregulated. Tracking translation activity canhelp to diagnose problems in the translation
process (e.g. below-average MT quality) and may
also provide translators with insights into theirworking patterns (Vieira, 2018). Depending on
the circumstances, however, these tools can also
1

The study was reviewed by the Faculty of Arts
E hic C mmi ee a he a hin i i n. F
ethical and confidentiality reasons, translators did notParticipants used activity-tracking tools within
their CAT software (memoQ and Trados Studio)
to calculate their productivity over a period of 16
weeks. Every week, they sent us quantitative selfreported project data such as translation speed andnumber of edited words,1 as well as qualitative
perception data related to productivity selfassessments via rating scales and open questions.
This combination of qualitative and quantitative
data was then complemented by focus-groupinterviews with all participants.
This presentation focuses on the weekly datacollection process, which provides valuable
insights into the work (and related productivity)of professional translators in real settings over an
extended period. Moreover, several relevant
share any information that could identify the texts
they worked with or the clients. They were remindedthat this was strictly prohibited.

Page 64

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

themes emerged from the analysis, including
issues related to tool-specific feedback,productivity definition and calculation issues, andethical implications. Most notably, it emerged thattranslators are not averse to activity-tracking
technology per se. In fact, most translators in ourstudy recognized as tantamount the importance of
kee ing ack f ne
d ci ihiling
CAT tools and related technologies. A coredifference in translator attitudes was noted
be eenn ane lacking nenproductivity vs. being required to do so by a clientor company owner, where only the latter was
highlighted as potentially problematic. It became
clear that issues with activity tracking often
revolve around the client-translator relationship,especially in cases of power imbalances andmarket structures that risk compromising
an laagenc . The e en a i n ill add e
the above-mentioned themes and discussan lae ce i nfd c i i . S me
tentative recommendations on the practice ofactivity-tracking will also be put forward, in order
to rai e a a ene
f he echn l g
implications and highlight situations wheretranslators deemed it to be useful and worthwhile.

Resources and Evaluation (LREC'16) (pp. 2326). P, Sl enia. 23-28 May 2016.

John Moran. 2018. Extensive CAT Tool Logging Big
Brother or language technology evaluation
panacea? In: ITI Research Network e-book 2018.
The Human and the Machine (p. 3). Institute ofTranslation and Interpreting.
Jared Tabor. 2010. Productivity for Translators: an
Overview. Proz.co Wiki. Available from:https://wiki.proz.com/wiki/index.php/Productivity_
for_translators:_an_overview [accessed June 2020]
Fotini Vallianatou. 2020. CAT Tools and Productivity:
Tracking
Words
andHours.
TranslatonDirectory.com. Available from:https://www.translationdirectory.com/article752.ht
m [accessed June 2020]
Lucas Nunes Vieira. 2018. Human Challenges in the
Use of Machine Translation in Professional
Translation Processes. In: ITI Research Network
e-book 2018. The Human and the Machine (p.
7). Institute of Translation and Interpreting.
Andy Way. 2018. Quality Expectations of MachineTranslation. In: J. Moorkens, S. Castilho, F.
Gaspari, and S. Doherty (Eds.), Translation

Quality Assessment. From Principles toPractice (pp. 159-178). Springer International
Publishing.

References
Lena Marg. 2016. The Trials and Tribulations ofPredicting Machine Translation Post-Editing
Productivity. Proceedings of the Tenth(498 words without references; 662 words includingreferences)

International Conference on Language

Page 65

Book of Abstracts: Translation in Transition (TT5)October 15 - 17, 2020

Researching Automatic Speech Recognition on the Process andQuality of Written Translation: Discussions of Indicators

Wenchao Su

Xiaoxing Zhao

Guangdong University of Foreign Studies

Sun Yat-sen University

suwenchao0617@126.com

zhaoxiaoxingsysu@163.com

Abstract

support the use of ASR in written translation islimited, and more systematic research is needed to
examine the function of ASR in the process andproduct of written translation on various language
pairs using a variety of ASR tools (Ciobanu &Secara 2019).

Automatic Speech Recognition (ASR) has beenused by professional translators for some years.
Instead of typing the translations, professionaltranslators prefer to speak out their translations,for they find it faster to produce high-quality
texts in this way (Bower 2002) .With
technologies maturing, more and more
technological tools have been playing a central
role in the practice of translators andinterpreters in recent years. Technologicalcompetence is now regarded as one of theessential components that define a qualified
professional translator or interpreter. ASR is
one of the technologies that deserve theattention of translators and interpreters.
Some studies have assessed the performance ofdifferent ASR systems (Zapata & Kirkedal2015), and other studies have investigated theapplication of ASR to simultaneous interpreting(Li & Wang 2018), post editing (Zapata et al.,2017) and written translation (Carl et al. 2016;
Dragsted et al. 2011; Baxter 2017). However,
the empirical data that

This study explores the application of ASR tool in
written translation from English to Chinese. Usinga within-subjects design, a group of undergraduateden
i h ne ea
f mal aining intranslation were invited to complete two tasks,
written translation with ASR and writtentranslation from scratch. Screen recording wasused to record the real-time process of writtentranslation. The data extracted from the screen
recording results, including the overall production
time, the time spent on pre-drafting, drafting andpost-drafting were used as the indicators of speed.
The number of changes in content (i.e.,
replacement, addition, and omission) and
expressions (i.e., punctuations, spellings, lexicalchanges, syntactic changes and textual changes)
were adopted as the indicators of revision

Page 66

Book of Abstracts: Translation in Transition (TT5)behaviors. Translation outputs were analyzed
and scored in terms of accuracy and target
language quality to suggest the ASR effect on
the quality of written translation.

References
Baxter, R.N. （2017）. Exploring the effects ofcomputerized sight translation on writtentranslation speed and quality. Perspectives,25(4):622-639.
Bower, L. (2002). Computer-aided Translation
Technology (p.34). Ottawa: University ofOttawa Press.
Carl, M., Aizawa, A., Yamada, M. (2016). Englishto-Japanese Translation vs. Dictation vs. Postediting: Comparing Translation Modes in a
Multilingual Setting. 10th Language Resources
and Evaluation Conference (LREC 2016).
P, Sl enia, May 2016.
Ciobanu, D., & Secara, A. (2019). Speech
recognition and synthesis technologies in the
an la i n
kfl . In M. O Hagan (Ed.), The
Routledge Handbook of Translation andTechnology (pp. 91-106). New York:
Routledge.

October 15 - 17, 2020

Dragsted, B., Mees, I.M., &Hansen, I.G. (2011).
S eakingan la i n: S den fi enc n e
with speech recognition technology. Translation &Interpreting, 3(1):10-28.
Zapata, J., & Kirkedal, A.S. (2015). Assessing the
Performance of Automatic Speech Recognition
System When Used by Native and Non-Native
Speakers of Three Major Languages in Dictation
Workflows. Proceedings of the 20th Nordic
Conference of Computational Linguistics (pp. 201210). Vilnius, Lithuania.
Zapata, J., Castilho, S., & Moorkens, J. (2017).
Translation Dictation vs. Post- editing with Cloudbased Voice Recognition: A Pilot Experiment.
Proceedings of MT Summit XVI, Vol2: Users andTranslators Track (pp. 123-136). Nagoya, Japan.
Li, X.L. & Wang, M. J. (2018). Construction andResearch of the Teaching Model of Using Automatic
Speech Recognition APP in SimultaneousInterpreting Training Course A Case Study ofVoice Note as an Auxiliary Tool. Technology
Enhanced Foreign Language Education, 179: 12-18.

Page 67