89

## Translation-Memory (TM) Research: What Do We Know and How Do We Know It?

## Abstract

It is no exaggeration to say that the advent of translation-memory (TM) systems in the translation profession has led to  drastic  changes in translators' processes and workfl  ow, and yet, though many professional translators nowadays depend on some form of TM system, this has not been the object of much research. Our paper attempts to fi  nd out what we know about the nature, applications and infl  uences of TM technology, including translators' interaction with TMs, and also how we know it. An essential part of the analysis is based on a selection of empirical TM studies, which we assume to be representative of the research fi  eld as a whole. Our analysis suggests that, while considerable knowledge is available about the technical side of TMs, more research is needed to understand how translators interact with TM technology and how TMs infl  uence translators' cognitive translation processes.

## 1. Introduction

The most widely used computer-assisted translation (CAT) tool for professional translation of specialised texts is translation-memory (TM) technology, which was fi  rst commercialised in the mid-1990s and has been widely used since the late 1990s (Bowker/Barlow 2008: 2; Garcia 2007: 56; O'Hagan 2009: 48; Somers 2003: 31). A TM is basically a database of segmented and paired source and target texts that translators can access in order to re-use previous translations while translating new texts (Christensen 2003: 183; Somers 2003: 31f). TM technology is now used in more jobs, for more languages, and by more clients and translators than ever before (LISA 2002, 2004; Lagoudaki 2006; Wheatley 2003). A clear indication of this is Gouadec's (2007: 151ff) fi  nding that, in 95% of the more than 430 job advertisements for professional translators that he surveyed, TM skills were mentioned as a prerequisite. The overwhelming argument for embracing TM technology appears to be the benefi  ts that the system can bring in terms of productivity, cost savings and quality of the translation output (Lagoudaki 2006; O'Hagan 2009: 48; Wheatley 2003). Indeed, it is no exaggeration to say that the advent of TM systems in the translation profession has led to drastic changes in translators' processes and workfl  ow (e.g. Garcia 2007: 56), and yet, though many professional translators nowadays depend on some form of TM system, neither CAT in general nor TM-assisted translation in particular has been the object of much research.

According to Dillon/Fraser (2006: 68), TM literature focuses on few and rather business-oriented aspects. Thus, for instance, translators' professional journals tend to publish TM-product reviews (see Benis 2004a and 2004b in the ITI Bulletin ; Davies 2004 in Communicator ) or discussions of general business issues such as rates and copyright on the translations (see EnriquezVeintimilla 2006 in the ITI Bulletin , for instance), whereas CAT journals tend to focus on more practical issues within workfl  ow management (see Guerberof 2008, Iverson 2003 and Levitt 2003 in Multilingual Computing and Technology , for instance) or on comparisons of different TM systems (see Hallett 2006 in Localisation Focus and Wa β mer 2005 and Zerfa β 2002 in Multilingual Computing and Technology ). Little research has been carried out on how translators interact with

* Tina Paulsen Christensen & Anne Schjoldager Department of Language and Business Communication Aarhus School of Business, Aarhus University Fuglesangs Allé 4 DK-8210 Aarhus V tpc@asb.dk - asc@asb.dk

TM and how TM systems affect the cognitive (internal) translation process, and very few studies of TM are empirical investigations.

Inspired by Mossop's (2007) review of empirical revision research, our paper attempts to fi  nd out what we know about the nature, applications and infl  uences of TM, including translators' interaction with the technology, and also how we know it. An essential part of this analysis is based on a small collection of rather disparate empirical TM studies, which we assume to be representative of the research fi  eld as a whole. Section 2 presents a brief history of TM technology, whereas section 3 discusses some basic defi  nitions and applications. Section 4 reviews nine empirical TM studies and explains our criteria for selecting these studies. Section 5 concludes our paper with a summary of what we have found as well as an attempt to suggest where and how empirical TM research could proceed.

## 2. A brief history

The conception of TM technology may be traced back some 30 years. In a paper on the origins and basic components of the professional translator's workbench, Hutchins (1998: 294) attributes the original idea for TM to Arthern (1979) in a paper on the potential use of computer-based terminology systems in the European Commission. Pointing out that translators working for the European Commission were wasting valuable time by retranslating (parts of) texts that had already been translated, Arthern suggested the compilation of a computerised storage of source and target texts that could easily be accessed by translators for re-use in current translations. Arthern (1979: 94) referred to this mode of translation as 'translation by text-retrieval'. Another monumental idea was conceived by Kay and described in a 1980 paper entitled 'The proper place of Men and Machines in Language Translation' (Kay 1980/1997). Unlike many of his contemporaries, who still assumed that machine translation (MT), i.e. fully automated translation, would soon be a feasible alternative to human translation, Kay argued that human translators should stay in control of the translation process and that the way forward was to develop computerised tools that could assist human translators in their work. Establishing a network of terminals linked to a mainframe computer, Kay's basic idea was to develop and add specifi  c translation tools to existing textprocessing technology. These tools should include various means for translators to keep track of earlier decisions and to access previous translations of the same or similar source-text passages, for instance (see Hutchins 1998: 295ff). According to Hutchins (1998: 297), another fundamental idea for the development of TM technology was proposed by Melby (1981) when he suggested computer-generated bilingual concordances as a tool for translators and documented that such a tool would enable translators to identify text segments with potential translation equivalents in relevant contexts.

Hutchins (1998) mentions other important steps in the development towards modern TM technology. For instance, in the late 1980s, the Automated Language Processing System (ALPS) offered on its Multilingual Word Processor a simultaneous display of source and target texts combined with a facility for 'repetitions extraction' (Hutchins 1998: 300). Another important step towards modern TM technology was Harris' (1988) bi-text conception. In continuance of Melby's (1981) bilingual concordance, a bi-text links source and target texts and, in connection with a new translation project, it facilitates an electronic search for and a retrieval of previous translations (Hutchins 1998: 301).

The realisation of commercial TM systems was fi  rst made possible when tools for text alignment made bilingual databases of translations possible. Based on these ideas and technical developments, four commercial TM systems appeared on the market in the early 1990s: The TranslationManager from IBM, the Transit system from Star, the Eurolang Optimizer and the Translator's Workbench from Trados (Hutchins 1998: 303). Since the appearance of the fi  rst TM systems on the market, the dissemination of this particular technology has kept on growing (Christensen 2003: 190).

## 3. Defi  nitions and applications

Computer-assisted translation (CAT) covers human-aided machine translation (HAMT) and machine-aided human translation (MAHT). In HAMT, translation is essentially carried out by the program itself, but humans are required to resolve specifi  c language problems arising from the source text or to correct the resulting target text. In MAHT, translation is carried out by a human translator, but computer assistance is seen as an integral part of the process.

TM technology is the computer tool that is most widely used by individual translators, translation agencies and other organisations involved in translation (Garcia 2007: 56; O'Hagan 2009: 48; Somers 2003: 31). While TM systems may differ in internal processes that govern segmentation, alignment, indexing, searching and match retrieval, they all share the basic function of deploying existing translation resources in a new translation project. In some cases, this is combined with other types of software such as word processors or terminology databases (Alcina 2008: 96ff).

As already mentioned, a TM is basically a database of segmented and paired source and target texts that the translator can access in order to re-use previous segments while translating. The TM continuously provides the translator with so-called matches, which are translation proposals stored in its database. TM systems operate with three kinds of matches: exact, fuzzy and no matches. The identifi  cation of these matches relies on an automatic comparison of character chains. Exact matches are found if the character chain of a new source-text segment is identical to that of a stored source-text segment; fuzzy matches are found if source-text segments are merely identifi  ed as similar; and no matches are found if no source-text segment is identifi  ed as (suffi  ciently) similar. As far as we are aware, the thresholds between these matches tend to be preset by the TM programme, but may also be set by individual translators. Most TM systems defi  ne an exact match as a 100% correspondence between source-text segments; a fuzzy match is defi  ned as a 70-99% correspondence; and no matches are found if the correspondence is below a 70% threshold, in which case the target-text segment is left empty. It is worth pointing out that the use of character chains for match identifi  cation is an entirely form-based process, which does not consider semantic, pragmatic or contextual aspects.

TM segments tend to be sentences or sentence-like units such as titles, headings, list items and table cells. Some researchers discuss the negative consequences of this and argue that TM segmentation below the sentence level may be more useful. Thus, for instance, Schäler (2001) draws attention to a possible relation between TM segmentation and translator productivity, Dragsted (2004, 2006) studies the relation between sentence-based segmentation and the translator's cognitive process, and Colominas (2008) experiments with chunk-based TM systems (see also sections 4.3 and 4.9, below).

Translators may engage with TM technology in different ways. In most commercial systems, the translator works with an interface that is characterised as interactive. This means that sourcetext segments are presented one at a time, giving the translator the option to translate this segment, or, if an exact or a fuzzy match is retrieved from the TM, to accept, revise or reject the previous translation of this segment (Gouadec 2007: 274). Translators may also engage with TM technology using a so-called pre-translation mode. This means that TM technology is applied to the source text prior to its translation, resulting in a hybrid output of exact and fuzzy matches from the TM plus some empty segments. Using a pre-translation mode, the translator concentrates on translating the empty segments and editing the target-text segments that have been suggested by the TM. Rather recently, TM technology also facilitates what Garcia (2007: 58ff) refers to as a web-interactive translation mode, in which translators interact with an online (server-based) TM. For an overview of some current TM systems, see Lagoudaki (2006: 18). For a detailed description of various aspects of TM technology, see Reinke (2004).

## 4. Empirical TM studies

As mentioned in the introduction, the aim of our paper is to fi  nd out what we know about the nature, applications and infl  uences of TM technology and also how we know it. We shall now concentrate on fi  nding out what has been documented empirically by studying a collection of empirical TM studies. These were found by means of the bibliographical references of the TM literature that we have been able to get hold of and via a search for publications that contain CAT, TM or their derivatives in titles, abstracts or keywords in online bibliographical databases such as the Translation Studies Bibliography , the Translation Studies Abstract , the Bibliography of Translation Studies and the Modern Language Association (MLA) Bibliography. Since TM technology was not common until the late 1990s, we assume that relevant publications did not appear until the turn of the millennium. Apart from this, we selected publications that fulfi  l the following three criteria.

(1) The publication is a research report . As pointed out by Vandepitte (2008: 574), following and adjusting Holmes' (1972/2000) map of Translation Studies, research is a 'knowledge-oriented type of study' aimed at describing, explaining and/or predicting a given phenomenon. By way of operationalising this criterion, we shall focus on reports that were published within the academic world, i.e. primarily aimed at academics as opposed to being aimed at the profession (see also Williams/Chesterman 2002). Consequently, though literature on TM technology contains an abundance of texts that convey much interesting and useful information about our topic, few of them are seen as research reports in this academic sense. Thus, for instance, though the Translation Memory Surveys that were carried out for the Localization Industry Standards Association (LISA 2002 and 2004) are both well-founded, well presented and have other academic virtues (which some of our selected studies may be said to lack), these surveys cannot be considered for this part of our paper because they were carried out for a professional association and aimed at the profession.

( 2) The topic of the publication is TM research . Only those studies that focus on TM technology are considered for this part of our paper. Examples of studies that are excluded from our list below are Mossop (2006), Cruz-Lara et al. (2008) and Gauton (2008), who deal with various aspects of CAT, but do not focus on TM technology as such.

(3) The method is empirical . By defi  nition, empirical papers analyse and discuss data. In empirical TM research, relevant data appear to be source texts, target texts, surveys and translation processes (both internal and external). While Bowker/Barlow (2008) make an interesting evaluative comparison between bilingual concordances and TM systems, their study is theoretical and is therefore excluded from our list below.

As a result of our search for empirical TM studies that were published in 2000 or later, we came up with the following list of nine studies, presented in chronological order (some of them reported in more than one publication).

- Lange/Bennett (2000) 1.
- Christensen (2003) 2.
- Dragsted (2004) and (2006) 3.
- Fulford/Granell-Zafra (2005) 4.
- Dillon/Fraser (2006) 5.
- Lagoudaki (2006) 6.
- Mandreoli et al. (2006) 7.
- O'Brien (2006) and (2008) 8.
- Colominas (2008) 9.

In sections 4.1-4.9, below, we shall review these nine studies with a view to discovering their aims, methods, data and fi  ndings.

## 4.1.  Lange/Bennett (2000)

Using a six-month professional project carried out at Baan Company as a case study, Lange/Bennett (2000) investigate the drawbacks and benefi  ts of combining MT and TM. According to one of its websites, Baan develops automated software solutions for manufacturing and business systems, including 'multilingual support with translations of its software in more than 20 languages'. The aim of the professional project was to investigate how computer technology may usefully assist the translation of online help texts in order to 'produce high-quality results in shorter times' (2000: 203) with a view to reducing 'throughput time for translation by 50 percent through the use of automated translation' (2000: 204). The project was carried out in four phases:

- (1)  analysis of MT,
- (2)  analysis of a combination of MT and TM,
- (3)  analysis and readjustment of the translation workfl  ow and
- (4)  enhancement of the translation output.

Though this is not quite clear from the paper itself, Lange/Bennett seem to have employed a mixture of evaluative and descriptive (quantitative) methods.

By way of conclusion, the authors observe that translators' productivity may indeed increase if TM and MT are combined, but perhaps their most interesting observation is this: productivity will only increase in this way if translators are comfortable with their new role as post-editors of machine-controlled translations. One of Lange/Bennett's (2000: 209) analyses even suggests that translators' post-editing process may take longer than human translation if the translator involved is negatively disposed towards MT. These observations sound plausible and appear to be well-founded, but Lange/Bennett's paper lacks signifi  cant details about the context and analytical methods of the case study (such as how many translators were involved), which we need to assess the validity of their fi  ndings.

## 4.2.  Christensen (2003)

Christensen's (2003) dissertation on TM as a tool for legal translation is based on the assumption that, because of the complexity and culture specifi  city of legal communication, TM technology will be less useful for legal translation than for technical translation, for which it was fi  rst designed. The basic aim of the dissertation is therefore to evaluate the usefulness of a TM for legal translation. Christensen selects and combines relevant theories and methods within legal studies (jurisprudence), translation studies, computer science and text linguistics. Her research questions are answered by means of theoretical and empirical analyses. She asks three general research questions:

- (1)  Is TM technology helpful in connection with those translation problems that typically arise in legal translation?
- (2)  How useful is TM technology for legal translation in general?
- (3)  Is it necessary to combine a TM (parallel corpus) with a reference corpus of authentic target-language texts that serve similar functions?

Christensen's theoretical fi  ndings are that, as it enables translators to access and re-cycle parts of a potentially unlimited number of previous translations within the same genre, TM technology will undoubtedly be of some use to legal translators. However, she also fi  nds that TM technology is not well suited for solving those problems that typically arise in legal translation. According to Christensen, the usefulness of TM technology for legal translation should improve considerably if it is combined with a corpus of authentic, functionally equivalent target-language segments. TM technology should also become more useful for legal translation if it includes an authoring mem- ory that standardises the source-text segments by way of pre-editing before the alignment with target-text segments (fi  rst suggested by Allen 1999, among others).

Christensen's empirical data derive from two corpora. One corpus comprises a TM of sourcetext segments from the Danish articles of 11 companies ('vedtægter') 1 and their authentic German translations (2003: 77). Another corpus comprises a reference corpus of model articles ('Musterverträge'), which are taken from a standard German handbook on German company law 2 , and which are assessed as functionally equivalent to a selection of source-text segments in the TM (2003: 89). In the empirical part of the dissertation, Christensen asks four specifi  c questions:

- (1)  Is the wording of Danish company articles suffi  ciently repetitive - in connection with certain standardised legal formulations that refer to specifi  c legal speech acts, for instance - to be useful as source texts in a TM?
- (2)  Can a TM identify segments that are functionally identical?
- (3)  How  can  the  process  of  fi  nding  matches  to  these  source-text  segments  be improved?
- (4)  Can the translation units in the TM be regarded as functionally equivalent to the segments found in the German model articles?

As far as Christensen's empirical study is concerned, her fi  ndings are as follows: The wording of the Danish articles does not show suffi  cient repetitiveness. The TM consisting of Danish articles and their translations fails to identify suffi  cient matches of source-text segments that are functionally identical. The identifi  cation of such matches may indeed be improved by means of an authoring memory, as suggested in her theoretical part. The translation units in the TM cannot be regarded as functionally equivalent, which means that the quality of legal translations may, to some degree, be improved if the translator has access to functionally identical segments in an authentic target-language (reference) corpus.

As she points out herself, Christensen's results may not be valid for all kinds of legal translation (2003: 299f), but her fi  ndings and suggestions are certainly relevant for the application of TM technology in general and defi  nitely deserve further empirical investigation.

## 4.3.  Dragsted (2004) and (2006)

Dragsted (2004) is a PhD dissertation on segmentation in human translation and TM systems. Based on data from her PhD dissertation, Dragsted (2006) investigates further how the enforced sentence-based segmentation of the TM system affects the translators' cognitive (internal) process. We shall regard this research as one single study. Dragsted asks three research questions:

- (1)  How do translators segment text naturally?
- (2)  How does the integration of TM systems (with enforced sentence-level segmentation) affect the cognitive translation process?
- (3)  How  can  TM  systems  be  optimised  to  conform  better  to  translators'  natural segmentation and to render higher match values?

1 According to the law of most countries, including Denmark and Germany, the articles of a company govern the relationship between shareholders and directors, forming an important part of the constitution of that company. This document is referred to as the 'articles of association' in Britian and the 'articles of incorporation' in the US.

2 Christensen's source for the reference corpus is Münchener Vertragshandbuch (1996). By way of explanation, we could mention that the German model articles of this handbook appear to play a similar role to that of the model articles included in the UK Companies Act 2006. Referred to as 'Table A', the UK model articles are used by companies as a basis for their own articles or apply by default if companies do not register their own articles.

The study comprises data from several experiments involving six professional translators (with at least two years of professional experience) and six students (in their fi  nal year of MA studies of specialised translation at the Copenhagen Business School). Two kinds of experiments were carried out. In one experiment, Dragsted studied human translation, which was recorded by means of key-stroke logging (Translog). In another experiment, she studied TM-assisted translation (Trados). In addition, Dragsted draws on retrospective verbalisations about the translators' own understanding of their choices and the text that they were given to translate. In her analyses, Dragsted focuses on the translators' revision time, the extent to which the source-text sentence structure is changed and the way in which translators appear to segment the source text. Pauses recorded by means of key-stroke logging are seen as indicators of the translators' segmentation.

Dragsted's fi  ndings are as follows. The sentence does not constitute a central unit in translators' cognitive segmentation, though this may be truer for professional translators than for students 3 . In both groups, there were detectable changes in the translators' production and revision time, in the way that they paused during the task (segmentation behaviour) and in the way that they tended to structure the target-text sentences. Professional translators were much more aware of the effects of the TM on their translation process. Revision behaviour was affected in both groups, but professionals spent relatively more time on revising, compared with what they did during human translation (2006: 460). TM systems could be modifi  ed to re-use translations of sentence fragments by integrating linguistic analysis (syntactical and morphological parsing) and possibly also by semantically based techniques (2004: 302). Consequently, according to Dragsted's fi  ndings, TM systems will need to be adjusted to suit translators' natural segmentation processes.

## 4.4.  Fulford/Granell-Zafra (2005)

Fulford/Granell-Zafra (2005) study freelance translators' uptake of information and communication technologies, including TM systems. In their paper, they report on the fi  ndings of the fi  rst phase of a research project set up to investigate the adoption (in late 2003) of information and communication technologies by UK freelance translators. The study adopts an exploratory questionnaire survey approach. The data of the study derive from responses from 591 UK freelancers. The most interesting result is that TM technology and other CAT tools are less widely used than we might expect: just under half of the respondents say that they are not really familiar with CAT tools at all, and only 28% of respondents state that they actually use TMs and other CAT tools. The survey also found that TM users tend to be specialised in the translation of technical and scientifi  c texts (2005: 4ff). The main conclusion is that the uptake of general-purpose software applications - such as word processing software, graphical software and desktop publishing software is more common among UK freelancers than the uptake of special-purpose software applications - such as TM and terminology management tools.

## 4.5.  Dillon/Fraser (2006)

Dillon/Fraser (2006) provide a snapshot of UK-based professional translators' personal views on TM technology. Reporting on one of very few studies that focus on the translators' perspective, the paper sets up three hypotheses:

- (1)  Novice translators have more positive perceptions of TM technology than more experienced translators.
- (2)  Translators with TM experience have more positive perceptions of the technology than translators without such experience.
- (3)  Translators' perceived IT profi  ciency is not the main infl  uence on their perception

3 While Dragsted's professional subjects complain that the sentence-by-sentence segmentation enforced by Trados 'has a constraining effect on their cognitive behaviour and mental representation of the text and thus changes the translation task', her student subjects are rather unaware of this constraint (2004: 302).

## of TM technology.

Data are derived from an online questionnaire survey that was carried out in August 2004. The authors received 59 usable responses (2006: 71). Inspired by Moore/Benbasat's (2001) instrument for measuring personal perceptions, Dillon/Fraser asked translators to respond to 24 statements that expressed various attitudes towards TM technology - such as, 'I would lose out on work if I did not have TM software' or 'The disadvantages of TM far outweigh the advantages'. According to their fi  ndings, the fi  rst two hypotheses may be true. Thus, newly qualifi  ed translators and translators with TM experience seem to be more positive towards TM technology than others. The third hypothesis appears to be falsifi  ed, as translators with strong IT skills also appear to be more likely to have positive perceptions of TM technology. These fi  ndings lead the authors to suggest that a lack of understanding and knowledge of TM technology and its possibilities - rather than the nature and applications of TM technology itself - may be an important reason why some translators reject it altogether.

## 4.6.  Lagoudaki (2006)

Like Fulford/Granell-Zafra (2005), Lagoudaki (2006) reports on a survey of the adoption of TM technology and, like Dillon/Fraser (2006), she studies users' attitudes towards TM technology. By means of an online questionnaire, Lagoudaki obtained responses from 699 translation professionals (translators, terminologists, project managers, reviewers, subtitlers, etc) from 54 countries. Unlike in Fulford/Granell-Zafra's (2005) survey, the adoption of TM technology appears to be considerable: the percentage of respondents using a TM system is 82.5%. In line with Fulford/ Granell-Zafra's (2005) fi  ndings, Lagoudaki asserts that those who specialise in technical texts are more likely to use TM tools, followed by those who specialise in fi  nancial and marketing content. Those who report a legal specialisation are also likely to use TM tools, but less so than the above-mentioned groups, which concurs with Christensen's (2003) fi  ndings. In line with Christensen's expectations, Lagoudaki fi  nds a relationship between high levels of textual repetition and high adoption of TM technology. Unlike Dillon/Fraser (2006), Lagoudaki does not fi  nd any striking difference in TM adoption between different age groups and between those with and without work experience. However, like in Dillon/Fraser's study, Lagoudaki fi  nds that high IT profi  ciency is linked with high adoption of TM technology. When asked why they use a TM, most respondents answer that it saves time (86%), that it ensures consistency in terminology (83%), and that it improves quality (70%). Other benefi  ts are cost savings (34%) and the effi  cient exchange of resources such as glossaries and TM databases (31%). A rather surprising result is that, though they own a TM tool, some respondents (16%) have not been able to learn how to use it yet.

## 4.7.  Mandreoli et al. (2006)

Mandreoli et al. (2006) evaluate the design of their own TM, namely the Example-based TRanslation Assistant (EXTRA). In comparison with most TM systems relying on artifi  cial intelligence, the search engine of the EXTRA system is founded on advanced information retrieval techniques executing two processes: fi  rst, the document to be translated undergoes a syntactic analysis, and then it is compared with the TM data using so-called edit distance. This procedure is applied in order to ensure a good trade-off between the effectiveness of the results and the effi  ciency of the processes involved (2006: 169). The aim of the paper is to present, analyse and test the effectiveness and effi  ciency of the system (2006: 169f). The investigation is based on theory and research on Example Based Machine Translation (EBMT). Data derive from statistical simulation experiments (2006: 167), which document that EXTRA is able to support effectively and effi  ciently the translation of texts in western languages (2006: 194). Based on a test run of the EXTRA system, the authors conclude that their 'results show the goodness of EXTRA in retrieving useful suggestions and of the two processes constituting it, suggesting the suitability of their stand-alone employment also in contexts that are not strictly related to translation' (2006: 194). We may not fully appreciate the technical intricacies of their investigation, but the authors are certainly right in asserting that CAT in general and TM in particular may currently 'represent one of the most promising translation paradigms' (2006: 168).

## 4.8.  O'Brien (2006) and (2008)

Investigating translators' cognitive load in connection with various TM match types, O'Brien's is one of very few TM studies of translators' interaction with TM tools. In her 2006 study, O'Brien sets out to investigate whether eye-tracking in general is a useful research methodology for investigating translators' interaction with TM, and, in order to investigate this generic question, whether eye-tracking makes it possible to identify differences in cognitive effort with different TM match types. The study is an experimental pilot study involving four professional translators, who translated a text using a TM (SDL Trados Translator's Workbench) and then commented on their translation process in retrospective verbalisations. The study investigates the cognitive effort required from translators for different TM match types by analysing quantitative data from eye-tracking (provided by Tobii 1750) supplemented with qualitative screen-capture data (provided by Camtasia) and with subjects' retrospective verbalisations about what they were doing during the translation task (2006: 189). The cognitive effort is measured using processing speed (words per second) and pupil dilation. The analysis is performed as a comparison of processing speed for each match type and the percentage change in pupil dilation for each match type, supplemented by the retrospective verbalisations. The fi  ndings suggest a strong correlation between percentage change in pupil dilation and processing speed for different match types. The study also shows that exact matches exert the least cognitive load on translators, while no matches exert the greatest load. Furthermore, though the relationship is not a linear one, it is demonstrated that the cognitive load increases as fuzzy-match values decrease (2006: 199ff). As a general conclusion, O'Brien suggests that her method of eye-tracking in combination with retrospective protocols is well suited for translation process research (2006: 200).

Inspired by her 2006 fi  ndings, O'Brien (2008) investigates in more detail why the relationship between fuzzy-match values and cognitive effort is not as linear as one might expect. She carried out an experimental study in which eight students translated a technical text using a TM (SDL Trados Translator's Workbench). The subjects' eye movements and pupil dilations were recorded using an eye tracker (Tobii 1750). Because only fi  ve subjects' eye movements could be accurately tracked, the study was limited to these fi  ve. Once the subjects had fi  nished the translation task, they were presented with a paper-based survey that included the same source segments and fuzzy matches that they had just seen on the screen. Subjects were not shown the fuzzy-match values that were assigned by the TM system, nor were any differences between the new source text and the source text in the TM highlighted. Subjects were asked to rate their perceived editing effort for each match using a fi  ve-point scale. Based on processing speed alone, the study demonstrates that decreasing fuzzy matches mean increasing effort, whereas the picture that emerges if pupil dilation is used as a measure of cognitive load is less clear. When seen as a median measurement across all subjects, dilations increase as match values decrease until the 60-69% match class is reached. Below this match class, decreased pupil dilation is noted. According to O'Brien, this might be due to the fact that subjects reached a baseline cognitive effort when they reached the 60-69% match class.

## 4.9.  Colominas (2008)

Like Dragsted (2004, 2006), Colominas (2008) assumes that sentence-based segmentation is unnatural and should at least be supplemented with sub-sentential segmentation, as the repetition of a whole sentence is generally rare. She therefore sets out to investigate the usefulness of subsentential segmentation. With a view to evaluating both recall (target-text proposal) and precision (usability), two experiments are carried out based on different TMs: a multilingual corpus extracted from the proceedings of the European Parliament (the Europarl corpus) and an EnglishSpanish corpus built up from United Nations documents (2008: 346). Segmentation below the sentence level, especially noun-phrase (NP) segmentation, is employed and analysed. The experiments are reported as showing that 'sub-sentential segmentation clearly shows a signifi  cant better recall', though NP segmentation 'should be much more refi  ned in order to be really useful' (2008: 352). These fi  ndings, which should be considered in future program improvements, are logical and probable, but the experiments are discussed in fairly technical terms and concepts are not suffi  ciently defi  ned and operationalised.

## 5. Conclusion

We set out to discover what we know about the nature, applications and infl  uences of TM technology, including translators' interaction with it. Based on the literature on TM technology in general, we have sketched the history of the conception of TM tools as an aid for professional translators and we have provided an overview of some basic defi  nitions and applications. While TM systems may differ as far as applications, internal processes and interactive modes are concerned, all TM technology shares the basic function of deploying existing translation resources in a new translation project, and, though the translation process is controlled by a human translator, the assistance of the TM tool is always regarded as an integral part of the translation process.

Hoping to establish what has been documented so far by means of empirical TM studies, we analysed a collection of nine empirical TM studies. As current TM systems only became commercially available in the 1990s, we decided to concentrate on studies that were published in 2000 or later. Searching the TM literature at large, we came up with a modest list of nine empirical studies (some of them reported in two publications). This list may not contain all available empirical studies of TM, but we hope that it may be seen as representative of empirical TM research so far. We shall now summarise some specifi  c knowledge documented by the empirical studies that were reviewed in section 4.

All studies either take for granted (Mandreoli et al. 2006; Colominas 2008; Lange/Bennett 2000;  Christensen  2003;  Dragsted  2004,  2006;  O'Brien  2006,  2008)  or  document  (Fulford/ Granell-Zafra 2005; Dillon/Fraser 2006; Lagoudaki 2006) that TM technology is here for good, as all types of professional translators seem to be aware of it and many - though not all - make good use of it.

Most practitioners seem to take for granted that TM technology speeds up production time and improve translation quality, but there are no studies that actually document this. Lange/Bennett (2000) suggest that translators' productivity may increase even more if TM technology is combined with machine translation (MT). Dillon/Fraser (2006) show that professional translators' levels of experience and IT competence may infl  uence how they approach and use TMs, though this is not confi  rmed by Lagoudaki (2006).

There can be no doubt that TM technology also has some serious drawbacks, which software developers need to address in future applications. Thus, for instance, Christensen (2003) argues that TM technology may be more useful for some genres than for others: TM technology is probably more useful when you translate texts that tend to be standardised and repetitive - technical documentation, for instance - whereas it may be less useful when you translate more complex texts - legal documents, for instance. To improve its usefulness for legal translators, Christensen (2003) suggests that TM technology is combined with a reference corpus of authentic, functionally equivalent target-language segments and, perhaps, also an authoring memory that can preedit and standardise source-text segments to facilitate better matches. As pointed out by O'Brien (2006, 2008), only exact matches require little effort on the part of the translator, whereas fuzzy matches require more effort, though the correlation between effort and match values is not a linear one. Another drawback inherent in TM technology is pointed out by Colominas (2008) and Dragsted (2004, 2006), who argue that it forces translators to work in a way that is cognitively unnatural: most commercially available TM systems force the translator to work with sentencebased segments as translation units, whereas (experienced) translators naturally tend to work with sub-sentential translation units.

Our review of empirical TM research shows that various methods are employed. Mandreoli et al. (2006) and Colominas (2008) base their fi  ndings on technical test runs. Christensen (2003) combines technical test runs with analyses of textual corpora. Lange/Bennett (2000) conduct a case study, in which they (presumably) apply a mixture of evaluative and descriptive (quantitative) methods. Fulford/Granell-Zafra (2005), Dillon/Fraser (2006) and Lagoudaki (2006) carry out rather large-scale questionnaire surveys. Dragsted (2004, 2006) and O'Brien (2006, 2008) use experiments with simulated translation situations. They both attempt to document translators' cognitive processes by means of advanced recording software, namely key-stroke logging (Translog) and eye-tracking (Tobii), respectively; and they both make use of translators' retrospective verbalisations.

Our analysis suggests that empirical TM research may be divided roughly into three areas according to authors' interests and orientation:

- (1)  Mandreoli et al. (2006) and Colominas (2008) seem to be interested mainly in the technical side of TM-assisted translation. This kind of empirical research may be referred to as technology-oriented.
- (2)  Lange/Bennett (2000) apply a mixture of datalinguistic methods to a case study, but their main interest obviously lies with workfl  ow issues. Their research may be referred to as workfl  ow-oriented TM research.
- (3)  Christensen (2003), Dragsted (2004, 2006), Fulford/Granell-Zafra (2005), Dillon/ Fraser (2006), Lagoudaki (2006) and O'Brien (2006, 2008) share a preoccupation with translation-theoretical issues and they all appear to take the translator's perspective. We may refer to this kind of research as translation-theoretical TM research.

Empirically documented knowledge about the nature and applications of TM systems and translators' interaction with them is both scarce and fragmented. In particular, more research is needed on how translators interact with TM technology and on how it infl  uences translators' cognitive processes. The translation profession itself will also welcome more knowledge about the translators' perspective on TM technology. We therefore suggest that translation-theoretical empirical TM research should be particularly encouraged.

## References

Alcina, Amparo 2008: Translation technologies. Scope, tools and resources. In Target 20(1), 79-102.

- Allen, Jeffrey 1999: Adapting the concept of 'Translation Memory' to 'Authoring memory' for a Controlled Language writing environment. In the proceedings of The 21 st  Conference of Translating the Computer, sponsored by ASLIB, 10-11 November 1999 [online]. http://www.transref.org/default.asp?docsrc=/u-articles/allen2.asp (accessed 15 Dec. 2009), 7 pages.
- Arthern, Peter 1979: Machine Translation and Computerized Terminology Systems: A Translator's Viewpoint. In Snell, B.M. (ed.), Translating and the Computer, Proceedings of a Seminar, London 14 th  November 1978 . Amsterdam: North-Holland Publishing Company, 77-108.
- Benis, Michael 2004a: The leader's progress. Review of Trados TM 6.5.5 and Trados TM Freelance. In ITI Bulletin (Sept/Oct), 26-32.
- Benis, Michael 2004b: A mixed bag of tricks. In ITI Bulletin (Mar/Apr), 22-27.
- Bowker, Lynne/Barlow, Michael 2008: A comparative evaluation of bilingual concordances and translation memory systems. In Yuste Rodrigo, Elia (ed.), Topics in Language Resources for Translation and Localisation. Amsterdam/ Philadelphia: John Benjamins, 1-22.

- Christensen, Tina Paulsen 2003: Translation memory-systemer som værktøj til juridisk oversættelse. Kritisk vurdering af anvendeligheden af translation memory-systemer til oversættelse af selskabsretlig dokumentation. Ph.D. dissertation. Syddansk Universitet.
- Colominas, Carme 2008: Towards chunk-based translation memories. In Babel 54(4), 343-354.
- Cruz-Lara, Samuel/Bellalem, Nadia/Ducret, Julien/Kramer, Isabelle 2008: Standardising the management and the representation of multilingual data: The Multi Lingual Information Framework. In Yuste Rodrigo, Elia (ed.), Topics in Language Resources for Translation and Localisation. Amsterdam/Philadelphia: John Benjamins, 151-172.
- Davies, Iwan 2004: SDLX 2004 TM and Trados 6.5 TM. In Communicator (Autumn), 32-36.
- Dillon, Sarah/Fraser, Janet 2006: Translators and TM: An investigation of translators' perceptions of translation memory adoption. In Machine Translation 20(2), 67-79.
- Dragsted, Barbara 2004: Segmentation in Translation and Translation Memory Systems - An Empirical Investigation of Cognitive Segmentation and Effects of Integrating a TM System into the Translation Process. Copenhagen: Samfundslitteratur.
- Dragsted, Barbara 2006: Computer-aided translation as a distributed cognitive task. In Pragmatics & Cognition 14(2), 443-464.
- Enriquez-Veintimilla, Nelly 2006: Translation memory issues - ownership and discounts. In ITI Bulletin (Jan/Feb), 32-36.
- Fulford, Heather/Granell-Zafra, Joaquin 2005: Translation and Technology: a Study of UK Freelance Translators. In Journal of Specialized Translation 4, 2-17.
- Garcia, Ignacio 2007: Power-shifts in web-based translation memory. In Machine Translation 21, 55-68.
- Gauton, Rachélle 2008: The use of corpora in translator training in the African language classroom: A perspective from South Africa. In Yuste Rodrigo, Elia (ed.), Topics in Language Resources for Translation and Localisation. Amsterdam/Philadelphia: John Benjamins, 90-106.
- Gouadec, Daniel 2007: Translation as a profession . Amsterdam/Philadelphia: John Benjamins.
- Guerberof, Ana 2008: Post-editing MT and TM. A Spanish case. In Multilingual Computing 19(6), 45-50.
- Hallett, Tim 2006: Transit and TRADOS: Converging functions, diverging approaches. In Localisation Focus 5 (4), 9-11.
- Harris, Brian 1988: Bi-text, a new concept in translation theory. In Language Monthly 54, 8-10.
- Holmes, James S. 1972/2000: The name and nature of translation studies. In Venuti, Lawrence (ed.) The Translation Studies Reader . London/New York: Routledge, 172-185.
- Hutchins, John 1998: The Origins of the Translator's Workstation. In Machine Translation 13, 287-307.
- Iverson, Steven P. 2003: Working with translation memory: When and how to use TM for a successful translation project. In Multilingual Computing and Technology 14(7), 35-37.
- Kay, Martin 1980/1997: The proper place of Men and Machines in Language Translation. In Research Report CSL-8011 , Xerox Palo Alto Research Center, Palo Alto, CA. Reprinted in M achine Translation 12, 3-23.
- Lagoudaki, Elina 2006: Translation Memories Survey 2006. Translation Memory systems: Enlightening users' perspective. Imperial College London [online]. http://www3.imperial.ac.uk/portal/pls/portallive/docs/1/7307707.PDF (accessed 8 October 2009), 36 pages.
- Lange, Carmen Andres/Bennett, Winfi  eld Scott 2000: Combining machine translation with translation memory at Baan. In Sprung, Robert C. (ed.) Translating into success , Amsterdam/Philadelphia: John Benjamins, 203-218.
- Levitt, Garry 2003: Internet-based sharing of translation memory: New solutions that are changing translation and localization workfl  ows. In Multilingual Computing and Technology 14(5), 38-41.
- LISA 2002: LISA Translation Memory Survey 2002. LISA The Localization Industry Standards Association [online]. http://www.lisa.org/Translation-Memory-S.523.0.html (accessed 8 October 2009), 19 pages.
- LISA 2004: LISA Translation Memory Survey 2004. LISA The Localization Industry Standards Association [online]. http://www.lisa.org/Translation-Memory-S.518.0.html (accessed 8 October 2009), 22 pages.
- Mandreoli, Federica/Martoglia, Riccardo/Tiberio, Paolo 2006: EXTRA: a system for example-based translation assistance. In Machine Translation 20, 167-197.
- Melby, Alan 1981: A Bilingual Concordance System and its Use in Linguistic Studies. In Gutwinski, Waldemar/Jolly, Grace (eds.), The Eighth LACUS Forum 1981, Columbia, SC: Hornbeam Press, 541-549.
- Moore, G.C./Benbasat I. 2001: Development of an instrument to measure the perceptions of adopting an information technology innovation. In Inform Sys Res 2, 192-221.
- Mossop, Brian 2006. Has Computerization Changed Translation? In Meta 51:4, 787-793.

- Mossop, Brian 2007. Empirical Studies of Revision: What We Know and Need to Know. In The Journal of Specialised Translation 8 [online]. http://www.jostrans.org/ (accessed 20 Oct. 2009), 5-20.
- Münchener Vertragshandbuch, Band 1, Gesellschaftsrecht, 4. Aufl  age 1996. München: Heidenhain/Meister.
- O'Brien, Sharon 2006: Eye-tracking and translation memory matches. In Perspectives: Studies in Translatology 14(3), 185-205.
- O'Brien, Sharon 2008: Processing fuzzy matches in translation memory tools: an eye-tracking analysis. In Göpferich, Susanne/Jakobsen, Arnt Lykke/Mees, Inger M. (eds.), Looking at eyes. Eye-tracking studies of reading and translation processing . Copenhagen: Samfundslitteratur, 79-102.
- O'Hagan, Minako 2009: Computer-aided translation (CAT). In Baker, Mona/Saldanha, Gabriela (eds.), Routledge Encyclopedia of Translation Studies . London and New York: Routledge, 48-51.
- Reinke, Uwe 2004: Translation memories. Systeme - Konzepte - linguistische Optimierung. Frankfurt am Main: Peter Lang.
- Schäler, Reinhard 2001: Beyond Translation Memories. In Proceedings of the Workshop on Example-Based Machine Translation [online]. http://www.eamt.org/events/summitVIII/papers/schaeler.pdf (accessed 16 December 2009).
- Somers, Harold 2003: Computers and Translation . A translator's guide. Amsterdam/Philadelphia: John Benjamins.
- Vandepitte, Sonia 2008. Remapping Translation Studies: Towards a Translation Studies Ontology. In Meta: Translators' Journal 53(3), 569-588.
- Wa β mer, Thomas 2005: Trados 7 and SDLX 2005. In MultiLingual Computing and Technology 16 (8), 27-32.
- Wheatley, Alan 2003: eCoLoRe (eContent Localization Resources for Translator Training) A Major Breakthrough for Translator Training. In TranslationDirectory.com [online]. http://www.translationdirectory.com/article450.htm (accessed 18 December 2009).
- Williams, Jenny/Chesterman, Andrew 2002: The Map. A beginner's Guide to Doing Research in Translation Studies . Manchester, U.K./Northampton MA: St. Jerome.
- Zerfa β , Angelika 2002: Comparing Basic Features of TM tools. In Language Technology Supplements, MultiLingual Computing and Technology 51 , 13(7), 11-14.

102