## User Consultation Behaviour in Internet Dictionaries: An Eye-Tracking Study

## Abstract

The purpose of this paper is  to  explore  and  discuss  user  consultation  behaviour  on  the  basis  of  eye-tracking  data and interview data. To date the focus has been almost exclusively on the use of log fi  les in Internet lexicography -  an  approach  which  is  questioned  in  this  article.  The  paper  is  based  on  empirical  data  from  an  exploratory  eyetracking study of the user consultation behaviour of six participants and on interview data from a follow-up post-study interview of the participants. The paper elucidates and discusses the consultation behaviour in Internet lexicography and shows not only at what the participants looked, but also how they accessed lexicographic data. The paper presents a suitable method for using eye-tracking studies in Internet lexicography and advocates an increased use of this method to produce empirical data upon which additional theoretical considerations on the information and data access process can be developed. Finally, the implications for further research in user consultation behaviour are briefl  y explored.

## 1. Internet Dictionary Usage and User Consultation Behaviour: Eye-tracking

Several theoretical contributions on the usage of Internet dictionaries rely on log fi  le analysis. The user's behaviour (key strokes and mouse clicks) is logged and tabulated in automatic, server-generated log fi  les, and this technology has no doubt contributed to theory development and user-prompted update and revision of the content of the dictionaries in question, cf. for example De Schryver/Joffe (2004), Bergenholtz/Johnsen (2005), De Schryver et al. (2006), Bergenholtz /Johnsen (2007), Hult (2008) and Almind (2008), who all make a strong case for the use of log fi  les in internet lexicography.

However, log fi  les only show what characters the users have typed on the keyboard and log fi  les do not show where users of an internet dictionary have looked.

The eye-tracking method proposed in this article is suitable for the study of the user consultation behaviour in connection with the use of an internet dictionary, and it can be used to learn more about what the user looks at, and how his eyes move and fi  xate during for example search operations and processing operations. Systematically collected data, which include data on eye fi  xations and eye movements, can be converted into valuable knowledge about Internet dictionary usage, and can be used in the pre-design, design and redesign phases of a dictionary project and in controlled dictionary reviews.

Finally, it is argued that eye-tracking studies in Internet lexicography are important in the ongoing development of theoretical considerations on information and data access in lexicography, because eye-tracking data may help us gain insight into what users are looking for, how they access data in specifi  c situations and to what extent the user profi  le plays a role in user consultation behaviour.

This article has three overall objectives.

First, it introduces and discusses a new type of usage study designed for Internet dictionaries. Second, it demonstrates and discusses how a user of an Internet dictionary locates lexicographic data in specifi  c situations and discusses the user consultation behaviour on the basis of eye-track-

* Henrik Køhler Simonsen Copenhagen Business School Dalgas Have 15 DK-2000 Frederiksberg hks.isv@cbs.dk

75

ing data and interview data. Third, it discusses the possible implications of this study for lexicographical data processing, Internet dictionary design and offers a number of theoretical considerations on search paths and search sequences.

The theoretical and practical relevance of this type of user survey in Internet lexicography is apparent, because eye-tracking data may help us learn more about the user, who was once characterized as the 'bekannten Unbekannten' by Wiegand (1977: 59). User survey research is admittedly faced with a number of challenges and inherent weaknesses, some of which are raised by Tarp (2006), who questions whether user surveys can uncover the needs of the user. Despite the fact that Tarp (2006) to some extent has a point, when it comes to an entirely objective identifi  cation of user needs, it is argued that a study of the type presented in this article does produce useful data of relevance for lexicographers, lexicographic web editors and publishers, especially today where focus is on needs-adapted information and data access. Desktop research alone does not suffi  ce, a view which is shared by Farø/Gottlieb (2007: 195), who criticize Tarp (2006) for his 'intuitions and desktop research' (my translation).

## 2. Defi  nitions and Delimitation

For the purpose of this discussion, we fi  rst need to defi  ne the term internet dictionary. A useful defi  nition is offered by De Schryver et al. (2003: 4), who describe an internet dictionary as an 'online dictionary for which the data are stored in databases, no matter where these databases are located, and which can be consulted from a search screen by anyone from anywhere through the Internet'.

The important eye-mind assumption proposed by Just/Carpenter (1980) also needs to be defi  ned. The eye-mind assumption is based on the widely recognized assumption that there is a high correlation between long fi  xation durations and effortful processing in the user's brain. Just/Carpenter's assumption has later been confi  rmed by Rayner (1998), who also demonstrates that eye fi  xation and gaze time data refl  ect cognitive processes in the user's brain. In this study it is also hypothesized that user consultation behaviour and especially effortful cognitive data processing can be measured and analysed by means of eye fi  xation data.

Furthermore, a number of eye-tracking-specifi  c concepts need to be defi  ned. The term eye movement refers to the situation when a participant's eyes move from one location of the screen to another and eye movements are typically shown by means of saccades, that is, the scanning path of the participant's eyes across the screen. The term eye fi  xation is the moment when the eyes are near-stationary and are encoding information, and a commonly recognized technical defi  nition is offered by Duchowski (2007: 46) who defi  nes fi  xations as '...eye movements that stabilize the retina over a stationary object of interest'. Also the threshold (minimum time for a fi  xation) is relevant here as interpretations of fi  xation frequency depend on the selected time threshold of the eye-tracking system. A fi  xation duration may range from 80 ms to 600 ms, and eye fi  xations are typically seen as an indicator of attention, cf. also Duchowski (2007) for a detailed discussion of eye-tracking methodology. For the purpose of this eye-tracking study, the minimum threshold was set at 100 ms within a 40 pixel radius, because this study focused on a combination of user consultation processes (search, reading and processing tasks). The 100 ms fi  xation fi  lter setting, which is low in comparison to the setting used in for example translation and reading studies, was preferred because of the somewhat sporadic nature of the user consultation process in Internet lexicography.

Typically, eye fi  xations are shown by means of blue or red circles on the screen depending on the software used. The Tobii eye-tracking equipment used in this survey shows fi  xations by means of blue circles as is shown in e.g. Appendix D. Transitions are when a participant's eyes move from one area of interest to another area of interest on the screen and are used to describe shifts between AOIs. Finally, area of interest (AOI) refers to a specifi  c, designated area of the screen where fi  xations and user behaviour are recorded for subsequent data analysis, cf. also Poole/Ball

(2004) for useful defi  nitions of eye-tracking metrics. ClearView 2.6.3 shows all AOI transitions (both cross-AOI and AOI-internal transitions) as will appear from the discussion below by means of a transition matrix, which shows cross-AOI and AOI-internal transitions. Finally, according to Duchowski (2007: 47) saccades are considered manifestations of the desire to voluntarily change the focus of attention and can be both voluntary and refl  exive. A more technical defi  nition is that 'saccades are rapid eye movements used in repositioning the fovea to a new location in the visual environment', cf. Duchowski (2007: 42). The duration of saccades is typically very short, usually from 10 ms to 100 ms.

Finally, it is relevant to stress that this article discusses the results and theoretical implications of a delimited, exploratory survey, which was designed to demonstrate how six users use a specially selected Internet dictionary. The article focuses on Internet dictionary usage and on user consultation behaviour on the basis of data on the user's eye movements, eye fi  xations, saccades and transitions while performing a number of search and processing operations in connection with a translation task. The article only focuses on the lexicographic part of the translation task and is delimited to the actual look-up and processing process.

## 3. Existing Research in Dictionary Usage and Dictionary Users

This section of the article will focus on existing user survey research and outline a number of useful conclusions and fi  ndings from this research. As already noted above Wiegand (1977: 59) once described the user as the ' bekannten Unbekannten ', and since then much research has been carried out to learn more about the known unknown. A number of user surveys were carried out by for example Béjoint (1981), Benbow et al. (1990), and Nesi (2002), but the problem with these user surveys, as pointed out by Bergenholtz/Johnsen (2007), was that they were primarily based on introspection, which is an argument also raised by Simonsen (2002: 94-99), who reports on an analysis of 25 lexicographic user surveys. The use of introspection as a method in dictionary usage is questionable, because self-observation and consequent reporting of own thoughts and observations, either verbally or in writing, are inherently very subjective and not suitable for studying dictionary usage.

Another type of user survey was proposed by Wiegand (1985), who outlines a protocol survey involving foreign students of German. This contribution introduced the use of user protocols, which was later refi  ned and used by other researchers, for example Atkins/Varantola (1997), who used a combination of a questionnaire and a user protocol involving students, Bergenholtz (1988), who carried out a dictionary test involving students, Diab (1999), who discussed a case study of students at the University of Jordan with a combination of a questionnaire, interviews and a protocol, and Nesi (2002), who used a test involving students to mention just a few. The use of protocol surveys and especially dictionary tests constituted a big step forward, however the focus of these surveys was dictionary use and the target group was typically students. In addition Wiegand (1998) outlined a large number of studies of dictionary use, and Hartmann (1989) discussed the 'sociology of the dictionary user ' and called for further empirical studies of dictionary use. Hartmann's call for further user research resulted in a large number of studies, some of which are mentioned in Hartmann (2008), and Hartmann has played a major role in dictionary use research through the Dictionary Research Centre at the University of Exeter.

All these researchers have shed valuable light on the 'bekannten Unbekannten ' , however the dictionary users and their actions are to some extent still unknown, especially in Internet lexicography. The surveys briefl  y outlined above did not empirically measure what users looked at during the access process, because they were mainly based on partly introspective, subjective descriptions by the users. An eye-tracking survey objectively measures what the user really looks at and what he does - a user protocol subjectively includes what the user says he does, or believes he does.

The advent of log fi  les and log fi  le analysis, however, marked the beginning of a new era in Internet lexicography and they offered a new method of describing the 'bekannten Unbekannten'. The fi  rst, and probably most important, contribution on the use of log fi  le data in Internet lexicography was published by De Schryver/Joffe (2004), who discuss how electronic dictionaries are used on the basis of log fi  le data from the Sesotho Sa Leboa Dictionary (SeDiPro).

Another important contribution on the use of log fi  les in Internet lexicography is Bergenholtz/ Johnsen (2005) where the authors demonstrate that log fi  les can indeed be used to improve Internet dictionaries. The two authors make a very strong case for log fi  les in Internet lexicography and offer a long empirically substantiated discussion of how log fi  les can be used to improve the lemma selection of an Internet dictionary, and they call for further research into stronger search systems. Bergenholtz/Johnsen (2005) criticize De Schryver/Joffe (2004) for using a limited number of visitors and number of lookups in their discussion, a criticism, which De Schryver et al. (2006) later elegantly address in their discussion of the value of corpus-based lexicography.

A similar discussion is offered by Bergenholtz/Johnsen (2007), who argue that the data from log fi  les should be analysed on the basis of lexicographic functions and that data should be correlated with data on the user and the user situation. Other important contributions on log fi  les include Johnsen (2005), who offers a discussion of the use of log fi  les in the DANISH INTERNET DICTIONARY in relation to four other Internet dictionaries, and Hult (2008), who discusses log fi  le analysis of the Swedish LEXIN Internet dictionary. Other relevant contributions on web log research include Ling et al. (2001) and Ling et al. (2002) which both discuss query patterns and web logs.

The research based on log fi  les is a huge step forward, because it objectively measures the search operations of the dictionary users, but log fi  les only show what the dictionary users searched for not how they searched for and accessed data.

One example of eye-tracking surveys in Internet lexicography is Simonsen (2009a), who reports on an eye-tracking survey, where fi  ve participants were asked to perform a number of operations in two versions of the same Internet dictionary; the only difference being that the fi  rst version of the Internet dictionary had a vertical data presentation and the second version a horizontal data presentation. On the basis of eye-tracking metrics and think-aloud protocol data, Simonsen (2009a) concludes that a horizontal data arrangement is recommended in connection with cognitive lexicographical functions, that is, when data are to be processed with a view to acquire knowledge whereas a vertical data arrangement is recommended in connection with communicative lexicographical functions, that is when data are to be used in for example L1-L2 translation as is the case in the present eye-tracking study. Furthermore, Simonsen (2009b) discusses another eye-tracking survey with special emphasis on the lexicographic functions. On the basis of eyetracking metrics, Simonsen (2009b) argues that there is a correlation between the different lexicographic functions and the cognitive load measured as eye fi  xations and gaze times. Simonsen (2009b) found that cognitive lexicographic functions, such as L1 knowledge acquisition, incurred more cognitive load than communicative lexicographic functions, such as for example L1-L2 translation. In fact ORDBOGEN OVER FASTE VENDINGER cf. http://www.idiomordbogen.dk/ (Danish Dictionary of Fixed Phrases) uses such an approach as it allows users to search for Danish idioms and phrases on the basis of lexicographic functions. In conclusion, the two eye-tracking studies discussed above show that there is a link between how a user accesses data (search-related data) and how he uses data (function-related data), but they did not focus on the user consultation process, which is the focus of the eye-tracking survey discussed in this article.

The above brief literature review revealed that to date different aspects of the known unknown have been revealed, and recent developments on the use of log fi  le data in conjunction with lexicographic functions bode well for the future. The different eye-tracking surveys within the fi  eld of Internet lexicography briefl  y outlined above are also relevant for this survey because they have revealed different aspects of the user's dictionary usage. However, we still need to learn more about the user consultation behaviour.

## 4. Methodology and Experimental Set-up

The methods used in this article include the collection, tabulation and interpretation of eye-tracking data and interview data.

Eye-tracking data may help lexicographers decide on the best design of an Internet dictionary and on where to locate search fi  elds and how to order and present lexicographic data. A survey of the type discussed here may also supply a prospective subscriber with comparable and objective data on how a specifi  c number of users have used the Internet dictionary in a specifi  c number of situations. Finally, the research method proposed here could also be used by professional reviewers when reviewing an Internet dictionary in order to be able to review how the Internet dictionary in question allows users to access data.

A Tobii 1750 remote eye-tracker was used to register the eye movements of six participants, who were all professional translators with an MA degree in translation: for additional information on the occupation, company, experience, area of expertise and knowledge level of the Internet dictionary of the participants, see Appendix A. The six participants were invited to take part in the eye-tracking study and the interview on a specifi  ed date and time. On the day of the eyetracking study, the participants were asked to use the Danish-English Accounting Dictionary at http://www.regnskabsordbogen.dk/regn/dkgb/dkgbregn.aspx, and the experiment was carried out in a special eye-tracking lab. The eye-tracking software used was ClearView, cf. also www.tobii. com for further information on the Tobii 1750 and ClearView.

Each participant was asked to look up at least fi  ve lemmata, which corresponds to a total of 30 lemmata (5 lemmata X 6 participants) during a translation task. The recordings totalled 3 GB, but unfortunately, as is often the case in eye-tracking studies, the quality of some of the data collected was not satisfactory, despite the fact that two successful pre-surveys were carried out and despite the fact that the eye-tracking equipment was calibrated for each participant. Unfortunately, the eye-tracking data from the six participants were not comparable, because the quality of some parts of the recordings was not good enough. To ensure comparability three participants with each three successful recordings of comparable lemmata were selected for further analysis by means of the ClearView Software, see also the discussion below and Appendix C, which includes a hot spot chart for each of the three selected participants. The remaining three participants were eliminated from the study because the eye-tracking data recorded for these three participants were not comparable with the data for the selected three participants.

One reason why the inclusion rate is only 50% may be that this eye-tracking survey focuses on the analysis of eye movements across large text units or entire screen texts and not eye movements from one word to another, which is typical in reading and translation studies. Another reason why data from only three out of six participants were included may be that the measuring accuracy of the eye-tracking equipment was not good enough because of the time threshold setting or because of Brownian motions. The lemmata selected for further analysis were the terms 'direktion', 'gældende' and 'retvisende', which are all typical words used in a statement by the management on the annual report.

The participants were asked to translate a small text from Danish to English and in the process look up at least fi  ve Danish lemmata from the economic register during a L1-L2 translation situation. Some of the participants looked up fewer than fi  ve lemmata, other participants looked up more than fi  ve lemmata. Before the test the participant received a written instruction sheet and a number of oral instructions including information on the purpose of the experiment, the process of the experiment and the intended use of the data. The text used in the eye-tracking study is shown in Appendix B. The participants were all professional translators with different degrees of experience and different expertise areas. Anonymized participant profi  les are shown in Appendix A, which lists relevant information about the participants, including occupation, company, experience, expertise area and knowledge level of the test object.

Den Danck Iapetk Regeska!

hermet lapler O spee· Srd · Fonde · EAasd-

ester lar labsibsgral Regnskabsordbøgerne

The test object, which today is managed by www.ordbogen.com, was the Danish-English Accounting Dictionary at http://www.regnskabsordbogen.dk/regn/dkgb/dkgbregn.aspx. The actual user interface, with labelling of the two AOIs, is shown in Figures 1 and 2.

Figure 1. User interface of test object with the two AOIs X\_search and X\_article

<!-- image -->

In transition matrix experiment I, two overall AOIs were used as shown in Figure 1. The AOI Search was the search fi  eld of the internet dictionary and the AOI Article was the actual lexicographic article.

Figure 2. User interface of test object with the nine AOIs A1 to C3

<!-- image -->

9 . 0

1 5 - i-

9:0- In transition matrix experiment II, a somewhat more detailed AOI grid was used. The grid divided the user interface into nine AOIs with a view to analysing the information access of the user. The nine AOIs are named A1 to C3, respectively, as is shown in Figure 2.

The second type of data collected is interview data. The qualitative follow-up interviews with the participants were performed immediately after the eye-tracking experiment and a total of 78 minutes of interview data were recorded by means of a digital recorder. Selected questions asked and responses given by the interviewees are included in the discussion  below. The  questions asked were designed to encourage the interviewee to explain what he did and what he saw during the individual search operations and the subsequent information processing.

## 5. Findings and Analysis

The experiment aimed at examining how six Danish translators use a Danish-English Internet dictionary by tracking the eye movements and eye fi  xations during lexicographic search and processing operations. During the interview, the participants were asked a series of questions on their user consultation behavior.

For the purpose of this study, three measures are included in the analysis of the eye-tracking data: 1) the fi  xation count, which is the number of eye fi  xations in a given AOI, 2) the gaze time, which is the duration of eye fi  xations in a given AOI and 3) transition matrix data measuring transitions and measuring the scan path of the participant's eyes during consultation. The eye-tracking study is thus based on the measurement of the number of eye fi  xations, the number of milliseconds used on gazing, and fi  nally, the number of transitions between and inside the AOIs. For the fi  rst two measures two AOIs were used: AOI\_search, which consists of the top 1/3 of the screen and AOI\_article, which consists of the remaining 2/3 of the screen. The motivation for defi  ning two overall AOIs was to obtain data on two different operations: the search operation and the reading operation.

For the last measure, two experiments were conducted. Experiment I used the same two overall AOIs as used for the fi  rst two measures. In experiment II the screen was divided into nine AOIs forming a grid. The AOIs were named A1-A2-A3 (top area of the screen), B1-B2-B3 (middle area of the screen) and C1-C2-C3 (bottom area of the screen). Each of these AOIs measures approx. 7.5 X 7.5 cm, see also Figures 1 and 2 and Appendices D and E for further information on where the AOIs are placed on the actual user interface. To be able to study the scan path of the user nine small AOIs were defi  ned. In the following discussion the following abbreviations Participant A, Participant D and Participant F are used for participants A, D and F, respectively.

## 5.1.  Fixation Count

Fixation counts can be shown graphically by means of gaze plot charts, which plot the fi  xations of the participant. Th  e bigger the blue circle the longer the fi  xation duration. The numbers in the blue circles refer to the fi  xation number and these unique numbers are used to identify the fi  xation count, and in the analysis of the transition matrix data, the fi  xation order of fi  xations.

C Den Dansk Engelske Regnskabsordbog Windows Internet Explorer

e http://www.regnskabsordbogen.dk/regr/diob/diobregn.aspv

Google

8-

E Den Dansk Engelske Regnskabsordbog

Center for Leksikografi

- - * Dogmarler+9 Find - Kortroer ·E] Autoryid -

0 · a

] HHA | Campushet | Biblotekel

Pertentde Regnskabsordbøgerne

Figure 3 below shows a ClearView gaze plot of participant D's fi  xations on the screen for the Danish lemma 'retvisende'.

Engelsk

Engelsk-Dansk

<!-- image -->

Udfart

i start What You See

STUDY: Lex Survey January09

STIMULUS. Test\_1\_ Anja.

TIME SEGMENT: Only include fixations inside interval (368819.387529) ms.

Figure 3. Gaze plot of participant D's screen fi  xations for the Danish lemma 'retvisende'

Participant D had 31 fi  xations in and around the search fi  eld, which is referred to as  AOI retvisende\_ search and 44 fi  xations in and around the article fi  eld, which is referred to as AOI retvisende\_ar-ticle. Participant D's fi  xation counts in the two AOIs appear in Table 1 below. As is shown, the participant had more eye fi  xations in the article fi  eld than in the search fi  eld.

Table 1. AOI fi  xation count of participant D's screen fi  xations for the Danish lemma 'retvisende'

|                     |   AOI retvisende_search |   AOI retvisende_article |
|---------------------|-------------------------|--------------------------|
| Number of Fixations |                      31 |                       44 |

A similar fi  nding was made for participant A. Figure 4 below shows a ClearView gaze plot of Participant A's fi  xations on the screen for the Danish lemma 'direktion'.

-ex

g - logind -

·Funktioner ·

C Den Dansk Engelske Regnskahsordbog Windows Internet Explorer

·

· http://www.regnskabsordbogen.dk/regn/dlgb/dlobregn.aspx

Google

8-

e Den Dansk-Engelske Regnskabsordbog

Center for Leksikografi

....

"httehelskolen

- #· Dogmaerker- rind - "Kontroler - E Autofyld -

9 - a

wHA | Campussee | abicteket

Regnskabsordbøgerne

<!-- image -->

Udfart

W start C E O g Mette Jehnsen - Mar...

STUDY: Lex\_Survey\_January09

STIMULUS: Test\_1 Mette.

TIME SEGMENT: Only include fixations inside interval [59876,71021) ms.

Figure 4. Gaze plot of participant A's screen fi  xations for the Danish lemma 'direktion'

On the basis of the AOI analysis it was found that participant A had 12 fi  xations in AOI direk-tion\_search and 14 fi  xations in AOI direktion\_article, respectively, as is shown in Table 2 below.

Table 2. AOI fi  xation count of participant A's screen fi  xations for the Danish lemma 'direktion'

|                     |   AOI direktion_search |   AOI direktion_article |
|---------------------|------------------------|-------------------------|
| Number of Fixations |                     12 |                      14 |

Participant A had almost just as many fi  xations in the AOI direktion\_search as in the AOI direk-tion\_article. The very small difference in the number of fi  xations may be explained by comments made by participant A during the interview; for this term she says ' I just wanted to check a term'.

Participant F's consultation behaviour in terms of eye fi  xations was somewhat different from participant D and participant A, see Figure 5 below.

· Log ind ·

→ Side = funitioner ·

A Den Dansk Engelske Regnskabsordbog Windows Internet Explorer

· http://ww.regnskabsordbogen.d//regr/dob/dobregn.aspx

Google

8-

Den Dansh-Engelske Regnskabsordbog

Center for Leksikografi

Astur School ef Potens

lcdalahekolen

- - * Bogmarker= Find - Kortroler · E] Autoryid ·

9 · ·

HHA | Campushec | Biblioteket

*Regnskabsordbøgerne

start Ce ue,legensen-MeC oen oud-Calde...

STUDY: Lex\_ Survey\_January09.

STIMULUS: Test\_1\_Lise.

TIME SEGMENT: Only include fixations inside interval (195436.206323) ms.

Figure 5. Gaze plot of participant F's screen fi  xations for the Danish lemma 'gældende'

<!-- image -->

On the basis of the AOI analysis it was found that participant F had 8 fi  xations in AOI gældende\_ search and only 5 fi  xations in AOI gældende\_article as is shown in Table 3. In other words, participant F had more fi  xations in the search fi  eld than in the article fi  eld.

Table 3. AOI fi  xation count of participant F's screen fi  xations for the Danish lemma 'gældende'

<!-- image -->

|                     |   AOI gældende_search |   AOI gældende_article |
|---------------------|-----------------------|------------------------|
| Number of Fixations |                     8 |                      5 |

## 5.2.  Gaze Time

The gaze times of the participants are shown in Tables 4, 5 and 6. The gaze times are shown graphically in the gaze plot charts above in Figures 3, 4 and 5 by means of blue circles and the bigger the blue circle the longer the gaze time. The duration of a participant's fi  xations inside an AOI may help us understand how long it takes a user of an Internet dictionary to execute a specifi  c task measured in milliseconds. Furthermore, the gaze time refl  ects how much cognitive effort the participant puts into a specifi  c task based on the eye-mind assumption by Just/Carpenter (1980).

On the basis of the gaze time analysis, it was found that participant D looked considerably longer at the AOI retvisende\_article than at the AOI retvisende\_search as shown in Table 4 below.

Table 4. Gaze time data for participant D's screen gazes for the Danish lemma 'retvisende'

|           | AOI retvisende_search   | AOI retvisende_article   |
|-----------|-------------------------|--------------------------|
| Gaze time | 13496 ms                | 17880 ms                 |

(Goodie P ·

g logind -

→ Sde - © Funktioner ·

A somewhat different result was found for participant A, as shown in the gaze data in Table 5. Participant A fi  rst of all looked more at AOI direktion\_search than at AOI direktion\_article, which seems to suggest that participant A did not need to scrutinize the lexicographic data, because she is an expert on translating economic and fi  nancial texts. Participant A spent 3927 ms looking at AOI direktion\_search, but only 2531 ms at AOI direktion\_article. Another plausible explanation could be that participant A being an expert in this fi  eld did not need to look up the term and thus only looked up the term to check her own translation.

Table 5. Gaze time data for participant A's screen gazes for the Danish lemma 'direktion'

|           | AOI direktion_search   | AOI direktion_article   |
|-----------|------------------------|-------------------------|
| Gaze time | 3927 ms                | 2531 ms                 |

A similar fi  nding can be seen for participant F. Participant F also looked comparatively longer at AOI gældende\_search than at AOI gældende\_article , and again the participant spent more time gazing at the search fi  eld than at the lexicographic data. Again this is consistent with the interview data and her profi  le, which shows that she is an expert on accounting texts, which may be why she did not have to scrutinize the lexicographic data. The gaze data for participant F are shown in Table 6.

Table 6. Gaze time data for partipant-F's screen gazes for the Danish lemma 'gældende'

|           | AOI gældende_search   | AOI gældende_article   |
|-----------|-----------------------|------------------------|
| Gaze time | 4285 ms               | 1674 ms                |

## 5.3.  Transition Matrix Data

Transition matrix data are useful when measuring transitions between or inside AOIs. Two experiments were conducted, and the purpose of the two experiments was to analyse time-dependent, linear information access behaviour patterns of the participants. Information access path here refers to 'the scanning path of the participant's eye fi  xations' and is in line with scanpath which is defi  ned as …'usually a complete sequence of fi  xations and interconnecting saccades' by Poole/ Ball (2004) . The sequence of fi  xations and interconnecting saccades is shown in Figures 6-8.

## 5.3.1.  Transition Matrix Experiment I

In experiment I the two overall AOIs X\_search and X\_article were used again. These two AOIs were also used during the collection of the eye fi  xation and gaze time data and it was expected that there would be a high amount of cross-AOI transitions between the search fi  eld and the article fi  eld, but the opposite was found. As is shown in Tables 7, 8 and 9 below, almost all transitions were AOI-internal.

Participant D for example, had 24 transitions from AOI retvisende\_search to AOI retvisende\_ search (or 80.0%) and 38 transitions (or 86.4%) from AOI retvisende\_article to AOI retvisende\_ article. The AOI-external data will also appear from Table 7 below.

Table 7. Transition matrix data for participant D's transitions for the Danish lemma 'retvisende'

| From To                  | AOI 'retvisende_search'   | AOI 'retvisende_article'   |
|--------------------------|---------------------------|----------------------------|
| AOI 'retvisende_search'  | 80.0% (24)                | 13.6% (6)                  |
| AOI 'retvisende_article' | 20.0% (6)                 | 86.4% (38)                 |

For participants A and F, see Tables 8 and 9 below, similar data were found. As is shown in Table 8, participant A had 7 transitions (or 87.5%) from AOI gældende\_search to AOI gældende\_search and 4 transitions (or 100 %) from AOI gældende\_article to AOI gældende\_article.

Table 8. Transition matrix data for participant A's transitions for the Danish lemma 'gældende'

| From To                 | AOI 'gældende_search'   | AOI 'gældende_article'   |
|-------------------------|-------------------------|--------------------------|
| AOI 'gældende_search'   | 87.5% (7)               | 0.0% (0)                 |
| AOI ' gældende_article' | 12.5% (1)               | 100.0% (4)               |

The same overall picture was found for participant F as is shown in Table 9. Participant F had 9 transitions (or 81.8%) from AOI direktion\_search to AOI direktion\_search and 12 transitions (or 85.7 %) from AOI direktion\_article to AOI direktion\_article.

Table 9. Transition matrix data for participant F's transitions for the Danish lemma 'direktion'

| From To                  | AOI 'direktion_search'   | AOI 'direktion_article'   |
|--------------------------|--------------------------|---------------------------|
| AOI 'direktion_search'   | 81.8% (9)                | 14.3% (2)                 |
| AOI ' direktion_article' | 18.2% (2)                | 85.7% (12)                |

The data show that a high amount of the transitions were AOI-internal, and additional analysis of the data by means of the Gaze Replay option in the ClearView software supports the contention that the participants were prone to looking within small, designated areas, that is, primarily inside the search-related AOI or the article-related AOI, which in essence is what users do when looking for lexicographic data. This contention is supported by research published by Jacobsen (2008), who on the basis of a survey of website readers concluded that short, compact segments or boxes of text were preferred.

However, with a view to analysing the details of user consultation behaviour, an additional experiment was needed. Consequently, a more fi  ne-meshed grid of AOIs was designed to be able to analyse the participants' information access paths more closely. The user interface and the fi  nemeshed grid of AOIs are shown in Figure 2 above.

## 5.3.2.  Transition Matrix Experiment II

On the basis of the analysis of the data from experiment I, it was decided to design a more fi  nemeshed grid of AOIs, see also Figure 2 and Appendix E for further information on the AOI grid A1

B1

C1

10

12

Direktion - TS-D

A2

20

АЗ

Search-related fixations

used. As is shown in Appendix E, the screen was divided into nine quadrants named A1 to C3. In addition to analysing transitions between the nine AOIs, experiment II also looked at the participant's information access paths by plotting gaze plot data in the grid to be able to analyse the user consultation behaviour. The transition matrix data and the information access path data of three selected participants in connection with the Danish lemma 'direktion' are shown in Figures 6-8 and Tables 10-16. C3

For the purpose of experiment II, the gaze plot data were exported to an Excel spread sheet with nine quadrants as shown in Figures 6-8. Three types of fi  xations were identifi  ed: search-related fi  xations which occurred in or around the search fi  eld (especially AOI A2), processing-related fi  xations, which occurred in or around the actual lexicographic article (especially AOIs B1, B2, B3, C2 and C3) and fi  nally navigation-related fi  xations, which occur in or around menus.

<!-- image -->

Figure 6. Participant D's transition matrix data for the Danish lemma 'direktion'

Table 10. Participant D's search-related fi  xations

| Fix. No.   | 1   | 2   | 3   | 4   |
|------------|-----|-----|-----|-----|
| AOI        | A2  | B2  | B2  | B2  |

Table 11. Participant D's processing-related fi  xations

| Fix. No.   | 5   | 6   | 7   | 8   | 9   | 10   | 11   | 12   | 13   | 14   | 15   | 16   | 17   | 18   | 19   | 20   |
|------------|-----|-----|-----|-----|-----|------|------|------|------|------|------|------|------|------|------|------|
| AOI        | B2  | B1  | B2  | B2  | B2  | B1   | B1   | B1   | B2   | B2   | B2   | B2   | B2   | B2   | A2   | A2   |

The data shown in Figure 6 demonstrate that participant D's information focus is in the centre area of the screen, especially in quadrants A2 and B2. The information access path will also appear from Figure 6, where fi  xations seem to be in two groups: search-related fi  xations (fi  xations 1-4) and processing-related fi  xations (fi  xations 5-20). As expected, the information access path starts in or around the search fi  eld, see fi  xations 1-4, and after participant D typed the lemma in the search fi  eld, the information access path followed the lemma and the equivalent and continued along the structure of the lexicographic article, cf. fi  xations 5-19. The information access path ended in or around the search fi  eld in quadrant A2, cf. fi  xation 20.

A1

C1

113

16

17

21/20

Direktion - TS-A

A2

25| 2 7 3 1

A3

Search-related fixations

The information access sequence is shown in the Tables 10-11 above. The fi  rst line of the table shows the sequence of the fi  xation numbers indicated in the nine quadrants and the second line of the table indicates the quadrant name. As is shown in Tables 10-11 participant D had 3 searchrelated fi  xations in the B2 quadrant and 10 processing-related fi  xations in the B2 quadrant, and the access sequence was chronological as participant D fi  rst had four fi  xations in or around the search fi  eld and then 15 fi  xations on the lexicographic article. The sequence of fi  xations is also shown in Tables 10-11.

When studying the data for participant A, similar observations can be made. Participant A's information access path is in fact quite similar to that of participant D.

Again two primary areas of information focus can be identifi  ed, see Figure 7 below. Participant A's information access path also begins in and around the search fi  eld in quadrant A2, continues to quadrant B2 and ends its journey in quadrant A2 as was the case for participant D. Fixations 1-9 are all roughly in the search-related cluster and fi  xations 10-26 are in the data processing zone, respectively. As will appear from participant F's fi  xations in the data processing zone, her information access behaviour seems to be both horizontal and vertical, which seems to suggest that she processes the lexicographic data line by line. In other words, participant A processes the equivalent, continues to the defi  nition of the Danish lemma 'direktion' and then reads the examples line by line, cf. fi  xation numbers 10-13, 14-16, 17-19 and 20-22.

The information access sequence and the names of the grids are listed in Tables 12-13.

<!-- image -->

Figure 7. Participant A's transition matrix data for the Danish lemma 'direktion'

Table 12. Participant A's search-related fi  xations

| Fix. No.   | 1   | 2   | 3   | 4   | 5   | 6   | 7   | 8   | 9   |
|------------|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| AOI        | A2  | A2  | A2  | B2  | B2  | B2  | A2  | A2  | A2  |

Table 13. Participant A's processing-related fi  xations

| Fix. No.   | 10   | 11   | 12   | 13   | 14   | 15   | 16   | 17   | 18   | 19   | 20   | 21   | 22   | 23   | 24   | 25   | 26   |
|------------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|
| AOI        | B2   | B2   | B2   | B1   | B1   | B2   | B1   | B1   | B2   | B2   | B1   | B1   | B2   | B2   | B2   | A2   | B2   |

20

10 18

A1

22

B1

C1

Direktion - TS-F

A2

A3

Search-related fixations

Similar observations can be made for participant F, whose information focus is also in the centre area of the screen except for the four outlier fi  xations in quadrant C1, whose location is equivalent with the collapsed Microsoft Word document used in the experiment and fi  xation no. 22 which is a fi  xation on the menu of the Internet dictionary in question and thus a navigation-related fi  xation. Again the fi  xations can roughly be divided into search-related fi  xations (fi  xations 1-9), processing-related fi  xations (fi  xations 11-17 and 21) and navigation-related fi  xations (fi  xations 10, 1820 and 22), see Figure 8 and Tables 14-16 below. Participant F's information access path starts in or around the search fi  eld in quadrant A2, cf. fi  xations 1-9 and ends with a number of fi  xations in quadrant B2. From B2 participant F's information access path seems to follow an almost straight horizontal line, cf. fi  xations 11-17 and 21. Finally, fi  xations 10, 18-20 and 22 are navigation-related as already explained above.

<!-- image -->

Figure 8. Participant F's transition matrix data for the Danish lemma 'direktion'

| Fix No.   | 1   | 2   | 3   | 4   | 5   | 6   | 7   | 8   | 9   |
|-----------|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| AOI       | A2  | A2  | B2  | B2  | A2  | A2  | A2  | A2  | B2  |

Table 14. Participant F's search-related fi  xations

Table 15. Participant F's processing-related fi  xations

| Fix No.   | 11   | 12   | 13   | 14   | 15   | 16   | 17   | 18   | 19   | 20   | 21   |
|-----------|------|------|------|------|------|------|------|------|------|------|------|
| AOI       | B2   | B2   | B2   | B2   | B2   | B2   | B2   | C1   | C1   | C1   | B1   |

Table 16. Participant F's navigation-related fi  xations

| Fix No.   | 10   | 18   | 19   | 20   | 22   |
|-----------|------|------|------|------|------|
| AOI       | C1   | C1   | C1   | C1   | A1   |

In summary, experiment II demonstrated the user consultation behaviour of the three participants and their information access paths and the sequence of their fi  xations.

Looking back at the eye-tracking survey, I believe that a number of learning points could be highlighted. First, the number of participants involved in the two experiments should have been considerably higher. This would have increased the total amount of eye-tracking data collected and would have made it easier to produce conclusive evidence for contending that there is a connection between the user profi  le and the user consultation behaviour. Second, additional pre-survey tests should have been performed to fi  nd the optimum time threshold setting for this type of eye-tracking experiment. I believe that the time threshold of the eye-tracking system perhaps should have been higher, for example 200-300 ms, because the user consultation behaviour of Internet dictionary users resembles the processes measured in reading and translation studies. A different setting might have improved the quality and comparability of the data acquired and thus improved the total data basis. Third, additional pre-survey tests should have been performed to identify the optimum way of measuring the different phases of the user consultation behaviour and it might be argued that the access process involves additional sub-steps such as navigation, writing, search, reading, processing and navigation.

However, despite the weaknesses of the eye-tracking study discussed in this article, the survey constitutes a new and empirically based method, which admittedly should be developed further, but which is suitable and relevant in Internet lexicography to gain additional insight into the user consultation behaviour. So despite the few participants involved and the somewhat tautological results, which merely seem to confi  rm what we already intuitively know about the user and the user consultation behaviour, it is argued that this eye-tracking study seems to indicate that the user consultation behaviour depends on several factors such as lexicographic function, user situation and user profi  le etc. This eye-tracking study thus adds valuable insight into user consultation behaviour and thus also into the development of more effective information tools. One line of research that I would like to pursue in continuation of the work presented in this article is to further test the connection between the lexicographic function, the user profi  le and the user consultation behaviour.

## 6. From Eye-tracking Metrics to Improved Design and Needs-adapted Information Access?

A number of general surveys have already established that the reading patterns and information access of readers of Internet pages are dramatically different from that of readers of paper-based texts, see for example Poynter Institute's Eyetrack III study from 2004 in Poynter (2004) and Jacobsen (2008), who offered some interesting conclusions in a study of website readers: that all types of readers had diffi  culties fi  nding the information they needed, that short, compact segments or boxes of text were preferred by especially inexperienced readers, that most users preferred a centre column design and that most users did not like scrolling.

The data from this eye-tracking study, supported by the interview data, seem to suggest that the information access process is connected with the user profi  le and the user situation in question, and the data confi  rm some of the conclusions reached by Jacobsen (2008). Participant D, who has three years of experience within the technical register, performed meticulous horizontal processing of the lexicographic data and states during the interview that 'I think I spent most time reading or studying the data - or focusing on something to understand it.' when asked where her eyes fi  xated during the experiment, while participant A, who has more than ten years of experience within the economic register, performed more brief consultations and said during the interview that ' I am quite experienced in this fi  eld, so most of the times I used the dictionary to confi  rm my translation - I quickly looked down to confi  rm my approach'.

## 6.1.  Fixations

The fi  xation counts are shown graphically in Figures 3, 4 and 5 by means of the black numbers in the blue circles.

As is shown in Figures 3-5 and Tables 1-3 above, participant D and participant A had 31 and 12 fi  xations in the AOI search fi  eld and 44 and 14 fi  xations in the AOI article fi  eld, respectively. These data indicate that these participants had more fi  xations in the article fi  eld than in the search fi  eld. Participant F, however, had 8 fi  xations in the search fi  eld, but only 5 fi  xations in the article fi  eld. All this is supported by the interview data. When asked 'where did your eyes fi  xate during the experiment? ' Participant A for example said: 'I think my focus was after I looked up a word. I am quite experienced in this fi  eld, so most of the times I used the dictionary to confi  rm my translation - I quickly looked down to confi  rm my approach. If I did not know the translation I spent more time looking through the examples' and participant D said 'I think I spent most time reading or studying the data - or focusing on something to understand it.'

## 6.2.  Gaze times

The gaze time also appears from the gaze plot charts above in Figures 3, 4 and 5. As will appear from the data measured in milliseconds and tabulated in Tables 4-6 above, participant D looked longer at the AOI article fi  eld than at the AOI search fi  eld, while participant A and participant F spent considerably longer time looking at the AOI search fi  eld than at the lexicographic data in the AOI article fi  eld. Again these data are supported by the interview data, where both participant A and participant F stated that they spent most time scrutinizing data. Participant A states: 'I think I spent most time reading or studying the data - or focusing on something to understand it'.

## 6.3.  Transition matrix data - experiment I

As is shown in Tables 7, 8 and 9, almost all transitions were AOI internal. It was expected that the participants would have had considerably more cross-AOI transitions, because participants were expected to go back and forth between the two AOIs, but that was not the case. As explained above, transition matrix data show the transitions of a participant's eyes and together with the Gaze replay functionality in the ClearView software these data indicate potential information access paths and scanning sequences. The fact that there were relatively few transitions between the two overall AOIs seems to indicate that the user behavior can be divided into three overall operations: a search-related operation, a data-processing operation and a navigation-related operation, which transition matrix experiment II also showed.

The transition matrix data from experiment I showed that participant D for example had 24 AOI internal transitions from AOI search fi  eld to AOI search fi  eld and 38 AOI internal transitions from AOI article fi  eld to AOI article fi  eld, and very few AOI external transitions. In fact participant D only had 6 AOI external transitions from AOI search fi  eld to AOI article fi  eld and 6 AOI external transitions from AOI article fi  eld to AOI search fi  eld.

These data seem to indicate that the scanning paths of the participants are relatively fi  xed and focused on the operation in question, that is, whether or not the user is performing a search operation or processing data. In other words, the users' scanning paths were relatively fi  xed and restricted to a small area of interest. Similar conclusions were reached by Morrison et al. (1997), who found that eye-tracking data can be used to evaluate the design of decision support systems and to evaluate the presentation of information in Human-Computer Interaction displays in military combat information centres.

The very few cross-AOI transitions furthermore seem to suggest that Internet dictionary design should take into account the preference of Internet dictionary users for fi  xating within relatively small areas of interest and the scanning paths and scanning patterns of the participants also support this interpretation.

## 6.4.  Transition matrix data - experiment II

As already described above, the purpose of transition matrix experiment II was to analyse the participant's information access path and sequence of fi  xations. The research design used in experiment II was much more fi  ne-meshed as it was based on nine AOIs, cf. Appendix E.

The data from transition matrix experiment II support the conclusions from experiment I, but they also give a detailed overview of the entire access process of the participants.

As is shown in Figures 6-8 the majority of transitions were AOI internal. The transition matrix data show that the participants were prone to looking within small, designated areas especially in AOIs A1 and B2, see Figures 7-8 above. The transition matrix data also seem to show that the two primary information access processes: the search-related operation and the data processing-related operation are separated in time and location. Not surprisingly, participants use less time and fewer fi  xations in the search-related AOIs in comparison with the data processing-related AOIs, cf. Figures 6-8.

The search operation and the data processing operation are thus two separate operations. The search operation can be described as a writing process during which the participant focuses on entering the search string in the search fi  eld and the processing operation is to a larger degree an exploratory process during which users explore the various types of data. This is a natural process, and the theoretical implication of this is to further develop the integrated web page in Internet dictionaries to facilitate the two different processes: the writing/searching process, which is a very focused consultation behaviour and the reading/processing process, which is exploratory, cf. Figures 6-8. This indicates that there is a relation between the actual operation performed (searching or processing) and the information access structure needed.

In this study the participants had one objective with two sub-steps: to fi  nd the lexicographic data and to read, understand and extract the lexicographical information needed to translate a text from Danish into English. These two different needs result in two different access processes: a very focused search-operation where the user fi  xates in or around the relatively restricted area of the search fi  eld and a more unfocused data processing operation, where the user either scans the lexicographical article with a view to fi  nding, reading, understanding and extracting the English equivalent needed or meticulously reads the lexicographical article line by line following the usual direction of reading, cf. Figures 4-6 above from which it appears that participant A follows a horizontal and vertical information access path and participant F follows a predominantly horizontal information access path.

Finally, even though the data are far from conclusive, it is interesting, but not surprising, to fi  nd that the user profi  le and degree of experience also play a large role for not only the time consumption in connection with search and data-processing operations (fi  xations and gaze times), but also the user consultation behaviour (information access paths). The purpose of this survey was not solely to study the relationship between the user profi  le and the user consultation behaviour, but a number of interesting trends were nevertheless found. Participant A for example, is a very experienced translator with more than 10 years of experience with translation of texts from the fi  nancial and economic register. Her consultation behaviour is both horizontal and vertical as described above, which seems to indicate that she quickly identifi  es the equivalent, reads a few words and scans the lexicographic article, because she only needs to check what she already knew or become reassured of her own translation proposal.

A similar trend can be observed by analysing participant F's user profi  le and user consultation behaviour. According to Appendix A, which outlines the participant profi  les, participant F is also a certifi  ed translator, but with only two years of experience from the technical register. As is shown in Appendix A, both participant A and participant F report that their knowledge level of Den Dansk-Engelske Regnskabsordbog , cf. www.regnskabsordbogen.dk (Danish-English Accounting Dictionary) is high on a low-medium-high scale. Participant F's user consultation be- haviour is characterized as being primarily horizontal, which suggests that she meticulously reads line by line, because she feels she needs to be completely sure which equivalent to choose.

## 7. Conclusion

This paper outlined and discussed an eye-tracking study specially designed for Internet lexicography. The method developed and discussed is based on data from an eye-tracking study of six participants and post-survey interview data, and it is argued that empirical data on the user consultation behaviour are useful in the ongoing development of more effective information tools, especially because different users with different user profi  les seem to have different user consultation behaviours. Consequently, additional knowledge of this behavior may help us develop and design more effective information tools, whose data to a higher extent are specially adapted to the user and in sync with the user need, user situation and user profi  le.

The paper demonstrated how six users of an Internet dictionary located and processed lexicographic data in an L1-L2 translation situation and demonstrated not only at what the participants looked, but also how they accessed the lexicographic data to be able to translate. The eye-tracking study used a total of three measures to harvest data on the user consultation behavior: the number of fi  xations, the length of gaze times and the number of transitions. Two transition matrix experiments were also carried out to further analyse the scan path and scan sequence of the participants. The fi  rst transition matrix experiment was based on two AOIs to measure the overall consultation behaviour in search-related operations (search and writing) and article-related operations (reading and processing) and the second transition matrix experiment was based on nine AOIs to get more detailed data on the information access path or scan path of the participants.

The article discussed the analysis of two fundamental processes in lexicography: to locate the lexicographic data and to read, understand and extract the lexicographical information needed to translate a text from Danish into English. This results in two access processes: a very focused search-operation where the user fi  xates in or around a relatively small area of the search fi  eld and a more exploring data processing operation, where the user either scans the lexicographical article with a view to fi  nd, read, understand and extract the English equivalent needed or where the user meticulously reads the lexicographical article line by line following the usual direction of reading.

The analysis of the gaze time data showed that two of the three participants spent more time gazing at the search fi  eld than at the lexicographic article. The analysis of the eye-tracking data combined with the analysis of the interview data and the user profi  le characteristics indicate a correlation between the user profi  le (experience and area of expertise) and the user consultation behaviour and how the user accesses and processes data. Participants A and F for example are both experienced translators of accounting texts and they actually had more fi  xations and spent more time on the search process than on the data processing process, which was consistent with the interview data where they both stated that they just wanted to check a term and did not have to scrutinize data. Only participant D spent more time fi  xating on the lexicographic data which again is consistent with her user profi  le as she is an expert on translating technical texts. The gaze time data thus indicate a correlation between user profi  le characteristics and the user consultation behaviour.

The analysis of the transition matrix data support the above fi  ndings but also show a more detailed picture of how the participants accessed the lexicographical data and in which sequence. The data showed that most transitions were in fact AOI-internal, and additional analysis of the data by means of the Gaze Replay option showed that the participants were prone to looking within small, designated areas, that is, inside the search-related AOI or the article-related AOI.

The analysis of the transition matrix data also showed the scan paths and scan sequences of the participants and showed that there were three types of fi  xations: search-related fi  xations, which occurred in or around the search fi  eld, processing-related fi  xations, which occurred in or around the actual lexicographic article and navigation-related fi  xations, which occur in or around menus.

The data indicated that the scanning paths of the participants were relatively fi  xed and focused on the actual operation, that is, whether or not the user is performing a search operation or processing data.

The data indicated that users of Internet dictionaries process lexicographical data in accordance with not only the lexicographical function, but also in accordance with their user profi  le. Obviously, a much more extensive study is needed to provide conclusive evidence for this contention, but the theoretical implications of this study are to realize that the individual user profi  le, the lexicographic function and the specifi  c user situation play a crucial role for the development of the data access structure of Internet dictionaries.

## 8. Bibliography

## Literature:

- Almind, Richard 2008: Søgemønstre i logfi  ler. In LexicoNordica 15, 33-55.
- Atkins, Beryl T. Sue/Varantola, Krista 1997: Monitoring Dictionary Use. In International Journal of Lexicography 10:1 , 1-45.
- Béjoint, Henri 1981: The Foreign Student's Use of Monolingual English Dictionaries: A Study of Language Needs and Reference Skills. In Applied Linguistics 2, 207-222.
- Benbow, Timothy/Carrington, Peter/Johannesen, Gayle/Tompa, Frank/Weiner, Edmund 1990: Report on the NEW Oxford English Dictionary User Research. In International Journal of Lexicography 3:3, 155-203.
- Bergenholtz, Henning/Johnsen, Mia 2005: Log Files as a Tool for Improving Internet Dictionaries. In Hermes, Journal of Linguistics 34, 117-141.
- Bergenholtz, Henning/Johnsen, Mia 2007: Log fi  les can and should be prepared for a functionalistic approach. In Lexikos 17, 1-21.
- De Schryver, Gilles-Maurice 2003 : Lexicographers' Dreams in the Electronic-Dictionary Age. In International Journal of Lexicography 16:2, 143-199.
- De Schryver, Gilles-Maurice/Joffe, David 2004: On How Electronic Dictionaries are Really Used. In Williams, G. /Vessier. S. (eds.), In Proceedings of the Eleventh EURALEX International Congress, EURALEX 2004, Lorient, France, July 6-10, 2004, 187-196.
- De Schryver, Gilles-Maurice/Joffe, David/Joffe, Pitta/Hillewaert, Sarah 2006: Do Dictionary Users Really Look Up Frequent Words? - On the Overestimation of the Value of Corpus-based Lexicography. In Lexikos 16, 67-83.
- Diab, Turki 1989: The Role of dictionaries in English for specifi  c purposes: a case study of students at the university of Jordan. In James Gregory (ed.), Lexicographers and their works. University of Exeter, 74-82.
- Duchowski, Andrew. T. 2007: Eye tracking methodology: Theory and practice. Second Edition. London: SpringerVerlag Ltd.
- Farø, Ken/Gottlieb, Henrik 2007: Socioleksikografi   - eller: 'Det sekundære og afl  edte er ordbøgerne'. Ny dansk disputats. In LexicoNordica 14, 189-199.
- Hartmann, Reinhard Rudolph Karl 1989: Sociology of the Dictionary User: Hypotheses and Empirical Studies. In Franz Josef Hausmann/Oskar Reichmann/Herbert Ernst Wiegand/Ladislav Zgusta (eds.), Wörterbücher, Dictionaries, Dictionnaires. An International Encyclopedia of Lexicography, First Volume 1989 . Berlin/New York: Walter de Gruyter, 102-110.
- Hartmann, Reinhard Rudolph Karl 2008: Twenty-fi  ve Years of Dictionary Research: Taking Stock of Conferences and Other Lexicographic Events since LEXeter '83. In Proceedings of XIII EURALEX International Congress, Barcelona, 131-148.
- Hult, Anki 2008: Från ord till handling - en studie i ordboksanvändning på nätet. In Nordiske studier i leksikografi   10. Rapport fra Konference om leksikografi   i Norden. Island 22.-26. maj 2007. Akureyri: Nordisk forening for leksikografi   2007, 243-257.
- Jacobsen, Peter From 2008: Eyetrack: Sådan læser folk dine net-artikler [online]. http://www.update.dk/cfje/vidbase. nsf/ID/VB01751618 (available online 18 January 2009).
- Johnsen, Mia Steen 2005: Logfi  ler som leksikografi  sk analyseinstrument og hjælpeværktøj . Cand.ling.merc.-speciale, ASB [online]. http://theses.asb.dk/projekter/fbspretrieve/2040/000139835-139835.pdf (available online 12 November 2007).

- Just, M. A./Carpenter, P. A. 1980: A theory of reading: from eye fi  xations to comprehension. In Psychological Review 87, 329-354.
- Ling, Charles X/Gao, Jianfeng/Zhang, Huajie/Qian, Weining/Zhang, Hongjiang 2001: Mining Generalized Query Patterns from Web logs. In Proceedings of the 34th Annual Hawaii International Conference on System Sciences (HICSS-34) ,  5 . Washington,  DC:  IEEE  Computer  Society  Press  5020  [online].  http://research.microsoft.com/~jfgao/ paper/hicss01.pdf (available online 12 November 2007).
- Ling, Charles X/Gao, Jianfeng/Zhang, Huajie/Qian, Weining/Zhang, Hongjiang 2002: Improving Encarta Search Engine Performance by Mining User Logs. In International Journal of Pattern Recognition and Artifi  cial Intelligence 16:8, 1101-1116.
- Morrison, Jeffrey, G./Marshall, Sandra, P./Kelly, Richard, T./Moore, Ronald, A. 1997: Eye Tracking in Tactical Decision Making Environments: Implications for Decision Support Evaluation. In Proceedings from Third International Command and Control Research and Technology Symposium , National Defense University, June 17-20 [online]. http://all.net/journal/deception/www-tadmus.spawar.navy.mil/www-tadmus.spawar.navy.mil/Eye\_Trkr.pdf (available online 18 January 2009).
- Nesi, Hilary 2002: A Study of Dictionary Use by International Students at a British University. In International Journal of Lexicography 15:4, 277-305.
- Poole, A./Ball, L. J. 2004: Eye Tracking in Human-Computer Interaction and Usability Research: Current Status and Future Prospects. In Ghaoui, Claude (Ed.), Encyclopedia of Human Computer Interaction . Idea Group.
- Outing, Steve/Ruel, Laura 2004: Eyetrack III: What We Saw Through Their Eyes [online]. http://www.poynterextra. org/EYETRACK2004/index.htm (available online 4 January 2009).
- Rayner, Keith 1998: Eye Movement in Reading and Information Processing: 20 Years of Research. In Psychological Bulletin 1998 , vol. 124:3, 372-422.
- Simonsen, Henrik Køhler 2002: TeleLex - Theoretical Considerations on Corporate LSP Intranet Lexicography: Design and Development of TeleLex - an Intranet-based Lexicographic Knowledge and Communications Management System . Unpublished PhD thesis. Aarhus School of Business.
- Simonsen, Henrik Køhler 2009a: Vertical or Horizontal? That is the Question: An Eye-Track Study of Data Presentation in Internet Dictionaries [online]. http://www.cbs.dk/forskning/konferencer/eye\_to\_it (available online 10 April 2009).
- Simonsen, Henrik Køhler 2009b: Se - og du skal fi  nde: en eyetrack-undersøgelse med særlig fokus på de leksikografi  ske funktioner. In Nordiske studier i leksikografi   11. Rapport fra Konference om leksikografi   i Norden. Finland 3.-5. juni 2009. Tampere: Nordisk forening for leksikografi   2009, 274-288.
- Tarp, Sven 2006: Leksikografi  en i grænselandet mellem viden og ikke-viden: Generel leksikografi  sk teori med særlig henblik på lørnerleksikografi . Doktorafhandling. Aarhus School of Business.
- Wiegand, Herbert Ernst 1977: Einige grundlegende semantisch-pragmatische Aspekte von Wörterbucheinträgen. Ein Beitrag zur praktischen Lexikologie. In Kopenhagener Beiträge zur Germanistischen Linguistik 12, 59-149.
- Wiegand, Herbert Ernst 1985: Fragen zur grammatik in Wörterbuchbenutzungsprotokollen. Ein Beitrag zur empirischen Erforschung der Benutzung einsprachigen Wörterbuch. In Lexikographie und Grammatik. Akten des Essener Kolloquiums 1984, hrsg. von Henning Bergenholtz und Joachim Mugdan. Tübingen: Niemeyer, 20-98.
- Wiegand, Herbert Ernst 1988: Wörterbuchforschung: Untersuchungen zur Wörterbuchbenutzung, zur Theorie, Geschichte, Kritik und Automatisierung der Lexikographie. 1. Teilband. Berlin. Walter de Gruyter.

## Dictionaries:

- Den Dansk-Engelske Regnskabsordbog = Sandro Nielsen, Lise Mourier and Henning Bergenholtz in cooperation with Mads Melgaard, Trine Middelboe, Brit Sørensen, Mia Johnsen, Rie Bobjerg Nielsen, Jóna Ellendersen, Amalie Kofoed Stender and Vibeke Vrang. 2004-2007. Den Engelsk-Danske Regnskabsordbog / English-Danish Dictionary of Accounting. Database and design: Richard Almind. Implementation and encoding of web pages: Caspar Thomsen [online]. http://www.regnskabsordbogen.dk/regn/dkgb/dkgbregn.aspx (available online 12 November 2007).
- Lexin - et lexikon för invandrarundervisning = Språkrådet [online]. http://lexin2.nada.kth.se/lexin.html (available online 12 November 2007).
- Ordbogen over Faste Vendinger = Center for Leksikografi  : Ordbogen over Faste Vendinger . Århus: Handelshøjskolen i Århus [online]. http://www.idiomordbogen.dk (available online 4 January 2009).
- Sesotho Sa Leboa Dictionary = Sesotho sa Leboa (Northern Sotho) - English Dictionary [online]. http://africanlanguages.com/sdp/ (available online 12 November 2007).

## 9. Appendix

## Appendix A. Participant Data

| Name          | Occupation               | Company                                           | Experience          | Expertise Area         | Knowledge Level   |
|---------------|--------------------------|---------------------------------------------------|---------------------|------------------------|-------------------|
| ParticipantA  | Certified translator, MA | Bech Bruun www.bechbruun.com                      | 10 years experience | Economic               | High              |
| Participant B | Certified translator, MA | Dialog Translatørservice http://www.dialog-ts.dk/ | 3 years experience  | Legal                  | High              |
| Participant C | Certified translator, MA | Godt Sprog                                        | 12 years experience | Literature             | Medium            |
| Participant D | Certified translator, MA | COWI www.cowi.com                                 | 3 years experience  | Technical              | High              |
| Participant E | Certified translator, MA | Semler Sprog www.semler-sprog.com                 | 8 years experience  | Technical              | Medium            |
| Participant F | Certified translator, MA | Dantranslation www.dantranslation.com             | 2 years experience  | Technical and Economic | High              |

## Appendix B. Source Text and Lemmata

## Ledelsespåtegning

Movias direktion har behandlet og godkendt årsregnskabet for regnskabsåret 1. januar - 31. december 2007.

Årsregnskabet er aflagt i henhold til gældende lovgivning og efter de retningslinjer, der er fastlagt af Velfærdsministeriet  i  Budget-  og  Regnskabssystem  for  kommuner.  Vi  anser  den  valgte regnskabspraksis for hensigtsmæssig således, at årsregnskabet giver et retvisende billede af aktiver og passiver samt den finansielle stilling.

Movias årsregnskab er godkendt på bestyrelsesmødet den 22. maj 2008.

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Johannes Sloth

Administrerende direktør C. Den Dausk Fogolkar Reguskabsurdog - Windows Internet Exploner

2http://www.regrskabsordbogen.d/regn/dhcb/dcbregn.asux

Googie 8-

3001

eDen Dersh-Engelsie Regns absordboa

Chofer for Laksikografi

Handelshajskolen

i Arbus Regnskabsordbøgerne

## Appendix C. Hot Spot Charts

Regnakabsonliegerme

Must You See

Uctert

2 start

STUDY: Lex\_Survey\_January09

NUMBER OF RECORDINGS: 1.

TIME SEGMENT: Include data from 0 ms to 21061 ms.

Den Dansk-Engelske Regnskabsordbog

<!-- image -->

A hot spot chart offers a quick overview of user activity on a page. By means of colours it displays areas where users have had many and long fi  xations (red), areas where users have scanned briefl  y (green) and areas which users have completely ignored (grey).

rind " Kontroler mE Autoryld -

9. 0

HE X

Logind - C Den Dansk Fryp-lakar Rompuskatsurlg : Windows Internet Exploner http://www.regrskabsordbogen.d./repn/dhgb/dcbregn.asux

8-

Googie ei ben Denst-Engelste Regnst absordboa Center for Leksikografi Astur Schack it Sam Handelsbajskolen i Artus

44 X

9 - 0

HHA I Campistel I ablietel

Regnskabsordbøgerne

<!-- image -->

Qi- a0- t2 dogmerer= rind - " Kontroler ·E Autofyld -

Udtert

- start CE C

9 Mete son Me....

STUDY: Lex Survey January09

NUMBER OF RECORDINGS: 1.

TIME SEGMENT: Include data from 0 ms to 11187 ms.

- Logind -

Side = Furitioner ·

e Den Dansk-Engelske Regnskabsordbog - Windows Internet Explorer

@ http:j/www.regnskabsordbogen.dkjregn/dkgb/dkgbregn.aspx

Google

8 - Søg *

e Den Dansk-Engelske Regnskabsordbog

Center for Leksikografi

Handelshøjskolen

· = x (Google

& - · Log ind

· - 5 · · Esde - @:Funktioner -

HHÃ | Campusiiet | Biblioteket

Marath akok Regnskabsordbøgerne

) Lise Jørgensen - Micr.)

-start G O

STUDY: Lex\_Survey\_January09.

NUMBER OF RECORDINGS: 1.

TIME SEGMENT: Include data from 0 ms to 10935 ms.

<!-- image -->

S - m-

* Bogmærker - Find - * Kontroller - E AutoFyld -

& Den Dansk-Engelske Regnskabsordbog - Windows Internet Explorer

· E http://www.regrskabsordbopen.dk/regn/dkgb/dkpbregn.aspx

Google 8-

* $

e Den Dansk-Engelske Regnskabsordbog

Center for Leksikograli

MAndelbunkoben v ++

· · ·

HHÅ | CampusNet | Biblioteket

Regnskabsordbøgerne

## Appendix D. AOIs for Transition Matrix Experiment I

Regnskabsordbegerne Den Dansk Engelske Regnskabsordbog

<!-- image -->

Udlert

i start CE O de what You Ste

STUDY: Lex\_Survey\_January09

TIME SEGMÊNT: Only include fixations inside interval (0,450960) ms.

This screen dump from ClearView shows the two AOIs used in Experiment I. The screen dump also shows participant D's fi  xations and fi  xation numbers in connection with the lemma 'retvisende billede'.

Da - m- * Dogmaerker= Find - * Kontroler - AutoFyld - He x P -

9 - Logind -

- · Side · ( Funktioner ·

»

C Den Dansk-Engelske Regnskabsordbog - Windows Internet Explorer

http://www.regnskabsurubogen.dk/repn/dhgb/dkgbregn.aspx

Google 8 -

ll pen Dansk-Engelike Regnskabsordboo

Center for Leksikografi

Astur Scholl at a

Idelshajskolen

MArhus

5 · R

HA / Campuatiet | Biblioteket

Regnskabsordbøgerne

Appendix E. AOIs for Transition Matrix Experiment II

Den Dansk Engelske Regnskabsordbog

<!-- image -->

start Ce ode loc-M...

STUDY: Lex\_ Suvey\_January09

TIME SEGMENT: Only include fixations inside interval [73200 93204] ms.

This screen dump from ClearView shows the nine AOIs used in Experiment II. The screen dump also shows participant F's fi  xations and fi  xation numbers in connection with the lemma 'direktion'.

@ - - * Bogmarker-rind - ** Kontroler + E Autoryid - LB x Loo ind - ide · (Funktioner -