# Added Value of Eye Tracking in Usability Studies: Expert and Non-expert Participants

Marco C. Pretorius, Judy Biljon, Estelle Kock

► To cite this version:

Marco C. Pretorius, Judy Biljon, Estelle Kock. Added Value of Eye Tracking in Usability Studies: Expert and Non-expert Participants. Second IFIP TC 13 Symposium on Human-Computer Interaction (HCIS)/ Held as Part of World Computer Congress (WCC), Sep 2010, Brisbane, Australia. pp.110-121, 10.1007/978-3-642-15231-3_12. hal-01055488

HAL Id: hal-01055488

https://inria.hal.science/hal-01055488

Submitted on 12 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Added Value of Eye Tracking in Usability Studies:
Expert and Non-expert Participants

Marco C. Pretorius¹, Judy van Biljon¹, Estelle de Kock¹

¹ School of Computing
University of South Africa
Pretoria, South Africa

marco.pretorius@gmail.com, vbiljja@unisa.ac.za, dkocke@unisa.ac.za

**Abstract.** This paper investigates the value of eye tracking in evaluating the usability of a Learning Management System, at an open distance learning university where the users' computer and Web skills vary significantly. Eye tracking utilize the users' eye movements, while doing a task, to provide information about the nature, sequence and timing of the cognitive operations that took place. This information supplements, but does not replace standard usability testing with observations. This forces the questions of when the added value of eye tracking justifies the added cost and resources. Existing research has indicated significant differences in the usability experienced by experts and non-experts on the same system. The aim of this paper is to go one step further and shed light on the type and severity of the usability problems experienced by non-expert users. Usability testing with eye tracking is a resource intensive method but our findings indicate that eye tracking adds concise, summarised evidence of usability problems that justifies the cost when testing special groups such as users deficient in Web and computer skills. The contribution of this paper is to highlight the added value of eye tracking as a usability evaluation method in working with Web non-expert users. Furthermore, the findings improve our understanding of the knowledge differences between expert and non-expert Web users and the practical challenges involved in working with non-expert users.

**Keywords:** Usability, Eye tracking, expert, non-expert.

## 1 Introduction

The Web is evolving into the preferred delivery mode for Learning Management Systems (LMS) which offer services such as student enrolment, coursework notice boards and assignment submission. A LMS is defined as a Web-based application used by institutions and companies that want to get involved in e-learning either for providing services to third parties, or for educating and training their own people [1]. In developing countries the students' computer and Web skills vary, some are experts with sophisticated skills while others lack the most basic computer skills. The
problem is compounded by the fact that the system is used occasionally and under
time constraints such as registering or submitting an assignment before a due date and
even financial constraints where students use commercial internet facilities. When the
Web interface of a LMS becomes the only access point, usability becomes critical.
Designing for novice users has been well-researched. However, eye tracking as
evaluation tool for evaluating non-expert web users is novel. This raised the question:
What is the added value of eye tracking in evaluating non-expert web users?

We can predict that computer expertise will influence usability positively, but the use of eye tracking could possibly help to find the severity of the effect on non-expert users and if anything can be done to improve the usability for these users. To answer these questions we conducted a usability study on the assignment submission section of a university LMS purposively selecting expert and non-expert users respectively. In order to identify the added value of eye tracking, we captured data by different means including eye tracking, observation (direct observations and video recordings) and a post-test questionnaire. We analysed the data sets separately and compared the findings to see if eye tracking did contribute additional knowledge.

Eye tracking has been used to research the difference between experienced and less experienced users in information retrieval tasks [2] and different styles have been associated with experienced and less experienced users [3]. In general expert users perform faster, more accurate and have more defined search paths whereas non-experts waste time searching or looking at irrelevant information [4, 5]. However, eye tracking has not previously been widely applied to studies of behavioural aspects of students using a LMS. What is lacking is an explanation of the implications the difference between expert and non-experts could have for the usability of a LMS.

The main contribution of the paper is to identify the added value of eye tracking in
investigating usability problems experienced by non-experts in using a LMS. The
scope of the research is limited to the evaluation of one LMS but the data sets
captured through video and audio recordings, eye tracking video recordings, eye
tracking data files and post-test questionnaires are triangulated to provide more
reliability.

## 2 Related Usability Studies

Any research comparing expert and non-expert users of an interactive system has to
take cognisance of the fundamental distinction between these two, founded in their
knowledge and experience and therefore we first review knowledge categories before
discussing usability evaluation paradigms, focusing on heuristic evaluation and eye
tracking.

There are different empirical definitions of experts and non-experts but two (strongly
overlapping) criteria used for the differentiation between experts and non-experts are
*knowledge* and the *time* spent working with a particular system [6]. Given this fact
that expertise is related not only to time spent working with the system, we use the
term non-expert rather than novice. Furthermore, the term non-expert is useful for
grouping categories such as novices and intermediate users, both not expert.
Focusing on the knowledge component, Prumper et al. [6] state that experts operate
at higher cognitive levels, i.e. using knowledge from different categories than non-experts do.

Working with non-experts require more structure in terms of doing tasks, more assistance and more encouragement in reporting problems [7], but Fields, Keith et al. [8] emphasize the value of user groups with varying forms of expertise. Popovic [9] states that all interactive interface design should support and facilitate the transitional process from novice to expert user. Gorman [10] classifies four knowledge categories, i.e. declarative knowledge (what), procedural knowledge (how), judgement knowledge (when) and wisdom (why) that we will use as reference.

The way experts and non-expert users use interactive Web sites is differentiated by the difference in their knowledge (particularly domain-specific knowledge) [9] and also by the way they approach and solve problems [11]. Äijö and Mantere [7] conducted a study on the usefulness of non-experts in identifying the usability problems in an interactive Web service. They found that non-experts were capable of reporting usability problems on the interface but had difficulties in reporting usability problems related to interactivity with the system. This concurs with Kotze et al. [12] who state that although experts and non-experts may differ in all the categories, novices rely more on declarative and to a lesser extent on weak heuristics based on procedural knowledge while experts rely more on judgment knowledge.

We now review the use of eye tracking in usability studies. The human eye reads a line of text in discrete chunks through a series of fixations and fast eye movements to take in the next section [13]. Eye tracking is based on the fact that a record of a person's eye movements while doing a task provides information about the nature, sequence and timing of the cognitive operations that took place [14].

Eye tracking studies have been used in diagnosing the effectiveness of Website designs with point of interest detection and information transmission via eye movement (scan path) as two main indicators [15]. Based on this relation between cognition and eye behaviour the trace of navigation pathways and user attention patterns is used to study the cognitive processes involved in reading [3], picture perception [7, 13], visual search [16], problem solving [5], face perception [17] and many other tasks.

In HCI eye tracking has been used to study the usability of Web pages [18], menu searching, information searching from Web pages and search result evaluation [3]. Eye tracking studies on the difference between expert and non-expert users in HCI are difficult to find but some have been done in other fields. The eye movements of experts and novices were compared in a virtual aiming task done in a laparoscopic surgery training system. The results showed significant differences in performance as well as eye movement between the two groups [4]. A study in the field of aviation compared expert and novice pilots to find that experts have better defined, more consistent and efficient eye-scanning patterns [5].

Eye tracking has the advantage of providing visual evidence of usability problems but the method has been criticized for reliability and objectivity. Sutcliffe and Namoune [19] question the reliability of eye tracking for determining which features are attended because they found that users discovered areas of interest by short fixations or proximal fixations i.e., it is difficult to know exactly what the user was looking at. Further criticism on eye tracking is the objectivity: the analysis of patterns is mostly based on the opinion and interpretations of the individual evaluator [20].
Correlating eye-movement metrics with usability problems is proposed as a more
rigorous, verifiable mapping of eye tracking patterns to usability problems [20].

3 Method

The method for this usability and eye tracking study was based on the methodology
by Pretorius et al. [21]. Section 3.1 presents the participants' profile, section 3.2
discusses the procedure followed and section 3.3 provides more detail on how the
data was captured.

**3.1 Participant Profile**

The intended user group for the LMS is students who have to submit assignments online. A screening questionnaire was used to screen the participants for this evaluation. This questionnaire reflected the possible participant's LMS experience, computer and Internet experience, culture, age and gender. Regarding expert and non-expert users, results showed that individuals' ratings of their overall knowledge were better predictors than were estimations of frequency of use [22].

The screening questionnaire asked participants to rate their experience level as an
Internet user. The following ratings were available: *Never used the Web*; *Beginner* –
have read pages on the Web; *Novice* – have entered addresses and used bookmarks;
*Competent* – can use a search engine to find information; and *Proficient* – know way
around and have done Web transactions like e-banking.

From the 23 questionnaires completed, we selected 10 students, five who rated
themselves as proficient Internet users and another five who rated themselves between
a beginner and competent level. The first group were referred to as expert participants
and the latter group as non-expert participants.

The participants had all used some sections of the LMS, their experience ranged
from less than one year; between one and two years; and two years or more.

Our participants included five male and five female students. One student was
below 20; six between 21 and 25; one between 26 and 30; and two above 30. The
mother tongue (first language they learned to speak) of the 10 participants was as
follow (number of participants indicated in brackets): English (2); isiXhosa (2);
Afrikaans (2); SiSwati (1); Sesotho (1); Xitsonga (1); Yoruba (1).

**3.2 Conducting the Test**

One participant was tested at a time. On arrival, the participant was briefed about
the experiment, which was followed by an explanation of the equipment to be used.
The details of the material to be recorded were explained and the participant was
required to complete an informed consent form.

A variation of the think aloud protocol was used, where participants were asked to comment if they were looking for something and could not find it; if they liked something particular about the Website; and if they disliked something particular.
Participants completed three tasks available from a task list (discussed in Section 4). Following the tasks, a quick interview was conducted to ask the participants what they liked most about the system, what they did not like, as well as questions about issues that the administrator picked-up. This was followed by a post-test questionnaire and a debriefing where the participant was thanked and given the opportunity to see the data.

## 3.3 Data Collection Method

Data was collected and calculated as follows: live video recordings were captured, including the screen, participant's face and mouse/keyboard movements; notes were taken during the test as well as a full evaluation of the video at a later stage; audio in the form of the participant or the test administer speaking were included with the video files; eye tracking video recordings included a cursor which indicates the participant's eye movements; eye tracking data files; a post-test questionnaire was used to capture the participants' perception of the user interface and the system; and continuous monitoring of tasks.

The usability laboratory consists of an observer room and a participant room, separated by a one-way mirror. The participant room is equipped with a 17" TFT monitor with resolution of 1280x1024 and a Tobii 1750 eye tracker, allowing the eye movement of participants on the screen to be recorded. A 9-point eye tracking calibration was used at all times.

# 4 Results and Findings

This section discusses the results and findings obtained from the usability test.

## 4.1 Task 1

The participant had to submit a written assignment (a Word file) for this task. This was done by entering or browsing for the correct file, before selecting the appropriate file format from a drop-down box. The Word file format was the default choice.

The usability results, as depicted in Table 1, showed a clear distinction between expert participants and non-expert participants. The expert participants had no problems with this task with task completion time ranging between 27 and 50 seconds. Expert participants made no errors and no assistance were needed.

In contrast, the non-expert participants struggled a great deal with task completion times significantly longer. Four of these participants made errors during this task; three needed assistance; and one was not able to complete the task. The participant who could not complete the task received assistance, but this did not help; a complete run through of the task was needed. The task completion time of this participant was removed from the sample (exceeded nine minutes). This task had to be turned into a training task that would help the participant to complete the subsequent tasks in the
task list. This non-expert participant did not understand computer terms such as "minimise" and "desktop".

<table><caption>**Table 1:** Usability results for task 1</caption><thead><tr><td></td><td>Completed</td><td>Time (in seconds)</td><td>Errors</td><td>Assistance needed</td></tr></thead><tbody><tr><td>Expert</td><td>5</td><td>27-50</td><td>0</td><td>0</td></tr><tr><td>Non-Expert</td><td>4</td><td>68-161</td><td>4</td><td>3</td></tr></tbody></table>

None of the non-expert Web participants could comprehend the term "File Name", where the name of the task file is entered or selected. One participant asked what a file name is and another entered the course code as the file name. The participant asked "how do I load a file from my PC to the LMS?" with a confused facial expression. After clicking "continue" without entering a file name, the participant received an error message which read: "ERROR: The type of file does NOT match the selected file type. (xxxxx!=DOC)". The participant did not understand the error message.

One participant clicked on "continue" without entering a file name; after the error message the participant entered the course code as the file name. Another participant entered his own name in the file name text box. The participant clicked 'continue' and received an error message; repeating the same mistake three times.

The eye tracking results provide clear evidence of the difference in usability between expert and non-expert participants. The scan paths for non-expert participants (Figure 1) are much longer and have more fixations scattered over the screen. This shows us that participants searched and did not know exactly what to do.

As stated in the usability results, all five non-expert Web participants could not comprehend the word "File Name".

<figure><img src="image_2.png" /><figcaption>**Figure 1:** Non-expert participant: long scan path with many fixations.</figcaption></figure>
The scan paths for expert users (Figure 2) are much shorter with fewer fixations. Eye tracking allows an evaluator to see what the participant was reading (and not reading) thereby identifying the problem areas on the screen as well as the information that goes unnoticed.

Throughout the task list, eye tracking data showed that the majority of participants did not read the instructions. Figure 3 demonstrates such an example where a participant did not read the instructions. The instructions are indicated by a rectangle. The phrase "Here is your final chance to check that your assignment is correct" is included in these instructions. Only one of ten participants read these instructions.

<figure><img src="image_3.png" /><figcaption>Figure 2: Expert participant: short scan path with few fixations.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 3: Participant gaze plot: not reading instructions.</figcaption></figure>
## 4.2 Task 2

The participant had to submit a written assignment that included selecting a file in the correct (PDF) file format. The usability results showed a 70% completion rate but all participants made errors in not selecting the correct file format, with four participants needing assistance. Once the participants wanted to proceed with the task, without selecting the correct file format, the following error message was displayed: "ERROR: The type of file does NOT match the selected file type. (PDF!=DOC)".

All non-expert participants received this error at least twice. One participant received the error five times. The PDF file was opened, but the participant could not distinguish between a Word and PDF file. Consequently, this participant could not complete the task. One participant was given assistance after receiving the error four times. The assistance given was that the file is a PDF file and not a Word file. This did not help the participant, as the difference between PDF and Word was not understood. The participant shook her head in frustration during the task. Two participants submitted Word files instead of the PDF file after receiving the error twice.

Like in the first task, one non-expert participant entered his own name under file name again. After receiving the error six times, instructions were given as where to select the file format. The participant proceeded to guess the file format, by selecting each one available, until PDF was reached and accepted. This shows that the file format and PDF terminology is not understood. Three expert participants received the error message once only, correcting their mistake very quickly. One expert proceeded to submit the wrong file, a Word file. The last expert participant searched in Help and FAQ to find a solution.

The evidence from the eye tracking data supports the problem with understanding technological terms. Eye tracking data showed that all participants read the full error message; however only three participants (experts) continued to complete the task correctly after the first error message. None of the non-expert participants fixated on the word "File Format" or on the drop-down box arrow where the file format had to be selected. This indicates that the error message was not comprehended.

## 4.3 Task 3

Task 3 was the same as task one and would give an indication of the learnability of the Website. The usability results showed that eight of the participants completed the task faster than in task one. The average task time went from 68.67 to 34.70 seconds. Task completion was 100%. This demonstrates that once a participant knows the basics of this Website, they use it without problems.

The expert participants had no problems with this task with task completion time ranging between 10 and 60 seconds. Expert participants made no errors and no assistance was needed. In this case, the two participants whose times did not improve were experts who double checked the files they were submitting.
<figure><img src="image_5.png" /><figcaption>Figure 4: Improved scan path and less fixations compared to same participant in Figure 1.</figcaption></figure>

All non-experts completed the task faster with task completion time ranging between 22 and 63 seconds. One non-expert participant struggled with the file type again, selecting PDF instead of Word. This participant learnt the process, but still struggled with the term *file type*, another non-expert struggled with the term *file name*.

Figure 4 demonstrates the same stage of the task as given in Task 1 for the same participant (a non-expert). Notice the improved shorter scan path and much less fixations. This demonstrates that the participants quickly learnt how to use this system. Research shows that a large number of fixations point to a less efficient search [17]. Once the participant knew what to do, the process was much faster.

## 4.4 Post-test Questionnaire

All the participants were positive about the system despite the evidence that some of them found it difficult to use. Results showed that all questions had a median of 4 (5-point Lickert scale) or above. Only one question regarding error messages given by the Website had a median of 3. This discrepancy between their experience and response highlights the usefulness of eye tracking in identifying problems that would not have been reported. We can only speculate that this may be due to social desirability bias, quiescence or participants forgetting some of the difficulties they experienced. Participant comments in the post-test questionnaire included:

* "Think it would be difficult for a person who rarely uses computers. They may have a problem understanding things like the file type."

* "It will be very useful if users could be shown how to use this Website. I had a friend who had to show me how to work it."
These comments provide further evidence that it is necessary to distinguish between expert and non-expert users.

# 5 Discussion

The aim of this research was to examine the findings from the different usability methods namely, usability testing with observation, eye tracking and the post-test questionnaires. Looking at the findings for the three tasks evaluated, the eye tracking data supported the observations by providing concise, objective evidence of what was observed. Hence, the value of eye tracking lies in triangulating and corroborating the observations rather than in replacing the observations made on the user testing.

Different user's data can be overlaid to highlight specific problems. This summarised format is useful in providing the data to clients and managers who do not have the time to view all the recordings. In both cases the expertise of the observer or evaluator can influence the results but the scan paths and heat maps from eye tracking data provides a form of concise, summarised data that is difficult to match with other evaluation methods.

Eye tracking equipment is expensive and dedicated expertise is needed to use the equipment optimally as we have also experienced in our study. Therefore the overhead of cost and expertise needs to be carefully considered against the possible gains. Furthermore, our findings corroborate the fact that eye tracking is a tool to supplement usability evaluations and it does not replace standard usability testing with observations. This returns us to the questions of when the added value of eye tracking is justified.

The value of eye tracking, based on our findings is that it provides summarised, concise evidence. Furthermore, it is especially useful in capturing problems where users may be hesitant or unable to identify usability problems. For example, the disparity between the user-satisfaction reported in the post-test questionnaires, the user-satisfaction captured in the observations and the user-satisfaction captured in the eye tracking data. Despite the usability problems encountered, and captured by the objective measures, all the non-expert participants reported satisfaction with the system. In contrast the experts were more critical in the user satisfaction questionnaire while their efficiency and effectiveness usability measures rated higher. At this point we can only speculate about the reasons for the discrepancy between the objective and subjective results for non-experts. Data capturing errors such as the halo effect and social desirability bias could play a role since users have been found to make different judgments about the interfaces of scenarios for serious educational use [23]. This raises questions about the reliability of using subjective measures (post-test questionnaires in this study) for usability testing and emphasizes the importance of adding eye tracking data as objective evidence.

From a design perspective, error messages should be as short and simple as possible, when designing for non-expert users. From a practical perspective our study showed that participants had trouble with the most basic of terms e.g. *File type*. In designing a LMS it would be difficult to avoid using basic terms and therefore we recommend the use of a training video to provide a standardised, minimum body of
knowledge before attempting any task. In the repeat task all non-experts' times improved; this means that the students learnt the system but they needed assistance in getting past the critical errors. From the design perspective, the negative effects of errors were compounded by the use of domain specific terms and the length of the error messages. It has been noted that non-experts provide valuable information about users' individual reactions and they are an important group in usability evaluations [7]. Our findings support the usefulness of non-expert input on the declarative (identifying problems with terminology) and procedural (identifying problems with instruction flow) knowledge levels.

## 6 Conclusion and Future Work

This study investigated the added value of eye tracking in usability testing of expert and non-expert Web users while doing LMS tasks. The usability and eye tracking data showed differences in terms of task time (considerably longer for non-experts), scan paths (longer for non-expert participants with more fixations scattered over the screen) and comprehension of terminology and error messages (non-experts had more difficulty understanding and kept repeating mistakes). Non-experts are known to have longer task times and scan paths than experts [4, 5], but what is surprising is the difficulty non-experts had in comprehending the most basic terminology and error messages to the point where it rendered the system unusable.

The added value of eye tracking was to pin-point the particular problem areas through users' scan paths, i.e., the problematic terms that caused many fixations as well as the instructions that were not read. Eye tracking data also provided visual evidence to back-up the observations made by the usability evaluator. In this study, where the findings from the post-test questionnaire and the usability testing observations were in conflict, eye tracking data provided evidence to allow resolution that the users were indeed experiencing usability problems.

We conclude that the additional cost of eye tracking is justified when working with users where usability problems may be purposely or unintentionally disguised. The number of participants used was satisfactory for a usability study, where five to eight participants are deemed sufficient [24, 25]. However, we plan to validate these findings further by having more participants completing a more comprehensive set of tasks. Future research will test these findings on different Learning Management Systems and also investigate the use of eye tracking with other user groups.

## References

<footnote n="1">1. Avgeriou, P., Papasalouros, A., Retalis, S., Skordalakis, M.: Towards a Pattern Language for Learning Management Systems. Educational Technology & Society, **6**(2), pp. 11-24 (2003)</footnote>

<footnote n="2">2. Dillon, A. and Song, M.: An empirical comparison of the usability for novice and expert searchers of a textual and a graphic interface to an art-resource database. Journal of Digital Information, **1**(1). (1997)</footnote>

<footnote n="3">3. Aula, A., Majaranta, P., K.J. Räihä.: Eye-Tracking Reveals the Personal Styles for Search Result Evaluation, INTERACT 2005, LNCS 3585 Paternò, Editor. pp. 1058 – 1061. (2005)</footnote>
4. Law, B., Atkins, M.S., Kirkpatrick, A.E., Lomax, A.J., Mackenzie, C.L.: Eye Gaze Patterns Differentiate Novice and Experts in a Virtual Laparoscopic Surgery Training Environment. Association for Computing Machinery (2004)

5. Kasrskis, P., Stehwien, J., Hickox, J., Aretz, A.: Comparison of expert and novice scan behaviours during VFR flight. 11th International Symposium on Aviation Psychology Columbus, OH: The Ohio State University (2001)

6. Prumper, J., Frese, M., Zap D., Brodeck, F.: Errors in computerized office work : Differences between novices and expert users. SIGCHI Bulletin April 1991, 23(2): pp. 63-66. (1991)

7. Äijö, R. and J. Mantere.: Are non-expert usability evaluations valuable? 18th International Symposium on Human Factors in Telecommunications (HfT'01). Bergen, Norway. (2001)

8. Fields, B., Keith, S., Blandford, A.: Designing for Expert Information Finding Strategies. Technical Report: IDC-TR-2004-001 January (2004)

9. Popovic, V.: Expert and Novice user differences and implications for product design and usability Human Factors and Ergonomics Society Annual Meeting Proceedings. 6(4): pp. 933-936. (2007)

10. Gorman, M.E.: Types of Knowledge and Their Roles in Technology Transfer. Journal of Technology Transfer. 27: pp. 219-231. (2002)

11. Tabatabai, D., Luconi, F.: Expert-Novice Differences in Searching the WebAIS Electronic Library (AISel). (1998)

12. Kotze, P., Renaud K., Van Biljon J.: Don't do this - Pitfalls in using anti-patterns in teaching. Computers & Education (2006)

13. Beymer, D., Orton, P.Z., Russell, D.M.: An Eye Tracking Study of How Pictures Influence Online Reading, in INTERACT 2007, LNCS 4663, IFIP 2007, C.B.e. al., Editor. pp. 456 - 460. (2007)

14. Rudmann, D.S., McConkie, G.W. Zheng, X.S.: Eye tracking in Cognitive State Detection for HCI. ICMI'03, November 5-7, Vancouver, British Columbia, Canada. (2003)

15. Yoneki, E.: Sentient Future Competition: Ambient Intelligence by Collaborative Eye Tracking, in European Workshop on Wireless Sensor Networks (EWSN). Zurich, Switzerland. (2006)

16. Bednarik, R., Tukiainen M.: An eye-tracking methodology for characterizing program comprehension processes. ETRA 2006. San Diego, California: ACM 1-59593-305-0/06/0003 (2006)

17. Karn, K.S., Jacob R.J.K.: Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. in The mind's eye, cognitive and Applied Aspects of Eye movement Research. Elsevier, Amsterdam (2003)

18. Jacob, R.J.K., Karn, K.S.: The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research ed. R.D. Hyona.: Elsevier Science. (2003)

19. Sutcliffe, A., Namoune, A.: Investigating User Attention and Interest in Websites, in INTERACT 2007, LNCS 4662, C.B.e. al., Editor. pp. 88-101. (2007)

20. Ehmke, C., Wilson, S.: Identifying Web Usability Problems from Eye-Tracking Data. Proceedings of HCI: People and Computers XXI (2007)

21. Pretorius, M.C., Calitz, A.P., van Greunen, D.: The Added Value of Eye Tracking in the Usability Evaluation of a Network Management Tool. Proceedings of the 2005 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries. White River, South Africa. (2005)

22. Vu, K.L., Hanley, G.L., Strybel, T.Z., Procto, R.W.: Metacognitive Processes in Human-Computer Interaction: Self-Assessments of Knowledge as Predictors of Computer Expertise. International Journal of Human-Computer Interaction, 12(1): pp. 43-71. (2000)

23. De Angeli, A., Sutcliffe A., Hartmann, J.: Designing Interactive Systems archive Proceedings of the 6th conference on Designing Interactive systems, pp. 271 - 280. (2006)

24. Barnum, C.: Usability Testing and Research. The Allyn and Bacon Series in Technical Communication, (2002)

25. Nielsen, J.: Why You Only Need to Test With 5 Users. Alertbox 2000 [cited 2009/01/12]; Available from: http://www.useit.com/alertbox/20000319.html.