The Evaluation of E-government Web Portal in Turkey through Eye-tracking Method
Creators: name: Demir, Ömer
name: Tuluk, Arik
name: Ay, Kerem
name: Özçelik, Fatma
name: Tüzün, Hakan
Publisher: name: figshare
Year: 2017
Subjects: Education

description: Advances in ICTs have rendered tremendous changes in the understanding of public administration mandatory. To keep pace with these continuous changes, governments have been trying to offer some of their services online over the last two decades. Governments all over Europe and world design and create comprehensive web sites to fulfill these purposes.E-Government web portal in Turkey:<br>E-government web portal in Turkey can be given as a representative example of one of these web sites. E-government web portal is a comprehensive web site designed by paying attention to user needs with the aim of offering government services to the citizens. It aims to reduce institutions’ and clerks’ work load, make government services more easily available, and provide quality, fast, non-stop and secure services for all citizens. E-government web portal in Turkey came into use on 18th of January 2008. It became available on mobile platforms on 7th of October 2009. A total of 1739 services are provided with the collaboration of 289 government institutions (E-Government web portal, 2016). The number of users enrolled in the web site is over 31 million now. As for another statistics, 53.2% of the Internet users in Turkey benefit from E-government services (TUIK, 2015). The number of services and users continuously increase. With the help of these improvements, Turkey has been ranked as the 8th amongst 33 countries in terms of being user focused according to e-government measurement study (European Union, 2015, as cited in Demirel, 2016).Usability:With the help of proliferation of the tools that can be used, the concept of usability gained traction. ISO (International Organization for Standardization) defined usability as the extent to which a product ensures effectiveness, efficiency, and satisfaction in a specified context of use for achieving specified goals by specified users (ISO 9241-11, 1998). Likewise, Shneiderman and Plaisant (2004) focused more on efficiency and satisfaction components of usability. They explained the concept of usability with the time required for users to learn system to successfully carry out a specified task, retention of the information learnt during the interaction with the system, performance of the system, mistakes made by users, and users’ subjective satisfaction level. In usability studies, many different methods such as participatory-based design, expert evaluation, think aloud procedure, scales, and eye-tracking are used (Yeniad, Mazman, Tüzün, &amp; Akbal, 2011).Significance:Determining usability of a web site is of great significance since users’ perceived ease of use and satisfaction indicates the success of design of web sites (Insfran &amp; Fernandez, 2008). However, little is known if citizens from various background in Turkey can effectively and efficiently make use of e-government web portal. If an average citizen cannot utilize this web site, this endeavor is doomed to yield dismal results. All in all, it is lame to try to be e-government without making its citizens e-citizens first. Therefore, if they are not found to be so, relevant precautions such as arranging information literacy courses to address this issue might have to be taken. On the other hand, European e-Government (2016) initiatives aim to ensure digital transformation of member governments by means of modernizing public administration in the upcoming 5 years. In this context, it is very significant to determine if citizens can use this web site. Hence, the purpose of this study is to help authorities improve e-government web portal in Turkey by eliciting usability problems of it. To reach this goal, threeresearch questions were determined.1) How effectively do authentic participants use E government web portal?2) How efficiently do authentic participants use E-government web portal?3) How satisfied are authentic participants with using E-government web portal?MethodThe method of this study is usability testing. The prime purpose of usability tests is to enhance the usability of the product being tested (Dumas &amp; Redish, 1999). Besides, eye-tracking method was employed amongst usability testing methods.Study Group:In usability studies, it is indicated that 5 authentic users are sufficient to identify 85% of all usability problems (Nielsen, 1993, 2000). Therefore, 7 volunteering authentic users participated in this study. Participants’ ages range between 18 and 49, 5 of whom were male.Data Collection Tools:Five data collection tools were utilized in this study, the first of which was participant information form. It includes personal data of users such as occupation, age, and gender. Second one was the satisfaction form containing 3 open-ended questions. Third one was authentic tasks form which incorporated 5 authentic tasks. These three forms were checked by all researchers and a field expert. The fourth one was the eye-tracking device. Using this device, eye-movements of the participants were recorded. To do that, Tobii T120 desktop eye-tracking device and Tobii Studio 3.3.0 software were used. As for the fifth data collection tool, an observation form was used. That is to say, researchers observed authentic participants trying to complete authentic tasks and took notes.Data Collection Process:First, participants were informed about the aim of the study. After that participants filled in the participant information form. Then participants read the introduction and eye-tracking device was calibrated. After that, authentic tasks were given to the participants. After the study, participants were asked to fill in the satisfaction form.Data Analysis:Completed tasks and tasks’ completion duration were determined. Frequencies, percentage, minimum, maximum, mean, and total durations were reported. Analyzing eye-tracking data, metrics of fixation count, fixation duration, heap map, gaze plot and area of interests were used. Besides, after reaching a consensus with other researchers 5 area of interests were created. These are (1) main page, (2) search and personal, (3) menus, (4) current information, (5) announcements and mostly used. To analyze the qualitative data of the study, first, all the answers given to the questions in satisfaction form were transcribed. In this way, a 395-word document was obtained. Afterwards, every researcher separately analyzed the documents. To keep the data anonymous, names of the participants were coded.Expected OutcomesP1 and P4 fixated on the profile area (which is in search and personal area, and necessary to fixate on to be able to complete T1 many times; however, they were not able to complete the T1. This is because they were not able to detect this area was clickable. Therefore, it can be suggested that this area should be larger and has some indications showing that it is clickable. Current information area covers much more area than the menu, and search and personal ones do. What is worse, current information area is located at the very center of the page. Considering the aim of the current information area is to inform people about the newest added services on E-government web portal and no one reported they sign in E-government web portal for that purpose, it can be recommended that current information area should be made smaller and placed anywhere away from the center of the screen. Participants using search bar conducted faster searches. For example, after P6 detected search bar, its search performance soared. Similarly, P2 could not detect the search bar, but knowing its affordances it invented its own method. It used Windows shortcut CTRL+F to carry out search, which is not feasible because only the open page can be searched within this method. In short, menu and search bar areas can be expanded and current information area can be shrinked. Besides, search bar can be made more visible by moving it to the center of the screen. Actually this modification to the E-government web portal uttered by P6: “Tasks are easily get done through search bar. But search bar should be more in sight.” In conclusion, E-government web portal can be designed like Google search engine in order to maximize the visibility and search efficiency.