<figure><img src="image_1.png" /></figure>

HAL
open science

# The Treatment of Temporal Data in Web-Based Reservation Systems: An Inspection-Based Evaluation

Gerhard F. Knolmayer, Lukas E. Helfenstein, Viola Sini

► To cite this version:

Gerhard F. Knolmayer, Lukas E. Helfenstein, Viola Sini. The Treatment of Temporal Data in Web-Based Reservation Systems: An Inspection-Based Evaluation. 13th International Conference on Human-Computer Interaction (INTERACT), Sep 2011, Lisbon, Portugal. pp.323-339, 10.1007/978-3-642-23768-3_27. hal-01597006

HAL Id: hal-01597006

https://inria.hal.science/hal-01597006

Submitted on 28 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# The Treatment of Temporal Data in Web-based Reservation Systems: An Inspection-Based Evaluation

Gerhard F. Knolmayer, Lukas E. Helfenstein, and Viola Sini

University of Bern, Institute of Information Systems
Engehaldenstrasse 8, CH-3012 Bern, Switzerland
{gerhard.knolmayer, lukas.helfenstein, viola.sini}@iwi.unibe.ch

**Abstract.** Web-based reservation systems realize a broad variety of different and often inadequate ways of handling temporal data in their user interfaces. We compiled possible procedures for treating temporal data in reservation systems in a Morphological Box, ending up with 49 treatment options for 15 features. We initiated an usability inspection by 15 usability experts, asking for evaluation of the user-friendliness of these options and the relevance of their differences. After discussing the results we use them to develop an evaluation model. We draw profile lines to compare existing systems, weight the scores of the options by relevance factors of the features, and compute aggregate usability indicators to rank the systems. In our analysis of 60 airlines, Delta Air Lines and American Airlines reach the best results. We also show that companies that belong to the same group differ remarkably in the usability of their reservation systems.

**Keywords:** Temporal Data, Morphological Analysis, Reservation Systems, Usability, Usability Indicators, Usability Inspection.

## 1 Introduction

Web-based reservation systems have become a highly relevant distribution channel in the travel industry [1]. Nevertheless, the poor usability of these systems is often criticized [2-8]. Handling temporal data in Web interfaces is an essential feature of reservation systems because each reservation is time related. However, temporal data is treated by Web-based reservation systems quite differently, sometimes in user-unfriendly ways and in contradiction to usability guidelines, and in some cases even erroneously [9].

In this paper we focus on handling temporal data in Web-based reservation systems and, thus, consider a single element of the user interface in more detail than previous analyses. Of course appropriate treatment of temporal data is only one aspect in the evaluation of user interfaces and the usability of reservation systems. However, inadequate treatment of temporal data seems to carry considerable potential to demotivate a client, may result in aborting a planned reservation, and can result in additional transaction costs or loss of revenues. It is widely accepted that a typical website carries so
many usability problems that only a fraction of them will be identified in an empirical
analysis; thus it seems legitimate to focus on one special aspect.

The remainder of this paper is organized as follows: In Section 2 we reference the large body of research on treating time in information systems with special emphasis on usability issues. In Section 3 we discuss Morphological Boxes and apply them to establish a systematic approach on discussing the options for treating temporal data in user interfaces. We also give examples for reservation systems which apply the different options. Section 4 describes an inspection-based evaluation of these options by 15 usability experts, discusses some details, and visualizes the results. In Section 5 we develop usability indicators for treating temporal data and determine the associated indicator values for selected reservation systems. Section 6 summarizes the results and gives an outlook.

2 Temporal Data as Research Object

2.1 Temporal Data in General

The adequate treatment of temporal data has been a major research area as well in Computer Science as in Management of Information Systems. There are thousands of papers which are referenced in at least 13 bibliographies on different aspects of handling temporal data in information systems [10, 11]. The most widely covered area is temporal databases; a recently published Encyclopedia of Database Systems contains more than 80 contributions related to temporal data [12, 13]. Temporal Logic and Temporal Reasoning are important research areas in Artificial Intelligence. Only a few papers focus on temporal data in Web-based systems. Furthermore, the potential problems of inadequate treatments of temporal data aroused broad public interest in connection with solving the Year 2000 (Y2K) problem.

2.2 Usability Aspects of Temporal Data

Fabre and Howard [14] argued that good practices for handling temporal data should find their way into style guides, screen design guidelines, and repositories of reusable interface designs; more generally, Temporal Aspects of Usability (TAU) should be given more attention. TAU have been discussed in several workshops, but most definitions of usability at best leave temporal aspects implicit, and at worst omit any consideration of time [14]. Several design guides for date entry and display exist [15-18]. Bainbridge [19] provides detailed recommendations to system developers for handling temporal data in reservation systems. Information engineering concepts like services and patterns could be useful also for treating temporal data. Temporal ser- vices are compiled in [20], temporal patterns discussed, e.g., in [21].
## 3 Morphological Boxes for Treating Temporal Data

A Morphological Analysis supports an ordered way of looking at things [22, 23]. A Morphological Box represents relevant features and the associated options in existing or forthcoming products and services. It can be applied to provide a systematic overview of elementary courses of action and how they are or can be combined.

In [24] we developed a Morphological Box to show the options for handling temporal data in reservation systems. The rows of the Box describe main features of treating temporal data in reservation systems and the corresponding table entries show the options available for each feature. The features were found by literature analysis and by a critical review of many reservation systems and are described in some detail in [24].

The research documented in this paper aimed to evaluate the options described in the Morphological Box by Usability Experts to allow well-founded statements about the usability of different reservation systems and to show their improvement potentials. When we designed the questionnaire we found that some items used in [24] would need refinement. Thus we rearranged the Morphological Box which now contains 15 instead of 10 features and 49 instead of 33 options. The features in Table 1 differ from those in [24] as follows:

* We now distinguish between initial entry of temporal data and changes.

* The feature “Temporal Integrity Constraints” is refined into 3 subcases of keeping temporal integrity (cf. [9]).

* We distinguish setting default values on the initial screen for entering temporal data from setting nontrivial default values after user entry.

* We also differentiate whether temporal flexibility is offered at the entry page or after the search procedure.

We number the features in the first column of Table 1 and the options for each feature in the table elements. Table 1 also gives examples of Web-based reservation systems that realize the different options (as of March 2011). In our extensive Web searches we found examples for 47 of the 49 options. This result indicates that existing systems use a broad variety of different procedures for treating temporal data.

When developing a Morphological Box one has to consider that for each solution one and only one marking should appear in each row of the Box. In this case the markings can be connected to obtain a profile line for a certain solution. It does not suffice to present elementary options in the Morphological Box if these may also be combined. For instance, if keyboard, dropdowns, and calendars are the 3 basic options for entry and changes of temporal data, one could assume that the relevant number of combinations is

$$ \binom{3}{1} + \binom{3}{2} + \binom{3}{3} = 7 $$
But providing a dropdown typically excludes entering dates via the keyboard. Thus, the number of relevant options for Features 3 and 4 in Table 1 is

$$ \binom{3}{1} + \binom{3}{2} - 1 = 5 $$

However, in our analysis of reservation systems we also found some strange exceptions of this assumption (cf. Section 5.2).

Formally, the upper bound on the number of combinations in Table 1 is almost 25 millions but not all of them are feasible because some entries in Table 1 are related. For instance, the option “No calendar” exists for Features 5 and 6. If “No calendar” is true for Feature 5, only this option is feasible for Feature 6; instead of 4*3=12 only 3*2+1=7 combinations result from joining Features 5 and 6.

# 4 Evaluating the Options

## 4.1 Design of the Evaluation

Each Web-based reservation system can be characterized by a profile line which connects the markings in the Morphological Box that are associated with the options realized by the system. To evaluate the treatment of temporal data in a given system one has to appraise the options described in the Morphological Box and also to evaluate the relevance of the features. The latter is similar to classifying usability problems on a scale of severity in a usability inspection [25].

For evaluating the usability of systems one can refer to users or to usability experts. As in other evaluations of information system design, the number of reviewers should be limited to a rather small number of persons. For usability studies, the participation of experts with broad background in usability evaluation and human-computer interface design is recommended. It is often assumed that no more than 5 experts are needed to obtain sufficient results [26, 27].

In our case we had to modify the typical procedure applied in an Inspection-Based Evaluation [28] because we did not want to have one specific solution evaluated but the experts should evaluate all options described in Table 1. Therefore we tried to find a larger number of evaluators than suggested for practical applications. We were able to interest 5 experts working in consultancy and interface development firms and 4 academics who are doing research in connection with evaluating the business benefits of websites to participate in the study. Another 3 evaluators were very familiar with Web-based reservation systems. Three Ph.D. students with a broad range of practical experiences also participated in the study. In total 15 persons evaluated the options described in Table 1. We explained the options in detail and gave, whenever possible, references to existing systems, showing screenshots, and providing links to systems in which this option was implemented.
<table><caption>Table 1. Morphological Box for Treating Temporal Data (Part 1).</caption><thead><tr><th>Number</th><th>Feature</th><th colspan="4">Characteristics / Examples</th></tr></thead><tbody><tr><td>1</td><td>Definition of Booking Interval</td><td>① t<sub>s</sub>, t<sub>e</sub><br>Most systems</td><td>② t<sub>s</sub>, d<br>www.hotelpronto.com</td><td>③ d, t<sub>e</sub><br>No system found</td><td>④ t<sub>s</sub>, d, t<sub>e</sub><br>hotels.myswitzerland.com/<br>?idlng=1</td></tr><tr><td>2</td><td>Entry of Temporal Data</td><td>① Only Keyboard<br>ww1.shanghai-air.com/<br>salnewweb-en/index.aspx</td><td>② Only click<br>in dropdowns<br>www.evaair.com/html/b2c/english/</td><td>③ Only click in<br>calendar<br>www.german<br>wings.com/en</td><td>④ Keyboard or<br>click in calendar<br>www.swiss.com</td><td>⑤ Click in<br>dropdown<br>or calendar<br>www.ana.co.jp/<br>asw/wws/sz/e/</td></tr><tr><td>3</td><td>Changes of Temporal Data</td><td>① Only Keyboard<br>ww1.shanghai-air.com/<br>salnewweb-en/index.aspx</td><td>② Only click<br>in dropdowns<br>www.evaair.com/html/b2c/english/</td><td>③ Only click in<br>calendar<br>www.german<br>wings.com/en</td><td>④ Keyboard or<br>click in calendar<br>www.swiss.com</td><td>⑤ Click in<br>dropdown<br>or calendar<br>www.ana.co.jp/<br>asw/wws/sz/e/</td></tr><tr><td>4</td><td>Display of Date Format</td><td>① Not indicated<br>www.southwest.com</td><td>② Abstractly<br>(e.g., mm/dd/yy)<br>www delta.com</td><td>③ Numerically<br>(e.g., 09/11/11)<br>www.united.com</td><td>④ With months shown<br>as text<br>www.qantas.com</td></tr><tr><td>5</td><td>Display of Calendars</td><td>① No calendar<br>www.evaair.com/html/<br>b2c/english/</td><td>② After clicking in<br>date entry field<br>www.swiss.com</td><td>③ After clicking on<br>calendar icon<br>www.lufthansa.com</td><td>④ Without user action<br>www.airberlin.com</td></tr><tr><td>6</td><td>Representation of and Access to Temporal Data</td><td>① No calendar<br>www.evaair.com/html/<br>b2c/english/</td><td>② Sequential access to months<br>www.austrian.com</td><td>③ Direct access to months<br>www.qantas.com</td><td></td></tr></tbody></table>

Legend:

t<sub>s</sub> ... Start of reservation interval; d ... Duration; t<sub>e</sub> ... End of reservation interval
<table><caption>Table 1. Morphological Box for Treating Temporal Data (Part 2).</caption><thead><tr><th>Number</th><th>Feature</th><th colspan="3">Characteristics / Examples</th></tr></thead><tbody><tr><td>7</td><td>Temporal Integrity of Start Date</td><td>① Not supported<br>www.singaporeair.com</td><td>② Past dates are excluded in dropdowns or calendar<br>Many systems, e.g., www.swiss.com</td><td></td></tr><tr><td>8</td><td>Temporal Integrity of End Date</td><td>① Not supported<br>www.ryanair.com/en</td><td>② Infeasible dates are excluded in dropdowns or calendar<br>Many systems, e.g., www.swiss.com</td><td></td></tr><tr><td>9</td><td>Temporal Integrity of Date Relationships</td><td>① Not supported<br>www.easyjet.com/asp/en</td><td>② Defining infeasible dates is excluded<br>www delta.com</td><td>③ Avoided by setting default values<br>www.united.com</td></tr><tr><td>10</td><td>Default Temporal Data on Initial Screen</td><td>① No default values<br>www.jetblue.com</td><td>② Default values set<br>Most systems</td><td></td></tr><tr><td>11</td><td>Nontrivial Default Temporal Data after User Entry</td><td>① No nontrivial default values<br>www.lufthansa.com</td><td>② Nontrivial default values added without warning<br>www.united.com</td><td>③ Nontrivial default values added with warning<br>www.easyjet.com/asp/en</td></tr><tr><td>12</td><td>Error Messages</td><td>① Avoided by setting nontrivial default values<br>www.united.com</td><td>② Immediate at data entry (e.g., by erasing previous date entries)<br>www.kulula.com</td><td>③ Delayed until subsequent user action (e.g., Search)<br>www.singaporeair.com</td></tr><tr><td>13</td><td>Travel Day Flexibility on Entry Page</td><td>① No flexibility offered<br>www.austrian.com</td><td>② Flexibility interval(s) &gt;0, but undefined<br>www.ryanair.com</td><td>③ Flexibility with preset interval(s) (e.g., ± 3 days)<br>www.lufthansa.com</td></tr><tr><td>14</td><td>Travel Day Flexibility after Search</td><td>① No flexibility offered<br>www.austrian.com</td><td>② Several days shown but only one price per day<br>www.lufthansa.com</td><td>③ Several days shown with several prices per day<br>www.airberlin.com</td></tr><tr><td>15</td><td>Temporal Data after “Major Changes”</td><td>① Temporal data is lost<br>www.ana.co.jp/asw/wws/sz/e/</td><td>② Temporal data is kept<br>www.lufthansa.com</td><td></td></tr></tbody></table>
<figure><img src="image_3.png" /><figcaption>Fig. 1. Average Usability Evaluation by Usability Experts.</figcaption></figure>
We wanted to have all options *j* of feature *i* evaluated as $a_{ij}$ on a 7 point scale ($1 \le a_{ij} \le 7$), in which a subset of values was verbally described as

$$a_{ij} = \begin{cases} 1 & \dots \text{totally unsuitable, annoying} \\ \vdots & \dots \text{could be better, but acceptable} \\ 4 & \dots \text{perfect, no improvement potential} \\ \vdots & \dots \\ 7 & \dots \end{cases}$$

The introduction to the questionnaire emphasized that if an expert cannot suggest a better option than those described in the Morphological Box, he should evaluate at least one of the options with 7. Unfortunately not all respondents followed this hint and we were not able to perform a second round of evaluations.

When we want to evaluate existing systems, not only the evaluation of the options is relevant but also an estimation of the relevance $r_i$ of feature *i*. We therefore also asked each expert *k* how he estimates the relevance $r_{ik}$ of feature *i* for users on a {1..7}-point scale.

## 4.2 Selected Results

Main results of the study are shown in Figure 1, which visualizes the average values obtained for each option. An option positioned in the right part of Figure 1 is evaluated as better usable than one which appears on the left side. Figure 1 also gives the average relevance factors the experts assigned to the features. In the following we describe some details of the evaluation results that seem worth to discuss.

*Feature 1: Definition of the Booking Interval:* In Table 1 we did not differentiate whether the booking should occur for a hotel or a flight. We assumed that there might be a difference, in particular with respect to the entry of the relevant booking interval. In Figure 2 we compare the average evaluations obtained for these two types of reservations. For hotel bookings defining the start day and the duration of the stay (Option 2 of Feature 1) is evaluated quite well ($a_{12}=4.87$) whereas for flights this option is regarded as inappropriate ($a_{12}=3.07$). We obtained no feedback that the difference between hotel and flight bookings may also be relevant for the other features. However, the differences shown in Figure 2 remind that the situational context of Human-Computer Interaction should always be kept in mind.

Rather surprisingly Option 4 for Feature 1 (entry allowed for all three temporal elements $t_s$, d, and $t_e$) was evaluated second-best for booking of flights. This evaluation may be based on the paradigm of recommending to give users as much freedom in communication with the system as possible. However, we are not sure whether the experts really were aware of the integrity problems resulting from allowing direct changes of all three temporal elements. If, for instance, the user entered $t_s$ and d and the system computed $t_e$, how should the system react if the user changes $t_s$ in a succeeding step: Should d or $t_e$ be adjusted? In this context it is remarkable that the evaluation of Option 4 for Feature 1 showed the highest standard deviation of all evaluations (cf. Table 2).
<figure><img src="image_4.png" /><figcaption>Fig. 2. Evaluations for Date Entry differ between Hotel and Flight Reservations.</figcaption></figure>

<table><caption>Table 2. Options with Low and High Standard Deviation (Flight Reservations).</caption><thead><tr><td>Feature</td><td>Option</td><td>Mean usability indicator value</td><td>Standard deviation of usability indicator value</td></tr></thead><tbody><tr><td>15</td><td>2</td><td>6.73</td><td>0.59</td></tr><tr><td>9</td><td>2</td><td>6.60</td><td>0.74</td></tr><tr><td>7</td><td>2</td><td>6.67</td><td>0.82</td></tr><tr><td>...</td><td></td><td></td><td></td></tr><tr><td>12</td><td>3</td><td>4.07</td><td>2.09</td></tr><tr><td>11</td><td>1</td><td>4.87</td><td>2.23</td></tr><tr><td>1</td><td>4</td><td>3.60</td><td>2.38</td></tr></tbody></table>

*Feature 4: Display of Date Format:* The experts favored a representation in which the months are abbreviated by letters (e.g., 05-Sep-2011), thus avoiding the problems related with pure numerical date representation (cf. [29, 30]). Such a representation is rarely used; it has the advantage that the intercultural differences of sequencing days and months [31] become irrelevant but may be difficult to interpret if the user has not sufficient skills in the language used on the Web site.

*Feature 5: Display of Calendars:* The experts preferred sites in which calendars are displayed without user action. Thus, they did not give too much weight to aspects of brimful pages and the potentially related information overload. Most websites do not present calendars immediately.

*Feature 6: Representation of and Access to Temporal Data:* The direct access to the month (Option 3 of Feature 6) is clearly evaluated best but also not realized too often. Direct access can be provided by dropdowns or by a clickable element of the calendar.

*Features 7 to 9: Temporal Integrity of Start Date, of End Date, and of Date Relationships:* With respect to the 3 temporal integrity features the experts favor representations that do not allow entering infeasible dates. If the user enters, e.g., a return date which lies before the outward date, the experts clearly refuse automatic adjustments without warnings.
*Feature 12: Error Messages*: In accordance with ISO 9241 but in contrast to the results presented in [32] the experts favored an immediate system reaction to entries that violate temporal integrity. However, if the user plans to correct both the start date and the end date of a reservation, an immediate error message may be inconvenient.

*Features 13 and 14: Travel Day Flexibility on Entry Page and/or after Search:* With respect to temporal flexibility on the entry page the experts prefer user-defined intervals (Options 4 and 3 of Feature 13) and give a high priority to showing several travel options around the originally selected dates after the search (Options 3 and 2 of Feature 14).

*Feature 15: Temporal Data after "Major Changes":* The loss of temporal data if the user changes, e.g., travel destinations is seen as inadmissible (Option 1 of Feature 15). If a system provides an explicit "go back" button, we used this button for back-tracking; in other systems we had to use the Browser's back button or even return to their homepages. Some of the losses of temporal data are based on "breaking the back button", which is regarded as a perpetual usability problem of Web systems [33].

Table 2 shows the standard deviations for the three options with the lowest and the three options with highest standard deviations. Least consensus between the experts exists with respect to the option of entering three temporal elements, of providing no nontrivial default values, and whether an error message should be delayed until a user action (such as Search). Most individual evaluations coincide quite well and their aggregates seem to be suitable as a basis for evaluating reservation systems.

When we look to the relevance indicators in the right column of Figure 1 we find that, on average, the experts evaluated all features at least with 4 on the {1..7}-point scale. The features considered most relevant are (in descending order)

* showing alternatives to the originally selected dates, preferably also with price information (Feature 14),

* the user-friendly definition of travel dates (Feature 1),

* immediate error messages (Feature 12),

* keeping temporal data in case of "major changes" (Feature 15), and

* (avoidance of) nontrivial default values (Feature 11).
# 5 An Empirically Based Evaluation Method

## 5.1 Defining Usability Indicators

For each option *j* of feature *i* we determined the average value

$$ \bar{a}_{ij} = \sum_{k} a_{ijk} / K \qquad (1) $$

where $a_{ijk}$ is the evaluation value assigned by expert *k* to option *j* for feature *i* and *K* is the number of evaluators. Furthermore, we computed the average value for relevance of feature *i* as

$$ \bar{r}_i = \sum_k r_{ik} / K \qquad (2) $$

and the relative relevance of feature *i* as

$$ \tilde{r}_i = \bar{r}_i / \sum_i \bar{r}_i \qquad (3) $$

The sum of the $\tilde{r}_i$ values adds up to one; we use these values as weights for the scores determined for Feature *i*.

In Figure 1 the options *j* for each feature *i* are positioned according to their $\bar{a}_{ij}$ values. If the profile lines for two systems differ and no point of the profile line for system X is left to any corresponding point of the profile line for system Y, system X dominates Y and is in all features evaluated as better or at least equally usable than system Y. However, when considering 15 features the dominance criterion will rarely hold.

We have no hints that the experts interpreted the scales used in the evaluation as nonlinear. Therefore we suggest computing a usability indicator $M_s$ for system *s* which is determined as

$$ M_s = \left[ \sum_i (\bar{a}_{ij(s)} * \tilde{r}_i) - 1 \right] / 6 \qquad (4) $$

where $ij(s)$ symbolizes the option *j* realized by system *s* for feature *i*. The subtraction of 1 in formula (4) is necessary to transform values 1 to 7 to values 0 to 6; this transformation and the division by 6 lead to standardized usability indicator values between 0 and 1.

Even a system which realizes for each feature the best evaluated option would not receive an indicator value $M_s=1$ because this value can only be obtained if for each feature all experts prefer the same option and evaluate it with 7. Such consensus is highly unlikely. In our study the best evaluated options did not obtain average values of 7 but received values between 5.40 (Option 3 of Feature 11) and 6.73 (Option 2 of Feature 15).
An alternative scale results if we take the score of the best evaluations for each feature into account. Let $h_i$ be the highest score obtained for feature $i$ on the {1..7} scale. We can define an alternative usability indicator $M'_s$ for system s as

$$M'_{s} = \left[ \sum_{i} (\bar{a}_{ij(s)} * \tilde{r}_{i}) - 1 \right] / \left[ \sum_{i} (h_{i} * \tilde{r}_{i}) - 1 \right] \quad (5)$$

which has the advantage that the maximum value 1 is achievable and, thus, may be more motivating in practical applications. The relative attractiveness of the systems is not influenced by this transformation because of the linear function

$$M'_{s} = \alpha M_{s} \qquad (6)$$

When comparing selected systems with respect to their implemented options we used the indicator values $M_s$.

## 5.2 Comparison of Selected Web-Based Reservation Systems

We selected the reservation systems of airlines to be analyzed with reference to several data sheets which define different classes of airlines (Table 3). The selection looks a bit arbitrary but is affected by the availability of data sheets. We were, e.g., interested whether very big airlines or highly rated airlines provide reservation systems that are more user-friendly with respect to handling temporal data than the systems of other companies. We had to adjust some of the lists; e.g., the list of the Top 10 Airlines in the USA and Canada (Class 3) mentions Skywest, which does not operate an own reservation system, and ExpressJet, which focuses on customized offerings for corporate clients and does not allow online booking. In the list of European Budget Airlines Scanderbeg seems no longer to exist and Eurofly was acquired by Meridiana fly. Pacific Blue is member of class 5 but its reservation system switches to Virgin Blue's system, which is also a member of class 5.

Many airlines belong to more than one class. For instance, Delta Air Lines is member of classes 1, 2, and 3; Emirates belongs to the classes 2, 4 and 5. Altogether we rated 60 airlines mentioned in at least one of the 6 classes defined in Table 3 for our statistics. However, additional evaluations were also conducted and some observations are mentioned later.

For each of the 60 airlines we analyzed its reservation system with regard to its treatment of temporal data. Whenever an English site was available, we evaluated this site. Because parallel sites of an airline sometimes do not always behave identical, this may introduce a bias against an airline which considers a site in its local language as its main site.

We found some cases which do not allow a unique marking in the Morphological Box. For instance, Delta's system shows a calendar as well after clicking into the date entry field as after clicking on a calendar icon (Feature 5). The Qantas system immediately adjusts the month if the user enters a return month prior to the departure month but does not immediately react if the return day entered is before the departure day of the same month (Feature 9). Temporal integrity may be checked after clicking calendar
dar entries but not if the user changes the entry via the keyboard (Features 7 to 9). In such situations we split the assignments, in the case described with 0.7 for controlling temporal integrity for calendar entries and with 0.3 for neglecting it for keyboard entries. The effect of such splits can be seen, e.g., in Figure 3 where for Feature 7 the profile line for Delta Air Lines does not pass through a circle.

<table><caption>**Table 3.** Classes of Airlines analyzed.</caption><thead><tr><td># of class</td><td>Class</td><td>Link</td><td># of airlines</td><td>Average M's</td></tr></thead><tbody><tr><td>1</td><td>Scheduled Passengers Carried (as of 2010)</td><td>http://www.iata.org/ps/publications/Pages/wats-passenger-carried.aspx</td><td>10</td><td>0.776</td></tr><tr><td>2</td><td>Scheduled Passenger - Kilometres Flown (as of 2010)</td><td>http://www.iata.org/ps/publications/Pages/wats-passenger-km.aspx</td><td>9</td><td>0.811</td></tr><tr><td>3</td><td>Top 10 Airlines in the USA and Canada (Number of passengers) (as of 2010)</td><td>http://airtravel.about.com/od/basedinnorthamerica/tp/top10na.htm</td><td>8</td><td>0.800</td></tr><tr><td>4</td><td>The World's Top 10 airlines (2010 World Airline Awards)</td><td>http://www.worldairlineawards.com/main/2010Awards.htm</td><td>10</td><td>0.746</td></tr><tr><td>5</td><td>Official World Airline Star Ranking (5 and 4 Star Airlines)</td><td>http://www.airlinequality.com/StarRanking/ranking.htm</td><td>36</td><td>0.732</td></tr><tr><td>6</td><td>Low Cost, Budget Airlines in Europe</td><td>http://airtravel.about.com/od/basedineurope/tp/lowcostbudgetairlinesEurope.htm</td><td>17</td><td>0.729</td></tr></tbody></table>

A dropdown usually impedes keyboard entry; but if a user of, e.g., Easyjet's system enters "J" in the month field via keyboard several times, the system switches between January, June, and July; if he enters, e.g., "2" in the day field, the display switches between 2, 20, 21, ..., 29. With respect to displaying the date format (Feature 4) we found that Asiana Airlines displays the date (on its English site) in one field as YYYYMMDD, without any separator. Even if we would have anticipated such effects before designing the Morphological Box it would make no sense to define and evaluate rather implausible options. We even found differences in handling date relationships when we used different Browsers.

Table 4 shows the usability indicator values $M'_s$ in descending order (as of March 31, 2011) for the 5 airlines which are ranked best, the 5 airlines evaluated least favorable, and for the airline which obtained an average evaluation with respect to treating temporal data.
<table><caption>**Table 4.** Usability Indicators for selected Reservation Systems (as of March 31, 2011).</caption><thead><tr><td> Reservation system of </td><td> Link </td><td> Member of classes </td><td> M'<sub>s</sub> </td></tr></thead><tbody><tr><td> Delta Air Lines </td><td> www.delta.com </td><td> 1, 2, 3 </td><td> 0.934 </td></tr><tr><td> American Airlines </td><td> www.americanairlines.ch/intl/ch/ index_en.jsp </td><td> 1, 2, 3 </td><td> 0.904 </td></tr><tr><td> Virgin Atlantic </td><td> www.virgin- atlantic.com/en/eu/index.jsp </td><td> 5 </td><td> 0.884 </td></tr><tr><td> Lufthansa </td><td> www.lufthansa.com/de/en/Homepage </td><td> 1, 2, 5 </td><td> 0.883 </td></tr><tr><td> Jet Blue Airways </td><td> www.jetblue.com </td><td> 3, 5 </td><td> 0.871 </td></tr><tr><td> ...... </td><td> ...... </td><td> ...... </td><td> ...... </td></tr><tr><td> Finnair </td><td> www.finnair.com </td><td> 5 </td><td> 0.738 </td></tr><tr><td> ...... </td><td> ...... </td><td> ...... </td><td> ...... </td></tr><tr><td> Aer Lingus </td><td> www.aerlingus.com </td><td> 6 </td><td> 0.660 </td></tr><tr><td> Hongkong Airlines </td><td> www.hongkongairlines.com/eng </td><td> 5 </td><td> 0.659 </td></tr><tr><td> Hainan Airlines </td><td> global hnair.com/en </td><td> 5 </td><td> 0.604 </td></tr><tr><td> Eva Air </td><td> www.evaair.com/html/b2c/english </td><td> 5 </td><td> 0.576 </td></tr><tr><td> China Southern Airlines </td><td> www.csair.com/en </td><td> 1, 5 </td><td> 0.526 </td></tr></tbody></table>

Figure 3 shows a detailed comparison of three reservation systems. We present the profile lines for the system which was evaluated best (Delta Air Lines), worst (China Southern Airlines), and for a system which represents average usability (Finnair). However, the “poorer” systems are not dominated by the “best” because of Features 5, 7, and 13.

According to our evaluation, Delta's system would achieve a usability indicator $M'_s = 1$ if it would enhance the displays of the date and the calendar (Features 4 and 5), support temporal integrity also for keyboard entries (Feature 7), and offer travel day flexibility already on its entry page (Feature 13). Thus, even the system which was evaluated best did not realize the best evaluated options in 4 of 15 features.

The last column of Table 3 shows the average indicator values $M'_s$ obtained by the airlines that are members of a certain class. Reservation systems of large carriers (Classes 1 to 3) obtain better average usability indicator values than those of less powerful carriers. Companies gaining awards (Classes 4 and 5) do not receive better evaluations than the large carriers. North American-based companies are evaluated better than those located in other regions.
<figure><img src="image_5.png" /><figcaption>Fig. 3. Comparison of Options for treating Temporal Data in the Reservation Systems of Delta Air Lines, Finnair, and China Southern Airlines.</figcaption></figure>

Legend:

<figure><img src="image_6.png" /></figure>
It is worth mentioning that major differences between the usability values of systems exist even if the owners belong to the same group. For instance, Lufthansa owns Swiss and Austrian Airlines but the usability of their reservation systems differs. Table 5 summarizes the realized options and shows for which features we had to use split values. Only in 3 of 15 features all three systems realize the same option. None of the three airlines realized the best evaluated options for Features 5, 10, 11, and 13.

<table><caption>Table 5. Options realized by Airlines belonging to the same Group.</caption><thead><tr><th rowspan="2">Feature</th><th colspan="4">Options realized by</th></tr><tr><th>Lufthansa</th><th>Swiss</th><th>Austrian</th><th>"Best" option</th></tr></thead><tbody><tr><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><td>2</td><td>4</td><td>4</td><td>5</td><td>4</td></tr><tr><td>3</td><td>4</td><td>4</td><td>5</td><td>4</td></tr><tr><td>4</td><td>3</td><td>3</td><td>4</td><td>4</td></tr><tr><td>5</td><td>3</td><td>2</td><td>3</td><td>4</td></tr><tr><td>6</td><td>3</td><td>3</td><td>2</td><td>3</td></tr><tr><td>7</td><td>2</td><td>1 or 2</td><td>2</td><td>2</td></tr><tr><td>8</td><td>2</td><td>1 or 2</td><td>2</td><td>2</td></tr><tr><td>9</td><td>1 or 2</td><td>3</td><td>3</td><td>2</td></tr><tr><td>10</td><td>2</td><td>2</td><td>2</td><td>1</td></tr><tr><td>11</td><td>1</td><td>1 or 2</td><td>2</td><td>3</td></tr><tr><td>12</td><td>2 or 3</td><td>1 or 3</td><td>1</td><td>2</td></tr><tr><td>13</td><td>3</td><td>3</td><td>1</td><td>4</td></tr><tr><td>14</td><td>2</td><td>3</td><td>1</td><td>3</td></tr><tr><td>15</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td>Usability indicator M'<sub>s</sub></td><td>0.883</td><td>0.774</td><td>0.687</td><td></td></tr></tbody></table>

# 6 Summary and Conclusions

In this paper we enhanced previous research on usability aspects of handling temporal data by an empirical study in which usability experts were asked to evaluate the options defined in a Morphological Box. We discussed the results of the empirical study and used them to define usability indicators that allow evaluating the procedures implemented in different systems from a usability viewpoint. We computed usability indicator values for 60 reservation systems and developed profile lines for easy comparisons of systems. In all systems some improvement potential exists. The best results with respect to handling temporal data reach the systems of Delta Air Lines, American Airlines, Virgin Atlantic, Lufthansa, and Jet Blue Airways. We also showed that the systems provided by Lufthansa, Swiss, and Austrian Airlines differ remarkably in handling temporal data.
Even today information engineering concepts like temporal patterns or the reuse of existing services still seem to be widely neglected by the developers of Web-based reservation systems. The approach suggested in this paper may help system designers in selecting appropriate options for handling temporal data. It could also contribute to better conformance in properties where individual system design does not improve the market position of service providers but may prevent potential customers from booking if unfriendly interfaces annoy users.

## References

1. Marcussen, C.H.: Trends in European Internet Distribution - of Travel and Tourism Services (2009)
http://www.crt.dk/uk/staff/chm/trends.htm

2. Selvidge, P.: How Usable are the Airline Websites? Usability News 1, (1999)
http://www.surl.org/usabilitynews/11/Usability.airline.asp

3. Crichton, E., Frew, A.J.: Usability of Information and Reservations Systems: Theory or Practice? In: Fesenmaier, D.R., Klein, S., Buhalis, D. (Eds.) Information and Communication Technologies in Tourism 2000, pp. 408-417. Springer, Wien (2000)

4. Chariton, C., Choi, M.-H.: Enhancing Usability of Flight and Fare Search Functions for Airline and Travel Web Sites. In: Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC'04), pp. 320-325. IEEE Computer Society, Los Alamitos (2004)

5. Malizia, A.: Adding flexibility to B2C booking systems using a virtual intermediate travel agent. In: Proceedings of the 2005 IEEE Symposium on Visual Languages and Human-Centric Computing, pp. 337-338. IEEE Computer Society, Los Alamitos (2005)

6. Zhou, Q., DeSantis, R.: Usability Issues in City Tourism Website Design: A Content Analysis. In: 2005 IEEE International Professional Communication Conference Proceedings, pp. 789-796. IEEE Press, Piscataway (2005)

7. Essawy, M.: Testing the Usability of Hotel Websites: The Springboard for Customer Relationship Building. Information Technology & Tourism 8, 47-70 (2006)

8. Lituchy, T.T., Barra, R.A.: International issues of the design and usage of websites for e-commerce: Hotel and airline examples. Journal of Engineering and Technology Management 25, 93-111 (2008)

9. Knolmayer, G.F., Helfenstein, L.E.: Usability Aspects of Ensuring Temporal Integrity Constraints: A Critical Appraisal. In: Modave, F., Dascalu, S., Moallem, A. (Eds.): Computational Intelligence, Proceedings of the 5th International Conference on Human-Computer Interaction, pp. 271-278. ACTA Press (2010)

10. Grandi, F.: An Annotated Bibliography on Temporal and Evolution Aspects in the World Wide Web, TimeCenter Technical Report TR-75 (2003)
http://timecenter.cs.aau.dk/TimeCenterPublications/TR-75.pdf

11. Grandi, F.: Introducing an Annotated Bibliography on Temporal and Evolution Aspects in the World Wide Web. SIGMOD Record 33, 84-86 (2004)

12. Jensen, C.S., Snodgrass, R.T. (Eds.): Temporal Database Entries for the Springer Encyclopedia of Database Systems, TimeCenter Technical Report TR-90 (2008)
http://timecenter.cs.aau.dk/TimeCenterPublications/TR-90.pdf

13. Liu, L., Özsu, T.M. (Eds.): Encyclopedia of Database Systems. Springer, Berlin (2009)
14.Fabre, J., Howard, S.: Introduction to the Special Issue on Temporal Aspects of Usability. Interacting with Computers 11, 1-7 (1998)

15.W3C: Date formats (2007) http://www.w3.org/International/questions/qa-date-format

16.NHS: CUI CAPS Development Lifecycle (2008)
http://www.isb.nhs.uk/docs/cui-1/cui-draft/SubCUI.pdf

17.Microsoft: Design Guidance, Date and Time Input, Version 3.0.0.0 (2010)
http://www.mscui.net/DesignGuide/Pdfs/Design Guidance -- Date and Time Input.pdf

18.Microsoft: Design Guidance, Date Display, Version 3.0.0.0 (2010)
http://www.mscui.net/DesignGuide/Pdfs/Design Guidance -- Date Display.pdf

19.Bainbridge, A.: Hotel Date Entry: Design & Usability Report. Travel UCD (2002)
http://www.tourcms.com/company/research/pdf/date_entry_hotel_july2002.pdf

20.Lennartz, S.: Online Calendars and Date Pickers. Smashing Magazine (2007)
http://www.smashingmagazine.com/2007/10/23/online-calendars-and-date-pickers/

21.Fowler, M.: Temporal Patterns (2005)
http://www.martinfowler.com/eaadev/timeNarrative.html

22.Zwicky, F.: Morphological Astronomy. The Observatory 68, 121-143 (1948)

23.Zwicky, F.: Discovery, Invention, Research Through the Morphological Approach. Macmillan, Toronto (1969)

24.Knolmayer, G., Borean, A.: A Morphological Box for Handling Temporal Data in B2C Systems. In: Cellary, W., Estevez, E. (Eds.): Software Services for e-World, Proceedings of the 10th IFIP Conference on e-Business, e-Services, and e-Society, pp. 215-225. Springer, Berlin (2010)

25.Rosson, M.B., Carroll, J.M.: Usability Engineering. Morgan Kaufmann, San Francisco (2002)

26.Virzi, R.A.: Refining the test phase of usability evaluation: How many subjects is enough? Human Factors 34, 457-468 (1992)

27.Nielsen, J.: How to Conduct a Heuristic Evaluation (no date)
http://www.useit.com/papers/heuristic/heuristic_evaluation.html

28.Cockton, G., Woolrych, A., Lavery, D.: Inspection-Based Evaluations. In: Sears, A., Jacko, J.A. (Eds.): The Human-Computer Interaction Handbook, 2nd ed., pp. 1171-1208. CRC Press, New York (2008)

29.Nielsen, J.: Designing Web Usability. New Riders, Indianapolis (2000)

30.Nielsen, J., Tahir, M.: Homepage Usability. 50 Websites Deconstructed. New Riders, Indianapolis (2002)31.

31.Hanna, J.: Date & Time Formats on the Web (no date)
http://www.hackcraft.net/web/datetime/

32.Bargas-Avila, J., Oberholzer, G.: Online form validation: Don't show errors right away. In: Rauterberg, M., Menozzi, M., Wesson, J. (Eds.): Human-Computer Interaction INTER-ACT'03, pp. 848-851. IOS Press, Amsterdam (2003)

33.Nielsen, J., Loranger, H.: Prioritizing Web Usability. New Riders, Berkeley (2006)