# The Application of Preference Mapping in Aesthetic Website Evaluation

Eleftherios Papachristos, Nikolaos Avouris

► To cite this version:

Eleftherios Papachristos, Nikolaos Avouris. The Application of Preference Mapping in Aesthetic Website Evaluation. 13th International Conference on Human-Computer Interaction (INTERACT), Sep 2011, Lisbon, Portugal. pp.616-619, 10.1007/978-3-642-23768-3_96 . hal-01596932

HAL Id: hal-01596932

https://inria.hal.science/hal-01596932

Submitted on 28 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# The application of Preference Mapping in aesthetic website evaluation

Eleftherios Papachristos, Nikolaos Avouris

Human-Computer Interaction Group, Electrical and Computer Eng. Dept.,
University of Patras, GR-265 00 Rio Patras, Greece.
{epap, avouris}@ece.upatras.gr,

**Abstract.** The objective of this study was to apply a technique called preference mapping to the context of aesthetic website evaluation. Preference mapping is a method in which evaluators and stimuli are simultaneously represented in the same multidimensional space. User segmentations and drivers of preference can easily be identified. We argue that this technique is particularly suited for website design evaluation especially for alternative prototype comparisons. The application of this method to an actual dataset resulted in a better understanding of participant preferences that could not be reached through simple comparison of average ratings.

**Keywords:** Web design, aesthetic evaluation, preference mapping.

## 1 Introduction

The importance of aesthetics in web design has been demonstrated by several studies [1, 2]. This realization underlines the need for visual website evaluation methodologies during initial stages of development. In this paper we tried to demonstrate that preference mapping is an appropriate method for the evaluation of website aesthetics. Preference mapping is referred to as a group of multivariate statistical techniques aimed at gaining deeper understanding of preference toward products by taking also participants' heterogeneity into account [3]. It is not a new technique since it is used widely in sensory analysis and marketing for many years. The main goal is to find relationships between descriptive data (usually provided by domain experts) and hedonic judgments of evaluators. This methodology can be encountered in HCI literature or website evaluation only rarely with the exception of [1].

There are two main preference mapping techniques which are generally referred to as internal (IPM) and external (EPM) analysis. There is as ongoing debate about the advantages and disadvantages of these variations of preference mapping [4]. In this study we used IPM alongside hierarchical cluster analysis to identify participant preference segmentation. The goal of IPM is to derive a multidimensional representation of objects and evaluators. This is usually accomplished by conducting principal component analysis (PCA) on a data matrix consisting of objects in rows and participants (treated as variables) in columns. In a variation of the technique called extended IPM, external descriptive data (usually provided by expert panel) are
projected onto the preference space. This can be done by projecting the attribute mean
scores onto the map coordinates through regression [3].

## 2 Method

The dataset used was previously acquired by a study regarding subjective and
objective factors that influence website design evaluation [5]. In this study, 53
volunteer students (35male, 18 female, mean age = 22.2) had to choose a new website
design for their university department among six prototypes. The test designs were
evaluated by two sources: the volunteer students and a trained panel of 9 design
experts. Student participants first viewed all the website screenshots in a random
order and then rated them on *visual appeal*, *credibility*, *perceived usability*, *novelty*
and *overall preference* on a linear, unmarked scale (from 0 to 100). The trained panel
first helped in generating a list of 14 design related descriptive attributes and then
rated the test websites on them.

## 3 Results

PCA was applied to participant's preference data for the websites. A two factor
solution explained 68.6% of the variance in participant preference (Fig. 1).

<figure><img src="image_2.png" /><figcaption>**Fig. 1** Bi-plot of websites and individual user preference in the derived two-dimensional space.</figcaption></figure>

Designs that are close together were perceived as similar regarding preference. Each
dot represents an individual user. A line from the center of the axes to that dot shows
general direction of preference for that participant. If all users were pointing to
approximately the same direction then their preferences would be homogenous and no
segmentation would be necessary. Although almost all preference is pointing to the
right, there is a significant spread across the vertical axis and sample segmentation
seemed required. Hierarchical cluster analysis on preferences revealed 3 groups of participants (table 1) with very different preferences. Group 1 gave moderate to high ratings to most websites except of the first two. Group preferred these first two designs the most and clearly disliked designs C and F. Finally, Group 2 which was also the most populated, was closer to the overall preference ratings, with clear preference to the last 3 designs and extreme dislike of design C.

<table><caption>**Table 1.** Mean design preference scores for the whole sample and the identified subgroups.</caption><thead><tr><th></th><th>A</th><th>B</th><th>C</th><th>D</th><th>E</th><th>F</th></tr></thead><tbody><tr><td>All users (n=53)</td><td>50,9</td><td>48,6</td><td>24,9</td><td>68,9</td><td>60,5</td><td>50,4</td></tr><tr><td>Group 1 (n=15)</td><td>38,3</td><td>39,6</td><td>56,6</td><td>63,1</td><td>50,9</td><td>66,6</td></tr><tr><td>Group 2 (n=22)</td><td>39,7</td><td>44,8</td><td>13,3</td><td>84,6</td><td>77,8</td><td>55,4</td></tr><tr><td>Group 3 (n=16)</td><td>78,1</td><td>62,2</td><td>11,3</td><td>52,7</td><td>45,9</td><td>28,5</td></tr></tbody></table>

In order to understand the drivers of preferences for our sample as a whole and for the different subgroups the derived preference dimensions have to be interpreted. This can be accomplished either by visual inspection of the websites and their position in the map or by projection of external descriptive data. The first method is more subjective but extremely useful if no external descriptive data is available. The projection of descriptive data requires the regression of their mean ratings in the dimensional space. We projected the evaluative constructs from the participants and the descriptive attributes from our expert panel that fitted significantly. The result of the attribute projection is depicted in Figure 2.

<figure><img src="image_3.png" /><figcaption>**Fig. 2** Attribute projection in the preference space. Arrows represent direction of preference for the user clusters.</figcaption></figure>
Examination of the final IPM map revealed that the overall preference direction and therefore the main driver for the participants as a whole, was *visual appeal*. Group 1 seems to be more influenced by *novelty*, group 2 by *credibility* and group 3 by *perceived usability*. Constructs that are close to each other, such as *credibility* and *visual appeal*, are highly correlated. Analysis of descriptive attribute positions reveals that increased *order*, *balance* and *simplicity* results in more *perceived usability* but also decreases perceptions of *visual appeal* and *novelty*. *Novelty* perceptions are driven by increased *saturation*, *contrast* and *colorful*. *Visual appeal* and *credibility* seem to require balance on the above attributes. The preference direction of overall preference and of the most populated group is towards designs E and D. The examination of table 1 reveals that design D is highly rated by each group and therefore is the safest pick as the final design choice.

# 4 Discussion

Our goal was to demonstrate the advantages of preference mapping in the context of visual web design evaluation by alternative prototype choice. The application of this technique resulted in deeper understanding of user perceptions and their key drivers of preference. The methodology helped us make better informed decision about final choice and also gave us instructions for design improvement towards desired directions. In addition the process of user clustering regarding their preference revealed distinct groups of participants with different opinions about final choice.

However, the above results cannot be generalized and are specific to the test sample of our case study. Our goal was to demonstrate an evaluation methodology which could be tailored to the context of different studies. To conclude, preference mapping seems promising in the visual web design evaluation context and flexible enough to be used in several different conditions.

# References

1. Schenkman, B.N., Jonsson, F.U.: Aesthetics and preferences of web pages. Behavior and Information Technology 19 5, 367--377 (2000)

2. Tractinsky, N., Katz, A.S., D. Ikar, D.: What is beautiful is usable, Interacting with Computers 13, 127--145 (2000)

3. Meullenet, J., Xiong, R., Findlay, C., & Knovel.: Multivariate and probabilistic analysis of sensory science problems.Ames, USA: Blackwell Publishing (2007)

4. van Kleef, E., van Trijp, H.C.M. and Luning, P. Internal versus external preference analysis: An exploratory study on end-user evaluation. Food Quality and Preference 17, 387--399 (2006)

5. Papachristos, E., Avouris, N.: The Subjective and Objective Nature of Website Aesthetic Impressions. In: Gross, T. et al. (eds.) INTERACT 2009, Part I, LNCS 5726, pp.119--122.Springer, Heidelberg (2009)