# Do Usability Professionals Think about User Experience in the Same Way as Users and Developers Do?

Torkil Clemmensen, Morten Hertzum, Jiaoyan Yang, Yanan Chen

► To cite this version:

Torkil Clemmensen, Morten Hertzum, Jiaoyan Yang, Yanan Chen. Do Usability Professionals Think about User Experience in the Same Way as Users and Developers Do?. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.461-478, 10.1007/978-3-642-40480-1_31. hal-01501764

HAL Id: hal-01501764

https://inria.hal.science/hal-01501764

Submitted on 4 Apr 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Do Usability Professionals Think about User Experience in the Same Way as Users and Developers Do?

Torkil Clemmensen<fnref n="1" />, Morten Hertzum<fnref n="2" />, Jiaoyan Yang<fnref n="3" /> and Yanan Chen<fnref n="3" />

**Abstract.** In this paper, we study how usability professionals' thinking about system use relates to that of system developers and end users. We conducted 72 repertory-grid interviews to capture how usability professionals, developers, and users describe their system use. The participants in each stakeholder group were from China, Denmark, and India. Our results indicate that usability professionals focus on emotion-related aspects of system use, while users focus more on context in terms of utility and degree of usage. There are no interactions between stakeholder group and nationality, although both stakeholder group and nationality independently influence how participants think about usability and user experience. We recommend that to understand users' concerns, researchers should study context more.

**Keywords:** Usability professionals; UX professionals; user experience; stakeholder differences; cultural differences.

## 1 Introduction

In this paper, we focus on how usability professionals' thinking about system use is different from other stakeholder groups, in particular system developers and end users. By usability professionals, we mean people with titles such as usability professional, user experience manager, user experience specialist, user researcher, customer-experience architect, human factors specialist, interaction designer, usability engineer, and more. Differences between usability professionals and other stakeholders in their thinking about system use can be expected for several reasons. First, usability professionals' and system developers' thinking about system use may differ from that of end users, because professionals have systems as the objects of their professional work, whereas users apply systems to do their activities. That is, the "computer application, from the user's perspective, is not something that the user operates *on* but something that the user operates *through* on other objects or subjects." [1, p. 1]. Second, usability professionals' thinking about system use may differ from that of developers and users, because usability professionals will tend to be in an evaluation mode (trained to reflect explicitly about use), in contrast to system developers and users. Third, there are
adfa, p. 1, 2011.

<footnote n="1">¹Department of IT Management, Copenhagen Business School, Denmark
tc.itm@cbs.dk</footnote>

<footnote n="2">²Computer Science and Informatics, Roskilde University, Denmark
mhz@ruc.dk</footnote>

<footnote n="3">³Institute of Psychology, Chinese Academy of Science, Beijing, China
{chenyn, yangjy}@psych.ac.cn</footnote>
cultural differences in how people experience the use of interactive systems and prod-
ucts, and also in the way people respond to usability evaluation techniques, and in
how they share their experiences of using a system [22].

The fit between the different stakeholder groups’ view of system use is a basic raison d’être for the usability profession. A better fit between users and usability professionals in their thinking about system use, compared to users and system developers, has been the main argument for a usability profession since the beginning of the profession [20]. According to Putnam and Kolko [17, p. 2021] a HCI professional [their term] should be able to “walk in the users’ shoes”. The usability profession emerged because design-oriented developers did not have such empathy with the users.

Recently, two developments have made it imperative to study in depth how usability
professionals think differently than users and developers about the use of systems.
First, the usability professionals' identity has expanded into a much broader User
Experience (UX) professional identity [3]. According to Dumas [3], some usability
professionals with backgrounds in psychology and human factors perceived this de-
velopment as a degradation of the skills of the profession, while other usability pro-
fessionals accepted it as an expansion of the profession. To reconcile such incompati-
ble beliefs, more knowledge about what distinguishes the thinking of a usability pro-
fessional from that of developers and users is much needed.

Second, the globalization of the interactive systems industry has led to a worldwide
quest for locally available usability and UX skills. The tasks of a global UX team can
include designing a new interactive system for global users, localizing an existing
system for national users, designing completely new system concepts for the local
users, providing UX management for local developers, and more [19]. In such pro-
jects, the geographical location of stakeholders and the availability of specialized UX
skills in a particular geographical location are important. UX skills are not distributed
evenly across geographical locations but depend, among other things, on the mix of
multidisciplinary university and professional training available locally [19]. Accord-
ing to Rosenberg and Kumar [19], local UX professionals may be needed to study
domain specific requirements, while general issues like UI patterns and guidelines
might be shared across countries. It appears to be a best practice in the computing
industry to hire local professionals because they are assumed to be in some way dif-
ferent from professionals in other countries.

In this paper, we investigate differences between usability professionals, users, and
developers in how they think about system use. We do this based on 72 repertory-grid
interviews. We also examine whether nationality (China, Denmark or India) affects
the differences between the three stakeholder groups. We use four UX classifications
to make a content analysis of the interviews. This paper contributes with insights into
the unique operational constructs held and used by usability professionals, compared
to the operational constructs used by other stakeholder groups in systems develop-
ment. These insights are important to usability professionals' practices when they
design and evaluate UX. The insights should also improve communication about UX
between stakeholder groups in systems development.
## 2 Background

Redish and Barnum [18] stated that from the late 1990s the usability profession opened up to technical communicators, information architects, interaction designers, content strategists, and others who were not part of the human factors or software engineering traditions from the beginning of the usability profession. As a part of this evolution, the usability profession gradually became a broader UX profession, culminating with Usability Professionals' Association (UPA) changing its name to User Experience Professionals Association (UXPA). Dray [2, p. 5] described how it became more difficult to identify a usability professional as one who practiced the usability test methodology: "In the past, usability could lay claim to a professional identity based on a particular set of practices or methodology—usability testing.". So what is it that distinguishes the usability professional from other IT professionals? Putnam and Kolko [17] proposed that we should study the differences between professionals who bear user-oriented job titles such as usability engineer and those who have design-oriented job titles such as developer. In a pilot study, they identified three professional groups: (1) design-oriented such as developers and designers, (2) UX oriented such as UX architects and usability professionals, and (3) management oriented. They found significant differences between the groups in their use of a range of methods to study and communicate about end-users, for example, the UX group reported a higher average use of contextual inquiry methods. They also found that the design-oriented group showed more empathy in how they considered users, compared to the UX oriented group, as measured by scales of emphatic concerns and personal distress [17]. However, the differences in empathy were not significant. Putnam and Kolko [17] concluded that we need to study differences, including regional differences, among the professionals who create products and those who conduct user research.

According to Lindgaard [15], usability professionals traditionally had their basic training in the behavioural and social sciences. Shackel [20] described the usability profession as being particularly focused on the evaluation of users' task completion with interactive tools in specified environments. Thus, while being a part of the design process, usability professionals' work was from the beginning of the profession defined as having to do with the analysis of user tasks and needs, and the forwarding of the resulting knowledge to developers. According to Lindgaard [15], the focus was on performance measures all the way from usability metrics of job task effectiveness and efficiency to broad business-level concerns. To the extent that the users' emotions have been in focus, the efforts have been directed toward helping the user avoid frustration, see for example [14]. Among others, Dumas [3] and Lindgaard [15] point out that although the concepts of attitude and satisfaction toward an interactive system were part of the classic understanding of usability, the broader field of user experience was still to come.

Early in the history of the profession, 'user' was the term used to denote the stakeholder group that interacted with the system to obtain some goal, and who did not have much interest in, or knowledge about, the technology itself. As Shackel [20, p. 339] put it: "For many users the informatics system is essentially the terminal or workstation which they are using, and that is the central computer as they see it". Thus, although the user was not attending to the system as such, s/he was acutely
aware that she or he was using a system. In contrast, Shackel [20] claimed that devel-
opers tended to forget about the user and focus on the technological artefact. Shackel
[20, p. 342] describe this: "Too easily this [setting goals and specifying attributes of
the design] may be seen solely as the work of the designers... [but] to achieve usabil-
ity in the ultimate design it is essential that users and managers are fully involved".
The stakeholder groups of user, developer, and usability professional were thus de-
fined as being different in their relation to each other and to the technology.

Irrespective of the recent efforts of the HCI community to define usability and insti-
tute UX standards [10], people relate to systems through their personal experience and
concepts. Hertzum, Clemmensen, Hornbæk, Kumar, Shi and Yammiyavar [9] ana-
lysed 48 repertory-grid interviews with developers and users to see how their personal
usability constructs were affected by nationality (Chinese, Danish, or Indian) and
stakeholder group (developer or user). For stakeholder groups, a significant overall
difference between developers and users was found. The differences included that
users associated ease of use with leisure and, conversely, difficulty in use with work-
relatedness, while developers expressed the reverse associations. There was no signif-
icant overall difference across nationalities in participants' thinking about usability.

Hertzum and Clemmensen [8] analysed how 24 usability professionals' thinking about usability fitted with analytic usability definitions, and found that usability professionals made use of more utilitarian than experiential, i.e. UX related, constructs. This indicates that goal-related performance was central to usability professionals' thinking about usability, whereas they had less elaborate sets of experiential constructs. The usability professionals tended to construe usability at an individual level, rather than at organizational and environmental levels. However, the usability professionals' constructs went considerably beyond what was captured by the ISO 9241 definition of usability, thereby indicating a discrepancy between this definition of usability and the thinking of the professionals concerned with delivering usability.

We are unaware of studies in which usability professionals are compared systemati-
cally with other stakeholder groups. Thus, it remains unknown whether usability pro-
fessionals think more like users than developers do. We need to know more about the
usability constructs of usability professionals and other stakeholder groups, and in
particular, how they may differ in the way that they think about user experience.

3 Method

In this paper, we merge the data from our two previous studies [8, 9] and, for the first
time, provide an analysis of how usability professionals' thinking about user experi-
ence compare to that of developers and users. As described above our previous stud-
ies have analysed usability professionals in isolation [8] and compared developers and
users [9]. While the data in the current paper are re-used from the two previous stud-
ies, the analysis of the data is completely new.

We analyse repertory-grid interviews with 72 people to capture how they described
their system use. The repertory-grid technique was developed by Kelly [11] as part of
his personal construct theory. It has been successfully used in many studies of sys-
tems development, design, and use [7, 21]. We used local interviewers in the three involved countries to help make the data collection cross culturally valid.

## 3.1 Participants

The 72 participants were 24 usability professionals, 24 system developers, and 24 end users. Within each of these three stakeholder groups there were eight participants from China, eight from Denmark, and eight from India (see Table 1). To ensure that the participants were representative of the general cultural context of computer use in their country, participants from each country had to be a resident of the specific country and have been raised in the country. We recruited through our personal networks, and the participants were thus a convenience sample. All participants had good English skills. All participants used text-processing, web applications, and email every day, or nearly every day. They all stated that their computer hardware was sufficient, that they were comfortable with computers, and that they felt neither anxious nor relaxed/indifferent when they ran into a problem with a computer or application.

<table><caption>**Table 1.** Demographic information about the 72 participants</caption><thead><tr><th rowspan="2">Group</th><th colspan="2">Gender</th><th colspan="2">Age (years)</th><th colspan="2">IT experience (years)</th></tr><tr><th>Male</th><th>Female</th><th>M</th><th>SD</th><th>M</th><th>SD</th></tr></thead><tbody><tr><td>Chinese usability prof.</td><td>5</td><td>3</td><td>26.6</td><td>±3.2</td><td>9.8</td><td>±2.4</td></tr><tr><td>Chinese developers</td><td>5</td><td>3</td><td>31.5</td><td>±1.9</td><td>10.6</td><td>±1.7</td></tr><tr><td>Chinese users</td><td>5</td><td>3</td><td>27.3</td><td>±1.9</td><td>8.4</td><td>±1.9</td></tr><tr><td>Danish usability prof.</td><td>3</td><td>5</td><td>32.5</td><td>±4.6</td><td>18.6</td><td>±6.1</td></tr><tr><td>Danish developers</td><td>5</td><td>3</td><td>36.6</td><td>±5.8</td><td>19.3</td><td>±5.8</td></tr><tr><td>Danish users</td><td>5</td><td>3</td><td>36.8</td><td>±6.2</td><td>16.9</td><td>±3.6</td></tr><tr><td>Indian usability prof.</td><td>7</td><td>1</td><td>29.9</td><td>±1.6</td><td>9.5</td><td>±2.7</td></tr><tr><td>Indian developers</td><td>8</td><td>0</td><td>29.6</td><td>±1.7</td><td>9.9</td><td>±2.5</td></tr><tr><td>Indian users</td><td>5</td><td>3</td><td>29.0</td><td>±4.0</td><td>7.0</td><td>±2.1</td></tr></tbody></table>

## 3.2 Procedure

The procedure was similar to the procedure proposed by Kelly [11]. Participants were interviewed individually at their workplace by a local interviewer; that is, a person with the same nationality as the participant. Participants filled out a questionnaire about their background and tried to elicit constructs with the repertory-grid technique on a couple of training tasks. Then the actual repertory-grid interview was conducted. It consisted of two steps: selection of systems and elicitation of constructs.

In selecting systems, the participant was asked to consider “the array of computer applications you use for creating, obtaining, revising, managing, and communicating information and documents in the course of your day-to-day activities.” We maintained a focus on the participants’ work by interviewing them at their workplace and by encouraging them to look for candidate systems at their workplace computer (in the start menu). Participants had to select a system within each of six categories: my
text processing system, my email, a useful system, an easy-to-use system, a fun system, and a frustrating system.

To elicit construct-contrast words, the participant was given groups of three of the selected systems and asked: “Can you think of some important way in which your personal experience using these three systems makes two of the systems alike and different from the third system?” Having indicated the two similar systems, the participant wrote down a short phrase that explained how these two systems were alike – the construct – and another short phrase that explained how the third system differed – the contrast. Then, a seven-point rating scale was defined with this construct/contrast pair as its end points, and the participant rated all six systems according to this rating scale. The construct-elicitation step was repeated for all twenty combinations of three systems, in random order, or until the participant was unable to come up with a new construct for two successive combinations. In this way, each participant elicited his or her personal grid of constructs.

Following local customs, Danish and Indian participants received no compensation for their participation in the study while Chinese participants were paid a small amount in RMB for their participation. Each interview lasted about 1.5 hours.

The repertory-grid interviews were conducted by three local interviewers: a Chinese, a Dane, and an Indian. The interviewers were all HCI researchers. We followed closely the prescriptions from Kelly [11] for doing repertory grid interviews. After a pilot interview, we met for sharing experiences and creating consensus about how to do the interviews, while maintaining the benefits of having local interviewers.

## 3.3 Data Analysis

Microsoft Word was the dominant choice of text-processing system among the participants, and Microsoft Outlook was the most frequently selected email system. For the four other types of system, there was more diversity among the selected systems. Most of the selected systems were software that is used all around the world. The selection of these systems did not appear to be biased by the participants’ nationality.

We analysed the elicited constructs by categorizing them according to four UX classifications, Table 2. The first classification distinguished between system, user, and context of use. The ISO standard for UX [10] lists these three factors as those that influence the subjective UX. The second classification contrasted subjective and objective experiences, by following the suggestion in Han [5, 6] that objective performance and subjective image/impression are both important dimensions. This classification was chosen because it gave detailed definitions of the two categories and could be expected to capture broadly any UX related constructs [5, 6]. In addition, the classification seemed to reflect the utilitarian/experiential dimension used by other researchers. The third and the fourth classifications were chosen to maximize the differences among the four classifications. We included Kujala, Roto, Väänänen-Vainio-Mattila, Karapanos, and Sinnelä [12]’s classification of UX into general relation to the system, attractiveness of the system, ease of use, utility, and degree of usage because all of these categories were defined with a focus on the IT system. In contrast to this system-oriented UX definition, we chose McCarthy and Wright [16]’s four
threads of experience: compositional, sensual, emotional, and spatio-temporal, because they focus on human experience in general.

<table><caption>**Table 2.** The classifications used in categorizing the constructs</caption><thead><tr><th>Classification</th><th>Category definitions</th></tr></thead><tbody><tr><td colspan="2"><em>ISO 9241-210 user experience</em></td></tr><tr><td>System-related</td><td>All components of an interactive system that provide information and controls for the user to accomplish specific tasks with the interactive system</td></tr><tr><td>User-related</td><td>Emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviours and accomplishments that occur before, during and after use.</td></tr><tr><td>Context of use-related</td><td>Other people, tasks, equipment, and the physical, social, temporal, organizational and cultural environments in which a product is used</td></tr><tr><td colspan="2"><em>Objective vs. subjective UX</em></td></tr><tr><td>Performance</td><td>How well users perceive and interpret the interface of a system, how fast the users get used to the product and how well they remember it, and the users control activity and its results;</td></tr><tr><td>Image/Impression</td><td>Basic sense (the primitive and direct image/impression stemming from the design characteristics), description of image of a system, and evaluative feeling</td></tr><tr><td colspan="2"><em>System-oriented UX</em></td></tr><tr><td>General relationship and user experience with the system</td><td>Any experience with systems that users find meaningful and important</td></tr><tr><td>Attractiveness of the system</td><td>General attractiveness (appeal) of the product in the users' own eyes and those of their friends; more than users' rational or practical experiences</td></tr><tr><td>Ease of use of the system</td><td>Product is easy and effortless to use</td></tr><tr><td>Utility of the system</td><td>Product serves an important function for the user</td></tr><tr><td>Degree of usage of the system</td><td>Degree of usage which affects user experience over time, related to quality of experience</td></tr><tr><td colspan="2"><em>Human experience of technology</em></td></tr><tr><td>Compositional</td><td>The way that different elements of experience form a coherent whole; the narrative structure, action possibility, plausibility, consequences and explanations of actions</td></tr><tr><td>Sensual</td><td>The concrete, palpable (to take in your hand), and visceral (in flesh and blood) character of experience that is grasped pre-reflectively in the instant situation</td></tr><tr><td>Emotional</td><td>Value judgments (e.g., frustration and satisfaction) that ascribe importance to other people and things with respect to the user's needs and desires</td></tr><tr><td>Spatio-temporal</td><td>What draws attention to the quality and sense of space-time that pervades experience</td></tr></tbody></table>
We categorized all constructs according to the first classification before we proceeded to the second classification, then categorized all constructs according the second classification, and so forth. For each classification, the categorization of the constructs involved five steps. First, the first author thoroughly discussed the classification with to the two coders, authors three and four, in order to ensure a common understanding of the classification. Second, a randomly selected training set, consisting of 20% of the constructs, was categorized by each coder independently. Each construct was assigned either to one of the categories of the classification or to an ‘other’ category. Third, all disagreements in the coders’ categorizations of the training set were discussed to reach consensus about the categorization of the constructs. The rationale for the second and third step was to create a shared understanding of how the classification was used. Fourth, the remaining 80% of the constructs were categorized by the two coders independently to be able to assess the quality of the categorization. Fifth, all disagreements in the coders’ categorizations of these 783 constructs were discussed and a consensus was reached, in order to arrive at the final classification. used in the analysis.

The Kappa values of the agreement between the coders in their coding of the 783 non-
training constructs were 0.55, 0.62, 0.61, and 0.68 for the ISO 9241-210 UX, objec-
tive vs. subjective UX, system-oriented UX, and human experience of technology
classifications, respectively. Whereas all four Kappa values indicate statistically sig-
nificant agreement, the value of 0.55 for the ISO 9241-210 UX classification is below
the minimum threshold of 0.60 recommended by Lazar et al. [13]. Since this classifi-
cation was important, and the agreement was approaching the threshold, we decided
to keep the classification in the analysis.

4 Results

The 72 participants elicited a total of 977 construct/contrast pairs, corresponding to an average of 13.57 pairs per participant. The minimum number of construct/contrast pairs elicited by a single participant was three, the maximum 20. Below, we first analyse the constructs for effects of stakeholder group, then look into effects of the participants’ nationality, and finally investigate interrelations across the classifications.

4.1 Differences between Stakeholder Groups

To analyse whether usability professionals had different user experiences than devel-
opers and users Table 3 gives, for each stakeholder group, the average percentage of
constructs in a category across the participants in that stakeholder group. We used
multivariate ANOVAs to test for stakeholder differences in the distribution of con-
structs across all categories in a classification and univariate ANOVAs for the indi-
vidual categories. The statistical analysis was performed on the percentage distribu-
tions of the constructs (columns 2 to 4 of Table 3); this was done to assign equal
weight to participants, irrespective of the number of constructs elicited by a partici-
pant. Before conducting the statistical analyses, the percentage values for each partic-
ipant were arcsine transformed because percentages cannot be assumed normally
distributed [4]. All pair-wise comparisons reported below were Bonferroni adjusted to
compensate for multiple comparisons.

For the classification of the constructs into system-, user- and context-related ISO 9241-210 UX there was a significant effect of stakeholder group, Wilks' λ = 0.77, *F*(8, 132) = 2.35, *p* < 0.05. While there were no differences across stakeholder groups for the percentage of system-related constructs (e.g., "Address register/No address register"), *F*(2, 70) = 0.99, *p* = 0.38, and other constructs (e.g., "Copyright protected (license needed)/Download from the internet for free"), *F*(2, 70) = 0.19, *p* = 0.83, there were significant differences across stakeholder group for user-related constructs (e.g., "Lot of fun to use/Hard job"), *F*(2, 70) = 5.55, *p* < 0.01, and for context- related constructs (e.g., "Communication tool/Work tool"), *F*(2, 70) = 6.36, *p* < 0.05. Pair-wise comparisons showed that usability professionals had a higher percentage of user-related constructs than both developers (*p* < 0.05) and users (*p* < 0.01); developers and users did not differ significantly from each other. In contrast, usability professionals had a *lower* percentage of context-related constructs than users (*p* < 0.001).

For the classification of the constructs according to objective vs. subjective UX there was a significant effect of stakeholder group, Wilks' λ = 0.81, *F*(6, 134) = 2.47, *p* < 0.05. While there were no differences across stakeholder groups for the percentage of performance constructs (e.g., ""Efficiency/Productivity""), *F*(2, 70) = 1.79, *p* = 0.18, and other constructs (e.g., "Environment driven...peer group team members involved in its usage/Personal, self-driven""), *F*(2, 70) = 2.06, *p* = 0.14, there was a significant difference across stakeholder group for image/impression constructs (e.g., "Overwhelming interface [means that the interface has too many features]/Enjoyable, rich interface""), *F*(2, 70) = 5.60, *p* < 0.01. Pair-wise comparisons showed that usability professionals had a higher percentage of image/impression constructs than developers (*p* < 0.05) and users (*p* < 0.01); developers and users were not significantly different from each other.

The classification of the constructs according to system-oriented UX showed a signif-
icant effect of stakeholder group, Wilks' λ = 0.63, *F*(12, 128) = 2.82, *p* < 0.01. We
found significant differences for the categories of attractiveness (e.g., "Nice-looking
UI/Ugly UI"), *F*(2, 70) = 7.83, *p* < 0.01, and utility (e.g., "For playing movies/Do
scientific analysis"), *F*(2, 70) = 6.87, *p* < 0.01. Pair-wise comparisons showed that
usability professionals had a higher percentage of attractiveness constructs than de-
velopers (*p* < 0.05) and users (*p* < 0.01); developers and users were not significantly
different from each other. In contrast, usability professionals had a *lower* percentage
of utility constructs than users (*p* < 0.01). There were no differences across stakehold-
er groups for general relationship and user experience (e.g., "Bright and fluid [means
good to look at...organic]/Static [means very rigid in structure...not organic]"), ease
of use (e.g., "Demands more clicks/Demands fewer clicks"), degree of usage (e.g.,
"Frequent use/Seldom use"), and other (e.g., "Install package is small/Size is too
large"), *Fs*(2, 70) = 2.35, 1.50, 0.94, and 0.94 respectively (all *ps* > 0.1).

For the classification of the constructs according to human experience of technology
there was a significant effect of stakeholder group, Wilks' λ = 0.76, *F*(10, 130) = 1.95,
*p* < 0.05. While there were no differences across stakeholder groups for the categories
compositional (e.g., "Can be used independent of other applications/Have to be used
with other applications"), sensual (e.g., "Complex/Simple"), spatio-temporal (e.g. "Available everywhere/May not be available everywhere"), and other (e.g., "On cd/Downloaded"), F(2, 70) = 1.15, 1.33, 0.65, and 1.01, respectively (all ps > 0.2), there was a significant difference in the percentage of emotional constructs (e.g., "Live, active/Dead"), F(2, 70) = 5.80, p < 0.01. Pair-wise comparisons showed that usability professionals had a higher percentage of emotional constructs than developers and users (both ps < 0.05).

<table><caption>**Table 3.** Frequency (N = 977 constructs) and percentage (N = 72 participants) of constructs within each classification, averaged across participants from the same stakeholder group.</caption><thead><tr><th rowspan="2">Category</th><th rowspan="2">Frequency</th><th colspan="2">Usability</th><th colspan="2">Development</th><th colspan="2">Users</th><th colspan="2">Total</th></tr><tr><th>M</th><th>SD</th><th>M</th><th>SD</th><th>M</th><th>SD</th><th>M</th><th>SD</th></tr></thead><tbody><tr><td colspan="10"><strong>ISO 9241-210 UX*</strong></td></tr><tr><td>System-related</td><td>306</td><td>33</td><td>±18</td><td>34</td><td>±14</td><td>27</td><td>±20</td><td>31</td><td>±17</td></tr><tr><td>User-related**</td><td>204</td><td>30</td><td>±19</td><td>19</td><td>±15</td><td>16</td><td>±12</td><td>22</td><td>±17</td></tr><tr><td>Context-related**</td><td>427</td><td>33</td><td>±18</td><td>44</td><td>±20</td><td>52</td><td>±19</td><td>43</td><td>±20</td></tr><tr><td>Other</td><td>40</td><td>4</td><td>±5</td><td>3</td><td>±6</td><td>4</td><td>±6</td><td>4</td><td>±6</td></tr><tr><td colspan="10"><strong>Objective vs. subjective UX*</strong></td></tr><tr><td>Performance</td><td>414</td><td>38</td><td>±14</td><td>47</td><td>±21</td><td>44</td><td>±13</td><td>43</td><td>±17</td></tr><tr><td>Image/impression**</td><td>353</td><td>44</td><td>±13</td><td>34</td><td>±16</td><td>31</td><td>±14</td><td>36</td><td>±15</td></tr><tr><td>Other</td><td>210</td><td>18</td><td>±10</td><td>20</td><td>±15</td><td>25</td><td>±14</td><td>21</td><td>±13</td></tr><tr><td colspan="10"><strong>System-oriented UX**</strong></td></tr><tr><td>General relationship and user experience with the system</td><td>77</td><td>11</td><td>±8</td><td>7</td><td>±9</td><td>6</td><td>±8</td><td>8</td><td>±9</td></tr><tr><td>Attractiveness of the system**</td><td>75</td><td>14</td><td>±13</td><td>6</td><td>±6</td><td>4</td><td>±6</td><td>8</td><td>±10</td></tr><tr><td>Ease of use of the system</td><td>178</td><td>22</td><td>±11</td><td>17</td><td>±16</td><td>16</td><td>±13</td><td>18</td><td>±14</td></tr><tr><td>Utility of the system**</td><td>405</td><td>30</td><td>±16</td><td>41</td><td>±23</td><td>54</td><td>±20</td><td>42</td><td>±22</td></tr><tr><td>Degree of usage of the system</td><td>46</td><td>3</td><td>±4</td><td>7</td><td>±7</td><td>4</td><td>±4</td><td>5</td><td>±5</td></tr><tr><td>Other</td><td>196</td><td>20</td><td>±14</td><td>22</td><td>±17</td><td>16</td><td>±15</td><td>20</td><td>±16</td></tr><tr><td colspan="10"><strong>Human experience of technology*</strong></td></tr><tr><td>Compositional</td><td>547</td><td>53</td><td>±14</td><td>59</td><td>±17</td><td>57</td><td>±14</td><td>57</td><td>±15</td></tr><tr><td>Sensual</td><td>95</td><td>12</td><td>±8</td><td>10</td><td>±9</td><td>8</td><td>±7</td><td>10</td><td>±8</td></tr><tr><td>Emotional**</td><td>94</td><td>15</td><td>±11</td><td>7</td><td>±6</td><td>7</td><td>±9</td><td>10</td><td>±9</td></tr><tr><td>Spatio-temporal</td><td>178</td><td>15</td><td>±11</td><td>18</td><td>±14</td><td>20</td><td>±11</td><td>18</td><td>±12</td></tr><tr><td>Other</td><td>63</td><td>5</td><td>±6</td><td>5</td><td>±8</td><td>7</td><td>±9</td><td>6</td><td>±8</td></tr></tbody></table>

## 4.2 Nationality Differences

We tested for interactions between nationality and stakeholder group using multivariate ANOVAs, but found no overall interactions between nationality and stakeholder group for any of the four classifications. Stakeholder group and nationality appeared to be two independent factors, each influencing how participants think about UX.

<footnote>*p < 0.05, **p < 0.01, indicate significant differences between stakeholder groups.</footnote>
Regarding nationality, Table 4 gives, for each nationality, the average percentage of constructs in a category for a single participant.

The international standard for UX lists three factors as those that influence subjective UX (system-, user-, and context-related). For the classification of the constructs into these three ISO 9241-210 UX categories there was a significant effect of nationality, Wilks' $\lambda = 0.54$, $F(8, 132) = 5.88, p < 0.001$. There were significant differences across nationality in the percentage of system-related constructs, $F(2, 70) = 12.44, p < 0.001$; user-related constructs, $F(2, 70) = 3.63, p < 0.05$; context-related constructs, $F(2, 70) = 3.30, p < 0.05$; and other constructs, $F(2, 70) = 5.68, p < 0.01$. Pair-wise comparisons showed that Chinese participants had a higher percentage of system-related constructs than Indian and Danish participants (both $p < 0.001$) and a lower percentage of user-related ($p < 0.05$) and, tentatively, context-related ($p = 0.05$) constructs than Indian participants. Chinese participants had a higher percentage of other constructs than Indian participants ($p < 0.01$).

<table><caption>**Table 4.** Percentage of constructs in the categories of each classification, averaged across participants with the same nationality, N = 72 participants</caption><thead><tr><th rowspan="2">Category</th><th colspan="2">Chinese</th><th colspan="2">Danish</th><th colspan="2">Indian</th></tr><tr><th>M</th><th>SD</th><th>M</th><th>SD</th><th>M</th><th>SD</th></tr></thead><tbody><tr><td colspan="7"><strong>ISO 9241-210 UX<sup>***</sup></strong></td></tr><tr><td>System-related<sup>***</sup></td><td>44</td><td>±16</td><td>26</td><td>±12</td><td>25</td><td>±17</td></tr><tr><td>User-related<sup>*</sup></td><td>15</td><td>±11</td><td>24</td><td>±13</td><td>27</td><td>±22</td></tr><tr><td>Context-related<sup>*</sup></td><td>35</td><td>±17</td><td>46</td><td>±14</td><td>48</td><td>±26</td></tr><tr><td>Other<sup>**</sup></td><td>6</td><td>±6</td><td>4</td><td>±6</td><td>1</td><td>±3</td></tr><tr><td colspan="7"><strong>Objective vs. subjective UX</strong></td></tr><tr><td>Performance</td><td>44</td><td>±17</td><td>38</td><td>±14</td><td>47</td><td>±18</td></tr><tr><td>Image/impression</td><td>40</td><td>±16</td><td>39</td><td>±11</td><td>30</td><td>±17</td></tr><tr><td>Other</td><td>16</td><td>±8</td><td>24</td><td>±12</td><td>23</td><td>±18</td></tr><tr><td colspan="7"><strong>System-oriented UX<sup>***</sup></strong></td></tr><tr><td>General relationship and user experience</td><td>6</td><td>±7</td><td>11</td><td>±9</td><td>8</td><td>±9</td></tr><tr><td>Attractiveness</td><td>7</td><td>±6</td><td>8</td><td>±9</td><td>10</td><td>±13</td></tr><tr><td>Ease of use</td><td>22</td><td>±12</td><td>13</td><td>±10</td><td>20</td><td>±16</td></tr><tr><td>Utility<sup>*</sup></td><td>32</td><td>±16</td><td>44</td><td>±19</td><td>50</td><td>±27</td></tr><tr><td>Degree of usage<sup>***</sup></td><td>5</td><td>±6</td><td>7</td><td>±5</td><td>2</td><td>±4</td></tr><tr><td>Other<sup>***</sup></td><td>29</td><td>±15</td><td>17</td><td>±14</td><td>12</td><td>±12</td></tr><tr><td colspan="7"><strong>Human experience of technology<sup>*</sup></strong></td></tr><tr><td>Compositional<sup>***</sup></td><td>59</td><td>±14</td><td>49</td><td>±13</td><td>62</td><td>±16</td></tr><tr><td>Sensual</td><td>12</td><td>±10</td><td>11</td><td>±6</td><td>7</td><td>±7</td></tr><tr><td>Emotional</td><td>8</td><td>±6</td><td>11</td><td>±9</td><td>11</td><td>±12</td></tr><tr><td>Spatio-temporal<sup>**</sup></td><td>13</td><td>±9</td><td>24</td><td>±13</td><td>16</td><td>±12</td></tr><tr><td>Other</td><td>9</td><td>±9</td><td>5</td><td>±7</td><td>4</td><td>±7</td></tr></tbody></table>

The classification of the constructs according to objective vs. subjective UX showed

<footnote>*p < 0.05, **p < 0.01, ***p < 0.001.</footnote>
no effect of nationality, Wilks' $\lambda = 0.85$, $F(6, 134) = 1.84, p > 0.05$. Thus, Chinese, Danish, and Indian participants did not display a different distribution of their constructs. The percentages of performance and image/impression constructs tended to be roughly similar.

The classification of the constructs according to system-oriented UX showed a significant effect of nationality, Wilks' $\lambda = 0.55$, $F(12, 128) = 3.73, p < 0.001$. Significant effects were found for three categories: utility, $F(2, 70) = 4.36, p < 0.05$, degree of usage, $F(2, 70) = 6.88, p < 0.01$, and other $F(2, 70) = 9.71, p < 0.001$. Pair-wise comparisons showed that Indian participants had a higher percentage of utility constructs than Chinese participants ($p < 0.05$), and that Danish participants had a higher percentage of degree-of-usage constructs than Indian participants ($p < 0.01$). Chinese participants had a higher percentage of other constructs than Danish ($p < 0.05$) and Indian ($p < 0.001$) participants. As much as 29% of the Chinese participants' constructs were not covered by the system-oriented UX classification.

For the classification of the constructs according to human experience of technology there was a significant effect of nationality, Wilks' $\lambda = 0.72$, $F(10, 130) = 2.38, p < 0.05$. There were significant differences in the percentages of compositional constructs, $F(2, 70) = 5.00, p < 0.01$, and spatio-temporal constructs, $F(2, 70) = 5.32, p < 0.01$. Pair-wise comparisons showed that Indian participants had a higher percentage of compositional constructs than Danish participants ($p < 0.01$), and that Danish participants had a higher percentage of spatio-temporal constructs than Chinese participants ($p < 0.01$).

As stated above, we did not find any overall interaction effects between stakeholder group and nationality. Since this was an interesting and surprising result, we decided to analyse whether the influence on UX of stakeholder group and nationality were also independent for each category of the classifications. We found only two interaction effects when looking at the individual categories of the UX classifications. First, for the ISO 9241-210 classification, there was a significant interaction effect between nationality and stakeholder group for system-related constructs $F(4, 68) = 2.95, p < 0.05$. Chinese users and Chinese usability professionals used more system-related constructs than the same stakeholder groups from India and Denmark, while for developers there was no difference. Second, for the objective vs. subjective UX classification, there was a significant interaction effect between nationality and stakeholder group for other constructs $F(4, 68) = 4.16, p < 0.01$. Users from India were quite different from developers and usability professionals in the degree to which they construed UX in terms categorized as “other”. The objective vs. subjective UX classification was more exhaustive for all Chinese and Danish stakeholder groups.

## 4.3 Interrelations of Constructs across Classifications

To analyse the interrelations between categories in different classifications, Table 5 shows the ISO 9241-210 classification crossed with the three other classifications.

Comparing the ISO 9241-210 classification with the Objective vs. subjective UX classification, the differences across stakeholder groups were unevenly distributed across the Objective vs. subjective categories. The usability professionals had overall
most subjective image/impression constructs (139/105/109), and had in particular more user related (55/26/32) subjective image/impression constructs, compared to the other stakeholder groups. They had fewer context-related constructs in the performance (35/73/58) and other (46/78/61) categories, compared to users and developers.

Comparing the ISO 9241-210 classification with the system-oriented UX classification, the usability professionals had more user-related constructs, in particular about attractiveness (36/13/17) and ease of use (32/20/26). The usability professionals also had fewer context-related constructs, in particular about utility and degree of usage.

Comparing the ISO 9241-210 classification with the human experience of technology classification, the usability professionals had more user-related emotional constructs. Again, the usability professionals had fewer context-related constructs, in particular compositional constructs, compared to the other stakeholder groups.

<table><caption>**Table 5.** Cross-tabulation of stakeholder groups' ISO 9241-210 and the other UX classifications.</caption><thead><tr><th></th><th colspan="2">Subjective vs. Objective UX</th><th colspan="4">System-oriented UX</th><th colspan="4">Human experience of technology</th><th>Total</th></tr><tr><th></th><th>Subjective image/impression</th><th>Objective performance</th><th>General relationship</th><th>Attractiveness of system</th><th>Ease of use</th><th>Degree of usage</th><th>Utility</th><th>Other</th><th>Compositional</th><th>Sensual</th><th>Emotional</th><th>Spatio-temporal</th><th>Other</th></tr><tr><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th></tr></thead><tbody><tr><td colspan="14"><strong>ISO 9241-210 UX</strong></td></tr><tr><td>Usability professional</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>System</td><td>48</td><td>55</td><td>3</td><td>16</td><td>2</td><td>24</td><td>29</td><td>0</td><td>35</td><td>76</td><td>24</td><td>0</td><td>3</td><td>3</td></tr><tr><td>User</td><td>31</td><td>55</td><td>4</td><td>14</td><td>36</td><td>32</td><td>5</td><td>0</td><td>3</td><td>35</td><td>9</td><td>42</td><td>2</td><td>2</td></tr><tr><td>Context</td><td>35</td><td>26</td><td>46</td><td>4</td><td>2</td><td>8</td><td>63</td><td>11</td><td>19</td><td>55</td><td>3</td><td>2</td><td>45</td><td>2</td></tr><tr><td>Other</td><td>5</td><td>3</td><td>5</td><td>0</td><td>0</td><td>4</td><td>1</td><td>0</td><td>8</td><td>2</td><td>1</td><td>0</td><td>0</td><td>10</td></tr><tr><td>Total</td><td>119</td><td>139</td><td>58</td><td>34</td><td>40</td><td>68</td><td>98</td><td>11</td><td>65</td><td>168</td><td>37</td><td>44</td><td>50</td><td>17</td></tr><tr><td>User</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>System</td><td>44</td><td>49</td><td>1</td><td>5</td><td>0</td><td>24</td><td>33</td><td>0</td><td>32</td><td>63</td><td>18</td><td>0</td><td>2</td><td>11</td></tr><tr><td>User</td><td>27</td><td>26</td><td>2</td><td>11</td><td>13</td><td>20</td><td>8</td><td>1</td><td>2</td><td>22</td><td>3</td><td>22</td><td>2</td><td>6</td></tr><tr><td>Context</td><td>73</td><td>27</td><td>78</td><td>3</td><td>1</td><td>5</td><td>138</td><td>11</td><td>20</td><td>106</td><td>1</td><td>3</td><td>61</td><td>7</td></tr><tr><td>Other</td><td>10</td><td>3</td><td>3</td><td>1</td><td>0</td><td>6</td><td>4</td><td>1</td><td>4</td><td>7</td><td>2</td><td>0</td><td>2</td><td>5</td></tr><tr><td>Total</td><td>154</td><td>105</td><td>84</td><td>20</td><td>14</td><td>55</td><td>183</td><td>13</td><td>58</td><td>198</td><td>24</td><td>25</td><td>67</td><td>29</td></tr><tr><td>Developer</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>System</td><td>57</td><td>49</td><td>0</td><td>10</td><td>4</td><td>21</td><td>24</td><td>1</td><td>46</td><td>74</td><td>22</td><td>1</td><td>6</td><td>3</td></tr><tr><td>User</td><td>23</td><td>32</td><td>4</td><td>8</td><td>17</td><td>26</td><td>5</td><td>1</td><td>2</td><td>25</td><td>6</td><td>21</td><td>4</td><td>3</td></tr><tr><td>Context</td><td>58</td><td>23</td><td>61</td><td>4</td><td>0</td><td>5</td><td>94</td><td>19</td><td>20</td><td>81</td><td>1</td><td>3</td><td>50</td><td>7</td></tr><tr><td>Other</td><td>3</td><td>5</td><td>3</td><td>1</td><td>0</td><td>3</td><td>1</td><td>1</td><td>5</td><td>1</td><td>5</td><td>0</td><td>1</td><td>4</td></tr><tr><td>Total</td><td>141</td><td>109</td><td>68</td><td>23</td><td>21</td><td>55</td><td>124</td><td>22</td><td>73</td><td>181</td><td>34</td><td>25</td><td>61</td><td>17</td></tr><tr><td>Total</td><td>414</td><td>353</td><td>210</td><td>77</td><td>75</td><td>178</td><td>405</td><td>46</td><td>196</td><td>547</td><td>95</td><td>94</td><td>178</td><td>63</td></tr></tbody></table>
# 5 Discussion

## 5.1 Usability Professionals Differ from Other Stakeholder Groups

We find three main differences between usability professionals and the other stakeholder groups in their thinking about usability and UX: Usability professionals focus more on user-related constructs and subjective UX than developers and users, and they focus less on context-related constructs than users, as illustrated in Fig. 1 which shows selected information from Table 3. Below we discuss these differences.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Usability professionals focus more on users and subjective UX, less on context.</figcaption></figure>

Usability professionals focus more on user-relatedness and subjective UX than developers. This supports the main rationale behind the profession, which is that usability professionals compared to developers have more empathy with users and their situation [20]. The implication of this is that while developers might need to be convinced to become more user centred, usability professionals are ready for more advanced theory and techniques that go beyond simply being user centred.

Usability professionals focus more on user-relatedness and subjective UX than users. This finding is somewhat against the idea of usability professionals having the same view on system use as users. Usability professionals may focus more on subjective UX, emotions, attractiveness and so forth than users, because they as professionals have a greater range of constructs to describe and distinguish between UX-related emotions. The distinction between objective performance and subjective image/impression by Han [5] appears on the surface to be similar to a distinction between utilitarian and experiential constructs. However, in this study usability professionals attended a lot to subjective aspects of UX, whereas Hertzum and Clemmensen [8] found that usability professionals made use of more utilitarian than experiential constructs. A utilitarian/experiential distinction divides usability into a utilitarian factor concerned with goal-related performance and an experiential factor concerned with satisfaction [8]. Satisfaction is, partly, about freedom from discomfort, that is, when there is no frustration there may be satisfaction [14]. In contrast, in the objective/subjective classification, the subjective dimension was inspired by Kansei-engineering and focused on the emotions actually experienced by the user: “...collecting the subjective feelings of a product in words, extracting the key feelings...” [5, pp. 478-479]. Furthermore, the subjective dimension covers both the immediate image/impression that the user gets from interacting with the product in that moment, the user’s description of impressions of an interaction, and their judgment
and attitude toward this interaction [6]. Thus, a different set and range of emotional
processes involved in interacting with computers may be covered by the subjective
image/impression category, compared to an experiential dimension.

Usability professionals focus less on context-related constructs than users. We specu-
late that the difference in focus on context-relatedness may be related to the difference
in focus on subjective UX. Users may be concerned with getting the work done, a
context-related concern, to a larger extent than usability professionals, who in contrast
focus more on the emotions experienced during use, a less context-related concern.
According to Hertzum et al. [9], users associate ease of use with leisure and difficulty
in use with work-relatedness, while developers express the reverse associations. The
usability professionals in the present study rarely distinguished between work use and
other use (e.g., “Working Tools/For Fun”). Only three of the 107 context-related con-
struct/contrast pairs elicited by usability professionals concerned a work-leisure dis-
tinction. A possible explanation for the near absence of a distinction between work
and leisure in the usability professionals’ constructs may be that they consider UX-
related emotions relevant to leisure as well as work. If a system is sufficiently mature
in that most performance and utilitarian issues have been taken care of, the way to
improve the system further may be to focus on the emotional aspects of the user expe-
rience, irrespective of whether the system is for work or leisure. This explanation may
apply to our study because most of the systems selected by the participants were ma-
ture products, such as MS Word and Outlook. In addition, usability professionals may
be more business-oriented than users in their thinking about usability and UX.
Though we did not find any direct evidence of this, usability professionals may tend
to think about how the user experience can give a product a competitive edge on the
market, and therefore value emotional over context-related constructs.

## 5.2 Global Software Development and Inclusive UX Definitions

Stakeholder group and nationality independently influence how people think about
UX, as illustrated in Fig. 2. This means that usability professionals are different from
the other stakeholder groups, irrespective of nationality. The lack of significant inter-
action effects between stakeholder group and nationality suggests that the ways in
which usability professionals differ from local user groups will be somewhat similar
across national borders. This result is promising for the global IT industry because it
indicates that usability professionals' competences may be globally applicable, there-
by meeting a need pointed out by the global IT industry, see [19]. However, since we
found clear main effects of nationality – people from different countries think differ-
ently about their system use – it seems to vary from one cultural context and country
to another what exactly usability professionals do when they, paraphrasing Putnam
and Kolko [17, p. 2021], "...walk in the [local] users' shoes".
<figure><img src="image_3.png" /><figcaption>Fig. 2. Effects of stakeholder group and nationality on the ISO UX classification (%).</figcaption></figure>

The present study confirms that UX is an inclusive concept. In the ISO definition, UX is a broad concept that aims to capture most, if not all, of what people experience when interacting with computer systems. ISO [10, p.3] defines the user experience as a “person’s perceptions and responses resulting from the use and/or anticipated use of a product”. The four UX classifications were all good or very good at capturing the 977 constructs used by the 72 participants to describe their everyday system use. All four UX classifications had better coverage (average 87%) compared to the coverage found in an earlier study [8] for the ISO usability definition (average 53%). This study provides empirical support that selected UX classifications, including the ISO definition of UX, match the variation in how core stakeholder groups think about their system use.

## 5.3 Limitations

Three limitations should be remembered in interpreting the results of this study. First, the stakeholder groups of developer, user, and usability professionals evolve over time. In particular, the usability profession may be evolving into a UX profession with a broader or different focus. Hence, we characterize usability professionals’ thinking about system use at a time when their profession is in a process of discussing its identity. Second, the Danish participants were older than the Chinese and Indian participants were, and the Indian participants were gender skewed toward males. The age
difference may reflect that in Denmark people are relatively older when they finish
their education. However, we acknowledge that these issues related to the sample and
recruitment of participants could have affected our results. Third, with respect to the
developers and usability professionals there may be a gap between their personal
constructs and their professional knowledge. While we asked participants for their
personal constructs, their descriptions of their user experiences might be influenced
by explicit definitions of usability and UX, learned during their education rather than
experienced during the use of the selected systems.

6 Conclusion

We find differences in how usability professionals think about their user experiences, compared to developers and users. Therefore, if the usability/UX profession in the future wants to continue to claim that they are the users' advocate, it may be worthwhile to review the profession's key constructs. To meet users' concerns, it may be time for HCI researchers to revisit the context of use and focus more on the physical, social, temporal, organizational, and cultural environments in which a product is used. In addition, people's nationality influences all stakeholder groups' thinking about system use, according to our study of a sample of Danish, Chinese, and Indian people. Finally, the four UX classifications used in this study have emerged as inclusive concepts that captured nearly all the ways in which our participants thought about system use. This inclusiveness is encouraging from the point of view of devising analytic UX conceptualizations that encompass most of the variability of different stakeholders' actual user experiences.

Future research may investigate how varying levels of professional experience influ-
ence usability professionals' thinking about UX. The usability professionals partici-
pating in this study were intermediate-level to experienced usability professionals.
Their thinking about UX may gradually have moved away from that of users and
developers. Novice usability professionals may think more like users, or they may
more directly apply textbook definitions of usability and UX.

Acknowledgements

This study was co-funded by the Danish Council for Independent Research. Thanks to
Institute of Psychology, Chinese Academy of Science, Beijing, for hosting the first
author during data analysis in 2012. Thanks to Jyoti Kumar, Qingxin Shi, Xianghong
Sun, Pradeep Yammiyavar, and the interviewees for taking part in this study.

References

1. Bødker, S.: Through the Interface: A Human Activity Approach to User Interface Design. Lawrence Erlbaum Associates Inc., Hillsdale, NJ, (1990).
2. Dray, S.M.: Engaged Scholars, Thoughtful Practitioners: The Interdependence of Academics and Practitioners in User-Centered Design and Usability. Journal of Usability Studies, 5(1), 1-7, (2009).

3. Dumas, J.: The Great Leap Forward: The Birth of the Usability Profession (1988-1993). Journal of Usability Studies, 2(2), 54-60, (2007).

4. Fleiss, J.: Statistical Methods for Rates and Proportions. Wiley, New York, (1981).

5. Han, S.H., Hwan Yun, M., Kim, K.J., Kwahk, J.: Evaluation of Product Usability: Development and Validation of Usability Dimensions and Design Elements Based on Empirical Models. International Journal of Industrial Ergonomics, 26(4), 477-488, (2000).

6. Han, S.H., Yun, M.H., Kwahk, J., Hong, S.W.: Usability of Consumer Electronic Products. International Journal of Industrial Ergonomics, 28(3), 143-151, (2001).

7. Hassenzahl, M., Wessler, R.: Capturing Design Space from a User Perspective: The Repertory Grid Technique Revisited. International Journal of Human-Computer Interaction, 441-459, (2000).

8. Hertzum, M., Clemmensen, T.: How Do Usability Professionals Construe Usability? International Journal of Human-Computer Studies, 70(1), 26-42, (2012).

9. Hertzum, M., Clemmensen, T., Hornbæk, K., Kumar, J., Shi, Q., Yammiyavar, P.: Personal Usability Constructs: How People Construe Usability across Nationalities and Stakeholder Groups. International Journal of Human-Computer Interaction, 27(8), 729-761, (2011).

10. ISO 9241-210: Ergonomics of Human System Interaction-Part 210: Human-Centred Design for Interactive Systems (Formerly Known as 13407). International Organization for Standardization (ISO). Geneva, Switzerland, (2009).

11. Kelly, G.A.: The Psychology of Personal Constructs. Volume One: Theory and Personality. Norton, New York, (1955).

12. Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., Sinnelä, A.: UX Curve: A Method for Evaluating Long-Term User Experience. Interacting with Computers, 23(5), 473-483, (2011).

13. Lazar, J., Feng, J.H., Hochheiser, H.: Research Methods in Human-Computer Interaction. Wiley, Chichester, UK, (2010).

14. Lazar, J., Jones, A., Shneiderman, B.: Workplace User Frustration with Computers: An Exploratory Investigation of the Causes and Severity. Behaviour & Information Technology, 25(3), 239-251, (2006).

15. Lindgaard, G.: Early Traces of Usability as a Science and as a Profession. Interacting with Computers, 21(5-6), 350-352, (2009).

16. McCarthy, J., Wright, P.: Technology as Experience. Interactions, 11(5), 42-43, (2004).

17. Putnam, C., Kolko, B.: HCI Professions: Differences & Definitions, in CHI 2012 extended abstracts. ACM, New York, (2012), pp. 2021-2026.

18. Redish, G., Barnum, C.: Overlap, Influence, Intertwining: The Interplay of UX and Technical Communication. Journal of Usability Studies, 6(3), 90-101, (2011).

19. Rosenberg, D., Kumar, J.: Leading Global UX Teams. Interactions, 18(6), 36-39, (2011).

20. Shackel, B.: Usability-Context, Framework, Definition, Design and Evaluation. Interacting with Computers, 21(5-6), 339-346, (2009).

21. Tan, F.B., Hunter, M.G.: The Repertory Grid Technique: A Method for the Study of Cognition in Information Systems. MIS Quarterly, 26(1), 39-57, (2002).

22. Walsh, T., Nurkka, P., Walsh, R.: Cultural Differences in Smartphone User Experience Evaluation. In: MUM '10, Limassol, Cyprus, pp. (Article No. 24). ACM, New York (2010).