# Experimentally Manipulating Positive User Experience Based on the Fulfilment of User Needs

Andreas Sonnleitner, Marvin Pawlowski, Timm Kässer, Matthias Peissner

► To cite this version:

Andreas Sonnleitner, Marvin Pawlowski, Timm Kässer, Matthias Peissner. Experimentally Manipulating Positive User Experience Based on the Fulfilment of User Needs. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.555-562, 10.1007/978-3-642-40498-6_45. hal-01510512

HAL Id: hal-01510512

https://inria.hal.science/hal-01510512

Submitted on 19 Apr 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Experimentally Manipulating Positive User Experience
Based on the Fulfilment of User Needs

Andreas Sonnleitner¹, Marvin Pawlowski¹, Timm Kässer¹ and Matthias Peissner¹
Fraunhofer Institute for Industrial Engineering, IAO
Nobelstr. 12, 70569 Stuttgart, Germany

[andreas.sonnleitner, marvin.pawlowski, timm.kaesser, matthias.peissner]iao.fraunhofer.de

**Abstract.** In this work, we prepared three variations of a prototype to experimentally manipulate parameters helping to improve User Experience (UX) of technological products. Based on a model considering the fulfilment of user needs (Fig. 1), two variations of a neutrally designed tool were developed to address two selected needs (*popularity, competition*) by slightly changing functionalities or design elements. The manipulation of UX is validated with real-time and retrospective subjective evaluation of UX, and objective data of user behaviour. Participants rated significantly higher positive UX and showed more active behaviour for designs of the prototype addressing the user needs *competition* and *popularity* compared to the *neutral* design. These findings show the importance of considering elements of UX in the early development process of technological products.

**Keywords:** User Experience, UX, User Needs, User-Driven Innovation

## 1 Introduction

User Experience (UX) is considered a key factor for the success of interactive products and systems. Looking at the definition of UX, it is “all aspects of the user’s experience when interacting with the product ... it includes all aspects of usability and desirability of a product ... from the user’s perspective” [1]. Next to this definition, various different interpretations exist due to the multidisciplinary nature of UX, ranging from a more psychological to a business perspective [2].

Our approach is based on the fulfilment of user needs during interaction with technological products derived from models describing general human needs [3,4]. The emphasis is placed on positive UX by meeting basic needs which is critical to the long-term success of technological products. Products that generated a lot of positive experiences will be used more frequently and will therefore be preferred [5]. In the context of work environments, UX can have a positive impact on skills and mental health and can also cause for motivated action, increased productivity and job satisfaction [6]. The developed model in Fig. 1 describes a holistic view of positive UX and will be taken as a basis for generating positive UX for interfaces. In addition to
the final evaluation of a product regarding attitude and behaviour of users, also the
experience itself and the cause for the use of the product are included in the detection
of UX. Usability, directness and comfort are integrated as moderators.

The aim of this study is to specifically address user needs and to experimentally
manipulate UX in a work environment, simulating both a more creative phase of idea
generation, as well as a blunt execution of tasks. For this, an idea management tool is
provided with differently shaped functionalities and design elements illustrating per-
sonal performance measures, idea evaluation and received achievements. In a first
step, we decided to investigate two user needs with small overlap: Competition, i.e. to
be better and outplay others, and popularity, i.e. to act dutiful and be liked by others.
The influence of various functionalities and design elements on user behaviour as
well as on real-time and retrospective subjective evaluation of UX was measured. The
research questions investigated in this study are:

* Does the specific fulfilment of user needs lead to a higher UX?

* Can experimentally crafted functionalities and design elements satisfy the addressed user needs *competition* and *popularity*?

* Do experimentally crafted functionalities and design elements have an influence on user behaviour?

<figure><img src="image_2.png" /><figcaption>Fig. 1. Model of Positive User Experience.</figcaption></figure>

## 2 Methods

### 2.1 Participants

In the pilot study nine participants were involved to test first conceptual designs (18-
74 years, mean: 35.9 years, 4 females). The sample of the main study consisted of 20
participants (18-70 years, mean: 38.5 years, 11 females, mean office working experi-
ence = 15.0 years). No subject participated in both studies.
Subjects were recruited from an in-house database in which volunteers for experiments are listed. Participation was voluntary, data were collected anonymously. Informed consent was obtained after the task had been explained. Participants were informed they had the option to end participation at any time without any type of penalty. After the study, they received 30€ for participation.

## 2.2 Pilot study

As a result of wireframes and user centered paper prototyping, clickable prototypes of the fictional idea management tool were used to get information about elements most perceived by the participants and about user needs addressed particularly. With a combination of *Valence method* [7] and *Laddering technique* [8], emotions and needs were associated with functionalities and design elements. These results were used to improve the final design variations by focusing on salient elements with a high effect on user's perception (see Tab. 1).

<table><caption>**Table 1.** Key differences for designs of the prototype (neutral, popularity, competition).</caption><thead><tr><th></th><th>Neutral (n)</th><th>Popularity (p)</th><th>Competition (c)</th></tr></thead><tbody><tr><td>Idea evaluation concept</td><td>Isolated positive rating (likes)</td><td>Interactive positive rating (likes)</td><td>Interactive pos. and neg. rating (likes/dislikes)</td></tr><tr><td>Idea evaluation icon</td><td>○<br>"circle"</td><td>♥<br>"heart"</td><td> likethumbsup<br>"thumbs"</td></tr><tr><td>Idea evaluation illustration</td><td>-</td><td>Alphabetic order</td><td>Ranking list</td></tr><tr><td>Performance concept</td><td>Own value</td><td>Own and team mean value</td><td>Own and each team member's values</td></tr><tr><td>Performance icon</td><td>◇<br>"diamond"</td><td>◇<br>"bulb"</td><td>◇<br>"bag of money"</td></tr><tr><td>Performance illustration</td><td>Me<br>45<br>"value based circle"</td><td>Me<br>45<br>"value based circles"</td><td>Me<br>45<br>"bar chart"</td></tr></tbody></table>

## 2.3 Main study

**Setting.** To show coherences between design, functionality and UX, three prototypes of an idea management tool were developed based on Adobe Flash. Since the basic prototype is designed as a *neutral* one, the other two designs focus on fulfilling selected user needs (*popularity, competition*). Tab. 1 exemplarily shows some of the key
differences of specific design and functionality characteristics. The aim of the study is to validate the assumption that slightly changing design and functionality of elements has an impact on the fulfilment of both user needs, and therefore has an impact on UX. The task is to brainstorm ideas to different topics in a team with four other simulated users that give input by a predefined script identical for all participants. Besides continuously brainstorming ideas to a total of nine topics (three per design), participants had to answer knowledge questions appearing on their screen.

On the main part on the right side of the screen, the current topic of the idea management tool is presented. Entered ideas are automatically arranged around the topic resembling the structure of a mind map. On the left side of the screen, personal information is presented including current performance measures, idea evaluation and received achievements. For entered ideas and correctly answered knowledge questions, the participants receive performance points. For idea evaluation, participants rate entered ideas by the team members positive and/or negative, depending on the functionality of the respective design (details, see Fig. 2 and Tab. 1).

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** 2a and 2b exemplarily show a complete screen of the neutral design. On the main part on the right side of the screen the ideas were arranged resembling the structure of a mind map (2b). The key differences on the left side of the screen are drafted for the *neutral design (2a)* and the designs addressing *popularity (2c)* and *competition (2d)*. See also Table 1.</figcaption></figure>

**UX Questionnaire.** Based on the model of positive UX, a questionnaire was developed to measure the UX of a technological product that assesses the amount to which a need is fulfilled during interaction with a product. The *User Needs Questionnaire (UNeeQ)* consists of five items regarding overall positive and negative UX and 30 items regarding the ten user needs shown in Fig. 1. As a result, two overall scores for positive and negative UX, as well as ten scores for user needs are calculated.

**Test Procedure.** After filling out a small socio-demographic questionnaire, participants had to rank the three screens presented in Fig. 2 in terms of aesthetics without any additional information about the designs. For the main study, they started to pass through every design of the prototype in a randomly given order balanced over all subjects. At the beginning of each design variation, participants were given a standardized instruction how to use the respective prototype. After each design of the pro-
totype, they were instructed to fill out the *UNeeQ*. After the main study, participants were again asked to rank the three screens presented in Fig. 2 in terms of aesthetics. To control the influence of individual competitiveness, participants had to fill out a German version of the Competitiveness Index (CI) at the end of the study [9].

**Experimental Design.** The study implied one independent variable, design of prototype (*neutral*, *popularity*, *competition*), and a covariate, user's competitiveness (*low*, *high*). The dependent variables were subjective ratings of UX (*UNeeQ*), subjective design-rankings (*preferences of designs*), and objective user behaviour (*frequency of likes/dislikes*, *number of ideas*).

A repeated-measures analysis of variance (ANOVA) was used for all within-subject comparisons to identify the effect of design of prototype and for between-subject comparisons to identify the effect of user's competitiveness on subjective ratings of UX and on objective user behaviour. Subjective design-rankings were investigated with Wilcoxon signed rank test. Only significant results are reported.

## 2.4 Hypotheses

* Participants report a higher *positive overall UX* in the *UNeeQ* for the designs addressing user needs *popularity* and *competition* compared to the *neutral* design.

* Participants report a higher fulfilment of the scales *popularity* and *competition* in the *UNeeQ* for the designs addressing user needs *popularity* respective *competition* compared to the *neutral* design.

* Participants show a more active behaviour for designs addressing user needs *popularity* and *competition* compared to the *neutral* design.

# 3 Results

## 3.1 Pilot study

During Valence Method, participants marked elements regarding the idea evaluation the most (58 markers: 41 positive, 17 negative). Performance elements were marked 22 times (11 pos., 11 neg.), elements regarding the topic 17 times (9 pos., 8 neg.) and elements regarding the ideas 13 times (6 pos., 7 neg.).

The user needs addressed most with the clickable prototype were *popularity* (31), *competition* (21) and *security* (20). The other user needs mentioned were *competence* (7), *relatedness* (7), *stimulation* (5), *influence* (3), *idealism* (2) and *self-expression* (1).
## 3.2 Competitiveness

To control the effect of user's competitiveness, participants were divided into two groups based on the median of the total score in the CI of all 20 participants. ANOVA showed no effects for any of the dependent variables. Also, correlation between total score of CI and overall positive and negative UX of UNeeQ did not show significant correlations.

## 3.3 Subjective ratings of UX

In Tab. 2, results of the ANOVAs are summarized. Overall positive and negative UX showed a significant difference for the variable design of prototype. Compared to the *neutral* design, overall positive UX was significantly higher and overall negative UX was reported significantly lower for the designs addressing user needs *popularity* and *competition*.

Single scales of UNeeQ showed a significant difference for user needs *popularity* and *competition*, as well as for the scale *influence*. All other scales of UNeeQ showed no significant difference for the variable *design of prototype*.

<table><caption>**Table 2.** Statistical results (ANOVA for repeated measures): Overall positive and negative UX and single scales of UNeeQ for the variable *design of prototype*.</caption><thead><tr><th rowspan="2">Measure</th><th rowspan="2"></th><th colspan="3">Main effect</th></tr><tr><th>F(2,38)</th><th>p</th><th>η<sup>2</sup></th></tr></thead><tbody><tr><td rowspan="2">Overall</td><td>Positive UX</td><td>10.415</td><td>&lt;.001</td><td>.354</td></tr><tr><td>Negative UX</td><td>7.534</td><td>.002</td><td>.284</td></tr><tr><td rowspan="3">User needs</td><td>Competition</td><td>11.637</td><td>&lt;.001</td><td>.380</td></tr><tr><td>Popularity</td><td>4.546</td><td>.017</td><td>.193</td></tr><tr><td>Influence</td><td>4.469</td><td>.018</td><td>.190</td></tr></tbody></table>

## 3.4 Subjective design-rankings

At the end of the study, participants preferred the design addressing *competition* (mean rank = 1.40) in the subjective ranking over the *neutral* design (mean rank = 2.45) and the design addressing *popularity* (mean rank = 2.15).

Interestingly, the design addressing *competition* is ranked significantly better after the study compared to rankings before the study (Z = -2.762, p=.006). For the design addressing *popularity*, rankings were significantly worse after the study (Z = -2.233, p=.026). Rankings for the *neutral* design stayed equal.

## 3.5 Objective user behaviour

To objectively compare user behaviour, frequencies of idea evaluation and number of ideas are compared. ANOVA showed a trend for the number of likes for the variable design of prototype ($F_{(2,38)}=3.084, p=.057$). Post-hoc analysis showed no significant
differences between designs (mean number of likes: *neutral* = 6.05, *popularity* = 8.05, *competition* = 8.20). For entered ideas, participants showed no significant differences (mean number of ideas: *neutral* = 6.20, *popularity* = 6.70, *competition* = 6.45).

# 4 Discussion

In line with our hypotheses, overall positive UX, as well as single UNeeQ-scales for *competition* and *popularity* were significantly higher for designs addressing user needs *popularity* and *competition* compared to the *neutral* design. Also, UNeeQ-score for *influence* showed significant difference. A valid explanation is the obviously higher influence in the process of idea evaluation due to an interactive rating compared to an isolated rating in the *neutral* design.

Generally, user needs can only be separated to a certain degree. A small overlap already came up in the beginning of the development of our prototype when asking experts to attribute functionalities addressing special user needs. Nevertheless, all other scales showed no significant differences for the variable design of prototype. Therefore, the developed prototypes only addressed the expected user needs and showed a very good differentiation between single scales of user needs, which was an important requirement in our consideration.

Next to subjective ratings of UX, the two variations addressing user needs stated better spontaneous design-rankings compared to the *neutral* design. Interestingly, they preferred the design addressing user need *popularity* before the study, whereas after the study, they preferred the design addressing *competition* most. We assume that the slightly higher UX led participants to rank the design addressing *competition* over the design addressing *popularity* after interaction with the prototype.

In the pilot study, expected user needs *popularity* and *competition* were already most prominent in the clickable prototype. Besides, user need *security* was often mentioned when asking participants using Laddering Technique. It was mostly used for negative aspects of the usability of the clickable prototypes. Suggestions for improvement were considered and integrated for the prototype in the main study.

Regarding objective user behaviour, the design of the three prototypes had no significant influence on the brainstorming of ideas, but showed a tendency towards a more active idea evaluation for designs addressing selected user needs. We assume that the higher fulfilment of user needs, indicated by significantly higher UNeeQ-scores, led to this more active behaviour. Getting personal feedback due to interactive idea evaluation motivated participants to rate entered ideas by the team members.

## 4.1 Future work

Future work aims at investigating all other user needs mentioned in the model of positive UX (Fig. 1). Variations of the described prototype will be developed to address the whole spectrum of user needs to define guidelines or specific parameters of elements to further improve UX for new products [10]. Another intended application of this prototype is to further investigate UX and emotions during interaction. Subjective
surveys are usually conducted in retrospect by using questionnaires and interviews. The personal experience cannot be directly measured and is also shaped by personal attitudes and prejudices [11]. To measure UX during interaction with a technical product, neurophysiological parameters are promising to objectively indicate positive and negative emotions during experiencing [2]. They should help to explore the subconscious processes of implicit cognitions, and to draw conclusions about human behaviour [12]. Neurophysiological methods allow for a real-time measurement of brain activity during the current experience without significantly affecting the situation [13]. The approach chosen here is to identify and apply methods that provide insight into the subjective emotional experience in the interaction with technology and predict certain aspects of user behaviour.

# 5 References

1. ISO 9241-210 (2010): Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems.

2. Roto, V.; Law, E.; Vermeeren, A.; Hoonhout, J. (2011): User Experience White Paper - Bringing clarity to the concept of user experience. In: V., Roto, E., Law, A., Vermeeren, and J., Hoonhout (Eds.): Dagstuhl Seminar on User Experience - 2010. Dagstuhl.

3. Sheldon, K.M; Elliot, A.J; Kim, Y.; Kasser, T. (2001): What's satisfying about satisfying events? Comparing candidate psychological needs. In: Journal of Personality and Social Psychology 80, S. 325–339.

4. Reiss, S. (2004): Multifaceted Nature of Intrinsic Motivation: The Theory of 16 Basic Desires. In: Review of General Psychology 8 (3), S. 179–193.

5. Kahneman, D. (1999): Objective Happiness. In: D. Kahneman, E. Diener and N. Schwarz (Ed.): Well-being. The foundations of hedonic psychology. New York: Russell Sage Foundation Press, S. 3–25.

6. Harbich, S.; Hassenzahl, M. (2011): Using behavioral patterns to assess the interaction of users and product. Int. J. Hum.-Comput. Stud. 69 (7-8), S. 496-508.

7. Burmester, M.; Mast, M.; Jäger, K.; Homans, H. (2010): Valence Method for Formative Evaluation of User Experience. In: K. Halskov and M.G Graves Petersen (Ed.): Proc. of the 8th ACM Conf. on DIS. Arhus, 16.-20. Aug. NY, USA: ACM (DIS ’10), S. 364–367.

8. Reynolds, T.J., Gutman, J. (1988): Laddering Theory, Method, Analysis and Interpretation. In: Journal of Advertising Research 28 (1), S. 11–31.

9. Smither, R. D.; Houston, J. M. (1992): The Nature of Competitiveness: The Development and Validation of the Competitiveness Index. In: Educational and Psychological Measurement 52 (2), S. 407–418.

10. Sproll, S.; Peissner, M.; Sturm, C. (2010): From product concept to user experience. Exploring UX potentials at early product stages. In: Proc. of the 6th NordiCHI: Extending Boundaries. Reykjavik: ACM, S. 473–482.

11. Amelang, M.; Schmidt-Atzert, L. (2006): Psychologische Diagnostik und Intervention. Heidelberg: Springer Medizin Verlag.

12. Bargh, J. A.; Ferguson, M. J. (2000): Beyond Behaviorism: On the Automaticity of Higher Mental Processes. In: Psychol Bull 126 (6), S. 925–945.

13. Spath, D.; Peissner, M.; Sproll, S. (2010): Methods from neuroscience for measuring user experience in work environments. In: W. Karwowski (Ed.): Conf. Proceedings of the AHFE International. 17-20 July 2010. Miami, USA.