# User Experience of Driver State Visualizations: A Look at Demographics and Personalities

Michael Braun, Ronee Chadowitz, Florian Alt

► To cite this version:

Michael Braun, Ronee Chadowitz, Florian Alt. User Experience of Driver State Visualizations: A Look at Demographics and Personalities. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.158-176, 10.1007/978-3-030-29390-1_9. hal-02877686

HAL Id: hal-02877686

https://inria.hal.science/hal-02877686

Submitted on 22 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# User Experience of Driver State Visualizations:
a Look at Demographics and Personalities

Michael Braun<fnref n="1" />,<fnref n="2" />, Ronee Chadowitz<fnref n="1" />, and Florian Alt<fnref n="2" />,<fnref n="3" />

michael.bf.braun@bmw.de, ronee.chadowitz@bmw.de, florian.alt@unibw.de

**Abstract.** Driver state detection is an emerging topic for automotive user interfaces. Motivated by the trend of self-tracking, one crucial question within this field is how or whether detected states should be displayed. In this work we investigate the impact of demographics and personality traits on the user experience of driver state visualizations. 328 participants experienced three concepts visualizing their current state in a publicly installed driving simulator. Driver age, experience, and personality traits were shown to have impact on visualization preferences. While a continuous display was generally preferred, older respondents and drivers with little experience favored a system with less visual elements. Extroverted participants were more open towards interventions. Our findings lead us to believe that, while users are generally open to driver state detection, its visualization should be adapted to age, driving experience, and personality. This work is meant to support professionals and researchers designing affective in-car information systems.

**Keywords:** Affective Computing · Emotion Detection · Demographics · Personality · Driver State Visualization · Automotive User Interfaces

## 1 Introduction

Human-computer interaction has been experiencing a trend of personal data analytics in recent years: fitness trackers, smart watches, and wearables allow consumers to monitor themselves during sports, to track sleep patterns, to watch their calories, and much more. This phenomenon is widely known as the quantified self movement [11]. The technology behind this trend, namely pattern recognition through machine learning, can also be applied to other data sources. In automotive research, this has been happening for some time in the form of fatigue detection from steering and video data [12, 34] or the deduction of cognitive load from pupil diameters [37]. Driver states on an emotional level have also been extracted from physiological data, speech recordings, and facial images [23, 38]. Detecting driver states in real-time might be one of the big upcoming topics in future cars, as it enables personalized and affective interfaces [17]. At the same time, it is yet unclear whether this state should be communicated to the

<footnote n="1">¹ BMW Group Research, New Technologies, Innovations, Munich, Germany</footnote>

<footnote n="2">² LMU Munich, Munich, Germany</footnote>

<footnote n="3">³ Bundeswehr University, Munich, Germany</footnote>
driver, and if personalization can help promoting acceptance of such new fea-
tures. In our vision, user-aware automotive UIs can inform the driver if they
show signs of hazardous affective states and ultimately increase road safety by
raising awareness for the impacts of the driver state on driving performance.

We designed three approaches for the graphical representation of driver states based on successful examples from previous work in the automotive and other domains. They were evaluated in a public driving simulator experiment (N=328). We report insights on the participants’ user experience and emotional facial expressions, analyzed by age groups, personality traits, and driving experience.

According to our core findings, we can see that older and inexperienced
drivers are in favor of reduced graphical representations as they fear distrac-
tion. Users with high scores in neuroticism like a continuous display of their
driver state, while drivers with high scores in extraversion rather prefer noti-
fications in case their state could threaten safe driving. We also show that a
constant display of the driver state in a GUI works more efficiently in reducing
negative states than visual notifications. This work is meant as a foundation for
practitioners and researchers working on user state displays in the automotive
context and other multi-task environments.

2 Background and Related Work

The idea of visualizing information on user states has been long established
by computer gaming and in recent years also found its way into everyday life
through step counters or fitness trackers. We introduce this concept into an
automotive environment. In this process we have to consider the challenges and
opportunities automotive user interfaces bring to the table.

## 2.1 Affective Automotive User Interfaces

Cars are the perfect environment to detect the user’s state, as everything happens
in a confined, easily observable space. In-vehicle applications can also consult
information collected from their environment sensing systems to understand a
situation’s larger context. An example of this was shown in an early approach for
an integrated driver safety and wellness system presented by Coughlin et al. [13].
They propose a sequence framework built on the three steps: “detect, display,
alert”. Especially while driving, the user should not be distracted or strained
with visual load, which can be achieved through a mix of continuous display and
proactive interventions [22, 27].

A detection of the driver state, for example on an emotional level, can be
realized using image-video processing techniques and speech analysis [26]. This
approach can also be combined with physiological data [21]. Thirunavukkarasu
et al. have shown a prototype for an in-car HMI which detects emotions and
synchronizes the data with information from an on-board diagnosis (OBD) in-
terface in order to trigger interventions [48]. We take up from their work with
an approach of classification with camera-based detection of facial action coding units [16, 30] as previously demonstrated by, for example, Abdic et al. [1].

Emotional states can be related to critical behavior behind the wheel: not only anger, but also sadness has been connected to poorer driving performance [6, 24]. Several approaches for automotive UIs have worked with this knowledge, enhancing driving safety by adapting the UI to the driver's emotional state [34] or increasing interactivity by mirroring the driver's emotions in natural language interfaces [35]. Apart from these reactive approaches, prophylaxis of negative states could also be interesting to investigate. We can learn from experiments in other domains, for example, Tsujita and Rekimoto's smart home appliance which achieved positive influences on emotions by making users smile [49], or we consider gamification approaches in automotive environments, which try to bring joy and driver safety together by focusing the driver's attention to the driving task in a playful manner [43, 46]. One current challenge in automotive research is bringing user experience and safety together, as measures to make the driver more aware of their current state might raise reactance when users feel patronized by the system [9].

## 2.2 Visualizing the User State

Visual interfaces are the most regulated information channels in the automotive domain as they can demand visual resources and thus distract the driver from events on the road [39]. To avoid dangerous interactions in situations with increased workload, in-car information systems should make use of mostly static visualizations and ideally include the driving context into the timing of notifications [47]. In current in-car systems, a limited number of notifications are used to communicate the car's states (low fuel level, speed limits, etc.), which we need to adapt to when designing interventions based on the driver state.

Related work on driver state visualization is scarce, which is why we also looked at concepts from other domains. We especially searched for ways to make drivers aware of their current state and, if possible, induce positive change. This is based on work by Choe et al., who describe the promotion of self-reflection and the support of self-experimentation as the primary goals for the design of supporting systems [11]. Fan et al. provide guidelines for quantified-self visualizations, which suggest the use of graphs or abstract geometry, preferably placed within an interaction space of daily usage (like the car) [18]. This approach was used for our concept *Quantified Self*. A second concept (*Gamification*) was inspired by the idea of gamification, meaning the introduction of game elements to motivate users towards a specific behavior, as demonstrated by Steinberger et al. with a gamified speedometer to encourage safer driving [45]. Our third concept *Notification* is based on notification systems, which are used to nudge users into a change of behavior [14] or improve wellbeing through text interventions [25].
## 2.3 Personalization of User Interfaces

Automotive UIs are currently developed with little personalization features, although recent work suggests that an adaptation to user-specific characteristics, can increase personal reference and with it improve user experience and trust [4, 5, 41]. One widespread framework to classify such personality features is the big five inventory, also called OCEAN model due to the initials of its categories openness, conscientiousness, extraversion, agreeableness, and neuroticism [20, 31]. Chen and Donmez analyzed personality and driving data from 525 drivers and found that young drivers are rather influenced by normative behavior, while drivers over the age of 30 show interactions with their associated big five classifications [10]. They see higher risk-taking attitudes in drivers with high extraversion and low conscientiousness ratings, and increased fearfulness in drivers with high neuroticism. Arthur and Graziano looked at traffic accidents and found that high conscientiousness predicted less crashes [2]. Benfield et al. found aggressive driving to correlate with low scores in openness, agreeableness, and conscientiousness [3]. If we consider this framework for the evaluation of driver state visualizations, we might experience preferences for different systems by different personalities, which may also prove to positively modulate driving behavior.

# 3 Focus Group: Attitudes towards Driver State Detection

Despite the possibilities we see in driver state detection, we experienced mixed feedback and skepticism towards such concepts when talking to potential users in preparation for this study. We had the impression that users were afraid of being spied on by their cars, and similar fears have been identified in a survey by Schmidt and Brauner [40]. To understand where these notions come from, we conducted two focus groups with experienced drivers (> 10.000 km per year). The groups consisted of 24 participants from the US and 21 from Germany, as these are the two main vehicle markets we had access to. 25 of the participants were women, 20 men. All described themselves as generally open towards digital innovations. All participants used a smartphone daily. 8 owned a smartwatch, 14 regularly used a fitness tracker of whom 12 belong to the US sample.

The goal of the focus groups was to get an impression of how users receive the idea of user state detection in the car. Generally speaking, one third of the participants stated that they would agree to share personal data to use digital services, another third would agree only under certain conditions. US participants were more open towards data usage than Germans.

When confronted with the idea of an in-vehicle information system which can detect and act upon the driver's or passengers' emotions through sensors, both groups expressed curiosity. However, German users often described it as an "invasion of privacy" while US participants stated things like "I love this technology" and it could be a "good and handy offer". General privacy concerns and the fear of being patronized were stated as negative implications. On the other hand, an improvement of driver safety and enhanced smart suggestions were perceived positively.
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Our prototype of an affective interface showed driver state data in the instrument cluster. Focus group participants (US and Germany) expressed general curiosity and potential for improving driver safety, but also concerns regarding paternalism.</figcaption></figure>

Participants were then shown a prototype of an ordinary automotive UI with a speedometer and navigation, which we enhanced with visualizations of physiological features and emotion detection (Figure 1). This design is well known from fitness tracking apps and was proposed as a potential UI to indicate driver state. The visual prototype received more positive feedback than the verbalized ideas, as it was perceived as simple and intuitive to interpret.

## 3.1 Focus Group Insights

The feedback from this focus group was condensed into four key learnings using thematic analysis [8]. We incorporated these take-aways into the proposed concepts and used them to guide our main study.

**Show, Don't Tell** When asked to evaluate an idea, the visual concept received more positive feedback than a textual description. Since a participant's understanding of a concept profits from seeing a concrete application, we strive to keep theoretical input at a minimum.

**Respect Cultural Diversity** Participants from different cultures showed diverging concerns. Germans are more aware of data privacy, Americans were immediately keen to use the system. Different demographics might bring new insights and participants' needs may adapt (cf. GDPR in the EU/CSL in China).

**Beware of Patronism** User concerns about a feeling of being patronized should be taken seriously. 35 users stated to only want information or suggestions. Only 5 would accept a system trying to influence the driver subconsciously.

**Emphasize Safety Features** Participants often mentioned the advantages of driver state detection in case of a medical emergency and on long, tiring rides. Users are aware of the dangers on the road and might be more open to new technologies when they are seen in connection to safety improvements.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Concept *Notification*: a minimalistic background animation combined with text notifications in case of negative user valence.</figcaption></figure>

# 4 System Design

Based on the focus group, we designed three interface prototypes to be tested with a larger sample of participants. As a main requirement, each concept should be easy to use from the start, without further explanations. This way we make sure to gather feedback on the idea itself and not the technology behind it.

Participants should experience the prototype in a driving context, thus we have to embed the driver state visualizations into a realistic dashboard design and adhere to standards for in-car UIs, meaning the display should only require minimal active attention while driving. The concepts should also avoid appearing manipulative or patronizing. The ideas for each concept come from related work, which we have illuminated closer in Section 2.2.

## 4.1 Concept Notification

Concept *Notification* implements interventions to positively influence the driver without constant driver state display. In case the lower threshold for detected driver valence is reached, a text message meant to cheer up the driver is displayed (see Figure 2). This message stays on the screen for at least 3 seconds to prevent fidgetiness and only fades out as the detected valence levels rise. The main idea of concept *Notification* is to reduce visual load and to only interact with the driver in form of interventions when they are in a negative state.

## 4.2 Concept Gamification

Concept *Gamification* consists of four scales, representing car status, fuel consumption, driver health, and driver fun levels (Figure 3). The inspiration was taken from computer games, where players have health levels they can directly influence through power-ups. In our prototype, users could keep consumption low by driving responsibly or increase their personal fun factor by smiling. Fun levels decreased over time if no positive expressions were detected. The general idea stems from approaches of increasing driver engagement through gamification [45]
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Concept *Gamification* displays current stats of car and driver in an interactive way. The general idea is to involve the driver and make their ride more fun.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Concept *Quantified Self* visualizes user vitality and emotion data in circular diagrams. Users can see at one glance whether their condition is classified as positive (green), neutral (blue), or negative (red).</figcaption></figure>

and the notion that the act of smiling itself makes people feel happier [49]. In our prototype the values for driver health was simulated with random positive values as we did not collect health data during the experiment.

## 4.3 Concept Quantified Self

Concept *Quantified Self* is borrowed from the visualization style known from fitness tracking apps. This display variant follows the idea of a quantified self, allowing the user to experiment on themselves through behavioral changes with an intended outcome of self-improvement [11, 42]. Two circular diagrams display the vital and emotional states of the drivers, which are color coded to allow quick information extraction (Figure 4). When a dangerous driver state is detected, the outline turns red, which we refer to as intervention in the experiment. Here, again, we simulated driver health values, as we did not collect health data.
## 4.4 Prototype Implementation

The system was built using the *Affdex SDK* [32] which analyzes facial images to estimate an emotional user state. We process a live video feed of the driver's face and then classify the driver state conditional to the resulting metrics: measurements exceeding anger values of 60 (out of 100) or falling below a valence of -50 (from -100 to 100) triggered interventions in concepts *Notification* and *Quantified Self*. Users were classified as happy when joy was above 60 or when a smile was detected. Additionally, in concept *Quantified Self* the valence values were transferred one-to-one to the fill percentage of the ring. In each session only one, single, visualization concept was experienced by each user.

# 5 Deployment Study: Driver State Visualization

The described concepts were tested in a publicly accessible driving simulator deployed close to a busy shopping street in Nuremberg, Germany. We chose Germany over the US as German participants in our focus group were more critical towards the concept of driver state detection. Nuremberg was selected as the city has no direct connection to the automotive industry, so we expected a more generalizable sample than in other cities. In order to minimize bias effects, all company affiliations and characteristic design features were also concealed.

## 5.1 Methodology

The unique setting in a public space allowed us to open participation for all passers-by, providing us with a much more diverse participant sample than a standard lab test would allow for. The test ran for three months, Monday through Saturday, from 10 am to 6 pm, during which close to 800 user sessions were logged. We modeled the public simulator setup according to best practices from related work, e.g. sturdy hardware, easy setup for the user, and short, concise explanations for every single step within the process [7].

Each participant experienced one visualization approach assigned randomly, resulting in a between-subject design. As additional independent variables we queried user demographics (gender, age, driving experience) and personality traits (BFI-20 Big Five Inventory [15]). The dependent variables were attractiveness, design quality, and usability, assessed using the user experience questionnaire (UEQ [29]), and emotion recognition data based on facial features from a frontal video feed.

## 5.2 Demographics

During a period of 12 weeks, 798 sessions were logged, of which 328 contained complete and valid measurements and no exclusion criteria (e.g. too young to drive, intoxication, unserious behavior). For an overview of age and gender distributions, see Figure 5. When asked about driving experience, 6.4% of included
<figure><img src="image_6.png" /><figcaption>**Fig. 5.** A total of 109 women and 219 men with a mean age of 36 years finished the experiment with evaluable data.</figcaption></figure>

participants stated they never drive themselves, 12.5% drive once a month, 13.1% once a week, 26.1% every other day and 41.8% identified themselves as daily drivers. Of the final subset, 33.8% of participants experienced the concepts *No-tification* and *Gamification* and 32.4% experienced the concept *Quantified Self*.

## 5.3 Apparatus

The study was conducted in a white label seating box situated at a research booth at Fraunhofer JOSEPH'S in Nuremberg [19]. The concepts were shown on a curved wide screen display above the steering wheel. (Figure 6) A camera on top of the screen provided a live video stream for facial expression analysis. The simulation was controlled by the driver using a steering wheel and pedals. Participants were provided step by step instructions on a tablet placed in the center console. We provided general information on our research and explained the technology behind the prototype on a nearby information screen. Staff assured smooth operations by answering questions. Participation was possible without personal on-boarding.

## 5.4 Procedure

Participants were welcomed by a starting screen and introduced to the setting and tasks. They were presented questions on their demographics, driving experience and exclusion criteria (for example, alcohol consumption, legal driving age). The main task consisted of following a white car on a highway with moderate traffic, the secondary task was assessing the UI on the dashboard screen.

After completing the ride, which took about 5 minutes, users were presented two questionnaires; a UEQ questionnaire to rate the user experience of the driver
<figure><img src="image_7.png" /><figcaption>**Fig. 6.** Publicly deployed medium fidelity driving simulation showing a participant experiencing the concept *Gamification*, displayed above the steering wheel.</figcaption></figure>

state visualization [29] and the BFI-20 short version of the Big Five Inventory to classify the participant's personality [15]. We also logged the output of the emotion detection system to see whether interventions had an effect on participants. After the ride, participants were asked to rate the idea of driver state visualizations as negative, neutral, or positive and explain their decisions.

## 5.5 Limitations

The driving simulator experiment was conducted without further restrictions on participation and a rather short experiment duration. This is attributed to the public study design, which in return allowed for a larger sample than traditional driving studies. Furthermore, although we take into account cultural particularities, our focus group does not reflect input from China as the third big market. This is simply due to a lack of access. We can, however, expect an open attitude from Chinese users, as they are generally appreciative of new technology [28].

# 6 Results

We collected UEQ ratings on the tested visualization concepts, emotion recognition data from the Affdex SDK and an estimation of participants' personality characteristics provided by the BFI-20 questionnaire. The personality characteristics and demographic data were combined and used to create participant subsets in order to understand if they experienced and evaluated concepts differently. Results with *p* < 0.05 are reported as statistically significant.
<table><caption>**Table 1.** When we look at the entirety of participants, concept *Notification* is perceived significantly less attractive than the other tested concepts and concept *Gamification* received the noticeably worst ratings in *Design Quality*. Only concept *Quantified Self* scored well in all dimensions of the UEQ.</caption><thead><tr><th rowspan="2"></th><th colspan="2">Notification</th><th colspan="2">Gamification</th><th colspan="2">Quantified Self</th></tr><tr><th>Mean</th><th>SD</th><th>Mean</th><th>SD</th><th>Mean</th><th>SD</th></tr></thead><tbody><tr><td>Attractiveness</td><td><strong>2.66</strong></td><td>1.13</td><td>3.05</td><td>1.06</td><td>3.04</td><td>1.15</td></tr><tr><td>Design Quality</td><td>3.18</td><td>1.00</td><td><strong>2.93</strong></td><td>1.04</td><td>3.23</td><td>0.97</td></tr><tr><td>Usability</td><td>3.23</td><td>0.84</td><td>3.27</td><td>0.78</td><td>3.35</td><td>0.83</td></tr></tbody></table>

## 6.1 Subjective Feedback

Participants gave us an estimation on how they perceived the concept of driver state detection in the car. 12% said they would not like to use a system like the one exhibited. They voiced concerns regarding paternalism or data protection issues. 28% reacted indifferent, one of them said they liked the idea for special groups like novice drivers or to test for intoxication but they would not accept it for themselves. A majority of 60% rated the idea as positive. Feedback included the wish for the detection of fatigue or health-related conditions and an optional annexation of control through the car if the driver was found incapable. Other ideas were music selection according to the driver's mood or windshield filters when increased pupil sensitivity is detected.

Looking back at our initial focus groups, we expected the feedback to be more narrow-minded. The hands-on presentation might have convinced participants who would have disliked theoretical concepts. We also see that younger users within this group were more open to the driver state detection than older users. There were no differences between genders.

## 6.2 Concept Ratings

We first analyzed the UEQ data without differentiation between user subsets with an analysis of variance and pairwise comparisons for each concept rating (see Table 1). The data distribution fulfils requirements for parametric tests. Concept *Quantified Self* was generally liked by the entirety of participants. Concept *Gamification* and *Quantified Self* were rated significantly more attractive than concept *Notification* (*p* < 0.01), while concept *Quantified Self* also had significantly higher ratings in novelty than *Gamification* (*p* < 0.05). Feedback on the design quality shows a tendency towards a slight disregard for concept *Gamification*.

This general look at the data suggests that the quantified-self approach of concepts *Quantified Self* could be a good first step towards displaying driver states in the car. If we however look at sub-groups within the user sample, we can generate further insights on the opportunities of personalization.
<figure><img src="image_8.png" /><figcaption>**Fig. 7.** UEQ ratings visualized by corresponding user personality traits.</figcaption></figure>

**Demographic Influences** Our dataset contains demographic information on the subjects, which we used to create subsets based on gender, age, and driving experience. This deep dive into demographics was only possible due to our public setup, as it allowed us to include a high number of participants with diverse backgrounds. We discovered no gender differences but age played a role in the UEQ ratings: young drivers (aged 17–24) preferred the gamified concept, while older drivers (age groups 55–64 and 65+) favored concept *Notification*. This can be explained with preferences of younger drivers for gamification due to under-stimulation [44] and an increased sensitivity for visual load in older drivers [36].

Driving experience also had an effect on ratings as inactive drivers preferred the visually less demanding concept *Notification*, while experienced drivers have balanced ratings for all three concepts.

**Personality Preferences** Each user took a personality test which led to a personality placement in the five dimensions extraversion, conscientiousness, openness, agreeableness, and neuroticism. We did an analysis of the interplay between UEQ ratings and personality traits for each concept and found high variances in answers on attractiveness and dependability from participants with high values in neuroticism and openness. High openness also led to higher variances in perspicuity. Other groups were generally comparable in ratings. It is noticeable that extremes in neuroticism, extraversion, and openness show greater variances
than other categories which makes us think that these are the most important
traits when it comes to the personalization of driver state visualizations. We can
also see that users with high agreeableness tend to rate all concepts rather well
and might therefore be a less ideal demographic to chose from when conducting
evaluations.

It is interesting to see that concept *Notification*, which gives proactive feed-
back and behaves thus rather extroverted, was rated higher by extroverted users,
while concept *Quantified Self* achieved better ratings from participants with high
neuroticism. This makes sense since an emotionally reactive person may also be
more receptive to the constant display of their state (see Figure 7).

<figure><img src="image_9.png" /><figcaption>**Fig. 8.** An exemplary view of emotion recognition data for detected valence values of one user: moving mean smoothing is used to visualize trends, linear regression shows the overall slope to categorize positive or negative development. Intervention frames display how this user was twice alerted due to low valence and e.g. in the second instance instantly started smiling.</figcaption></figure>

6.3 Emotion Detection

We analyzed the data provided by the Affdex SDK for emotion detection with a focus on time intervals after interventions performed by the system. Concepts *Quantified Self* and *Notification* performed different actions when the detected valence of the user fell below a threshold of -50 (from -100 to 100). In concept *Notification*, a written message appeared which was designed to bring the driver into a more positive state and only disappeared when an improvement was measured. Concept *Quantified Self* constantly displayed the valence values and switched to an alerting red color scheme when the value was less than the threshold. Figure 8 shows exemplary valence data of a user performing a ride with concept *Notification*: they experienced two interventions of 6 and 7 seconds duration and both led to a brief improvement. In the second intervention they actively reacted with a smile as we can see from the high valence values.
<figure><img src="image_10.png" /><figcaption>**Fig. 9.** Interventions during the concepts *Quantified Self* (495 instances) and *Notification* (442 instances). Concept *Quantified Self* worked more effectively in inducing positive valence through interventions.</figcaption></figure>

We analyzed the development of valence values after each of the total of 937 interventions in our dataset and see significantly better performance in concept *Quantified Self* than in concept *Notification* (see Figure 9). Interventions in concept *Notification* also took significantly longer ($t_{(611.43)} = -16.527, p < 0.001$), supposedly because the written text was harder to process than the abstract representation of concept *Quantified Self*.

# 7 Discussion

In this paper we present three different approaches to visualizing driver states. Participants in our study were rather open towards the idea of user state detection in the car (only 12% opposed) with seniors being more sceptical than young drivers. Some users also expressed issues concerning privacy or data protection.

Concept *Quantified Self*, which incorporates quantified-self visualizations, is generally more preferred than *Notification* and *Gamification*. The gamified approach *Gamification*, however, scores well with the subset of young drivers (cf.
related literature [45]). Older and unpracticed drivers prefer the more subtle concept *Notification*. This shows that users are actually aware of the dangers visual distraction poses on driving. We think a system with real-time cognitive load measurement could recognize available resources and select the appropriate means to visualize information, e.g. an interaction-free visualization in demanding situations and a more involving approach when the driver is bored [44].

Participants with extreme ratings on the big five scales of openness, extraversion, and neuroticism showed the most diverging UEQ ratings. Extroverted users preferred proactive behavior, which resembles an extroverted agent. Neurotic users liked a constant display of their state, possibly because they see this as a way to get feedback on their condition, motivated through higher emotional reactivity. Users with high agreeableness scores liked all approaches, which was to be expected in a way, and suggests that we do not need to focus our future work on this character trait. Conscientiousness scores also did not have any influence on UX ratings. We however know from related work that this dimension is connected with driving performance, hence still important.

According to the analyzed valence data from image-based emotion detection we found that concept *Quantified Self* works faster and is more effective in decreasing negative emotions than concept *Notification* (concept *Gamification* did not utilize interventions). We see this as an important finding of our study, as current fatigue warning systems work with notifications. We see potential in defining a dedicated area within the in-vehicle UI to display the driver state to keep the driver always in the loop.

# 8 Conclusion & Future Work

We investigated user preferences on driver state visualization in the car. We conducted a focus group and evaluated three proposed GUIs in a public driving simulator study. The findings lead us to believe there is no one-fits-all solution to driver state visualizations. We propose that interfaces should adapt to age, driving experience, and personality. When testing with users, the personality dimensions openness, extraversion, and neuroticism should be at the focus of attention. We are also aware that personality is a highly complex field to explore.

The main contribution of our focus group and study are a high acceptance of driver state detection within our broad user sample, as well as the positive influences a constant display can have on the driver's affective state. We hope this affirms other researchers to keep working on user-aware interfaces.

This field offers plenty of directions for future work. One interesting question that emerged while we were working on this project, is whether users will accept such a system in long-term usage or if we will see an abandonment after habitual usage, as was reported for fitness trackers [12]. Furthermore, cultural differences for personalization might play an important role and should be investigated. Therefore, we would like to motivate future research on the long-term usage of user-aware in-vehicle UIs and try to transfer these research questions into culturally different markets.
References

1. Abdić, I., Fridman, L., McDuff, D., Marchi, E., Reimer, B., Schuller, B.: Driver frustration detection from audio and video in the wild. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence. pp. 1354–1360. IJCAI’16, AAAI Press, New York, NY, USA (2016). https://doi.org/10.1007/978-3-319-46073-4

2. Arthur, W., Graziano, W.G.: The five-factor model, conscientiousness, and driving accident involvement. Journal of Personality 64(3), 593–618 (1996). https://doi.org/10.1111/j.1467-6494.1996.tb00523.x

3. Benfield, J.A., Szlemko, W.J., Bell, P.A.: Driver personality and anthropomorphic attributions of vehicle personality relate to reported aggressive driving tendencies. Personality and Individual Differences 42(2), 247 – 258 (2007). https://doi.org/10.1016/j.paid.2006.06.016

4. Blom, J.: Personalization: A taxonomy. In: CHI ’00 Extended Abstracts on Human Factors in Computing Systems. pp. 313–314. CHI EA ’00, ACM, New York, NY, USA (2000). https://doi.org/10.1145/633292.633483

5. Braun, M., Mainz, A., Chadowitz, R., Pfleging, B., Alt, F.: At your service: Designing voice assistant personalities to improve automotive user interfaces. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI ’19, ACM, New York, NY, USA (2019). https://doi.org/10.1145/3290605.3300270

6. Braun, M., Pfleging, B., Alt, F.: A survey to understand emotional situations on the road and what they mean for affective automotive uis. Multimodal Technologies and Interaction 2(4) (2018). https://doi.org/10.3390/mti2040075

7. Braun, M., Roider, F., Alt, F., Gross, T.: Automotive research in the public space: Towards deployment-based prototypes for real users. In: Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. pp. 181–185. AutomotiveUI ’18, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3239092.3265964

8. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qualitative Research in Psychology 3(2), 77–101 (2006). https://doi.org/10.1191/1478088706qp0630a

9. Brehm, S.S., Brehm, J.W.: Psychological reactance: A theory of freedom and control. Academic Press (2013)

10. Chen, H.Y.W., Donmez, B.: What drives technology-based distractions? a structural equation model on social-psychological factors of technology-based driver distraction engagement. Accident Analysis & Prevention 91, 166 – 174 (2016). https://doi.org/10.1016/j.aap.2015.08.015

11. Choe, E.K., Lee, N.B., Lee, B., Pratt, W., Kientz, J.A.: Understanding quantified-selfers’ practices in collecting and exploring personal data. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1143–1152. CHI ’14, ACM, New York, NY, USA (2014). https://doi.org/10.1145/2556288.2557372

12. Clawson, J., Pater, J.A., Miller, A.D., Mynatt, E.D., Mamykina, L.: No longer wearing: Investigating the abandonment of personal health-tracking technologies on craigslist. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. pp. 647–658. UbiComp ’15, ACM, New York, NY, USA (2015). https://doi.org/10.1145/2750858.2807554

13. Coughlin, J.F., Reimer, B., Mehler, B.: Monitoring, managing, and motivating driver safety and well-being. IEEE Pervasive Computing 10(3), 14–21 (7 2011). https://doi.org/10.1109/MPRV.2011.54
14. Czerwinski, M., Gilad-Bachrach, R., Iqbal, S., Mark, G.: Challenges for designing notifications for affective computing systems. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. pp. 1554–1559. UbiComp ’16, ACM, New York, NY, USA (2016). https://doi.org/10.1145/2968219.2968548

15. Danner, D., Rammstedt, B., Bluemke, M., Treiber, L., Berres, S., Soto, C.J., John, O.: Die deutsche version des big five inventory 2 (bfi-2). Tech. rep., Zusammenstellung sozialwissenschaftlicher Items und Skalen (2016). https://doi.org/10.6102/zis247

16. Ekman, P., Rosenberg, E.L.: What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA (1997)

17. Eyben, F., Wöllmer, M., Poitschke, T., Schuller, B., Blaschke, C., Färber, B., Nguyen-Thien, N.: Emotion on the road: Necessity, acceptance, and feasibility of affective computing in the car. Adv. in Hum.-Comp. Int. **2010**, 5:1–5:17 (Jan 2010). https://doi.org/10.1155/2010/263593

18. Fan, C., Forlizzi, J., Dey, A.K.: A spark of activity: Exploring informative art as visualization for physical activity. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing. pp. 81–84. UbiComp ’12, ACM, New York, NY, USA (2012). https://doi.org/10.1145/2370216.2370229

19. Fraunhofer-Arbeitsgruppe für Supply Chain Services SCS des Fraunhofer-Instituts für Integrierte Schaltungen IIS: Josephs service manufaktur – megatrends. http://www.josephs-service-manufaktur.de/besucher/vorherige-themenwelten/themenwelt-megatrends/ (2018), [Online; accessed 16-April-2018]

20. Goldberg, L.R.: The structure of phenotypic personality traits. American psychologist **48**(1), 26 (1993)

21. Hassib, M., Pfeiffer, M., Schneegass, S., Rohs, M., Alt, F.: Emotion actuator: Embodied emotional feedback through electroencephalography and electrical muscle stimulation. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 6133–6146. CHI ’17, ACM, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025953

22. Häuslschmid, R., Klaus, C., Butz, A.: Presenting Information on the Driver’s Demand on a Head-Up Display, pp. 245–262. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-67684-5_15

23. Hoch, S., Althoff, F., McGlaun, G., Rigoll, G.: Bimodal fusion of emotional data in an automotive environment. In: Proceedings. (ICASSP ’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005. vol. 2, pp. ii/1085–ii/1088 Vol. 2. IEEE, Philadelphia, PA, USA (March 2005). https://doi.org/10.1109/ICASSP.2005.1415597

24. Jeon, M.: Don’t cry while you’re driving: Sad driving is as bad as angry driving. International Journal of Human-Computer Interaction **32**(10), 777–790 (2016). https://doi.org/10.1080/10447318.2016.1198524

25. Jeong, S., Breazeal, C.L.: Improving smartphone users’ affect and wellbeing with personalized positive psychology interventions. In: Proceedings of the Fourth International Conference on Human Agent Interaction. pp. 131–137. HAI ’16, ACM, New York, NY, USA (2016). https://doi.org/10.1145/2974804.2974831

26. Katsis, C.D., Rigas, G., Goletsis, Y., Fotiadis, D.I.: Emotion Recognition in Car Industry, chap. 20, pp. 515–544. Wiley-Blackwell (2015). https://doi.org/10.1002/9781118910566.ch20
27. Kushlev, K., Proulx, J., Dunn, E.W.: "silence your phones": Smartphone notifications increase inattention and hyperactivity symptoms. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 1011–1020. CHI ’16, ACM, New York, NY, USA (2016). https://doi.org/10.1145/2858036.2858359

28. Lachner, F., von Saucken, C., ‘Floyd’ Mueller, F., Lindemann, U.: Cross-cultural user experience design helping product designers to consider cultural differences. In: Rau, P. (ed.) Cross-Cultural Design Methods, Practice and Impact. pp. 58–70. Springer International Publishing, Cham (2015)

29. Laugwitz, B., Schrepp, M., Held, T.: Konstruktion eines fragebogens zur messung der user experience von softwareprodukten. In: Heinecke, A.M., Paul, H. (eds.) Mensch und Computer 2006: Mensch und Computer im Strukturwandel. pp. 125–134. Oldenbourg Verlag, München (2006)

30. Lien, J.J., Kanade, T., Cohn, J.F., Li, C.C.: Automated facial expression recognition based on facs action units. In: Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition. pp. 390–395 (Apr 1998). https://doi.org/10.1109/AFGR.1998.670980

31. McCrae, R.R., Costa Jr, P.T.: A five-factor theory of personality. In: Handbook of personality: Theory and research, vol. 3, pp. 159–181. Guilford Press New York, New York, NY (2008)

32. McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., Kaliouby, R.e.: Affdex sdk: A cross-platform real-time multi-face expression recognition toolkit. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. pp. 3723–3726. CHI EA ’16, ACM, New York, NY, USA (2016). https://doi.org/10.1145/2851581.2890247

33. Melnicuk, V., Birrell, S., Crundall, E., Jennings, P.: Towards hybrid driver state monitoring: Review, future perspectives and the role of consumer electronics. In: 2016 IEEE Intelligent Vehicles Symposium (IV). pp. 1392–1397. IEEE, Gothenburg, Sweden (June 2016). https://doi.org/10.1109/IVS.2016.7535572

34. Nasoz, F., Lisetti, C.L., Vasilakos, A.V.: Affectively intelligent and adaptive car interfaces. Information Sciences 180(20), 3817 – 3836 (2010). https://doi.org/10.1016/j.ins.2010.06.034

35. Nass, C., Jonsson, I.M., Harris, H., Reaves, B., Endo, J., Brave, S., Takayama, L.: Improving automotive safety by pairing driver emotion and car voice emotion. In: CHI ’05 Extended Abstracts on Human Factors in Computing Systems. pp. 1973–1976. CHI EA ’05, ACM, New York, NY, USA (2005). https://doi.org/10.1145/1056808.1057070

36. Owsley, C., Ball, K., Sloane, M., L. Roenker, D., Bruni, J.: Visual/cognitive correlates of vehicle crashes in older drivers. Psychology and Aging 6, 403–15 (10 1991). https://doi.org/10.1037/0882-7974.6.3.403

37. Pfleging, B., Fekety, D.K., Schmidt, A., Kun, A.L.: A model relating pupil diameter to mental workload and lighting conditions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’16, ACM, New York, NY, USA (May 2016). https://doi.org/10.1145/2858036.2858117

38. Riener, A., Ferscha, A., Aly, M.: Heart on the road: Hrv analysis for monitoring a driver's affective state. In: Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications. pp. 99–106. AutomotiveUI ’09, ACM, New York, NY, USA (2009). https://doi.org/10.1145/1620509.1620529

39. Riener, A., Jeon, M., Alvarez, I., Frison, A.K.: Driver in the loop: Best practices in automotive sensing and feedback mechanisms. In: Meixner, G., Müller, C. (eds.)
Automotive User Interfaces: Creating Interactive Experiences in the Car, pp. 295–323. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-49448-7_11

40. Schmidt, M., Brauner, P.: A survey on different means of personalized dialog output for an adaptive personal assistant. In: Adjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization. pp. 75–81. UMAP ’18, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3213586.3226198

41. Schneider, H., Schauer, K., Stachl, C., Butz, A.: Your Data, Your Vis: Personalizing Personal Data Visualizations, pp. 374–392. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-67687-6_25

42. Schraefel, M.C., Churchill, E.F.: Wellth creation: Using computer science to support proactive health. Computer 47(11), 70–72 (Nov 2014). https://doi.org/10.1109/MC.2014.339

43. Schroeter, R., Rakotonirainy, A., Foth, M.: The social car: New interactive vehicular applications derived from social media and urban informatics. In: Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. pp. 107–110. AutomotiveUI ’12, ACM, New York, NY, USA (2012). https://doi.org/10.1145/2390256.2390273

44. Steinberger, F., Moeller, A., Schroeter, R.: The antecedents, experience, and coping strategies of driver boredom in young adult males. Journal of Safety Research 59, 69 – 82 (2016). https://doi.org/10.1016/j.jsr.2016.10.007

45. Steinberger, F., Proppe, P., Schroeter, R., Alt, F.: Coastmaster: An ambient speedometer to gamify safe driving. In: Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. pp. 83–90. Automotive’UI 16, ACM, New York, NY, USA (2016). https://doi.org/10.1145/3003715.3005412

46. Steinberger, F., Schroeter, R., Lindner, V., Fitz-Walter, Z., Hall, J., Johnson, D.: Zombies on the road: A holistic design approach to balancing gamification and safe driving. In: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. pp. 320–327. AutomotiveUI ’15, ACM, New York, NY, USA (2015). https://doi.org/10.1145/2799250.2799260

47. Tchankue, P., Wesson, J., Vogts, D.: The impact of an adaptive user interface on reducing driver distraction. In: Proceedings of the 3rd International Conference on Automotive User Interfaces and Interactive Vehicular Applications. pp. 87–94. AutomotiveUI ’11, ACM, New York, NY, USA (2011). https://doi.org/10.1145/2381416.2381430

48. Thirunavukkarasu, G.S., Abdi, H., Mohajer, N.: A smart hmi for driving safety using emotion prediction of eeg signals. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). pp. 004148–004153 (Oct 2016). https://doi.org/10.1109/SMC.2016.7844882

49. Tsujita, H., Rekimoto, J.: Smiling makes us happier: Enhancing positive mood and communication with smile-encouraging digital appliances. In: Proceedings of the 13th International Conference on Ubiquitous Computing. pp. 1–10. UbiComp ’11, ACM, New York, NY, USA (2011). https://doi.org/10.1145/2030112.2030114