#635

In-Car HMI UX Evaluation& Benchmarking

UX
TABLE OF CONTENTS
Executive Summary
Features and Functionality

EVALUATION

In-car HMI UX
Evaluations

HMI Execution
Perceived Quality
ADAS

Renault Megane E-Tech

Infotainment
Navigation

The Megane’s OpenR link system is doubly surprising: firstly, a new system on a new EV platform that
demonstrates a relatively high level of stability, and secondly, a Renault system that manages to scoresignificantly higher than many industry leaders. With large screens, a positive user experience, a flat
information architecture, highly responsive interactions and a pleasing level of functionality, the overall
impression is of a premium system that for the most part succeeds in carrying over the slick experience ofa high-end tablet interface into the car.

Voice Recognition
Connected Features
Convenience

RELATED SBD REPORTS
619 – UX Benchmarking S eries
A precursor to the new In-Car HMI UX Evaluation& Benchmarking Series, SBD Automotive’s UX
Team evaluates the infotainment user experienceof over 40 vehicles.
806 – AD AS HM I E v aluations
A precursor to the new In-Car HMI UX Evaluation& Benchmarking Series, SBD Automotive’s
Autonomous Car Team evaluates the ADAS
performance and usability of over 20 vehicles.

COVERAGE

FREQUENCY

PUBLICATION FORMAT

PAGES

6
NA

CHINA

EUROPEANNUALLYQUARTERLYCARS PER YEARPD F

POWERPOINT

EXCEL

ONLINE

150+

#635/ In-Car HMI UX Evaluation & Benchmarking

Scoring> Features and functionality:
evaluating whether thesolutions provide featuresthat customers expect, needand solve problems (orprovide a wow factor).

This research is useful for

Do I have access?

> Reliability/stability: evaluating
the repeated usability andwhether the users can have asimilar (positive) experienceeach time.

> Usability:> Perceived quality: evaluating
evaluating whether thethe potential perception infeatures available are easy
quality of the HMI
to learn and use. This
components and how thisconsiders areas such as
contributes to the overall
ergonomics, legibility,
customer experience.
usability characteristics andhow the system implements
the various features.

PRODUCT PLANNERS

M A R K E T I NG

C- S UITE

USER E XPE R IENCEE NGI NE E R S

Portal access

Request a quote for
In-Car HMI UX Evaluation & Benchmarking Series

Renault Megane

July 2022

In-Car HMI UX Evaluation & Benchmarking
Renault Megane E-TechTable of Contents
Executive summary ≫4

Convenience domain ≫148

Introduction to this report ≫13

Support areas ≫152

Features and functionality ≫20

Contact us ≫159

Execution ≫26

Perceived quality ≫55

ADAS domain ≫63

Infotainment domain ≫87

Navigation domain ≫113

Voice recognition domain ≫130

Connected features domain ≫140

Introduction

Introduction

Aim of this report

One of the core goals of these studies is to provide a true indication ofwhat the final customer experience of each solution could be. To do thisevaluations are focused on providing scoring and analysis in the following
areas:

Brand perception

Consumer testing

Welcome to the 2022 HMI benchmarking report series. This report has
been created to provide a fair, unbiased and objective view of the latest
in-vehicle HMI solutions in the European, US and Japanese markets.
Evaluations are carried out by SBD usability experts with a deep
understanding of CASE domains such as the Connected Car and ADAS &autonomy domains.

▪ Features and functionality: evaluating whether the solutions provide
features that customers expect & need, and solve problems (orprovide a wow factor)▪ Reliability/stability: evaluating the repeated usability and whether theusers can have a similar (positive) experience each time

▪ Perceived quality: evaluating the potential perception in quality of the
HMI components and how this contributes to the overall customer
experience
SBD supports clients throughout the development of new HMI and
products from a relatively simple companion app to a more complex
multi-domain infotainment solution. The methodologies used in thesereports take into account many years of experience with consumer
testing and custom client projects to provide a fair and, as much as
possible, objective methodology.
All viewpoints and analysis within the report are aimed defining areas ofconcern through a data driven approach. This report aims to benchmark
6 score solutions whilst also being able to provide actionable
andrecommendations to design and development teams.

BRAND
APPEAL

Aesthetics and engagement“Do I have a positive emotional attachment to thesolution?”

EMOTIONAL
APPEAL

Repeated usability

RELIABILITY

Expert testing(the focus of this report)▪ Usability: evaluating whether the features available are easy to learn
and use. This considers areas such as ergonomics, legibility, usability
characteristics and how the system implements the various features“Do I have a positive emotional attachment to thebrand?”“Can I have a similar experience each time?”

Ergonomics, usability characteristics & execution“Are the features available easy to learn
and use?”USABILITY

Domain functionality &prioritisation“Does the system provide features I
want and need to use?” i.e. “Does it
solve problems for me?”FUNCTIONALITY

SBD’s view on the hierarchy of needs for CX benchmarking
6Introduction

Scope of report: focus on in-car HMI evaluations
The scope of evaluations in this report are constrained to the in-car HMI
experience, in both static and dynamic conditions. One notable element isdriver distraction which SBD covers at only a high level in this study as
carrying out a full driver distraction evaluation requires biometrics test
equipment to ensure the collected data is unbiased and objective.

Prepurchase

Purchase

Activation

In-car HMI

Driverdistraction

IoT

A full evaluation of the end-to-end customer experience is not within scope of
this report, but it is something which SBD has may years experience in fromboth a consumer and expert perspective. Other areas such as the companion
app, online portal and in-home smart devices are not in scope as they aredefined as “out of car” experiences.
Within the vehicle, any HMI element the user interacts with is evaluatedincluding steering wheel controls, touch screen displays, voice control, HUDs
and digital keys. The features and services on offer have been broadlygrouped into the following domains (or test areas):▪ ADAS domain▪ Infotainment domain▪ Navigation domain▪ Voice recognition domain▪ Connected services domain▪ 7Convenience domain
7

Introduction

2022 vehicle list
SBD has chosen six cars to evaluate in 2022, based on two selection categories. New/interesting UX focuses on systems with to never-seen-before features orfunctionality, or the implementation of a solution that has previously been a challenge or pain-point for end-users. New mass-market UX includes vehicles insegments that are sold in high numbers and are entering a new generation of UI for that vehicle. While we make best efforts to adhere to the chosen cars andschedule, the last year has seen release dates slipping significantly, so it may be necessary to make substitutions.

Cars tested

Lucid Air

Rivian R1T▪ Potential disruptor▪ New disruptor▪ Multiple displays▪ Appears to have aninnovative approach to
HMI▪ US market test

▪ US market test

Awaiting test

Renault Megane
E-Tech▪ LG’s new Android
Automotive IVI▪ French market test

Xpeng P5
▪ High level of ADAS
▪ Advanced voicerecognition▪ Configurable avatar▪ China market test

GMC HummerEV▪ GM’s new Android OS
system▪ Unreal Engine graphics▪ US market test

Lexus NX
▪ New Lexus system▪ Cloud-based navi
▪ New voice recognition▪ German market car,UK test8
8

Introduction

SBD experience through years of testing in-car solutions
Over the last nine years SBD has evaluated 96 solutions from a Connected Car or ADAS perspective for our public report series (many more for privateclient evaluations). This current report series is an evolution of both test methodologies to provide a holistic view of in-car HMI. Furthermore, custom
evaluations methodologies used across the globe for SBD clients have been included where applicable to enhance to overall approach.

99Introduction

One page methodology overview
One of SBD’s core goals of this report is to be as objective, fair
and as transparent as possible. To achieve this, various
methodologies are used throughout the testing to evaluate
different areas of the solution in various conditions.
These methodologies are a mix of different types of tests:▪ Objective tests: where the value provided is not influenced
by a tester’s viewpoint e.g. response time

Type of tests
Test area

Objective

First impressions

Subjective

Task based

P

Freeform

Scoringrange

P

Static tasks

P

P

P

P

Dynamic tasks

P

P

P

P

Random freeP

P

Navigation specific tests

P

P

P

Voice recognition

P

P

P

▪ Scoring range: ranges and definitions of how to score a testelement e.g. poor depth and accuracy score = the results
provided are not in line with what is reasonably expected by
the user

Performance & responseP▪ Static: tests are carried out when the vehicle is not moving

Final SBD UX score▪ Dynamic: tests are carried out when the vehicle is moving invarious road conditions and locations e.g.
motorways/highways, cities, villages, country roads etc.

▪ Subjective tests: the test score is based on the expert
testers’ viewpoints e.g. task ease of use▪ Task-based: evaluations carried out based on a predefinedtask list e.g. navigate to a pizza restaurant near location X▪ Freeform: random free testing by the tester with no clear
pre-defined task list. This allows the testers flexibility to dig
deeper into various parts of a solution when needed▪ Misuse/failures: carried out to evaluate the stability of thesolution in unusual conditions e.g. repeatedly pressing thevoice command button

This document does not provide a detailed description of themethodology and this page serves to provide an overview of
the approach.
For a detailed discussion and presentation of SBD’s
10methodology
please contact us.

System Usability Scale
(SUS)

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

ADAS

P

P

UX heuristics

P

P

P

P

P

P

Perceived Quality (PQ)

P

P

P

P

P

Legibility & readability

P

P

P

P

P

P

P

P

P

Misuse/failures

P

P

Ergonomics

Dynamic

P

P

Execution

Static

P

P

P

P

10

Example slides from thefull 150+ page reportFeatures & Functionality

Delight and Performance features

Few delight features, performance features mostly well implemented
Delight feature considerations:

Performance feature considerations:•

The Megane has few delight features. The only particularly noticeable one is thestart-up sequence, which greets the driver as the system starts.

•

Performance features are mostly well implemented and expected to please users,
particularly the connected media offering.

•Some IoT functionality is offered via Google Assistant, but this is a relatively
invisible offering and requires specific commands to be able to operate successfully.

•

The display configuration is expected to delight customers, with two large, clear,high-resolution screens.

•

A final minor feature is the accurate representation of external lights on the ADAS
display in the instrument cluster. While only a small point, it emphasises theattention to detail evident throughout the system.

•

The ADAS display is executed effectively and clearly, offering a good balance of
information.

•

The lack of a HUD is a disappointment, although not specifically an expectation.

•

The parking camera system is more difficult to access than it should be.

Start-up sequence is a minor ‘wow’ feature
The Megane features a visual and auditory start-up sequence that begins about a second after
the driver’s door is closed, timed to occur when the driver is sitting in the seat. The sequencespans both screens effectively, feels on-brand and raises anticipation, particularly for new
owners. This is expected to appeal to the majority of users.

Impressive display configuration
The system comprises two HD (167 PPI) screens: a 12.3-inch instrument cluster and a 12-inch
portrait display, slightly angled towards the driver for improved viewing and reach. Both screens
offer a high level of contrast, good colours, deep blacks and barely any lag was experienced
throughout testing. Overall, the interface is very pleasing to use, delivers a high level ofperceived quality and gives the impression of a system consistent with a higher segment car thanthe Megane.

1212

ExecutionⅼReliability issues

Issue with parking sensors

3. ADAS

On one occasion, the error message ‘Parking sensors
unavailable’ showed on the instrument cluster in white
text, followed by ‘Check parking sensors’ in yellow. Thisremained for a few minutes, then was not seen again.

Frequency

Low

MediumHigh

Severity

Minor

Major

Critical13
13

ExecutionⅼErgonomics highlights

Key lowlights

Column stalks cluttered
Initial misoperation of wipers likely, obscuration issue
On the right-hand side of the steering column, three stalks plus a paddle areprovided in this limited area of space.
Particularly when a novice user is initially learning the HMI layout and hasn’t
fully adapted to the functionality of this vehicle, there is a reasonably strong
chance that users will bring prior experience of other vehicles (e.g. Tesla Model
3, Mercedes-Benz) with a single stalk on the right which is used to select thedrive position.
With three stalks in the Megane, this means that this prior experience can leadto the user operating the incorrect stalk – often the wiper stalk whenattempting to select the drive position.

The stalk layout is cluttered and partially obscures the
instrument cluster

A similar and related concern exists with the audio control stalk, which is
uncommon practice in the automotive landscape today and risks adding
distraction for users not familiar with the control location or operating logic.
A further issue with this configuration is that the instrument cluster is partiallyobscured by the drive position stalk.

14
14

Perceived quality

Tactile highlights

Perceived Quality: Tactile
Level 1

TactileDisplays are rigidly
mounted. No
issues with touch
input

Stiffness & looseness: No wobble was experienced at all, other than avery small amount in the steering wheel paddles, however this wasconsidered acceptable. Overall, the HMI gives the impression of being
appropriate to a higher segment vehicle.

SBD viewpoint

Force feedback: No haptic feedback for the display, but responsive
and consistent touch input increases sensation of a quality interface.
Feedback from steering wheel and HVAC controls is firm and pleasing,giving the impression of a high-quality interface.
Material quality: The feel of all buttons/stalks/displays provides animpression of good quality, exceeding expectations. All plastics infrequently touched areas meet or exceed expectations.
Material harmony: Tactile quality is consistently high throughout thevehicle.
Geometric & Positioning: All controls convey a sense of good quality
and for the most part surpass the brand identity.

All physicalcontrols give
impression of highquality

HVAC controls arefirm, reassuring
and lack any
freeplay

Level 2 scoring
Stiffness &looseness

Force feedback

Material quality

Material
harmony

Geometric &positioning

Good

Good

Good

Good

Good
15
15

ADAS Domain

SAE L0 – ADAS images – System usage

SAE Level 0 ADAS: System usage
System usage: LKA

System usage: BSM

System usage: RCTA

Visual warning after lane deviation with assistancedisplay

LED warning shown in mirror with ISO icon

Good visual warning in infotainment

System usage: LKA

Lack of individual lane tracking and warning when notin assistance display

16
16

Infotainment Domain ⅼPositive points

Pleasing screen configuration
Category

Infotainment

Description

Large, high-resolution instrument cluster and central display

The Megane OpenR link system comprises two HD (167 PPI) screens:a 12.3-inch instrument cluster and a 12-inch portrait (or 9-inch in thelower configuration) TFT central display, provided by Continental.
• Brightness is adaptive and the instrument cluster features 'microblinds' technology, similar to desktop privacy screens, to shield thescreen from sunlight.

SBD
viewpoint• In use, the screens mostly function very effectively with a high levelof contrast, good colours, deep blacks and barely any lag
experienced throughout testing.

Large amount of screen estate offered• The central display is angled slightly towards the driver to improvereach and viewing.

• The only issue observed was that in direct sunlight (rarely
experienced) the central display's high-gloss finish suffered fromhigh reflectivity, showing fingerprints excessively and making thescreen barely legible.
Overall, both displays are very pleasing to use, deliver a high level ofperceived quality and give the impression of a system consistent witha higher segment car than the Megane.

UX impact
17

Major
negative

Minor
negative

No impact

Minor
positive

Major
positive17

Infotainment Domain ⅼ

Negative points

Source selection screen is confusing and requires prioritisation
Category

Radio/Media

Description

Sources screen is unintuitive and hard to use
The sources screen lists all sources alphabetically without prioritisation.
• Because sources are listed alphabetically, if a new source isinstalled, it will displace some or all of the other sources (depending
on how close its initial is to the beginning of the alphabet). This
causes issues as users form a mental image of the page layout (inthe example the Radio shortcut is middle-right) and installing new
apps disrupts this.
• Cycling through the sources using the stalk control on the steeringcolumn currently goes through every source.
• SBD recommends several modifications to the current format:

SBD
viewpoint

1. Show a ‘priority apps’ section at the top with space for up to
eight apps. Cycling through sources would just show these.
2. Provide a system default setting with e.g. Radio, Bluetooth
Audio and Spotify (if installed) at the top.
3. Show all other apps below (e.g. divided by a horizontal bar).
4. Allow users to move apps between the two sections as well
as changing the order within sections.
5. Add any new apps to the end of the lower list.
6. Move Android Auto and CarPlay to the apps screen.
Implementing these changes would make this section far easier to use
and maintain consistency as apps were added over time.

UX impact
18

Major
negative

Minor
negative

No impact

Minor
positive

App listing changes each time an app is added

Major
positive18

Navigation Domain

Negative points

Touchscreen inputs should be enlarged to meet recommendations
Category

User input

Description

Touchscreen inputs are smaller than recommended

Input elements within Maps are smaller than recommended.
• The navigation search box is approximately 9mm tall, below therecommended size which is a minimum of 12-13mm.
• Further buttons and inputs are also less tall than optimal, and allelements can be hard to operate when driving.
SBD
viewpoint• When Android Auto is run on the same system, the search box isapproximately 12mm tall. While this gives a ‘lower resolution’impression, it is far more usable, particularly when driving.
It is important to ensure that input areas are sufficiently large to meetguidelines. The current configuration requires more time andconcentration to interact with than Android Auto which is likely to leadto an increased level of distraction.

UX impact
19

Major
negative

Minor
negative

No impact

Minor
positive

Comparison of input boxes on Android Automotive (left) and Android Auto (right)

Major
positive19Request the price for thefull 150+ page reportContact Us

Contact SBD Automotive

Do you have any questions?
If you have any questions or feedback about thisresearch report or SBD Automotive’s consulting services,
you can email us at info@sbdautomotive.com or discusswith your local account manager below.

info@sbdautomotive.com

Book a meeting

USA

Garren Carr
North America
garrencarr@sbdautomotive.com+1 734 619 7969

21

UK

Germany

Luigi Bisbiglia
UK, South & West Europe
luigibisbiglia@sbdautomotive.com+44 1908 305102

Andrea Sroczynski
Germany, North & East Europe
andreasroczynski@sbdautomotive.com+49 211 9753153-1

India

China

Japan

SBD China Sales TeamChina
salesChina@sbdautomotive.com+86 18516653761

SBD Japan Sales TeamJapan, South Korea & Australia
postbox@sbdautomotive.com+81 52 253 6201
21