#635

In-Car HMI UX Evaluation& Benchmarking

UX
TABLE OF CONTENTS
Executive Summary
Features and Functionality

EVALUATION

In-car HMI UX
Evaluations

HMI Execution
Perceived Quality
ADAS

Xpeng P5

Infotainment
Navigation

First impressions of the Xpeng P5 (China market car) are very positive: a minimalistic, high quality andclutter-free cockpit dominated by a 12.3-inch instrument cluster and a 15.6-inch portrait-configuration
central display occupying the entire centre stack. The only immediately obvious physical controls are on
the steering wheel, but even these are minimalistic and thoughtfully designed.

Voice Recognition
Connected Features
Convenience

In use, the system doesn't disappoint with a high level of responsiveness, functionality and stability, leading
to the highest HMI UX score achieved in recent times. There is of course always room for improvement.

RELATED SBD REPORTS
619 – UX Benchmarking S eries
A precursor to the new In-Car HMI UX Evaluation& Benchmarking Series, SBD Automotive’s UX
Team evaluates the infotainment user experienceof over 40 vehicles.
806 – AD AS HM I E v aluations
A precursor to the new In-Car HMI UX Evaluation& Benchmarking Series, SBD Automotive’s
Autonomous Car Team evaluates the ADAS
performance and usability of over 20 vehicles.

COVERAGE

FREQUENCY

PUBLICATION FORMAT

PAGES

6
NA

CHINA

EUROPEANNUALLYQUARTERLYCARS PER YEARPD F

POWERPOINT

EXCEL

ONLINE

150+

#635/ In-Car HMI UX Evaluation & Benchmarking

Scoring> Features and functionality:
evaluating whether thesolutions provide featuresthat customers expect, needand solve problems (orprovide a wow factor).

This research is useful for

Do I have access?

> Reliability/stability: evaluating
the repeated usability andwhether the users can have asimilar (positive) experienceeach time.

> Usability:> Perceived quality: evaluating
evaluating whether thethe potential perception infeatures available are easy
quality of the HMI
to learn and use. This
components and how thisconsiders areas such as
contributes to the overall
ergonomics, legibility,
customer experience.
usability characteristics andhow the system implements
the various features.

PRODUCT PLANNERS

M A R K E T I NG

C- S UITE

USER E XPE R IENCEE NGI NE E R S

Portal access

Request a quote for
In-Car HMI UX Evaluation & Benchmarking Series

Xpeng P5

September 2022

In-Car HMI UX Evaluation & Benchmarking
Xpeng P5

Table of Contents
Executive summary ≫4

Convenience domain ≫140

Introduction to this report ≫13

Support areas ≫146

Features and functionality ≫20

Contact us ≫153

Execution ≫26

Perceived quality ≫45

ADAS domain ≫53

Infotainment domain ≫83

Navigation domain ≫101

Voice recognition domain ≫117

Connected features domain ≫127

Introduction

Introduction

Aim of this report

One of the core goals of these studies is to provide a true indication ofwhat the final customer experience of each solution could be. To do thisevaluations are focused on providing scoring and analysis in the following
areas:

Brand perception

Consumer testing

Welcome to the 2022 HMI benchmarking report series. This report has
been created to provide a fair, unbiased and objective view of the latest
in-vehicle HMI solutions in the European, US and Japanese markets.
Evaluations are carried out by SBD usability experts with a deep
understanding of CASE domains such as the Connected Car and ADAS &autonomy domains.

▪ Features and functionality: evaluating whether the solutions provide
features that customers expect & need, and solve problems (orprovide a wow factor)▪ Reliability/stability: evaluating the repeated usability and whether theusers can have a similar (positive) experience each time

▪ Perceived quality: evaluating the potential perception in quality of the
HMI components and how this contributes to the overall customer
experience
SBD supports clients throughout the development of new HMI and
products from a relatively simple companion app to a more complex
multi-domain infotainment solution. The methodologies used in thesereports take into account many years of experience with consumer
testing and custom client projects to provide a fair and, as much as
possible, objective methodology.
All viewpoints and analysis within the report are aimed defining areas ofconcern through a data driven approach. This report aims to benchmark
6 score solutions whilst also being able to provide actionable
andrecommendations to design and development teams.

BRAND
APPEAL

Aesthetics and engagement“Do I have a positive emotional attachment to thesolution?”

EMOTIONAL
APPEAL

Repeated usability

RELIABILITY

Expert testing(the focus of this report)▪ Usability: evaluating whether the features available are easy to learn
and use. This considers areas such as ergonomics, legibility, usability
characteristics and how the system implements the various features“Do I have a positive emotional attachment to thebrand?”“Can I have a similar experience each time?”

Ergonomics, usability characteristics & execution“Are the features available easy to learn
and use?”

Domain functionality &USABILITY

prioritization“Does the system provide features I
want and need to use?” i.e. “Does it
solve problems for me?”FUNCTIONALITY

SBD’s view on the hierarchy of needs for CX benchmarking
6Introduction

Scope of report: focus on in-car HMI evaluations
The scope of evaluations in this report are constrained to the in-car HMI
experience, in both static and dynamic conditions. One notable element isdriver distraction which SBD covers at only a high level in this study as
carrying out a full driver distraction evaluation requires biometrics test
equipment to ensure the collected data is unbiased and objective.

Prepurchase

Purchase

Activation

In-car HMI

Driverdistraction

IoT

A full evaluation of the end-to-end customer experience is not within scope of
this report, but it is something which SBD has may years experience in fromboth a consumer and expert perspective. Other areas such as the companion
app, online portal and in-home smart devices are not in scope as they aredefined as “out of car” experiences.
Within the vehicle, any HMI element the user interacts with is evaluatedincluding steering wheel controls, touch screen displays, voice control, HUDs
and digital keys. The features and services on offer have been broadlygrouped into the following domains (or test areas):▪ ADAS domain▪ Infotainment domain▪ Navigation domain▪ Voice recognition domain▪ Connected services domain▪ 7Convenience domain
7

Introduction

2022 vehicle list
SBD has chosen six cars to evaluate in 2022, based on two selection categories. New/interesting UX focuses on systems with to never-seen-before features orfunctionality, or the implementation of a solution that has previously been a challenge or pain-point for end-users. New mass-market UX includes vehicles insegments that are sold in high numbers and are entering a new generation of UI for that vehicle. While we make best efforts to adhere to the chosen cars andschedule, the last year has seen release dates slipping significantly, so it may be necessary to make substitutions.

Cars tested

Lucid Air

Rivian R1T▪ Potential disruptor▪ New disruptor▪ Multiple displays▪ Appears to have aninnovative approach to
HMI▪ US market test

▪ US market test

Awaiting test

Renault Megane
E-Tech▪ LG’s new Android
Automotive IVI▪ French market test

Xpeng P5
▪ High level of ADAS
▪ Advanced voicerecognition▪ Configurable avatar▪ China market test

GMC HummerEV▪ GM’s new Android OS
system▪ Unreal Engine graphics▪ US market test

Lexus NX
▪ New Lexus system▪ Cloud-based navi
▪ New voice recognition▪ German market car,UK test8
8

Introduction

SBD experience through years of testing in-car solutions
Over the last nine years SBD has evaluated 96 solutions from a Connected Car or ADAS perspective for our public report series (many more for privateclient evaluations). This current report series is an evolution of both test methodologies to provide a holistic view of in-car HMI. Furthermore, custom
evaluations methodologies used across the globe for SBD clients have been included where applicable to enhance to overall approach.

99Introduction

One page methodology overview
One of SBD’s core goals of this report is to be as objective, fair
and as transparent as possible. To achieve this, various
methodologies are used throughout the testing to evaluate
different areas of the solution in various conditions.
These methodologies are a mix of different types of tests:▪ Objective tests: where the value provided is not influenced
by a tester’s viewpoint e.g. response time

Type of tests
Test area

Objective

First impressions

Subjective

Task based

P

Freeform

Scoringrange

P

Static tasks

P

P

P

P

Dynamic tasks

P

P

P

P

Random freeP

P

Navigation specific tests

P

P

P

Voice recognition

P

P

P

▪ Scoring range: ranges and definitions of how to score a testelement e.g. poor depth and accuracy score = the results
provided are not in line with what is reasonably expected by
the user

Performance & responseP▪ Static: tests are carried out when the vehicle is not moving

Final SBD UX score▪ Dynamic: tests are carried out when the vehicle is moving invarious road conditions and locations e.g.
motorways/highways, cities, villages, country roads etc.

▪ Subjective tests: the test score is based on the expert
testers’ viewpoints e.g. task ease of use▪ Task-based: evaluations carried out based on a predefinedtask list e.g. navigate to a pizza restaurant near location X▪ Freeform: random free testing by the tester with no clear
pre-defined task list. This allows the testers flexibility to dig
deeper into various parts of a solution when needed▪ Misuse/failures: carried out to evaluate the stability of thesolution in unusual conditions e.g. repeatedly pressing thevoice command button

This document does not provide a detailed description of themethodology and this page serves to provide an overview of
the approach.
For a detailed discussion and presentation of SBD’s
10methodology
please contact us.

System Usability Scale
(SUS)

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

ADAS

P

P

UX heuristics

P

P

P

P

P

P

Perceived Quality (PQ)

P

P

P

P

P

Legibility & readability

P

P

P

P

P

P

P

P

P

Misuse/failures

P

P

Ergonomics

Dynamic

P

P

Execution

Static

P

P

P

P

10

Example slides from thefull 150+ page reportFeatures & Functionality

Delight and Performance features

Many delight features, performance features mostly well implemented
Delight feature considerations:

Performance feature considerations:•

The P5 has many delight features, among which the ability to connect to a voicecontrolled drone is one that stands out.

•

No major issues were found in performance features. Most of the features are well
implemented.

•

Smart recommendation is a less common feature to find in the market. The P5 canpush suggestions to the head unit based on location and other conditions.

•Two large, high-definition displays with fast response lead to a smooth andpleasant user experience.

•

In-car games are also notable. Several are provided that do not require
touchscreen interaction and can be played with voice only.

•

The handwriting input area is located very low down due to the portrait design,
which makes it more difficult to access than it should be.

Connected drone
The head unit can be connected to a DJI Mavic 2 Pro drone using a USB cable. The drone does
not come with the car and has to be purchased separately. The app for the connected drone
however, is a native app that comes with the system. Once connected, the drone can becontrolled by the P5 voice assistant. Controllable movements include take off, return and several
flying modes.

In-car KTV
Catering to Chinese people’s tastes, in-car KTV is not uncommon in connected cars in the
Chinese market. However, most OEMs only provide one app, either Changba, WeSing or
Thunderstone KTV. Xpeng provides two KTV Apps (Changba and WeSing) for users to choose
from. A microphone can be bought in the Xpeng online shop with Bluetooth connection
supported.

12
12

ExecutionⅼReliability issues

Issue with parking sensors

3. ADAS

On one occasion, the error message ‘Parking sensors
unavailable’ showed on the instrument cluster in white
text, followed by ‘Check parking sensors’ in yellow. Thisremained for a few minutes, then was not seen again.

Frequency

Low

MediumHigh

Severity

Minor

Major

Critical13
13

ExecutionⅼReliability issues

Failed to trigger VPA when leaning towards the center console

2. Voice recognition

The Xpeng P5 has positional detection feature. Under
most conditions, it is very convenient to use, however,the downside is that the wake-up rate is significantly
lower when trying to wake up the VPA near the centre
console.

Frequency

Low

MediumHigh

Severity

Minor

Major

Critical14
14

ExecutionⅼErgonomics highlights

Key lowlights

Glancing down posing driving risks
Users must glance down to operate the central display
The central display (15.6-inch) is reasonably wide for alandscape orientation and occupies the entire centre stack.
While this works well for the visual appeal, it unfortunately
presented reach issues in operation for SBD’s testers. Whensitting in the driver’s seat, interacting with the central display
requires the tester to look down to the screen. Also, since thescreen is very large, locating desired information may betime-consuming. This can cause potential risks if the driver is
not focusing on the road for a long time.

Portrait display leads to bigger glance-down angle

15
15Perceived quality

Tactile highlights

Perceived Quality: Tactile
Level 1

TactileGood responsetime for thecentral display
touch screen

Stiffness & looseness: No wobble experienced during testing,everything feels secure and high quality.

SBD viewpoint

Force feedback: Most of the feedback experienced matches the user’s
expectations, including feedback from steering wheel buttons andstalks. However, there is no haptic feedback for the display and the
response time is good.
Material quality: The feel of all buttons/stalks/displays provides animpression of good quality.
Material harmony: Tactile quality is consistent throughout the vehicle.

Geometric & Positioning: The overall shape of the hard buttons andshifters conveys good quality and fits the brand identity.

All physicalcontrols give
impression of highquality

Level 2 scoring
Stiffness &looseness

Force feedback

Material quality

Material
harmony

Geometric &positioning

Good

Good

Good

Good

Good
16
16

ADAS Domain

SAE L0 – ADAS images – System usage

SAE Level 0 ADAS: System usage
System usage: LDW/LKA

System usage: BSM

System usage: RCTA

Visual warning after lane deviation

Warning icon in mirror

Warning in cluster is very subtle

System usage: LKA

Solid line in cluster

Broken line on road

Poor reflection of lane lines

17
17

Infotainment Domain ⅼPositive points

Smart recommendations and reminders
Category

Infotainment

Description

Contextual recommendations/reminders add value

When carrying out static testing, it was observed that the system caneither recommend activities that might be suitable to interact with, orgive reminders if something happens.

SBD
viewpoint• If the vehicle has been started and has remained motionless for aperiod of time, the system may recommend entertainment activities
such as video watching.

Shall we watch
movies together?
Smart fridge door
is open• If the smart fridge door is left open, it will remind the passengers toclose it promptly.

This is a pleasing feature that is unexpected and adds an element of‘wow’ to the user experience of the system.​Pop-up recommendation and reminder shown on screen

UX impact
18

Major
negative

Minor
negative

No impact

Minor
positive

Major
positive18

Infotainment Domain ⅼ

Negative points

Some apps and mini-programs are not well integrated
Category

Infotainment

Description

Third-party apps and mini-programs poorly integrated

Although the system features many third-party apps and miniprograms, not all of them are well integrated.
• In some cases, the text shown on the central display is extremely
small and cannot be easily read.
SBD
viewpoint• Some content is not optimized to fit the full screen and large spaces
are left unused.
• For mini-programs, the takeout-ordering platform (Eleme)malfunctioned during testing, with pages failing to load normally.
The above issues drive down the user experience, and should beprioritized for future OTA improvements.

Small text size

UX impact
19

Major
negative

Minor
negative

No impact

Minor
positive

Mini-program bug

Major
positive
19

Infotainment Domain ⅼ

Negative points

USB music playback lacks simplicity
Category

Infotainment

Description

USB music playback requires specific folder structure and file naming

During media source testing, a USB drive containing songs wasconnected to the system. On connection, a message was shown
saying users must create an “Xpeng music folder” in the root of theUSB drive, and the songs are recommended to be renamed with a“Singer, Song name” structure.
• This process of playing USB tracks is far more complex than itneeds to be.
SBDviewpoint• USB media playback is considered a hygiene feature, and should bean effortless process without any requirement for specific folder
structures or naming conventions.
• To be able to play files, users will have to create a folder on the USB
drive using a computer and then move all files into it before
potentially renaming each one individually.
The system should be able to read any standard music files copied to
the USB, irrelevant of naming convention or folder structure. Although
USB music is a less frequently used medium nowadays, the current
approach is likely to be viewed as a significant and unnecessary
inconvenience.

UX impact20

Major
negative

Minor
negative

No impact

Minor
positive

Major
positive

Cannot find USB music20

Request the price for thefull 150+ page reportContact Us

Contact SBD Automotive

Do you have any questions?
If you have any questions or feedback about thisresearch report or SBD Automotive’s consulting services,
you can email us at info@sbdautomotive.com or discusswith your local account manager below.

info@sbdautomotive.com

Book a meeting

USA

Garren Carr
North America
garrencarr@sbdautomotive.com+1 734 619 7969

22

UK

Germany

Luigi Bisbiglia
UK, South & West Europe
luigibisbiglia@sbdautomotive.com+44 1908 305102

Andrea Sroczynski
Germany, North & East Europe
andreasroczynski@sbdautomotive.com+49 211 9753153-1

India

China

Japan

SBD China Sales TeamChina
salesChina@sbdautomotive.com+86 18516653761

SBD Japan Sales TeamJapan, South Korea & Australia
postbox@sbdautomotive.com+81 52 253 6201
22