#635

In-Car HMI UX Evaluation& Benchmarking

UX
TABLE OF CONTENTS

EVALUATION

Executive Summary
Introduction to this ReportIn-car HMI UX
Evaluations

Features and Functionality
Execution
Perceived Quality
ADAS Domain

BYD Seal

Infotainment Domain
Navigation Domain

In this edition, the UX team is testing the BYD Seal.

Voice Recognition Domain

Overall, the system achieves a strong score in UX and a reasonable score for functionality. It
manages to impress in terms of voice recognition and extensive app offerings, but is prevented fromscoring higher by missing functionality and poor integration of some features.

Connected Features Domain
Convenience Domain
Support Areas

RELATED SBD REPORTS
619 – UX Benchmarking Series
A precursor to the new In-Car HMI UX
Evaluation & Benchmarking Series, SBD
Automotive’s UX Team evaluates the infotainment
user experience of over 40 vehicles.
806 – ADAS HMI Evaluations

A precursor to the new In-Car HMI UX
Evaluation & Benchmarking Series, SBD
Automotive’s Autonomous Car Team evaluates the
ADAS performance and usability of over 20
vehicles.

COVERAGE

FREQUENCY

PUBLICATION FORMAT

PAGES

6
NA

CHINA

EUROPEANNUALLYQUARTERLYCARS PER YEARPD F

POWERPOINT

EXCEL

ONLINE

142

#635/ In-Car HMI UX Evaluation & Benchmarking

Scoring> Features and functionality: > Reliability/stability:
evaluating whether theevaluating the repeatedsolutions provide featuresusability and whether thethat customers expect, needusers can have a similarand solve problems (or(positive) experience eachprovide a wow factor).
time.
> Usability:
evaluating whether the> Perceived quality:
features available are easy
evaluating the potential
to learn and use. This
perception in quality of theconsiders areas such as
HMI components and howergonomics, legibility,
this contributes to the overallusability characteristics andcustomer experience.
how the system implements
the various features.

This research is useful for

PRODUCT PLANNERS

MARKETING

C-SUITEDo I have access?

USER EXPERIENCEE N G I N E ER S

Portal access

Request a quote for
In-Car HMI UX Evaluation & Benchmarking Series

BYD SealJune 2023

635-23(23c)

In-Car HMI UX Evaluation & Benchmarking
BYD Seal

Contents
635 - In-Car HMI UX Evaluation & Benchmarking – BYD Seal
Executive Summary≫5

Features and functionality≫

22

Perceived quality≫▪ Overall Score▪ Overview▪ Introduction▪ SUS score by domain▪ Delight and performance features▪ Definitions▪ Kano analysis▪ Hygiene and navigation▪ Evaluation results▪ Positive and negative implementations▪ ADAS feature set▪ Highlights▪ Driver distraction▪ IoT integration, music, entertainment
and information▪ PQ Overview▪ CX recommendations

Execution≫▪ Potential improvements

Introduction to this report≫▪ Aim of report▪ Scope of report▪ 2023 vehicles▪ SBD experience▪ Methodology▪ Report structure▪ Overview

15

ADAS domain≫

28

47

55

▪ Summary▪ Failure modes▪ Misuse▪ System scoring▪ UX Heuristics▪ UX laws▪ SAE L0

▪ System performance▪ SAE L1&2

▪ Reliability▪ Ergonomics

Infotainment domain≫▪ Legibility and readability▪ Summary▪ UX heuristics▪ Key positive and negative78

Contents
635 - In-Car HMI UX Evaluation & Benchmarking – BYD Seal
Navigation domain≫

92

Connected features domain≫▪ Summary▪ Summary▪ UX heuristics▪ Key positive and negatives

118▪ Navigation specific scoring

Convenience domain≫▪ Key positive and negatives

125▪ UX heuristics

Voice recognition domain≫▪ Summary▪ Functionality

108▪ HVAC summary▪ HUD summary▪ User profile summary▪ Performance▪ Stability

Next steps≫

132

Contact us≫

141▪ SUS score▪ SBD UX principles▪ Command structure▪ Localization▪ Integration▪ Amazon Alexa integration

Total number of pages - 142

Introduction

Introduction

Aim of this report

One of the core goals of these studies is to provide a true indication of what
the final customer experience of each solution could be. To do thisevaluations are focused on providing scoring and analysis in the following
areas:▪ Features and functionality: evaluating whether the solutions provide
features that customers expect & need, and solve problems (or provide a
wow factor)

Brand perception

Consumer testing

Welcome to the 2023 HMI benchmarking report series. This report has beencreated to provide a fair, unbiased and objective view of the latest in-vehicle
HMI solutions in the US, European, and Japanese markets. Evaluations arecarried out by SBD usability experts with a deep understanding of CASE
domains such as the Connected Car and ADAS & autonomy domains.

▪ Usability: evaluating whether the features available are easy to learn anduse. This considers areas such as ergonomics, legibility, usability
characteristics and how the system implements the various features▪ Perceived quality: evaluating the potential perception in quality of the
HMI components and how this contributes to the overall customer
experience
SBD supports clients throughout the development of new HMI and products
from a relatively simple companion app to a more complex multi-domain
infotainment solution. The methodologies used in these reports take into
account many years of experience with consumer testing and custom client
projects to provide a fair and, as much as possible, objective methodology.
All viewpoints and analysis within the report are aimed defining areas ofconcern through a data driven approach. This report aims to benchmark andscore solutions whilst also being able to provide actionable recommendations
to design and development teams.
Please note that due to the ever-evolving automotive technology market,
SBD updates it’s methodology each year, but does not update scores from
7 previous years. Therefore, please assume a slight drop in scores for boththeuser experience (UX) and functionality from the previous year.

BRAND
APPEAL

Aesthetics and engagement“Do I have a positive emotional attachment to thesolution?”

EMOTIONAL
APPEAL

Repeated usability
RELIABILITY

Expert testing(the focus of this report)▪ Reliability/stability: evaluating the repeated usability and whether theusers can have a similar (positive) experience each time

“Do I have a positive emotional attachment to thebrand?”“Can I have a similar experience each time?”

Ergonomics, usability characteristics &execution“Are the features available easy to learn
and use?”USABILITY

Domain functionality &prioritization“Does the system provide features I
want and need to use?” i.e. “Does it
solve problems for me?”FUNCTIONALITY

SBD’s view on the hierarchy of needs for CX benchmarking
7

Introduction

Scope of report: focus on in-car HMI evaluations
The scope of evaluations in this report are constrained to the in-car HMI
experience, in both static and dynamic conditions. One notable element isdriver distraction which SBD covers at only a high level in this study as
carrying out a full driver distraction evaluation requires biometrics test
equipment to ensure the collected data is unbiased and objective.

Prepurchase

Purchase

Activation

In-car HMI

Driverdistraction

IoT

A full evaluation of the end-to-end customer experience is not within scope of
this report, but it is something which SBD has many years experience in fromboth a consumer and expert perspective. Other areas such as the companion
app, online portal and in-home smart devices are not in scope as they aredefined as “out of car” experiences.

Within the vehicle, any HMI element the user interacts with is evaluatedincluding steering wheel controls, touch screen displays, voice control, HUDs
and digital keys. The features and services on offer have been broadlygrouped into the following domains (or test areas):▪ ADAS domain▪ Infotainment domain▪ Navigation domain▪ Voice recognition domain▪ Connected services domain▪ 8Convenience domain
8Introduction

2023 vehicle list
SBD has chosen six cars to evaluate in 2023, based on two selection categories. New/interesting UX focuses on systems with to never-seen-before
features or functionality, or the implementation of a solution that has previously been a challenge or pain-point for end-users. New mass-market UX
includes vehicles in segments that are sold in high numbers and are entering a new generation of UI for that vehicle. While we make best efforts toadhere to the chosen cars and schedule, the last two years have seen release dates slipping significantly, so it may be necessary to makesubstitutions.

Cars tested

Jeep Grand
Wagoneer

Nissan Ariya▪ Passenger display▪ ProPilot Assist▪ Amazon Fire TV▪ Unique HMI andinterior▪ Active Drive Assist▪ US market test

Planned cars for 2023

▪ EV SUV▪ German Market Test

BYD Seal▪ Unique Home Screen
HMI▪ EV▪ Chinese market test

Peugeot 408

Fisker Ocean

Ford F-150
Lightning▪ Drive Assist 2.0▪ EV SUV▪ Android-based OS▪ 3D instrument cluster▪ Sustainable materials▪ EV Truck▪ German market test

▪ Large central display▪ US market test

▪ US market test99Introduction

SBD experience through years of testing in-car solutions
Over the last ten years SBD has evaluated over 100 solutions from a Connected Car or ADAS perspective for our public report series (many more for privateclient evaluations). This current report series is an evolution of both test methodologies to provide a holistic view of in-car HMI. Furthermore, custom
evaluations methodologies used across the globe for SBD clients have been included where applicable to enhance to overall approach.

1010Introduction

One page methodology overview
One of SBD’s core goals of this report is to be as objective, fair
and as transparent as possible. To achieve this, various
methodologies are used throughout the testing to evaluate
different areas of the solution in various conditions.
These methodologies are a mix of different types of tests:▪ Objective tests: where the value provided is not influenced
by a tester’s viewpoint e.g. response time
▪ Subjective tests: the test score is based on the expert
testers’ viewpoints e.g. task ease of use▪ Task-based: evaluations carried out based on a predefinedtask list e.g. navigate to a pizza restaurant near location X▪ Freeform: random free testing by the tester with no clear
pre-defined task list. This allows the testers flexibility to dig
deeper into various parts of a solution when needed▪ Scoring range: ranges and definitions of how to score atest element e.g. poor depth and accuracy score = theresults provided are not in line with what is reasonably
expected by the user

Type of tests
Test area

Objective

First impressions

Subjective
P

P

P

P

P

Dynamic tasks

P

P

P

P

Random freeP

P

Navigation specific tests

P

P

P

Voice recognition

P

P

P

Performance & response

P

System Usability Scale
(SUS)

P

P

UX heuristics

P

P

Legibility & readability

P

Perceived Quality (PQ)

P

P

P
P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

Ergonomics

P

Misuse/failures

P

P

ADAS

Dynamic

P

P▪ Dynamic: tests are carried out when the vehicle is moving
in various road conditions and locations e.g.
motorways/highways, cities, villages, country roads etc.

StaticP

P
P

ExecutionPPP

For a detailed discussion and presentation of SBD’s
11
methodology
please contact us.

Scoringrange

PFinal SBD UX score

This document does not provide a detailed description of themethodology and this page serves to provide an overview of
the approach.

Freeform

Static tasks▪ Static: tests are carried out when the vehicle is not moving▪ Misuse/failures: carried out to evaluate the stability of thesolution in unusual conditions e.g. repeatedly pressing thevoice command button

Task based

P

P

P

P

11

Example slides fromthe reportFeatures & Functionality

Delight and Performance features

Impressive rotating display, POI search and display poorly integrated• Delight features are limited to only a small selection of minor features.
• BYD App allows the user to remote control the vehicle backward and forward, andto turn left and right slowly.
• The main delight feature is the rotatable central display from landscape mode toportrait mode.

• RCTA lacks the expected level of visual warning by not providing any graphicalwarning in the central display.
• POI integration is considered poor and not effectively organized or presented tominimize user scrolling and navigating menu steps.

Rotating central display

Ineffective POI integration

The central display can be rotated from a landscape position to a portrait position using
the on-screen button or a button located on the steering wheel. When rotated the onscreen display will adjust automatically and change the layout of certain buttons andon-screen elements. Most apps are well integrated regardless of the screen
configuration, however some apps are not optimized effectively for both configurations
and do not effectively use the screen space in both rotations.

When adding POIs along the route with route guidance active, the search results are
not presented effectively. Only two POIs are shown per page forcing the user to scroll
repeatedly. This is not only cumbersome but also potentially distracting if performed
while driving. Information on each POI at a top level is also very limited, resulting in theuser having to select each POI to view basic information.

13
13

Features & Functionality

ADAS feature set

Stable performance, lack of commonly used functions
SBD tested five ADAS on the 2023 BYD Seal. These were Rear Cross Traffic Alert (RCTA), Lane Departure Warning/Lane Keeping Assist (LKA/LKA), Blind Spot Monitoring (BSM), Adaptive
Cruise Control (ACC) and Piloted Driving (PD).

Overall, most ADAS are well implemented with most system providing important status information at all times, regardless of the display chosen. Warning HMI is good for all system apartfrom RCTA which lacks any warning the central display. The most negative aspect of the ADAS suite is the absence of SAPA (Semi Automatic Parking Assist) or FAPA (Fully Automatic
Parking Assist). This is a feature that most users are likely to expect in this type of vehicle.

RCTA is automatically
activated when reversegear is selected. The
system utilizes the BSM
warning icons in each side
mirror. The system does
not provide warning in thecentral display and systemstatus is difficult tounderstand.

LKA/LDW provides lane
tracking status clearly and iscommunicated via thecluster and HUD. The
repeater icon gives a very
good level of detail andshows which side of the lane
is being crossed when themain display is not shown.
However, there is little
distinction between LKA andLDW support.

BSM provides adequate
warning in thecorresponding side
mirror when a vehicle isdetected. This warningchanges in color and an
audible warning is given
when indicating into thepath of the detectedvehicle. A good visual
warning is also given in
the cluster.

ACC is controlled via
buttons located on thesteering wheel. A goodlevel of visual feedback
is given in the cluster
and HUD using correctcolor logic. A good levelof support is given
during stop&go traffic.
However, informationsuch as headway is only
ever shown when
adjusted.

Overall PD provides goodvisual support in thecluster and HUD with clear
graphics. The multi laneview in the cluster provides
a good sense of thesurrounding environment
but suffers from some
glitches and erratic
graphics. Status is clearlycommunicated at all times.
Hands-off warning is clear
and robust.

14
14

ExecutionⅼLegibility & Readability

Display reading distance & text size
Smaller text size on both displays
The central display and IC perform very wellIn the central display the height of the text “PM2.5” in the top left portion of thescreen is 4mm. This is below the target size of 4.5 mm and scores as an“Emerging risk“.
In the instrument cluster, the text size of the unit readout “kW·h/100km” is
3mm. This falls below the recommended text height of 3.5 mm but above theminimum height of 2.8 and is an “Emerging risk“. The two text examples inboth displays have an acceptability rating of “concerns“ due to the risk they
may present while driving.

Display

Reading
distance

Text size

Target size

Final Rating

Central display

73cm

4mm

4.5mm

Emerging risk

Instrument cluster

63cm

3mm

4.1mm

Emerging risk

Severity / Driver distraction
RatingAcceptability Rating

Central display

Emerging risk

Concerns

Instrument cluster

Emerging risk

Concerns

Display

Text in both displays is below the target size

15
15Perceived quality

Tactile highlights

Perceived Quality: Tactile
Level 1

TactileTolerances between
buttons on the door
are tight andprevent wobble orunintended
movement

Gloss black plastic
buttons contributeto impression ofcheapness

Stiffness & looseness: Tolerances of most buttons and surfaces
are tight and prevent wobble or unintended movement.
Force feedback: Most of the feedback from buttons gave theimpression of cheapness, partly due to the hard, shiny plastic
materials used and the lack of sufficient damping. The volume
adjustment rotary feels well damped.
SBD viewpoint

Material quality: High gloss and black plastic are usedthroughout the cabin which contributes to an impression ofcheapness for some components and touch points.
Material harmony: Steering wheel buttons have the same feelingof quality as other components.
Geometric & Positioning: Many of the design elements andsurfaces within the cabin have an organic curved appearance.

Organic curved
design throughout
cabin

Level 2 scoring
Stiffness &looseness

Force feedback

Material quality

Material
harmony

Geometric &positioning

Fair

Fair

Fair

Fair

Good
16
16

ADAS Domain

SAE L1&2 - ADAS images – ACC

SAE Level 1 & level 2 ADAS – ACC
ACC

ACC headway is only shown when adjusted

ACC

ACC

ACC

Ask whether to adjust ACC speed limit when recognizing traffic sign(due to reflection current ACC speed limit cannot be seen clearly)

Actual speed and set speed appear grouped in thecluster.

A clear message is given when the driver performs athrottle override17
17

Infotainment Domain

Positive points

Multiple aspects of the system are linked to user profiles

Multiple aspects of the system and settings preferences can be linked
to user accounts.

SBD
viewpoint• Elements such as HMI theme settings and ambient lighting
preferences (e.g. color, brightness, etc.) can all be tied to a user's
profile in the central display. Mainstream 3rd party accounts
including Ximalaya FM, Yunting, Kugou Music, Tingban, and Auto
Map can also be linked to the user profile.
• When accessing the vehicle using NFC or cloud digital keys, therelevant BYD account will be automatically logged in as well as any
linked account.

Multiple HMI-related features and 3rd party account linking to a user
account allow users to personalize their own experience andconvenience levels while providing an intuitive UX. Convenience levels
are greatly enhanced and flexibility is provided if multiple users
regular drive the same vehicle.

UX impact
18

Major negative

Minor negative

Minor positive

Multiple aspects of the system are linked to user profiles

Major positive18

Navigation Domain

Positive points

Lane guidance is clear and precise

When there are multiple lanes on the road (with a set route), theblue lane guidance bar will be displayed on the top of the screen.
• All lanes on the road are shown in the bar with the suggested lane
highlighted in white. The lane guidance graphic also clearly
indicates which lanes on the road ahead are designated for buses
only.
SBD
viewpoint• The lane guidance graphic is presented in a timely manner without
obscuring any visual route guidance information on the map.
The visual lane guidance shown in the central display is of a good sizeand provides drivers with simple and intuitive lane guidance
instructions for junctions and intersections. However, when thedetailed junction view pop-up is also present, the lane guidance
display almost becomes redundant and duplicates information. In
these scenarios it could be removed to show more of the mapbehind.

UX impact
19

Major negative

Minor negative

Minor positive

Clear and precise lane guidance

Major positive19

Request price forthe full reportContact Us

Contact SBD Automotive

Do you have any questions?
If you have any questions or feedback about thisresearch report or SBD Automotive’s consulting services,
you can email us at info@sbdautomotive.com or discusswith your local account manager below.

info@sbdautomotive.com

Book a meeting

USA

Garren Carr
North America
garrencarr@sbdautomotive.com+1 734 619 7969

21

UK

Germany

Luigi Bisbiglia
UK, South & West Europe
luigibisbiglia@sbdautomotive.com+44 1908 305102

Andrea Sroczynski
Germany, North & East Europe
andreasroczynski@sbdautomotive.com+49 211 9753153-1

India

China

Japan

SBD China Sales TeamChina
salesChina@sbdautomotive.com+86 18516653761

SBD Japan Sales TeamJapan, South Korea & Australia
postbox@sbdautomotive.com+81 52 253 6201
21