#635

In-Car HMI UX Evaluation& Benchmarking

UX
TABLE OF CONTENTS
Executive Summary
Features and Functionality

EVALUATION

In-car HMI UX
Evaluations

HMI Execution
Perceived Quality

Lexus NX

ADAS
Infotainment
Navigation

In this edition, the UX team is testing the Lexis NX.

Voice Recognition

One of the most positive takeaways is the general improvement in user experience when compared toprevious Lexus systems. The navigation system as a whole is pleasing to use and more intuitive thanprevious iterations. The removal of a clunky touch pad for the central display and a shift towards dedicatedtouch input only improves the user interaction and makes for a more enjoyable system to use.

Connected Features
Convenience

RELATED SBD REPORTS
619 – UX Benchmarking S eries
A precursor to the new In-Car HMI UX Evaluation& Benchmarking Series, SBD Automotive’s UX
Team evaluates the infotainment user experienceof over 40 vehicles.
806 – AD AS HM I E v aluations
A precursor to the new In-Car HMI UX Evaluation& Benchmarking Series, SBD Automotive’s
Autonomous Car Team evaluates the ADAS
performance and usability of over 20 vehicles.

COVERAGE

FREQUENCY

PUBLICATION FORMAT

PAGES

6
NA

CHINA

EUROPEANNUALLYQUARTERLYCARS PER YEARPD F

POWERPOINT

EXCEL

ONLINE

150

#635/ In-Car HMI UX Evaluation & Benchmarking

Scoring> Features and functionality:
evaluating whether thesolutions provide featuresthat customers expect, needand solve problems (orprovide a wow factor).

This research is useful for

Do I have access?

> Reliability/stability: evaluating
the repeated usability andwhether the users can have asimilar (positive) experienceeach time.

> Usability:> Perceived quality: evaluating
evaluating whether thethe potential perception infeatures available are easy
quality of the HMI
to learn and use. This
components and how thisconsiders areas such as
contributes to the overall
ergonomics, legibility,
customer experience.
usability characteristics andhow the system implements
the various features.

PRODUCT PLANNERS

M A R K E T I NG

C- S UITE

USER E XPE R IENCEE NGI NE E R S

Portal access

Request a quote for
In-Car HMI UX Evaluation & Benchmarking Series

Lexus NX

December 2022

In-Car HMI UX Evaluation & Benchmarking
Lexus NX

Table of Contents
Executive summary ≫4

Convenience domain ≫138

Introduction to this report ≫13

Support areas ≫143

Features and functionality ≫20

Contact us ≫150

Execution ≫26

Perceived quality ≫48

ADAS domain ≫56

Infotainment domain ≫80

Navigation domain ≫100

Voice recognition domain ≫121

Connected features domain ≫131

Introduction

Introduction

Aim of this report

One of the core goals of these studies is to provide a true indication ofwhat the final customer experience of each solution could be. To do thisevaluations are focused on providing scoring and analysis in the following
areas:

Brand perception

Consumer testing

Welcome to the 2022 HMI benchmarking report series. This report has
been created to provide a fair, unbiased and objective view of the latest
in-vehicle HMI solutions in the European, US and Japanese markets.
Evaluations are carried out by SBD usability experts with a deep
understanding of CASE domains such as the Connected Car and ADAS &autonomy domains.

▪ Features and functionality: evaluating whether the solutions provide
features that customers expect & need, and solve problems (orprovide a wow factor)▪ Reliability/stability: evaluating the repeated usability and whether theusers can have a similar (positive) experience each time

▪ Perceived quality: evaluating the potential perception in quality of the
HMI components and how this contributes to the overall customer
experience
SBD supports clients throughout the development of new HMI and
products from a relatively simple companion app to a more complex
multi-domain infotainment solution. The methodologies used in thesereports take into account many years of experience with consumer
testing and custom client projects to provide a fair and, as much as
possible, objective methodology.
All viewpoints and analysis within the report are aimed defining areas ofconcern through a data driven approach. This report aims to benchmark
6 score solutions whilst also being able to provide actionable
andrecommendations to design and development teams.

BRAND
APPEAL

Aesthetics and engagement“Do I have a positive emotional attachment to thesolution?”

EMOTIONAL
APPEAL

Repeated usability

RELIABILITY

Expert testing(the focus of this report)▪ Usability: evaluating whether the features available are easy to learn
and use. This considers areas such as ergonomics, legibility, usability
characteristics and how the system implements the various features“Do I have a positive emotional attachment to thebrand?”“Can I have a similar experience each time?”

Ergonomics, usability characteristics & execution“Are the features available easy to learn
and use?”

Domain functionality &USABILITY

prioritisation“Does the system provide features I
want and need to use?” i.e. “Does it
solve problems for me?”FUNCTIONALITY

SBD’s view on the hierarchy of needs for CX benchmarking
6Introduction

Scope of report: focus on in-car HMI evaluations
The scope of evaluations in this report are constrained to the in-car HMI
experience, in both static and dynamic conditions. One notable element isdriver distraction which SBD covers at only a high level in this study as
carrying out a full driver distraction evaluation requires biometrics test
equipment to ensure the collected data is unbiased and objective.

Prepurchase

Purchase

Activation

In-car HMI

Driverdistraction

IoT

A full evaluation of the end-to-end customer experience is not within scope of
this report, but it is something which SBD has many years experience in fromboth a consumer and expert perspective. Other areas such as the companion
app, online portal and in-home smart devices are not in scope as they aredefined as “out of car” experiences.
Within the vehicle, any HMI element the user interacts with is evaluatedincluding steering wheel controls, touchscreen displays, voice control, HUDs
and digital keys. The features and services on offer have been broadlygrouped into the following domains (or test areas):▪ ADAS domain▪ Infotainment domain▪ Navigation domain▪ Voice recognition domain▪ Connected services domain▪ 7Convenience domain
7

Introduction

2022 vehicle list
SBD chose six cars to evaluate in 2022, based on two selection categories. New/interesting UX focuses on systems with to never-seen-before features orfunctionality, or the implementation of a solution that has previously been a challenge or pain-point for end-users. New mass-market UX includes vehicles insegments that are sold in high numbers and are entering a new generation of UI for that vehicle. Only one substitution was necessary this year: the Toyota bZ4X
had to be replaced with the Lexus NX due to unexpected delays to its scheduled release.

Cars tested

Lucid Air

Rivian R1T▪ Potential disruptor▪ New disruptor▪ Multiple displays▪ Appears to have aninnovative approach to
HMI▪ US market test

▪ US market test

Renault Megane
E-Tech▪ LG’s new Android
Automotive IVI▪ French market test

Xpeng P5
▪ High level of ADAS
▪ Advanced voicerecognition▪ Configurable avatar▪ China market test

GMC HummerEV▪ GM’s new Android OS
system▪ Unreal Engine graphics▪ US market test

Lexus NX
▪ New Lexus system▪ Cloud-based navi
▪ New voice recognition▪ UK test, Germanmarket car

88

Introduction

SBD experience through years of testing in-car solutions
Over the last nine years SBD has evaluated 99 solutions from a Connected Car or ADAS perspective for our public report series (many more for privateclient evaluations). This current report series is an evolution of both test methodologies to provide a holistic view of in-car HMI. Furthermore, custom
evaluations methodologies used across the globe for SBD clients have been included where applicable to enhance to overall approach.

99Introduction

One page methodology overview
One of SBD’s core goals of this report is to be as objective, fair
and as transparent as possible. To achieve this, various
methodologies are used throughout the testing to evaluate
different areas of the solution in various conditions.
These methodologies are a mix of different types of tests:▪ Objective tests: where the value provided is not influenced
by a tester’s viewpoint e.g., response time

Type of tests
Test area

Objective

First impressions

Subjective

Task based

P

Freeform

Scoringrange

P

Static tasks

P

P

P

P

Dynamic tasks

P

P

P

P

Random freeP

P

Navigation specific tests

P

P

P

Voice recognition

P

P

P

▪ Scoring range: ranges and definitions of how to score a testelement e.g., poor depth and accuracy score = the results
provided are not in line with what is reasonably expected by
the user

Performance & responseP▪ Static: tests are carried out when the vehicle is not moving

Final SBD UX score▪ Dynamic: tests are carried out when the vehicle is moving invarious road conditions and locations e.g.,
motorways/highways, cities, villages, country roads etc.

▪ Subjective tests: the test score is based on the expert
testers’ viewpoints e.g., task ease of use▪ Task-based: evaluations carried out based on a predefinedtask list e.g., navigate to a pizza restaurant near location X▪ Freeform: random free testing by the tester with no clear
pre-defined task list. This allows the testers flexibility to dig
deeper into various parts of a solution when needed▪ Misuse/failures: carried out to evaluate the stability of thesolution in unusual conditions e.g., repeatedly pressing thevoice command button

This document does not provide a detailed description of themethodology and this page serves to provide an overview of
the approach.
For a detailed discussion and presentation of SBD’s
10methodology
please contact us.

System Usability Scale
(SUS)

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

P

ADAS

P

P

UX heuristics

P

P

P

P

P

P

Perceived Quality (PQ)

P

P

P

P

P

Legibility & readability

P

P

P

P

P

P

P

P

P

Misuse/failures

P

P

Ergonomics

Dynamic

P

P

Execution

Static

P

P

P

P

10

Example slides from thefull 150 page reportFeatures & Functionality

Delight and Performance features

Unique FAPA complete sequence and configurable HUD•

Wow features in the Lexus NX are extremely limited. No real standout features
above and beyond those found in other vehicles in a similar segment.

•

The HUD provides a clear display of critical information. Three varying levels ofdisplay detail can be selected.

•

FAPA does provide a minor delight feature in the way it communicates thecompletion of a parking procedure. The unique animation is pleasing and is
expected to wow most users.

•

Some ADAS suffer from increased risk of misoperation due to unlabelled touch
buttons located on the steering wheel.

•

Routing accuracy and roundabout guidance are delivered and integrated well,giving support when needed, especially the roundabout guidance with well timed
and easily understandable announcements.

Parking complete animation
When a parking procedure using FAPA is complete, a unique 3D animation plays in the central
display. Using a computer generated representation of the vehicle, a camera flies out of theinterior and completes a 360° walkaround of the vehicle before ending. By utilising the surroundview camera, this small feature not only gives a useful perimeter check of the parked vehicle but
also adds a small wow factor.

HUD
The HUD delivers a crisp and clear display, reflected onto the windscreen. The depth of
information shown in the HUD depends on the settings chosen but can include, turn-by-turnnavigation, incoming calls, ADAS and steering wheel button layout. ADAS icons are replicatedexactly between the cluster and HUD while providing all the information necessary without
becoming overwhelming and cluttered.

1212ExecutionⅼReliability issues

Poor Incoming call teams call UI

2. Phone

Incoming Teams call showed the phone number pad in
the central display with the option to pause or reject thecall but not answer it. On rejecting the call, “Talking...
Adam Jefferson” remained stuck on screen. This
repeating issue became frustrating.

Frequency

Low

MediumHigh

Severity

Minor

Major

Critical13
13

ExecutionⅼReliability issues

Inconsistent traffic display

6. Navigation

When dragging the map view around or zooming in andout, an inconsistency in when traffic colouring isdisplayed. At some zoom levels, traffic colouring willclearly be present on the roadway, after zooming, thiscan disappear.

Frequency

Low

MediumHigh

Severity

Minor

Major

Critical14
14

ExecutionⅼErgonomics highlights

Key lowlights

Confusing dynamic steering wheel buttons
Touch buttons result in regular misoperation
The steering wheel buttons have no clear icon labelling at a glance. The user
must place a finger or thumb on the surface to reveal a diagram of the button
layout in the cluster or HUD. This layout can change depending on chosenconfiguration.

While driving, especially on roundabouts or bends, the user's hand issusceptible to brushing over the touch pad and led to significant frustration
due to the visibility of the diagram. This was a regular occurrence during
testing.

Unintuitive steering wheel buttons

1515Perceived quality

Tactile highlights

Perceived Quality: Tactile
Level 1

TactileCapacitive touchsteering wheel
controls lead tomisoperation

Stiffness & looseness: No noticeable concerns with buttons, switches,
stalks or panels being too stiff or too loose, all feel as expected.
Force feedback: Mechanical feedback is pleasing and feels as expected.
Some inconsistencies between the volume and temperature rotaries,
the temperature one has detents, whereas the volume rotary does
not, but it offers sufficient resistance to give an impression of goodquality.

SBD viewpoint

Material quality: All materials are pleasing to the touch and give an
impression of appropriate quality.
Material harmony: Tactile quality is pleasing throughout the vehicle
cockpit and meets expectations.
Geometric & Positioning: The placement of the capacitive touchsteering wheel controls leads to persistent misoperation, causinggraphics to show on the HUD on most corners and roundabouts. Thisis likely to cause significant annoyance and lower the perception of ahigh-quality implementation.

Minor inconsistencies
among temperatureand volume rotaries

Level 2 scoring
Stiffness &looseness

Force feedback

Material quality

Material
harmony

Geometric &positioning

Good

Good

Good

Good

Fair
16
16

ADAS Domain

SAE L0 – ADAS images – System usage

SAE Level 0 ADAS: System usage
System usage: LKA

Non ADAS display

System usage: BSM

System usage: RCTA

BSM icon illuminated in side mirror during warning

Good visual warning in central display

ADAS display

Lane tracking status shown at all times in cluster

System usage: LKA

LKA Warning

PD Active

LKA warning and PD active have very similar HMI

17
17

Infotainment Domain ⅼPositive points

Large central display is a wow factor
Category

General system

Description

Central display in top specification model is large and capable

The NX comes standard with a 9.8-inch touchscreen display, but the
top-specification model comes with a 14-inch touchscreen display.
• For mid-specification models, the 14-inch touchscreen display isoffered as a cost option.
SBD
viewpoint• The 14-inch touchscreen provides a wow factor in this vehicle,
particularly in navigation view.
• Also, this central display is slightly angled towards the driver, but
manages not to be detrimental for the passenger.

Large 14-inch central display for top specification model

The 14-inch central display trends on the larger side for screen size inthe market today, so this is viewed as an attention grabber andstandout feature.

UX impact
18

Major
negative

Minor
negative

No impact

Minor
positive

Major
positive18

Infotainment Domain ⅼ

Negative points

Overlapping on-screen buttons, negatively impacting usage
Category

General system

Description

Some on-screen buttons overlap content, effectively reducing usable
screen area

On the lower left side of the central display, some on-screen buttons
overlap content.
• The voice recognition button presents the most significant issue,
being visible on almost all screens.
SBD
viewpoint

On-screen buttons overlap content• Depending on the screen, an additional button may be shown next
to the voice recognition button, also overlapping content. For
example, on the radio/media screen, the ‘Sources’ button is shown,and on the phone screen, the same button is used for ‘Devices’.
These overlapping on-screen buttons block both visibility andfunctionality, effectively reducing usable screen area of the central
display.

UX impact
19

Major
negative

Minor
negative

No impact

Minor
positive

Major
positive19Request the price for thefull 150 page reportContact Us

Contact SBD Automotive

Do you have any questions?
If you have any questions or feedback about thisresearch report or SBD Automotive’s consulting services,
you can email us at info@sbdautomotive.com or discusswith your local account manager below.

info@sbdautomotive.com

Book a meeting

USA

Garren Carr
North America
garrencarr@sbdautomotive.com+1 734 619 7969

21

UK

Germany

Luigi Bisbiglia
UK, South & West Europe
luigibisbiglia@sbdautomotive.com+44 1908 305102

Andrea Sroczynski
Germany, North & East Europe
andreasroczynski@sbdautomotive.com+49 211 9753153-1

India

China

Japan

SBD China Sales TeamChina
salesChina@sbdautomotive.com+86 18516653761

SBD Japan Sales TeamJapan, South Korea & Australia
postbox@sbdautomotive.com+81 52 253 6201
21