Information Presentation Strategies to Promote Pedestrian Behavior
Change in Mixed Spaces with Automated Vehicle
Yuki Nishiura1 Hiroyuki Yamada2 and Ryo Kurazume3
Abstract— This paper introduces an information presentation
strategy for pedestrians, aiming to enhance traffic efficiency ina mixed pedestrian-automated vehicle environment, such as apublic road. While automated driving technology has maderemarkable progress, interactions with pedestrians on regularroads have mostly been studied in virtual environments using
virtual reality goggles. According to these studies, potential
traffic efficiency and safety issues arise from pedestrians’
limited understanding of automated vehicle behavior. To address this, we propose a human-machine interface employing
a head-mounted display (HMD) to mitigate traffic efficiency
degradation caused by pedestrians. The proposed system draws
upon behavioral economics principles to encourage pedestrians
to modify their behavior and develop better interactions with
automated vehicles. Simulations were conducted to identify
an information presentation strategy that strongly supports
learning, and its effectiveness was further validated throughexperiments involving a real vehicle. Notably, the experimental
results confirmed that the information presentation strategies
proven effective in simulations also facilitated pedestrian learning during real-world interactions.

I. INTRODUCTION
Interacting with pedestrians is crucial for Automated
Vehicles (AVs) when navigating public roads, except for
automobile-only roads. Typically, when an AV detects apedestrian in its path, it will try to slow down and stop
to ensure the pedestrian’s safety. However, the behaviorof pedestrians with a limited understanding of AVs during
this protection process can potentially compromise traffic
efficiency and safety[1]. Previous research on pedestrianAV interactions has primarily taken place in virtual environments, utilizing a human-machine interface (HMI) called
e-HMI attached to the vehicle. This approach has shownpromising results, enhancing pedestrian trust in the AV
and improving traffic efficiency[1][2]. Nevertheless, many ofthese studies have not thoroughly explored whether e-HMI
actually facilitates pedestrian learning of AV behavior.
This study introduces an HMI system designed to aid
pedestrians in comprehending the behavior of AVs andencourage them to adopt behaviors that enhance benefits forboth vehicles and pedestrians. By facilitating pedestrians’
understanding of AV behavior, the system aims to enhance
1 Yuki
Nishiura is with Graduate School of Information
Science and Electrical Engineering, Kyushu University, Japan.

nishiura@irvs.ait.kyushu-u.ac.jp
2 Hiroyuki Yamada is with Autonomous Control Research Department, Center for Sustainability - Connective Automation, Hitachi,Ltd. Research and Development Group, Ibaraki 312-0034, Japanhiroyuki.yamada.qt@hitachi.com
3 Ryo Kurazume is with Faculty of Information Science andElectrical Engineering, Kyushu University, Fukuoka 819-0395, Japan.

kurazume@ait.kyushu-u.ac.jp

traffic efficiency and reduce operational costs of e-HMI
systems for vehicles. This study investigates the use of abehavioral economics framework to influence pedestrians’
behavior positively, particularly in an open field setting
where interactions between pedestrians and AVs take place.
The study presents the outcomes of a behavior change
promotion system rooted in behavioral economics, enablingpedestrians to learn behaviors that optimize advantages forboth vehicles and pedestrians.
II. R ELATED W ORK
According to [3], in critical traffic situations, such as roadcrossings, pedestrians decide whether to cross based on the
speed and proximity of approaching vehicles. This holds true
even when the vehicle has an external information display.
However, in [4], an interview-style study conducted at an
actual traffic scene, pedestrians mentioned that eye contact
with the driver also influenced their crossing decisions.
Unlike interactions with human-driven vehicles, achieving
eye contact may be challenging in interactions with AVs.
Therefore, information presentation interfaces have beensuggested to aid pedestrian-AV interactions.
According to [5], many studies have investigated visualand auditory HMI. And these proposed ideas have been
mainly categorized by the physical triggers proposed in [6].
In this study, we attempted to design an HMI more based onpedestrian behavior change principles by further classifying
them as psychological triggers.
In the case of [1], the focus is on a scenario where apedestrian crosses an intersection simultaneously with an
AV entering an intersection without a traffic signal. In thissituation, visual and audio information presentation fromthe vehicle side enhanced the pedestrian’s receptivity to
the AV, both before and after exposure to the information
presentation. Similarly, in [2][7], the authors share the outcomes of an experiment in which the vehicle was equippedwith an information presentation interface using a display,while pedestrians were crossing an intersection without atraffic signal. The experimental results demonstrated thatthe information presentation significantly reduced the pedestrian’s crossing decision time. Moreover, [2] reported an
improvement in pedestrians’ acceptance of AVs, similar to
the findings in [1].
In [8], an attempt is made to convey information onpedestrians’ recognition status by AVs using an eye-likedevice attached to the vehicle. This resulted in a reduction in
the failure rate of pedestrian crossing decisions. Furthermore,[9] suggested that expressing ”anger,” ”fear,” ”happiness,”and ”sadness” by eye-like interface can elicit empathy andincrease agreement by pedestrians. This result suggests thatemotional expression by HMIs may contribute to improving
the benefits for both pedestrians and AVs. However, thesestudies[2][7][8][9] did not investigate how crossing decisiontime or safety changed after the experience of information
presentation by the interface. Therefore, it is not possible
to discuss whether pedestrians learned the behavior of thevehicles from these data alone.
Moreover, experiments conducted in virtual environments
may not fully capture the same level of attentiveness fromsubjects as in real environments due to the absence of real
danger [2]. To address these concerns, this study conducted
experiments using actual AV to verify the effectiveness ofthe proposed HMI in real traffic situations.
III. BASIC C ONCEPT OF THE P ROPOSED HMI
This study introduces a system that provides information
to pedestrians and helps them learn how to interact with
AVs. The primary goal of the proposed system is to present
information to pedestrians in a way that enables them toeffectively interact with AVs when crossing the street, even
without the information presentation. This chapter explains
the interface hardware and the core concepts of behavioral
economics that underpin the proposed information presentation.
A. Hardware
In this study, a head-mounted mixed reality device(Hololens2, Microsoft) was employed to provide information
to pedestrians. Utilizing augmented reality (AR) technology,
this device can overlay a virtual space onto the real world,allowing information to be presented without significant
alterations to the environment, such as the need for a physicaldisplay installation.
B. Behavioral Economics
Our research focuses on pedestrian learning. Therefore,
it is essential to modify the behavior of pedestrians who
are unfamiliar with or have a limited understanding of theinteraction. To achieve this goal, we proposed information–presenting ideas to create a synergistic effect that canbring about behavioral changes .
The triggers that induce behavioral changes are definedin the field of Shikakeology[6], a branch of behavioral
economics. These behavior change triggers, as proposed in
Shikakeology, are illustrated in Fig. 1. They are broadly
categorized into ”physical triggers” and ”psychological triggers”. We assume that when these two types of triggers
are combined, they produce a synergistic effect that caneffectively modify a person’s behavior. As mentioned in the
introduction section, by utilizing the idea generation framework proposed in Shikakeology to promote behavior change,
we believed that we could devise an effective method ofpresenting information to assist pedestrians in their learning
process.

Auditory
HapticFeedback

Olfactory
Taste

Physicaltrigger

VisualFeedforward

Challenge

Shikake
Trigger

DissonanceIndivisualcontext

Negative expectation
Positive expectation
Rewards

Psychologicaltrigger

Self-esteem
Socialcontext

Fig. 1.

Analogy
Perceived affordance

Being watchedSocial norm
Social proof

Shikake triggers proposed in the Shikakeology[6].

IV. I NFORMATION P RESENTATION P ROTOTYPE
In this study, we developed five behavior change ideas
based on behavioral economics, as depicted in Fig. 2, alongwith the corresponding Shikake triggers..
Due to hardware limitations, we utilized only Visual andPerceived affordance as physical triggers in this study. Now,
we will elaborate on the psychological triggers that werecombined with these physical triggers and their intentions.
First, in Idea 1, we combined Visual and Social Norm. Given
that pedestrians face difficulties in communicating with AVs,such as through eye contact, we deemed it crucial for themto learn about the intentions and efficiency of the vehicle.
To address this, we employed emotional icons as a communication medium. The utility of the AV is expressed through”happy” and ”sad” values, indicating efficiency based onwhether the AV can reach a pre-defined target arrival time.
Our intention is to stimulate communication between the AV
and pedestrians, leading to changes in social behavior, suchas ”giving way to a vehicle.”
In Idea 2, we combine Visual and Being watched triggers
to promote communication between vehicles and pedestrians
using a virtual eye. The virtual eye is indicated when thevehicle is within a certain range from pedestrians, signifyingthat the AV is aware of their presence. This idea, like Idea
1, aims to facilitate communication between vehicles andpedestrians. However, it differs from Idea 1 in two aspects:
first, it intends to deter selfish pedestrian behavior by evoking
the sense of being watched; second, it does not display the
AV’s efficiency to the pedestrian. These features allow us
to respect the pedestrian’s intentions and make the crossing
itself more challenging compared to Idea 1, thus enabling
appropriate adjustments to the learning difficulty.
In Idea 3, we combine Visual and Negative expectation
triggers. This method indicates the approaching AV’s direction using an arrow when it is within a certain range
from pedestrians. We anticipate that this will implicitly givethe driver an approximate distance to the AV and that theapproach warning from the indicator will effectively restrain
the driver’s behavior.
In Idea 4, we combine Perceived affordance and Negative
expectation triggers. Idea 4 utilizes perceived affordance
dynamically by displaying an arrow resembling a pedestriancrossing. It is expected that pedestrians will recognize thatthe system is presenting a crossing timing that does notcollide with the AV. The indication of crossing timing
is determined by an algorithm that calculates whether apedestrian and AV will coincide at the vehicle’s entry time
based on the pedestrian’s walking speed. If no collision is
projected, a green arrow appears on the street, creating aneasy problem setup to encourage pedestrians to proceed withthe crossing task.
In Idea 5, we combine Visual and Negative expectation
triggers. The concept involves displaying the level of danger
at various locations on the road based on the AV’s speed.
The danger level is calculated from the AV’s braking distance, and points within the AV’s braking distance range arehighlighted in red. Additionally, the AV’s planned route ispresented in yellow to indicate its intended direction. We
believe that this information will provide pedestrians withmore comprehensive safety details about the road, motivatingthem to optimize their behavior accordingly.

F =

Vp
Vv
+
Tp
Tv

(1)

In both the preliminary experiment and the actual experimentusing the actual device, a two-stage experiment consistingof a ”learning process” and a ”test process” was conducted
to measure the subject’s learning effect. In the ”learning
process,” subjects used the HMI for a certain period tolearn optimal interaction with the AV. Subsequently, a ”testprocess” was conducted where subjects attempted to interact
with the AVs on their own without using the HMI, to assess
whether they had indeed learned the interaction. For boththe preliminary and actual experiments, the learning process
consisted of 10 trials, while the test process had 5 trials.
The predicted learning effect of the subjects in advance(hypothesis) is illustrated in Fig. 3.

Fig. 3.

Subject’s learning hypothesis.

VI. P RELIMINARY E XPERIMENT

Fig. 2.

Behavior change ideas.

V. F ORMULATION OF INTERACTION EVALUATION VALUES
Since the objective of this study is to aid pedestrians
in learning to interact with AV, it was essential to havean objective measure for evaluating the interaction. In thisstudy, the interaction evaluation value, denoted as F -value,
was defined as shown in Formula (1). Here, Vp represents
the pedestrian’s travel time when no vehicle is present andno interaction occurs, Vv is the AV’s travel time when nopedestrian is present and no interaction occurs, Tp stands forthe pedestrian’s travel time when the AV is present, and Tv
corresponds to the AV’s travel time when the pedestrian ispresent. The calculation of Vp involved measuring the walking speed of each subject. Additionally, Vt was obtained bymeasuring the travel time again under the same experimental
conditions in an environment where pedestrians were notpresent.

As the ultimate goal of this study, we planned to conduct
an open-field experiment (real scene experiment) using an
actual AV. To ensure the experiment’s practicality, we first
conducted a preliminary experiment to further narrow down
the five behavior change ideas shown in Fig. 2, which were
implemented using an virtual AV in the preliminary phase.
In this preliminary experiment, we created a virtual space
simulating a real road on Hololens2. The configuration of thevirtual space used in the pre-experiment is depicted in Fig. 4.
We then evaluated the impact of these behavior change ideas
in this virtual space and selected the ones that we expected
to be effective in inducing behavior change when using theactual device.

Fig. 4. Virtual environment constructed during the preliminary experiment(interaction between a subject crossing a T-intersection (green arrow) andan AV attempting to make a left turn (red arrow)).

A. Scenario
The preliminary experiment took place in the scenario
depicted in Fig. 4, where a pedestrian (subject) crossed anintersection without a traffic light while an AV attemptedto make a left turn into the same intersection. The subjects
were given two instructions: first, that ”the AV is in ahurry,” and second, that ”You should complete the crossing
as quickly as possible within the range of not running.” To
ensure the subjects’ learning of interactions, we recognized
the importance of introducing randomness in the interactions
they experienced. As a result, the speed of the AV was
randomly selected for each experiment to incorporate thiselement of randomness.

TABLE IS UM OF F -values IN THE LEARNING PROCESS .
Idea
number
1
⃝
2
⃝
3
⃝
4
⃝
5
⃝

Results ofTrust in
Automationquestionnaires.
15.0
15.3
15.3
14.7
16.7

Sum of
F -values in
the learning
process.
13.85
13.47
14.77
14.07
14.61

Displacementof F -values
after learning.
0.30
0.21
-0.26
-0.03
0.32

B. Hypothesis in the Preliminary ExperimentVII. EXPERIMENT IN REAL SCENE
During the preliminary experiment, three conditions wereExperiments were conducted to confirm the effectivenessidentified as having effective behavior change effects for the of the behavior change ideas selected in the preliminary
main experiment: Hypothesis I, Hypothesis II, and Hypoth- experiment in real scene.
esis III.
A. Scenarios
Hypothesis I
It is important for users to be able to understandIn the experiment, conditions were randomly chosen forand predict AV behavior.
each trial from a pool of 18 possible combinations, comprisHypothesis II
ing 6 AV starting positions and 3 pedestrian walking starting
Learning occurs when the user is exposed to apoints. An overview of the placement of the vehicle starting
large number of iterations of near-optimal behavior. position and the pedestrian starting position is depicted in
Hypothesis III
Fig. 6. A scene of an experiment using actual AV is shown
It is important to learn that the learner canin Fig. 7.
break free from HMI involvement.
Each of the subjects who experienced ideas most in linewith each of these hypotheses would have the following
properties:
Hypothesis I
The results of the 5-point Likert scale subjective
evaluation of the Understanding/Predictability itemof the Trust in Automation questionnaire[10] will
Fig. 6. Overview of AV start positions and subject start positions.
have the highest score.
Hypothesis II
The sum of F -values in the ”learning process”
will be the highest.
Hypothesis III
The decrease in F -value after the end of
Fig. 7.
Scenery of an experiment using an actual automated vehicle.
learning process will be minimized.
(Example of a scenario in which the vehicle enters straight ahead).

C. Experiment Result
In the preliminary experiment, data were collected from15 subjects, aged between 21 and 30. Each behavior change
idea was experienced by three subjects. The experiment’s
outcomes are presented in Fig. 5 and Table I. Idea 5 aligns
most closely with Hypothesis I, while Idea 3 correlates wellwith both Hypothesis II and Hypothesis III.

B. Experiment Results
The experiment involved a total of five subjects. Eachbehavior change idea was experienced by two subjects using
the HMI, while one subject participated in 15 interactions
without any information presented. The experimental results
are presented in Figs. 8, 9, and 10.
In right-hand member of Formula (1), the left term isreferred to as the ”Pedestrian term,” denoting the efficiency
Vof the pedestrian as Tpp , while the right term is called the”AV term,” representing the efficiency of the vehicle as TVvv .
VIII. D ISCUSSION

Fig. 5. Results of preliminary experiment (F -value of each behavior
change idea).

This study examined and investigated the following threepoints based on the results of the experiment in real scene:• The impact of different experiment scenarios (vehicle
turning right or going straight) on the F -value.
• Transitions of F -value before and after the learning
process.

Fig. 8.

Transitions of F -value in experiments.

Fig. 10.

Transitions of AV term value in experiment in real scene.

determined using formula (2).
The F -values after normalization are shown in Figs. 12
and 11. Additionally, the results of the t-test for the distribution of F -values in each scenario group are presented again
in Table IV. Table IV demonstrates that the bias of F -values
due to scenarios was reduced.

Fig. 9.

•Transitions of Pedestrian term value in experiments.

The relationship between the two terms (Pedestrian termand AV term) that constitute the F -value.

A. The effect of the difference in the scenarios
The study verified the impact of variations in experimentscenarios (right-turn intrusion and going straight ahead forAVs) on the F -value, assuming that the F -value canaccurately assess the interaction’s optimality. Hence, it was
desirable to have a consistent distribution of F -value acrossscenarios in the experiment. The 15 trials conducted byeach subject were divided into two groups: one for AV start
positions (1)-(3) and another for AV start positions (4)-(6).
First, the effect of differences in experiment scenarios(right-turn intrusion and going straight ahead) on F -values
was examined, assuming that F -values could uniquely assess the interaction’s optimality. Consequently, it was preferable for the distribution of F-values to remain consistentacross the experiment scenarios. The 15 trials conductedby each subject were divided into two groups: a group forvehicle start positions (1)-(3) and a group for vehicle start
positions (4)-(6). The results of the statistical test for themean F -values in each scenario are presented in Table II.
The distribution of F -values for each group is illustrated inFig. 11. The significance level was set at 5%, and the results
indicate that the F -values exhibit different distributions
under different scenarios.
To mitigate the differences among scenarios, an effort
was made to normalize the F -values for each scenario.
The normality of the distribution of F -values for eachscenario was tested, and the results are displayed in Table
III. According to Table III, the distribution of F -values foreach group may exhibit normality. Therefore, the F ’-values,
which are values with reduced bias due to scenarios, were

Fig. 11. Distribution of F values for each scenario (AV turning right or
straight ahead along the road senario).
TABLE IIR ESULTS OF TESTS OF THE MEAN FOR EACH SCENARIO .
p-value of F-test
test result
t-valuep-value of T-test
test result

0.327(>0.05)
Inequality of dispersion was not observed.
5.653
2.882×10−7 (<0.05)
Significant difference in means is observed

TABLE IIIR ESULTS OF S HAPIRO -W ILK - TESTS FOR EACH SCENARIO .
p-value of right-turn
intrusion scenario
test result
p-value of straight ahead
along the road senario
test result

0.1804(>0.05)
Follow the normal distribution.
0.1484(>0.05)
Follow the normal distribution.

(−µR
F ′ = 1 + F3σ
Right T urn
R
(2)
F
−µS
Straight
F ′ = 1 + 3σ
S
B. Transitions of F -value before and after the learning
process
The study examined the change in F ’-values before
and after the learning process. The results, comparing the
average of the F ’-values from the first three trials of the”learning process” with the average of the F ’-values fromthe five trials conducted in the ”test process,” are presented
in Fig. 13. The data in Fig. 13 reveal that the two subjects

IX. CONCLUSIONS

Fig. 12. F ’-values were calculated for the experimental results to reducethe bias of F -values due to differences in scenarios.
TABLE IVR ESULTS OF TESTS OF THE MEANS FOR EACH SCENARIO .
p-value of F-test
test result
t-valuep-value of T-test
test result

0.004(<0.05)
Inequality of dispersion was observed.
-0.7920.4306(>0.05)
Significant difference in means
is not observed

who experienced behavior change Idea 5 showed a higher
improvement rate of F ’-values before and after learningcompared to the subjects who were not presented withinformation.

Fig. 13. Improvement rate of F ’-value: Graph showing the average F ’value after learning (11 to 15 trials) divided by the average F ’-value atthe beginning of learning (1 to 3 trials).

C. Relationship between Pedestrian term and AV term
The study investigated the relationship between the twoterms (Pedestrian term and AV term) that constitute the F value. The Pearson’s correlation coefficients of the Pedestrian and AV terms for each subject are presented in Fig.
14. From Fig. 14, it is evident that for behavior change Idea
5, the AV term and Pedestrian term exhibit a larger positive
correlation compared to the other subjects. This suggests thatpedestrians might have learned behaviors that strike a balance
between the benefits of AV and their own benefits.

Fig. 14. Graph showing correlation between Pedestrian term and AV term
during learning.

In this study, we introduced and created an HMI prototype
that aids pedestrians in learning to interact with AV throughthe application of AR technology and behavioral economics.
The preliminary experiments and main experiments demonstrated the potential of supporting effective communication
between pedestrians and AV. By providing relevant information, such as real-time displays of road safety during
the learning process, the HMI prototype encouraged natural
behavioral changes in pedestrians.
ACKNOWLEDGMENT
This research is the result of joint research by Hitachi,Ltd. and Kyushu University.
R EFERENCES[1] Shuchisnigdha Deb, Lesley J Strawderman, and Daniel W Carruth.
Investigating pedestrian suggestions for external features on fullyautonomous vehicles: A virtual reality experiment. Transportation
research part F: traffic psychology and behaviour, 59:135–149, 2018.
[2] Kai Holländer, Ashley Colley, Christian Mai, Jonna Häkkilä, FlorianAlt, and Bastian Pfleging. Investigating the influence of external
car displays on pedestrians’ crossing behavior in virtual reality. In
Proceedings of the 21st International Conference on Human-Computer
Interaction with Mobile Devices and Services, MobileHCI ’19, New
York, NY, USA, 2019. Association for Computing Machinery.
[3] Debargha Dey and Jacques Terken. Pedestrian interaction withvehicles: Roles of explicit and implicit communication. In Proceedingsof the 9th International Conference on Automotive User Interfaces andInteractive Vehicular Applications, AutomotiveUI ’17, page 109–113,
New York, NY, USA, 2017. Association for Computing Machinery.
[4] Matúš Šucha. Road users’ strategies and communication: driverpedestrian interaction. Transport Research Arena (TRA), 1, 2014.
[5] Manon Prédhumeau, Anne Spalanzani, and Julie Dugdale. Pedestrian
behavior in shared spaces with autonomous vehicles: An integrated
framework and review. IEEE Transactions on Intelligent Vehicles,8(1):438–457, 2023.
[6] Naohiro Matsumura, Renate Fruchter, and Larry Leifer. Shikakeology:
designing triggers for behavior change. AI & SOCIETY, 30:419–429,
2015.
[7] Sebastian Stadler, Henriette Cornet, Tatiana Novaes Theoto, and Fritz
Frenkler. A tool, not a toy: Using virtual reality to evaluate the communication between autonomous vehicles and pedestrians. Augmented
Reality and Virtual Reality: The Power of AR and VR for Business,
pages 203–216, 2019.
[8] Chia-Ming Chang, Koki Toda, Xinyue Gui, Stela H. Seo, and Takeo
Igarashi. Can eyes on a car reduce traffic accidents? In Proceedings ofthe 14th International Conference on Automotive User Interfaces andInteractive Vehicular Applications, AutomotiveUI ’22, page 349–359,
New York, NY, USA, 2022. Association for Computing Machinery.
[9] Yiyuan Wang, Senuri Wijenayake, Marius Hoggenmüller, Luke Hespanhol, Stewart Worrall, and Martin Tomitsch. My eyes speak:
Improving perceived sociability of autonomous vehicles in sharedspaces through emotional robotic eyes. Proc. ACM Hum.-Comput.
Interact., 7(MHCI), sep 2023.
[10] Moritz Körber. Theoretical considerations and development of aquestionnaire to measure trust in automation. In Proceedings ofthe 20th Congress of the International Ergonomics Association (IEA
2018) Volume VI: Transport Ergonomics and Human Factors (TEHF),
Aerospace Human Factors and Ergonomics 20, pages 13–30. Springer,
2019.