「第30回マルチメディア通信と分散処理ワークショップ論文集」 令和4年10月

Concept Design for Inclusive and Engaging Augmented
Reality Assisted Disaster Prevention Guidelines
De León A. Sergio1,a)Matsuda Yuki1

Yasumoto Keiichi1

Abstract: Disaster prevention guidelines are media aimed at the general public, that provide instructions to create
safer environments and improve people’s attitude during disasters, through graphics and easy to understand language.
Previous work has documented that the citizenry usually do not read them or follow their advice, speciﬁcally, in cases
when a user considers them hard to follow. Research in educational contexts have found Augmented Reality (AR)technologies to be eﬀective in capturing students’ attention. While in industrial contexts, AR technologies have beenable to signiﬁcantly reduce mistakes while workers follow instructions for maintenance processes. To increase public participation in disaster prevention activities, and produce better results while following its guidelines, this paper
proposes a novel AR-assisted disaster prevention guideline interface design. Consisting of a simpliﬁed checklist interface with gamiﬁcation elements to provide an easier-to-follow and more engaging guideline. Furthermore, a machinelearning object detection model is used to reduce the subjective judgment of the user while recognizing the targets ofinterest of guidelines’ instructions. Our work contributes new insight into the practical performance of AR and object
recognition models, and what they are able to achieve under the restrictions of commercial mobile platforms. Also,
since this targets a broad demographic, from young to elder adults, this interface also will able to survey the eﬀectiveness of accessibility design and gamiﬁcation features in AR application interfaces from an approach applicable tomultidisciplinary contexts.

1.

Introduction

Civil defense programs are government initiatives whose purpose is to increase the readiness of the citizens in a certain community against natural or man-made disasters. Around all ofJapan, Civil Defense programs actively try to reach and informthe local and visiting population on preparedness for typhoons,ﬁres or earthquakes [1], [2], [3]. Traditionally, printed and mass
media is used to communicate preparedness knowledge in these
programs. These documents are presented as a diverse collectionof formats like pamphlets, books, or ads found in public spaces,
television or the Internet. Their intention is to inform the public regarding actions that can be taken before, during and afterand disaster has occurred. In these media, simple advice is given
through easy to understand language, simple examples, and clear
illustrations.
The consequences of non, ill-informed or an unprepared population experiencing a natural disaster such as an Earthquake,can range from an increase in private property and business damages [4], and the increase in minor or grave injuries due secondary
hazards, such as falling objects or ﬁres [5].
In a previous survey study, we found that traditional media
used in civil defense programs, under-performs with regards topenetration in the population [6]. Less than half of the surveyed
general population, regardless of their age, marital status or nationality are not used to read disaster prevention guidelines as
1a)Nara Institute of Science and Technology
de leon aguilar.sergio.dm9@is.naist.jp©2022 Information Processing Society Japan

shown in Fig. 1. More importantly, we found their expert advice
being considered hard to apply in practice, even more in seniorsof the sampled population as shown in Fig. 2. To address these issues, several applications using a diverse set of technologies havebeen developed. Regardless, literature surveys show that studies
regarding their eﬀectiveness on the general public are still lacking [7].
This situation can be framed as a knowledge transfer problem
as found in Organizational theory; in which, sticky informationrefers to technical knowledge that is hard to transfer to a diﬀerent organizational unit [8]. In this comparison, the government
requires a transfer of technical knowledge — procedures, techniques, reference information — to the citizens who are expected
to apply expert’s advice into their surroundings.
In this document, we propose a novel interface design based onmodern technologies that can be deployed in mobile platforms.
Machine learning object detection models will be used to support
the user with expert judgement, and AR virtual elements will be
used to provide familiar elements expected to reduce information
stickiness. We are exploring the particular knowledge transfer
problem, where experts from an organization with a certain distance from its public — e.g. a local Disaster Management oﬃce
— try to convey their knowledge to a generic public that spans awide range of ages.

2.

Related Work

Augmented Reality (AR) technologies eﬀectiveness on improving learning rate and information retention has been widely181

「第30回マルチメディア通信と分散処理ワークショップ論文集」 令和4年10月

26
26
62% (16) 62% (16)

20-30
23
57% (13)

30-40
6

17
53% (9)Read
Guidelines
No
Yes
10

Amount of People

Amount of People

20

4

50-60

38% (10) 38% (10) 43% (10)+60

47% (8)

2

15-20

Blank

2
1
0

0
Blank

15−20

20−30

30−40

50−60

1

+60

Fig. 1

People usually do not read Disaster Prevention Guidelines(this ﬁgure taken from [6])studied on education, elderly care and support, and industrial
maintenance contexts [9]. It is found that even on traditionally
low engaging school subjects like History, AR technology successfully helps students to retain attention to the lesson [9]. Tzima et al. highlight how teachers’ opinion regarding AR in theclassroom is favorable, since in a laboratory practice context,
can also serve as a better instructor on proper equipment operation [10]. Taking into account the broad research on AR impact
on elder care and its ease to use, Leonardi et al. have noted theimpact of familiar elements in AR interfaces. Such familiar elements and methodology are able to reduce the barriers aﬀecting the elder population acceptance of AR technology, such as
the perception of technological superiority and the expectationof a low return on invested time used on new technology [11].
These familiar elements are noted relevant as the potential of AR
applications can be extended to the daily life of elderly people
outside of their homes. On this, Kong et al. showcase the potential of AR daily life guidance applications, where elderly people can be further supported in the operation of common interfaces such ATM’s, cinema ticketing machines or a cafeteria’s order taking machines [12]. In the case of industrial maintenancecontexts, Obermair et al. explored the diﬀerence between theperformance of AR-based and traditional instructions. They havefound how AR technology in maintenance tasks can signiﬁcantly
reduce misidentiﬁcations of the objects that need to be manipulated for a successful procedure [13]. Regardless of the beneﬁts,
Matthias et al. have tested these applications and found inherent problems in handheld devices, such as that their one-handmanipulation impedes semi-complex double-handed tasks [14],common in maintenance processes.
Thus, an inductive approach from the survey of the literatureshows that, the aforementioned AR-based applications present
each, diﬀerent sets of requirements needed to achieve their goals.
Requirements that reﬂect on each interface implementation design.
Clear examples shown in Eﬃe Lai-Chong et al. systematic re-©2022 Information Processing Society Japan

2

3

4

5

Apparent Diﬃculty

Age Range

Fig. 2 Older generations regard DPG’s as more diﬃcult than younger
groups (this ﬁgure taken from [6])view on usability and user experience of AR in school settings,
highlight the importance of teacher involvement during the usage
AR-based educational applications in the classroom [15]. Additionally, Dunleavy et al. identiﬁes an interesting and undesirable consequence of the game-like characteristics of AR applications: competition between students to ﬁnish the presented tasks
at hand [16], which they implicitly recommended to reduce. In industrial maintenance context as Matthias et al. point out, there arecases where workers found themselves in the need to be guided
around machinery. This produces the particular need of 3D exploration guidance, that is then addressed by the implementationof new interfaces elements, such as those reviewed and proposedby Perea et al. [17]. Lastly in the case of AR applications targeting the elder part of the population, there are informal and formal
documentation on accessibility, such as the guidance provided by
the W3C Web Accessibility Initiative [18]. These guidance attend
the particular needs that wide groups of elder ﬁnd needing due
their skew from the standard in cognitional and physical skills,
as Yu Fu et al. design considerations for AR-based games for
elderly population applies in their interface designs [19].
The interface design considerations for each context in the current literature, reﬂects the need of developers and researchers tospecialize each application to their intended audience.
The aforementioned specialization in AR-based applications
for instruction based tasks, have resulted in a gap in literature recognized by literature review as documented by Marion et al. [7].
Marion et al. identify the lack of research in the eﬀectivenessof applications’ interface design when targeting the general public, the public that disaster prevention guidelines are intended for.
The interface design proposed on this paper intends to serve as aplatform to further study this under-documented combination ofmobile technology and this speciﬁc diverse audience.

3.

Proposed Interface Design

To improve citizenry participation in Civil Defense activities
and ease the sticky information nature of expert advice found in182

「第30回マルチメディア通信と分散処理ワークショップ論文集」 令和4年10月

Fig. 3 Interface design and information integration ﬂow.

traditional disaster preparedness guidelines, we propose a new information delivery media and method, using mobile technology.
In this proposed interface design, advice on how to prepare
home appliances and furniture against earthquakes, is synthesizedin simple “Tasks to Complete.” Then, the user is shown AR virtual checklists over relevant objects identiﬁed by a machine learning object detection model (ODM). Through the integration ofexpert advice in the task collection engine, and the expert judgement in the ODM as shown in Fig. 3, we expect to improve thetransfer of knowledge from disaster management oﬃces to the
public.
3.1 Virtual Elements’ Familiarity
A checklist has been selected as a control element since it isa familiar form input known by both young and elder audiences,regardless of their familiarity with digital media. With this, we
expect to improve the acceptance of technology in elderly people by reducing perception of technological superiority and thenegative expectation in return of investment of product usage, as
described by Leonardi et al. [11]. Additionally, this element canbe enhanced with optional accessibility features, including:• Increased contrast.
• Bigger font size.

• Diverse color schemes for colorblindness variation.
• Ability of opt-out distracting interface elements to avoid distraction.
As recommended by W3C Web Accessibility Initiative guidelines [18]. A simulation of this AR overlay can be seen in Fig. 4.
3.2 Transfer of Expert Judgement
In the industry context, it has been found that object detectionengines on AR-based applications, are able to reduce errors on
the identiﬁcation of machinery parts during a maintenance process [13]. In our design, a machine learning object detection
model identiﬁes the Targets of Interest (ToI) that disaster prevention guidelines ask the public to pay attention.
Through the integration of a ODM, it is expected a decrease inusers’ mental load of trying to interpret and apply expert knowledge that is otherwise encoded as simpliﬁed graphics and limited
examples in traditional physical guidelines. Since the generalpublic is in principle less experienced than the experts writting
the guidelines, we expect an improvement in the user ability toidentify potential hazards in their everyday living environments.
3.3

Improving Engagement in Disaster Prevention Activities
The engagement of the activity of ﬁxing potential hazards
will be improved through gamiﬁcation elements to indicate theprogress of advancement in securing their environment. Game
like features, such as progress indicators, can be included to leverage the competitiveness attitude that appears during AR educational applications usage. This natural occurring behavior is regarded as undesirable in the classroom [20], but in the household context, it is expected to improve young family members’
involvement in disaster prevention activities.

4.

Expected Contributions

Augmented Reality applications’ knowledge transfer eﬀectiveness has been widely studied in diverse contexts, where the designed and tested applications present a specialized design foreach of such speciﬁc contexts. Through an inductive approachand our implementation of this interface design, we expect toshed light on the diverse features that can improve the eﬀectiveness of the transfer of expert knowledge to a diﬀerent public,
Fig. 4

Simulation of the AR checklist and progress status elements©2022 Information Processing Society Japan

183

「第30回マルチメディア通信と分散処理ワークショップ論文集」 令和4年10月

under-explored on the current literature.
The expected contributions of this design are the following:• Integrate design knowledge produced from the research on
AR applications used in education, maintenance and elderlycare.
• A design suited for applications that in practice, using
datasets with classes of high feature diversity, will be able
to test compresed object detection models generalization capacity.
• A positive social impact by increasing the public engagement and usage of disaster prevention guidelines.
4.1 Knowledge Integration
This design will integrate the knowledge gathered from studies
in the education, industry, and elderly care contexts. The subjects surveyed in those studies, reﬂect the wide age range of thegeneral population, users of this interface.
Consecuently, our design leverages a behavior that in the educational context is considered as undesirable: competition between students to ﬁnish ﬁrst each task. We think this behavior
described by Tzima et al., on using AR applications in the classsrom, will promote the participation of young family members ina household. On the other hand, including the accessibility design considerations found in research on AR applications for theelderly population, we expect to provide a comfortable and familiar experience to the elderly.
4.2 ODM Generalization Capacity
Using this design in a practical implementation in real-world
smartphone applications, will require the use of simpliﬁed versions of state-of-the-art machine learning object detection models. These simpliﬁed versions are a technical compromise, where
their complexity and layers are reduced, to improve their speed
in mobile devices [21]. We expect this interface to be suitable toapplications where an ODM needs to identify everyday objects,
that due their commonplace present a high variation in their appearance. By also targeting the general public, we believe it isa suitable platform for applications that can gather datasets big
and diverse enough to test these models’ feature generalization
capacity.
4.3 Social Impact
Public engagement in disaster prevention activities has beenfound low, in public surverys data [6]. Relevant factors include a
perceived diﬃculty in following their guidance. Through familiar smartphones and tablets, we believe our interface is capableto provide an easier-to-follow guidance, on securing the citizenry
households from inevitable natural disasters.
The integration of design advice from research on AR, in education and elder care, it is expected to improve a family engagement in disaster prevention activities.

will be achieved by three main elements:• Checklists as task completion forms, familiar to all age
ranges of the intended audience.
• An object detection model, to reduce the misidentiﬁcation of
relevant targets.
• Gamiﬁcation elements such as progress indicators, to leverage competitive behavior between young family members.
Integrating the knowledge in current research on AR-based
specialized applications in three main contexts: education, industrial maintenance and elder care.
This novel implementation of a disaster prevention guideline,
will address the issue of current disaster prevention media having,a low penetration in the general population.
Through a following study, we will explore the impact of ansmartphone application, that in particular, targets a public of highdiversity. We will measure and analyze their engagement, theuser experience, and their knowledge retention. Through this research, we will be able to get a broader perspective of the factorsthat aﬀect knowledge transfer, in governments’ civil defense programs by using common everyday mobile platforms.
References
[1]
[2]
[3]
[4]
[5]

[6]
[7]

[8]
[9]
[10]
[11]
[12]

[13]

[14]

5.

Conclusions

In this paper we propose a novel user interface intended to improve the eﬀectiveness of disaster prevention guidelines. This

©2022 Information Processing Society Japan[15]大阪府府庁. 防災・減災ポータルサイト. https://www.pref.
osaka.lg.jp/kikikanri/bousaiportal\_hp/, 2022. [Online;
accessed 28 January 2022].
Tokyo Metropolitan Government. 東京防災. https://www.bousai.
metro.tokyo.lg.jp/1002147/index.html, 2019. [Online; accessed 28 January 2022].
北海道総務部危機対策局危機対策課防災グループ. 北海道防災情報. http://kyouiku.bousai-hokkaido.jp/wordpress/, 2011.
[Online; accessed 1 February 2022].
Yu Xiao and Walter Peacock. Do hazard mitigation and preparedness
reduce physical damage to businesses in disasters: The critical role ofbusiness disaster planning. Natural Hazards Review, 12 2014.
Corinne Peek-Asa, J Kraus, Linda Bourque, D Vimalachandra, J Yu,
and J Abrams. Fatal and hospitalized injuries resulting from the 1994
northridge earthquake. International journal of epidemiology, Vol. 27,
pp. 459–65, 07 1998.
K. Yasumoto S. De León A., Y. Matsuda. Survey study on factors incivil defense participation. 03 2022.
Marion Lara Tan, Raj Prasanna, Kristin Stock, Emma E.H. Doyle,
Graham Leonard, and David Johnston. Understanding end-users’ perspectives: Towards developing usability guidelines for disaster apps.
Progress in Disaster Science, Vol. 7, p. 100118, 2020.
von Hippel Eric. ”sticky information” and the locus of problem solving: Implications for innovation. Management Science, Vol. 40, pp.
429–439, 4 1994.
Stéphane Lévesque. Why can t you just tell us? : Learning CanadianHistory with the Virtual Historian, pp. 43–65. University of Michigan
Press, 2014.
Stavroula Tzima, Georgios Styliaras, and Athanasios Bassounas. Augmented reality applications in education: Teachers point of view. Education Sciences, Vol. 9, No. 2, 2019.
Chiara Leonardi, Claudio Mennecozzi, Elena Not, Fabio Pianesi, andMassimo Zancanaro. Designing a familiar technology for elderly people. Gerontechnology, Vol. 7, No. 2, p. 151, 2008.
Junhan Kong, Anhong Guo, and Jeﬀrey P. Bigham. Supporting older
adults in using complex user interfaces with augmented reality. In The
21st International ACM SIGACCESS Conference on Computers andAccessibility, ASSETS ’19, p. 661–663, New York, NY, USA, 2019.
Association for Computing Machinery.
F. Obermair, J. Althaler, U. Seiler, P. Zeilinger, A. Lechner, L. Pfaffeneder, M. Richter, and Josef Wolfartsberger. Maintenance with augmented reality remote support in comparison to paper-based instructions: Experiment and analysis. pp. 942–947, 04 2020.
Matthias Eder, Maria Hulla, Felizian Mast, and Christian Ramsauer.
On the application of augmented reality in a learning factory working environment. Procedia Manufacturing, Vol. 45, pp. 7–12, 2020.
Learning Factories across the value chain – from innovation to service– The 10th Conference on Learning Factories 2020.
Augmented reality applications for k-12 education: A systematic re-184

「第30回マルチメディア通信と分散処理ワークショップ論文集」 令和4年10月

[16]
[17]

[18]

[19]
[20]
[21]

view from the usability and user experience perspective. International
Journal of Child-Computer Interaction, Vol. 30, p. 100321, 2021.
Matt Dunleavy and Chris Dede. Augmented reality teaching and learning. Handbook of Research on Educational Communications andTechnology, pp. 735–745, 01 2014.
Patrick Perea, Denis Morand, and Laurence Nigay. Spotlight on oﬀscreen points of interest in handheld augmented reality: Halo-based
techniques. In Proceedings of the 2019 ACM International Conferenceon Interactive Surfaces and Spaces, ISS ’19, p. 43–54, New York, NY,
USA, 2019. Association for Computing Machinery.
W3C Web Accessibility Initiative. Developing websites for older
people: How web content accessibility guidelines (wcag) 2.0 applies. https://www.w3.org/WAI/older-users/developing/,
2022. [Online; accessed 21 August 2022].
Yu Fu, Yan Hu, Veronica Sundstedt, and Yvonne Forsell. Conceptual
design of an extended reality exercise game for the elderly. Applied
Sciences, Vol. 12, No. 13, 2022.
Matt Dunleavy and Chris Dede. Augmented reality teaching and learning. Handbook of Research on Educational Communications andTechnology, pp. 735–745, 01 2014.
Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao.
Yolov4: Optimal speed and accuracy of object detection, 2020.

©2022 Information Processing Society Japan

185