BentoMuseum: 3D and Layered Interactive Museum Map for
Blind Visitors
Xiyue Wang

Seita Kayukawa

Miraikan – The National Museum of
Emerging Science and Innovation
Tokyo, Japanwang.xiyue@lab.miraikan.jst.go.jp

Miraikan – The National Museum of
Emerging Science and Innovation
Tokyo, Japanseita.kayukawa@lab.miraikan.jst.go.jp

Hironobu Takagi

Chieko Asakawa

Miraikan – The National Museum of
Emerging Science and Innovation
Tokyo, Japanhironobu.takagi@miraikan.jst.go.jp

Miraikan – The National Museum of
Emerging Science and Innovation
Tokyo, Japan
chieko.asakawa@miraikan.jst.go.jp

Figure 1: BentoMuseum, a 3D and layered design of a museum map that makes information accessible to visually impaired
visitors. (a) All foors can be stacked or separated. (b) A user taps the interactive label, which responds with an audio guide when
the foor is overlayed on an iPad app. (c) A user explores a structural attraction with fngers (a circular walkway named Oval
Bridge which goes around a “globe-like” display named Geo-Cosmos). (d) The Oval Bridge and Geo-Cosmos in the museum.

ABSTRACTCCS CONCEPTS

Obtaining information before a visit is one of the priority needs andchallenges for blind museum visitors. We propose BentoMuseum, alayered, stackable, and three-dimensional museum map that makes
complex structural information accessible by allowing explorations
on a foor and between foors. Touchpoints are embedded to provide
audio-tactile interactions that allow a user to learn the museum’s
exhibits and navigation when one foor is placed on a touch screen.
Using a tour design task, we invited 12 frst-time blind visitors to
explore the museum building, chose exhibits that attracted them,and built a mental map with exhibit names and directions. The
results show that the system is useful in obtaining information thatlinks geometric shapes, contents, and locations to then build a roughmental map. The connected foors and spatial structures motivatedusers to explore. Moreover, having a rough mental map enhancedorientation and confdence when traveling through the museum.

• Human-centered computing → Accessibility systems andtools; • Hardware → Tactile and hand-based interfaces; • Socialand professional topics → People with disabilities.

Author Version

KEYWORDS
information access, 3D structure, audio-tactile, touch screen
ACM Reference Format:
Xiyue Wang, Seita Kayukawa, Hironobu Takagi, and Chieko Asakawa. 2022.
BentoMuseum: 3D and Layered Interactive Museum Map for Blind Visitors.
In The 24th International ACM SIGACCESS Conference on Computers andAccessibility (ASSETS ’22), October 23–26, 2022, Athens, Greece. ACM, New
York, NY, USA, 14 pages. https://doi.org/10.1145/3517428.3544811

1

INTRODUCTION

As audience-centered institutions with a range of educational andsocial roles, museums are, more than ever, aware of the importance of delivering equality, diversity, and inclusion, as well as theirpower to bring about positive change in society [13, 33, 34]. Increased attention is being paid to the development of equal accessand multidimensional access, which not only refers to physicalaccess but also to multi-sensory, intellectual, fnancial, emotional,
cultural, educational as well as information access [2, 12, 24], allof which are important to the visually impaired community. At
present, many barriers impede the attempts of people with visualASSETS ’22, October 23–26, 2022, Athens, Greece

impairments to visit a museum. Alongside exhibition accessibility
and mobility assistance [2, 3], the limited provision of informationand orientation before a visit can lead to a negative overall experience and emotional isolation [2, 10]. Nevertheless, making detailed
and usable information available would create a welcoming andencouraging environment to blind visitors [30].
The museum’s unique environment brings challenges to blind
people in accessing information, diferent from other public spaces
such as neighborhoods, parks, and hallways in a building. First, contemporary museums have distinctive architecture, internal design,and “inter-foor structures” (i.e., stairs and walkways connecting thefoors, see an example in Fig. 1d) as a part of their exhibitions [31, 43].
Blind visitors have difculty grasping the museum’s complex “multidimensional information”, such as the overall geometric shape ofthe building, the inter-foor structures, and each exhibit section’s
name, description, approximate shape and size, and location. Second, even though some museums have a minimal structure witha pre-defned route, more museums contain freely arranged exhibits that may or may not have clear route indication [14]. In suchmuseums, visitors explore the museum and choose the exhibits
they examine based on their interests. By encouraging such “freeexplorations”, these museums efectively trigger a sighted visitor’scuriosity, but conventional posted information may cause blind
visitors access difculties and orientation frustrations.
Accessible maps are the means for visually impaired visitors tolearn about a site. Tactile maps are often available in public spaces
and institutions to provide information and assist in navigation,with the aim of helping the user to build a mental map before goingto a new place [45, 46]. Since the efectiveness and understandability of a tactile map largely depend on the user’s tactile skills andabilities [38], three-dimensional (3D) maps with volumetric symbols
have been developed for ease of understanding. Moreover, audiotactile labels have been proposed for seamless and autonomous
operation. The user is thus largely freed from either shifting attention between the map and the braille legend [25] or asking others for
explanations [16]. The current 3D-printed audio-tactile maps show
thrilling possibilities, but limitations persist. These maps usuallypresent a simple one-foor layout, which is insufcient to support a
structural mental map of a multidimensional museum.
Due to the fact that a museum contains a large amount of multidimensional information, and it is not a frequently visited place,blind visitors might feel it’s particularly challenging to obtain information, orient themselves, and build a mental map. Consequently,
they refuse to do this and give up the idea of a self-reliant visit. How
can museums that have a complex 3D structure and freely arranged
exhibitions provide information access to blind people before avisit? To bridge the gap between museums and blind visitors interms of information access, and to investigate the suitable formatof an accessible and inclusive museum map, the following researchquestions emerge:• RQ1. How can we make the vast amount of needed information (e.g., architecture and interior structures, exhibits,
facilities, locations, and route-fnding) accessible and understandable on a museum map?
• RQ2. Is building a mental map possible and signifcant in themuseum context?

Wang, et al.

Previous literature either developed foor plans in isolation [20,29] or reproduced external structures [26, 50], in which complex
multi-foor structures such as museums were rarely explored. Our
core concept and innovation is the design of stackable 3D foors to
capture the complex multidimensional nature of a museum. The
multidimensional information includes the external and internal
structures, exhibits and facilities, and their locations as well as
route-fnding. Using a participatory and user-centered approach,
we designed BentoMuseum, a 3D and layered museum map withaudio-tactile interactions, to support blind users in obtaining information and understanding the 3D attractions through tactile
explorations (Fig. 1). The system contains two main elements: the3D and layered foors (Fig. 1a), which can be either interlocked toallow vertical exploration between foors (Fig. 1c) or separated tosupport horizontal exploration of a single foor; interactive touchpoints on the foor that allow audio feedback by touch (Fig. 1b).
When one foor is placed on a touch screen, diferent levels of information and tactile navigation with audio support can be triggeredby tapping. Novel designs we propose include stackable foors, 3D
and 2D attributes that represent diferent types of contents, andsimulated navigation by tracing paths and intersections.
We invited 12 participants with severe vision impairment to bemuseum tour designers and instructed them to use the system aspart of an authentic museum tour. We let them explore as muchas possible, obtain information, choose exhibits of their interest,
and try to build a mental map. Participants expressed their map
exploration styles and elaborated on their needs for information
access before, during, and after the tour. Our results suggest: (1)Using the system, the participants were able to actively obtain
information that links shape, location, and content. Consequently,
they were able to choose exhibits of interest and build a roughmental map. (2) Touching inter-foor structures motivated blindusers to explore the museum map. Along with the navigation, it
supported them in building a 3D mental map. (3) Building a roughmental map beforehand was benefcial for the subsequent visit. It
provided orientation, enhanced the sense of safety and confdence
that they would not get lost while traveling through the museum,and led to a positive and inclusive museum experience.

2 BACKGROUND AND RELATED WORK
2.1 Museum Accessibility
Museums are not only institutions for the collection, preservation,and display of valued objects but also audience-centered spaces
with a wide range of social roles and responsibilities [2, 6, 27, 48].
Sandell suggested that museums should contribute to social inclusion on individual, community, and society levels by supporting
creativity and confdence; empowering independence, decisionmaking processes, and democratic structures; fostering acceptanceand respect; and challenging stereotypes [41].
However, barriers exist when people with visual impairments
attempt to visit a museum. It was noted that a blind person’s visit
experience begins even before entering the museum site [48]. The
provision of information has been a priority need in terms of service accessibility [11, 23, 48]. A United Kingdom survey concluded
that the basic accessibility information provided by a majorityof museum websites could not address the access needs of blindBentoMuseum: 3D and Layered Interactive Museum Map for Blind Visitors

or partially sighted persons who want to plan an independent
visit [10]. Argyropoulos et al. found that complex museum architecture/interior design also hindered access to a museum. The overall
inaccessibility led to a lack of motivation and negative emotions [2].
It was noted that providing haptic, touchable, and multi-sensory
objects is one of the most anticipated and efective ways of improving museum accessibility [2, 9, 51]. In terms of accessing information and orienting visually impaired visitors, Vas et al. suggested
presenting the museum space and exhibition through audio at the
beginning of the visit [49]. Tactile maps and 3D models have also
been developed to the needs of visually impaired visitors. Urbas etal. explored 3D printing techniques to develop physical foor plans
to be mounted on the museum walls for touch [47]. Holloway et
al. created 3D maps with distinct icons for an event, and they suggested that large maps require time to explore and thus should bemade available before an event or at the entrance in a comfortableenvironment [26]. Leporini et al. pointed out that being able toexplore and familiarize themselves with the structure and details ofa large cultural site is crucial for orienting visually impaired people.
This allows them to gain a global understanding and an overall
impression [29].

2.2Accessible Maps for the Visually Impaired

Maps that enable tactile explorations are designed for the visually impaired to learn the environment. One of the most common
methods is tactile maps created with a set of accessibility guidelines [4, 35, 36]. Such maps represent features with raised lines, symbols, keys, and orientation [38, 55], often with the same elevation(around 0.5mm) on swell paper or a greater height diference (up to
2-3cm) when thermoformed [25, 39]. Most tactile maps are createdfor information provision in a digestible form that can be easily
understood [39, 40, 55] and for mobility and navigation [25, 40, 46].
Although they are commonly used, tactile maps propose a set ofchallenges to visually impaired users. They are limited in creating
3D structures and a variety of heights [25]. The understandability ofa tactile map also largely depends on the user’s tactile ability [38, 40]
and training in the skill of reading tactile graphics [1, 45, 46]. These
maps also contain a relatively small amount of information, and normally a large area needs to be divided into diferent sections [38, 55].
Furthermore, the tactile maps need supplementary means to accessthe information, for example, a symbols glossary or legend [38].
To mitigate the understandability issues, research has exploredmaps with more distinct 3D structures. Voigt and Marten suggested
the use of 3D models of buildings to facilitate spatial orientation
and build a mental map [50]. Leporini et al. developed foor plansof indoor monuments of a cultural site to help both visually impaired and sighted people explore and familiarize themselves withelements before a visit [29]. Gual et al. found it was easier to memorize 3D volumetric symbols than 2D symbols [18]. Comparing
two tactile maps, one with only 2D elements and the other also
including volumetric symbols, Gual et al. found the use of 3D volumetric symbols signifcantly reduced both location-fnding timeand discrimination errors [19]. Holloway further compared thereadability of diferent 3D icons on a map and proposed guidelines
for icon design [26]. Pistofdis et al. tested a number of parameters
related to 3D shape and haptic performance [37]. Holloway et al.

ASSETS ’22, October 23–26, 2022, Athens, Greece

compared 3D printed maps with tactile maps and found that 3D
maps were preferred. The use of more easily understood icons andrelative heights of map elements facilitated improved memory inshort-term recall [25]. Gual et al. designed urban maps containing
both volumetric attributes and relief attributes and demonstrated
the value of the maps in terms of interpreting, memorizing, andunderstanding. However, they also found that the maps required
verbal support to be used autonomously [20, 21].

2.3

Touch Sensing and Audio-Tactile Labels

Audio-tactile maps using touch sensing or buttons have been proposed to further support understanding in addition to the tactile
sensations. Brock et al. designed an interactive map composed ofa multitouch screen, a raised-line overlay, and audio output. Compared with tactile maps, they showed that replacing braille withaudio-tactile interaction signifcantly improved efciency and satisfaction [7]. Comparing tactile maps with interactive small-scalemodels (SSMs) for learning, Giraud et al. showed that the interactive SSMs improved both space and text memorization and werealso adaptable to diferent situations and needs [16]. Several studies demonstrated that perceptible buttons that invoked diferent
levels of audio content promoted an interactive, autonomous exploration [25, 29] and increased emotional engagement [52]. Utilizing
printing technologies, automatic creation of 3D printable tactile
graphics, and touch screens, research has developed the ability to
instantly produce tactile-audio representations on a printed map
by implementing the map on a touch screen [17, 44, 53].
The previous literature has built a strong foundation for developing 3D maps with audio-tactile labels. However, those works
focused on one-foor settings with relatively small amounts of information. To the best of our knowledge, few research works have
explored accessible maps for complex multi-foor structures. We
fll this research gap by proposing stackable foor maps to accessboth the internal and external structures of an entire museum.

3

PARTICIPATORY SYSTEM DESIGN

The design concept is implemented in a science museum, Miraikan– The National Museum of Emerging Science and Innovation1 , with
distinctive structure and symbolic interior attractions. It is a sevenfoor building with a large-area atrium (with the 2nd, 4th, and 6th
foors mainly atrium space) and structural attractions such as aseries of escalators that directly connect all the foors (Fig. 3a), a
walkway called Oval Bridge that goes around a “globe-like” display
named the Geo-Cosmos (Fig. 1d)2n, and a Dome Theater with halfof it inside the building and the other half extended into the exterior(Fig. 3b). It also lacks maps that can be perceived by touch.
We employed a participatory and user-driven methodology todesign a map adapted to the museum. The design sessions include
seven interviews with the blind designer (once in prototype 1, threetimes in prototype 2, and three times in preparing for the fnaldesign), one event that involved twenty blind museum visitors
and three staf members, and one group meeting with those stafmembers.

1 https://www.miraikan.jst.go.jp/en/ASSETS ’22, October 23–26, 2022, Athens, Greece

Wang, et al.

Figure 2: The printout of an early iteration and the fnal design. (a) The realistic model of the 3rd foor that was used inPrototype 1 to start the discussion (Section 3.2). (b) The 3rd foor of the fnal model, with simplifed designs, audio-tactile labels
painted in black, and control buttons on a touch screen. (c) The fully stacked fnal model that shows the external structure.

3.1

Motivation: “What if I can open the modeland get information about the foors?”

One of the designers, P0, is an adult female who has complete
blindness, as well as being an interaction designer and researcher.
After being presented a 3D model of the museum, she expressed theneed to understand the interior: “I have heard about the symbolic
globe-like display and the Oval Bridge around it. But it’s so hard
to imagine them just through descriptions. I wish I could open themodel and touch them” 2 . This was the initial attribute of the map wehoped to investigate: a 3D model that contains internal structures.

3.2

Prototype 1: Feedback of a Realistic ModelAn early version of the map was a realistic 3D print of the museum
foor (Fig. 2a). We sliced the 3D model into foors and encapsulated
the detailed information such as the walls and tables at each exhibit
to a 24cm×14.5cm×1.2cm miniature foor. A tactile map resembling
the 3D map’s layout was developed and printed on swell paper
for comparison. During a two-day event called Inclusion Week,
two maps along with other 3D prints were explored in the wildby 20 blind visitors, for 5 to 10 minutes each person. From theircomments, we learned the following needs to satisfy in developing
an understandable map:
• Content: Simplifed and categorized forms were needed.
Users highly praised the understandable form of the Oval
Bridge on the 3D map but also pointed out that the detailed
depictions of exhibits were not digestible.
• Tactile exploration: A relatively smooth surface without
acute edges was preferred. Small and pointy objects (i.e.,
walls and tables) on the 3D map hindered hand scanning.
• Explanations: Automated audio-tactile interactions weredesired. Both maps were not understandable unless the museum staf gave explanations.
The two maps were then tested by P0 during a 30-minute interview.
Further requirements were confrmed based on her knowledge ofthe museum and expertise in design:
2 All communication with the participants was in their native language. In this paper,
we present any translated content in the form of “translated content.”• Facilities: While the current maps were highly focused on
the exhibits, the basic map elements such as restrooms, elevators, and escalators also needed to be included.
• Orientation and navigation: The map should support identifying the entrance, main route, each exhibit, and how tomove around the space. These elements support the development of a “mental map,” which is crucial for blind people.
The feedback revealed that the 3D map was good at delivering
structural impressions while the tactile map preserved scanning,
thus confrming the previous research [25]. Previous literaturesuggested 3D maps with audio-tactile labels ofer clear advantages,
including understandability, memorization, and efectiveness, all ofwhich are important for museums. Nevertheless, we did not fnd
a clear advantage to using the tactile map for the museum, so wewere motivated to focus on 3D maps. We utilize the advantagesof both volumetric attributes and relief attributes, learning fromprevious practices [18, 19, 25, 26, 29] while making our own design
innovations and adaptations.

3.3

Prototype 2: 3D Floors and Audio-tactile
Interactions

Based on the feedback, we categorized the museum’s multidimensional information into the following three types of information,
and we provided design criteria for each of them:• Structural attractions include inter-foor structures andsymbolic spatial structures (Fig. 1c, 1d, 3a, 3b). Our design
choices include: (1) Simplifying structures into primary forms
with understandable relative scales. For example, the parallel
escalators and stairs were made simpler into one slope with
textures (Fig. 3a). (2) Simplifying prominent walls into 1mm
tall and 3mm wide cuboids. (3) Embedding magnets to support easy stacking and lining up of foors, which has proved
to be efective in developing 3D objects for the blind [15].
Floors can be partially stacked to simulate how to walk
between them (Fig. 3a) or fully stacked to show a facade(Fig. 2c).
• The exhibits included booths, wall-divided spaces, and artifacts placed in open spaces. Our design choice was to simplify
them into outlines that were proportional to the real space

BentoMuseum: 3D and Layered Interactive Museum Map for Blind Visitors

ASSETS ’22, October 23–26, 2022, Athens, Greece

Figure 3: Designs for diferent types of information. (a) The escalators and stairs run parallel in the museum (left) and theirrepresentations on several foors (right). (b) The Dome Theater (left) and its representations on several foors (right). (c) One
exhibit area (left) and it’s representation on one foor (right). (d) Eight symbols that represent museum facilities.
they took (Fig. 3c). This design supports clear separation, differentiation, and scanning. The outlined shape was hollowed
to enable audio-tactile interaction (described later).
• The facilities in the museum were summarized into eightfrequent items. We represented them using volumetric symbols (Fig. 3d), with design guidelines from previous work
[18, 19, 26]. For those facilities that take a large space (e.g.,lobby and restaurant), their outlines were hollowed out toshow the area and enable touch interaction.

touchpoint (see an example in Fig. 1c). To tactually distinguishhollowed interactable exhibits from the atrium, we attached a paper with textures on the back of the foor (Fig. 2b). An app thatprocesses touch and provides voiceover information was developedin Unity. As learned from the previous work [7] and through ourown testing, we mainly adopted a double tap as the recognizedtouch to prevent accidental triggering during the exploration. Twomodes were developed to serve the needs of free exploration androute-fnding:To support orientation and navigation on the map, we further defned paths and intersections to indicate how the user can travel.

• In the Exploration mode, double-tapping a touchable area
triggers the audio explanations.
• In the Navigation mode, the user double-taps two exhibits
to select the destination and the start. A route with a start,
destination, and a number of intersections in-between is
generated. Next, The user is instructed to move a fnger
to the entrance of the start place, which is the location in
the exhibit reached by the path. Once the user moves there(without any tapping), she is directed to trace the path to thenext location of the route until she reaches the destination.

• The path is similar to the tactile paving [32], which suggests
a route on open ground. According to the actual layout andfow, we defned a main path in the center of the exhibition
space and sub-paths as routes that connect each exhibit’s
entrance to the main path. All of the paths are represented
by 1mm wide embossed lines (Fig. 2b).
• The intersection is represented as a 10mm × 10mm hollowed square located at the crossing of the paths, which isdistinguishably smaller than the exhibit areas (Fig. 2b).
To automate the explanations with diferent levels of detail, we
implemented audio-tactile labels using capacitive sensing on atouch screen. A 12.9-inch iPad Pro was used as a platform to sense
touch. When a foor is placed on it, a touch can be sensed directly on
the hollowed exhibits. On a structural attraction with a geometricshape (Oval Bridge, Geo-Cosmos, Dome Theater), the audio-tactile
label was implemented by redirecting touch from the screen to thesurface of the shape using conductive ink, following touch screen
redirection technical guidelines [42, 54]. A 3.5mm wide tube wascut out in the geometric shape, flled with the conductive ink, and
had its top and bottom painted with conductive ink. We also pasted
a 4mm wide circular tactile sticker at the center to indicate the

The fnal foors are created with a stacked area of approximately
32cm × 20cm × 13cm stacked, and 2.5cm tall, 1.5mm thick each foor
(Fig. 2c), which was at a 1/400 scale of the actual museum (seespecifc sizes and details in Fig. 4). This is the largest size that can
ft onto an iPad to support audio-tactile exploration. It is designedin Autodesk Fusion 360, and printed with Formlabs Form 3L SLA
3D printer, using Clear Resin material.

3.4

Final Design: Content and Customization

We then conducted a 1.5-hour group meeting with 3 museum stafmembers, who are not only profcient as museum guides but also
experienced in guiding blind users. We decided to include the following contents:

ASSETS ’22, October 23–26, 2022, Athens, Greece

Wang, et al.

Figure 4: The layout of all the 3D foors. The x and y axes are in cm. Diferent colors marked the foors, facilities, and paths.
• The audio guide for the 3D structure or the exhibit,
which speaks at one of two levels of detail in turn whentapped. The frst level contains name, keyword (e.g., universe, earth, life), and accessibility info (e.g., “Over there, you
can touch a 3D model of the rocket engine.” ) The second levelcontains a 15-second description about it.
• The audio guide for an intersection, which speaks thesurrounding information when tapped (e.g., “This intersection
is connected to an earth-type exhibit on the top and a universetype exhibit on the bottom. Eight exhibits are on the left. Fiveare on the right.” )in Table 1. They were recruited via an e-newsletter for people withvisual impairments, and compensated $75 plus travel expenses fortheir time. All of them were frst-time visitors who held minimalpreset knowledge about this museum where the study took place.
Six participants were frequent museum visitors who visited othermuseums more than once a year. Three visited other museums
every two to three years, and three rarely visited a museum. Allof them had experience with tactile materials, including tactile
graphics (P2, P4–P12) and 3D models (P1–P11).
Table 1: Demographic information of the participants.

The following updates were made to enable user customization:• Three physical buttons (stop the voiceover, modify the speed,and change Exploration/Navigation mode) were developed.
They are clipped onto the iPad and can be triggered at any
time using a double-tap (Fig. 2b).
• In the Navigation mode, the route explanation style can beswitched between the turn-by-turn (default) and the northup navigation.
In summary, all elements in our proposed map are as follows:(1) the 3D and layered foors, which include inter-foor structural
attractions, the outlined exhibits, and facilities shown by volumetricsymbols; (2) the audio-tactile interactions, which include the twolevel audio guide of exhibits, an audio guide at the intersections,and navigation by tracing the paths and intersections.

4

USER STUDY

We conducted a user study at the science museum1 to investigate
our research questions and evaluate the efectiveness of our proposed system. The staf who joined the fnal design process (Section 3.4) stressed that visitors came with diferent interests andexpectations. A fxed task and a rigorous evaluation of the performance might discourage the participants, who are also important
stakeholders. We came to agree that a tour design task should be
fexible to refect diferent user styles and support curiosity andautonomy, which are the museum’s important social roles. We included a tour after the map exploration to help visitors generate
feedback towards a real museum visit. Each individual study took
two to three hours in an order of tour design, conducting the tour,
and post-tour interview.

4.1

Participants

We recruited 12 blind participants (male = 5, female = 7) with ages
ranging from 24 to 71 years old (mean = 53.8, SD = 13.1), as listed

4.2

ID

Age

Blind
since

Navigation
Aid

Visiting
other Museums

P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12

58
49
63
60
42
49
57
24
71
43
61
68

41
45
50
45
32
16
53
3
60
3
56
35

Guide DogWhite Cane
Guide DogWhite CaneWhite CaneWhite CaneWhite CaneWhite CaneWhite CaneWhite CaneWhite Cane
Guide Dog

Once every 2–3 years
2–3 times/year
Once every 2–3 years
Once every 2–3 years
2–3 times/yearNever
2–3 times/year
Once/year
2–3 times/year
A few times
4 times/year
Never

Task and Procedure

4.2.1 Pre-Interview. The frst part of the study, tour design, took
place in a guest room located on the frst foor of the museum.
Before presenting them the system, we conducted a roughly 10minute pre-tour interview, hearing about their tactile experienceand previous preparations before going to other museums.
4.2.2 Structural Exploration. Next, we presented the fully stacked
BentoMuseum 3D model and allowed the participant to explore
freely by touch as we introduced the basic external structures. We
informed participants that the museum had the shape of a boat, andthe front “bow” should be kept on the left-hand side for a consistentorientation. We then introduced the “Bento Box” characteristics
and encouraged the participant to take the foors apart one by one.
Next, the foors were stacked back one by one, and we encouraged
the participant to touch the inter-foor structures (e.g., escalators

BentoMuseum: 3D and Layered Interactive Museum Map for Blind Visitors

and Oval Bridge) to learn how foors are connected. Finally, theparticipants were primed with a list of 10 icons (eight in Fig. 3d,
escalator and wall), which is separately prepared on a sheet. This
phase took roughly 10 minutes.
4.2.3 Training Phase. A training phase was conducted to familiarize participants with the audio-tactile interactions. The participant
was shown the 1st foor map on the touch screen in Exploration
mode. The following steps were taken: (1) double-tap the special
exhibit zone to listen to the name; (2) double-tap it again to hear
the details; (3) double-tap the intersection to hear information ofsurroundings; (4) fnd the guest room by double-tapping; (5) doubletap the speed buttons to adjust voiceover speed; (6) double-tap
the cancel button to stop the audio; (7) double-tap the navigation
button to change to the Navigation mode; (8) double-tap to set the
special exhibit zone as the goal and the guest room as the start, andtrace the route following the audio guide. This entire process tookaround 10 minutes.
4.2.4 Tour Design Task. A loosely structured tour design task was
conducted. The individual participant was asked to imagine thefollowing real-world scenario: The system is placed at the entranceof the museum, and they are using this system to select the exhibitsof interest and design a unique tour for themselves. With the helpof the staf, they can place any foor on the touch screen. From a
total of 28 exhibits, they were asked to select 6 (equivalent to a twohour tour) and to try to build a mental map with routes connecting
the spots within 45 minutes. Considering the real-world scenario,
they were also free to take notes. During the task, a researcher was
taking notes of the selected spots for later evaluation. We video andaudio recorded the session and saved app log data for later analysis.
When time was up or the participant was fnished, they wereasked to orally explain the (1) name and (2) orientation and locationof each spot. Based on their explanations after the task and during
the tour, we determined which level of mental map they possessed.
In this study, we defned fve levels of the mental map:
• Level 1: Hardly remember any exhibits they chose.
• Level 2: Remember some of the exhibits they chose.
• Level 3: Remember all of the exhibits they chose.
• Level 4: Remember all of the exhibits they chose and which
foor each exhibit is on.
• Level 5: Remember all of the exhibits they chose and thelocation of each exhibit.
We also asked the participants to give a self-evaluation of whatlevel of the mental map was needed.
4.2.5 Conducting the Tour. To validate their mental map in a realworld setting and gather feedback on the important factors before
visiting the exhibitions, we invited the participant to experience thedesigned tour. We shortened the two-hour tour to approximately 15
minutes, the time allotted for walking over the designed tour witha museum guide and listening to elaborated guidance of a chosen
exhibit to get a taste of the museum. We encouraged participants toconcentrate on validating their mental map, and they were allowed
to fully explore the museum after the study.
4.2.6 Post-Tour Interview. A roughly 30-minute interview was conducted after the participant was settled back in the guest room. The

ASSETS ’22, October 23–26, 2022, Athens, Greece

interview included two forms: a seven-point Likert rating, fromstrongly disagree (score = 1) to strongly agree (score = 7), andfree responses. Four sections composed the interview: (1) rating
the overall experience of using the system (Q1–Q3 in Fig. 5); (2)rating the overall system usability (Q4–Q6 in Fig. 8); (3) rating the
3D foors and audio-tactile interactions, which were further divided
into eight specifc elements in terms of A. understandability and B.
usefulness (Q7.A–Q14.B in Fig. 9); (4) free responses about using themap prior to the visit, the strengths and limitations of the system,
applications, and other fndings after the tour. For all of the ratings,
we asked participants to consider or imagine accessing information
by audio means, such as reading a homepage when preparing for avisit, as a baseline (score = 4).

5 RESULTS
5.1 Preparations before Going to the Museums
Eight out of nine who had visited museums said they would gather
information ahead of time through web pages and other means.
The information they hoped to collect included exhibit information(all), foor information (P2, P3, P7, P8, P10, P11), the detailed routeto get to the exhibits (P7, P8, P11), and facility information such as
restaurants (P4, P5) and opening hours (P5). Two participants said
it was difcult to acquire information they needed through web
pages, so they made calls (P5, P7).

5.2

Performance of Information Access5.2.1 Overall Experience. All participants successfully fnished thetour design task within the allowed time (45 minutes). The ratings
related to information access through the task are summarized inFig. 5 (Q1–Q3). The participants strongly agreed that by using thesystem they could get an overall image of the museum (median =7). They also agreed that they were able to grasp the details of themuseum (median = 6). All participants agreed (median = 7) that it
is important to decide on their own where to go. Participants were
excited about having good control of information and being able todesign the tour independently based on their own interests:
A1:“The museum visits are precious parts of my life. I really
don’t want to miss anything interesting. Thus I want both independent exploration and recommendations."
P9
A2:“It might be nice if there were a recommended course, but I
would still like to explore it myself. There is a sense of security
to control where I go."P11
5.2.2 User Exploration Styles. By analyzing the double-tap log data
during the Exploration mode, we identifed several hand movement
styles when the participants were exploring the foor. Research has
found that touch readers were taught to frst systematically scan ina circular pattern, and this efective scan strategy was used withboth tactile map and 3D models [25]. We were interested in learning whether did they naturally perform this exploration strategy.
Although all participants had tactile experience, we identifed thatthree participants (P8, P9, P12) under-explored the foor (see example in Fig. 6a). They only touched some of the exhibits, and nocircular pattern was formed. Three participants (P3, P5, P11) overexplored the foor (see example in Fig. 6b). Their tapping covered
most exhibits, but their fngers traveled randomly by long distances

ASSETS ’22, October 23–26, 2022, Athens, Greece

Wang, et al.

Figure 5: Questionnaire results of the overall experience of our system (Q1–Q3).

(a) Under-exploration (P8, 3rd foor)(b) Over-exploration (P3, 3rd foor)(c) Typical (P4, 3rd Floor)(d) Ideal (P4, 5th foor)

Figure 6: Diferent exploration styles. The location of frst level touches and their sequences during the Exploration mode areplotted on a 2D foor map. The arrow indicates the consequence, and the color shows the touch order.
on the map. A clear, circular pattern was hardly observed. The restof the participants showed a typical style (see example in Fig. 6c).
Even though they had slightly long traveling, they were able to build
a relatively circular and systematic pattern. In latter exploration,
one participant (P4) also showed an ideal style (see example inFig. 6d) with a very systematic circular pattern. After one exhibit
was explored, he moved to the closest exhibit. This style exhibitedthe “circular and complete” scanning strategy, which was noted as
one of the most efcient strategies for exploring a map [25].
We next analyzed the relationship between the identifed styles
and the quantitative performance, and the latter was used to identify

exploration efectiveness on tactile graphics [5]. These quantitative
measures include coverage percentage of the exhibitions, tap counts(perceived as a more reliable indication of time in our case comparedto the clock time), and exploration distance. We computed a Univariate ANOVA with a Bonferroni Pairwise post-hoc test to compare
the results. Mean coverage for under-exploration was signifcantly
lower than those for typical (p < .01) and over-exploration (p <.001) (Fig. 7a). There was no signifcant diference among the threestyles in terms of tap count (Fig. 7b). Mean exploration distancefor over-exploration was signifcantly higher than those for typical(p < .01) and under-exploration (p < .01) (Fig. 7c). The results show

BentoMuseum: 3D and Layered Interactive Museum Map for Blind Visitors

ASSETS ’22, October 23–26, 2022, Athens, Greece

Figure 7: Mean (a) coverage, (b) tap count, and (c) exploration distance for the typical, over-explored and under-explored groups.
p: p-value of the Univariate ANOVA with a Bonferroni Pairwise post-hoc test (*** and ** indicate the 0.001 and 0.01 levels ofsignifcance, respectively).
that among all the styles, those participants who hold a typical style
are having high coverage and short exploration distance, whichindicated relatively efcient explorations.

5.3

Performance of Mental Map Building

Among twelve participants, nine participants (75%) remembered
all the exhibits they chose and their locations (level 5), two (16.7%)
remembered the exhibitions and their corresponding foors (level4), and one (8.3%) remembered a part of the chosen exhibits (level2). Level 5 participants could explain the general location of eachexhibit (e.g., “Exhibit A is on the upper-left side of the 5th fooror “Exhibit B is located to the left as you exit Exhibit C), and wedetermined that they had built a “rough” mental map.
All participants noted that there could be a clear diference between with and without a mental map, and some commented that
3D and layered foors and the Navigation mode of the proposed system were efective for building a 3D mental map (see Section 5.4.2
and Section 5.4.3). The participants noted that there was an improvement in orientation (P2, P6, P9, P10) and confdence that they were
safe and would not get lost (P7–P9, P11) with a rough mental map
during the visit, compared to their previous museum experiences
without a mental map.
A3:“ Without the mental map, I didn’t understand where I waswalking. Controlling where I was going using the map, I walked
with a sense of accomplishment, and everywhere I went became
a lot more fun."P2
A4:“ Without a rough mental map, it just feels like being pulledaround and it’s simply boring and tiring."
P9
A5:“ If I don’t have a mental map [before following a course], I
don’t remember where I went, I don’t know how long I will walk.
Now when I notice where I am, I can calculate back from themental map, and I feel a completely diferent level of security." P8
A6:“ It feels safe to decide where I go, understand the relative
locations, and have a structure of the museum in my mind. It
makes the tour and the discussion easier. I might make mistakes
about the route, but I can soon integrate the new informationand easily correct my map."
P7
On the other hand, all participants contented themselves with thecurrent level of the mental map they built. The participants thoughtbuilding a higher level of mental map, which means remembering
the route clearly, would be unnecessary for the following reasons:(1) the museum is not a frequently visited place (P4); (2) there is toomuch intellectual information to remember (P9, P11, P12); and (3)they felt the complex environment is not yet ready to allow themto travel alone (P1, P5, P6).
A7:“It was easier to remember the routes than the exhibit
names. I wish the names could be replaced with another format,
such as numbers.”
P9

5.4

System Usability

5.4.1 Overall Usability. The results of three ratings related to usability are summarized in Fig. 8 (Q4–Q6).
All participants agreed that the system was useful (Q4, median= 6.5) and enjoyable compared with getting information from thehomepage (Q6, median = 6.5). In particular, seven participants (P1,P3, P4, P5, P7, P10, P12) commented that linking geometric or
outlined shape, location, and content using the 3D model and audiotactile labels was both efective and enjoyable.
A8:“I’m not good at building mental maps, but I somehow
managed to do it by touching and listening."
P1
A9:“Processing the audio-tactile information, I could understand the content and arrangement of the exhibit booths." P10
A10:“Compared to merely reading the homepage, touching andlistening made me excited about the following trip."P4
Three participants (P2, P11, P12) were also excited about the independence they obtained in the exploration.
A11:“The best thing is that I could explore independently without asking for help."
P11
The participants other than P12 leaned toward giving a ratingthat the system was easy to use (score equal to or greater than 5).
Six participants commented that the double-tap was not easy at
frst (P2, P4, P6, P8, P9, P12), which infuenced their score on Q5.
The participants pointed out that there was a learning curve, andit largely depended on the user’s profciency with mobile devices
and voiceover controls.
A12:“I wasn’t used to double-tapping and didn’t know where
I could tap at frst. But I got profcient after I spent more timewith it."P6
Three participants hoped to use a more explicit and seamless touch
to trigger the audio (P2, P6, P8), although they acknowledged thata single-tap would trigger unnecessary sounds (P2, P8).
A13:“The double-tap also reacted to other fngers during exploration. I think touch with a stronger force is better thandouble-tap. It would respond to more conscious movements." P6

ASSETS ’22, October 23–26, 2022, Athens, Greece

Wang, et al.

Figure 8: Questionnaire results of the overall usability of our system (Q4–Q6).

Figure 9: Questionnaire results of the usability of 3D and layered foor-related elements (Q7–Q10) and audio-tactile interfacerelated elements (Q11–Q14). A: [The element] is easy to understand, and B: [It] is useful in exploration and tour design.
5.4.2 Usability of 3D and Layered Floors. The results of ratings
related to 3D and layered foors are summarized in Fig. 9 (Q7.A–
Q10.B). In general, all 3D and layered foor-related elements wererated as understandable and useful (median >= 6). The fully stacked
3D building (Q7) and partially stacked 3D interlocking foors (Q8)received especially positive ratings (median = 7). The participants
felt it was especially benefcial and enjoyable to be able to stack andtouch structural attractions in the building, such as the Oval Bridge,
the Dome Theater, the atrium, and the series of long escalators (P2,
P4–P8, P10). They noted that they were attracted by the “Bento”
characteristics, which motivated them to learn structural details.
A14:“When building a mental map before, I could only make a
fat map for one foor. But using this system, I had a stronger
impression of 3D movement. I was so excited to walk [with afnger] in the 3D space."
P4
A15:“By exploring the structures in order, such as the entrance
and escalator, I feel like I was walking in the museum. It gaveme the sense of being immersed into the museum."
P6
A16:“The building can be separated, and it is easy to understand the details. In the case of a tactile map, it is difcult tounderstand how to move from foor to foor."
P8

Participants commented that touching the structural attraction’s
geometric shape on the model was the best way to understand its
actual structure (P2, P12). As a result, all of the participants chose
the Oval Bridge to be a part of the tour they designed.
A17:“It would be impossible to understand the structure of theOval Bridge without the 3D model."P7
About the outlined exhibits, not every participant associatedthem with the actual size and outline of the exhibition area. However, when they noticed the association, they were very positive
about this kind of information being provided.
A18:“One exhibit was a narrow and long chamber. When I wentin, I was like ‘That’s it!’ I remembered the shape clearly withmy fngers. The impression would not be that strong if I had
only heard from the voice guide that it was narrow and long."
P4
The volumetric symbols were understandable but not perceived
as especially useful due to the fact that the task was designing a tour.
They acknowledged that even though it was not very much used, itwould be absolutely important in the actual visit (P4). Participantsalso noted that facilities, especially the restroom and front desk,
possibly needed touch interactions (P4, P10, P11).

BentoMuseum: 3D and Layered Interactive Museum Map for Blind Visitors

A19:“I want to know more about the ticketing information andwhere the fush button is in the restroom."
P11
The study did not fnd particular challenges in stacking andorienting the foors due to the model’s irregular shape, the magnets’
support of a specifc lockup, and the maintained model orientation.
However, some participants reported difculties in recognizing
separable 3D structures. They noted that even though the interfoor structures (e.g., Oval Bridge and escalators) on the bottom
foor were self-explanatory, it was difcult to locate them on the
top foor alone (P1, P3, P6).
5.4.3 Usability of Audio-tactile Interface. The results of ratings
related to audio-tactile interactions are summarized in Fig. 9 (Q11.A–
Q14.B). The exhibition’s two-level audio information (Q11 andQ12) and Navigation mode (Q14) were rated as understandable anduseful (median >= 6.5). After obtaining information, all participants
included at least one exhibit with accessible content in their tour.
Two participants (P4, P11) chose the north-up navigation style, andthe rest used the default turn-by-turn style. Participants mentionedthat the navigation was helpful for them to develop the route intheir minds (P1–P4, P7, P11) and that using a fnger to trace theroute was an enjoyable experience (P1, P3, P4, P7, P10).
A20:“Thanks to the Navigation mode, I could learn relative
locations and build a mental map."
P1
A21:“It was fun to grasp the location relationship by tracing theroute with a fnger."
P3
Nevertheless, the need for the Navigation mode might dependon the tactile and orientation ability:
A22:“I don’t need the Navigation mode now because the layout
is easy. I can understand and remember it just by touch." P8
Participants somehow agreed that double-tapping the intersection was easy (Q13.A, median = 5) and useful (Q13.B, median =4.5). Four participants reported that the intersection was small fora double-tap (P5, P7, P9, P10). In terms of usefulness, we received avariety of comments and suggestions.
A23:“I can touch the exhibit to learn the needed information,
thus I didn’t use the intersection."
P8
A24:“I might have touched a number of exhibits, but I don’tknow which are untouched. I hope the system tells me what area
hasn’t been explored. Probably it is best at an intersection." P6
5.4.4 Free Comments and Suggestions. Participants freely expressed
where they wanted to use the system. The answers are categorized
as follows: locations containing many points of interest, such as
museums (P2, P5, P6, P7, P9, P11), amusement parks (P5, P11, P12),and department stores (P5, P9, P11); large and complex places, suchas convention halls (P6) and airports (P2, P10); and frequently visited places, such as train stations (P3, P8, P10, P11), hospitals (P1,
P11), city halls (P4, P11), schools (P4, P6), and in the train (P4).
Participants also raised a variety of hopes and suggestions ofwhat the system could ofer.
A25:“I want to bring the system with me, and let it explain
things to me just like a tour guide. Like ‘We arrived here. It
is about...’ I hope we can hold a discussion in-depth about theexhibition. It would also be nice to tell me where it is crowded
and where it isn’t."
P5

ASSETS ’22, October 23–26, 2022, Athens, Greece

A26:“The museum has a shop, right? I want to learn what are
popular souvenirs. I also want to know the restaurant menu. I
want to hear a lot of options in this museum!"
P11
A27:“I want to use it at the entrance when I come again. But
it’s not only for the visually impaired. Foreigners, children, it
can be useful for everybody!"
P12

6 DISCUSSION
6.1 RQs: Efectiveness in Information
Provision and Mental Map Building
6.1.1 RQ1. How can we make the vast amount of needed information
accessible and understandable on a museum map? While contemporary museums contain a massive amount of multidimensional
information, through the participatory design with stakeholders
we categorized the needed information into structural information(layered and stackable), exhibition (name, detailed description, andaccessibility), facilities, and their locations on the map.
Our proposed method, BentoMuseum, has been proved efective
in obtaining the above types of information (Section 5.4.1, Q1 and
Q2). It helped the participants to build knowledge that integrates
the structural information, location, and contents. All of the ratingsof the purposed system outperformed the baseline of accessing
information by reading a web page, the current prevalent method.
The multi-sensory method was noted to be more helpful than using
either a tactile map or an audio guide alone (A8 and A9). Participants
especially praised the innovative “Bento Box” characteristics of the
map as being curiosity-arousing, enjoyable and understandable(Section 5.4.2, Q7 and Q8). Enabling the users to travel from one
foor to another by touch when the map is stacked gave them a senseof immersion (A14 and A15). It also made the structural attraction’s
geometric shape, especially those crossing several foors, accessible
and understandable in a concrete way (A16 and A17).
Aside from the provision of information, we also found a setof advantages that BentoMuseum contributes toward the social
inclusion of the museum. It empowers users to achieve independent
exploration and decision-making, which are reported to be valuable(Section 5.2.1, A1 and A2) [41]. At the end of the study, participants
showed their gratitude for the museum becoming more accessible
and inclusive. Frequent museum visitors (P5, P11) ideated further
customization (Section 5.4.4, A25 and A26), and those who rarely
visited a museum (P12) expressed the desire to come again for an indepth exploration (A27). The fndings indicate that the system canhelp to bridge the understanding between the blind visitors and theinformation provider and contribute to the social role of museums,
that is, to welcome blind visitors and challenge stereotypes.
6.1.2 RQ2. Is building a mental map possible and significant in themuseum context? The results from a non-rigorous fve-level evaluation show that most participants could build a rough mental map
using our system (level 5 in Section 5.3). The proposed system was
helpful in the following ways: In addition to the other elements thatallowed them to explore the exhibits of interest, Navigation mode
was benefcial for drawing location relationships on a foor (A20 andA21), and the touchable inter-foor structures helped them to connect foors and build a 3D mental map (A14–A16). Diferent fromfrequently visited places where Orientation and Mobility (O&M)

ASSETS ’22, October 23–26, 2022, Athens, Greece

training has been conducted, the visitors exhibited resistance tobuilding a detailed mental map of the museum (Section 5.3). The results demonstrate the ability of our system to help build the current
type of rough mental map (A20 and A21) and indicate its ability tosupport route memorization (A7). Even though creating and maintaining a fner mental map imposes a cognitive load in addition
to information access and tour design, it is necessary for visually
impaired people’s independence [22]. A future work is to rigorously
measure spatial memory to support O&M training.
On the other hand, when we asked participants’ thoughts and impressions about the mental map after the museum tour, all of them
explicitly stated that the current rough mental map was benefcial
for their tour. First, it supported their orientation, which made their
tour meaningful (A3 and A4). Second, it gave them a sense of safety
and the confdence that they will not get lost (A5 and A6). This
confdence is an objective of navigation for the visually impaired
in an unfamiliar environment, which supports autonomy and selfreliance [8, 28]. Through such comments, we infer that the roughmental map is signifcant in improving the museum experience.

6.2

Limitations and Next Steps

Some usability issues are related to system design, which can beaddressed by further improvement: (1) Enlarge the small touch area
(e.g., the intersections) to ft diferent fnger sizes. (2) Refne 3D
details to make clear, inter-foor structures available on both top
and bottom foors. (3) Refne the audio content. The audio-tactile information at the intersection was noted as not useful (A23); instead,
it can be used to report the exhibit coverage as P6 suggested (A24).
Some participants also found double-tap difcult. This input method
was used to prevent unintentional touches but might be unnaturalwhen all fngers are used for exploration. Seamless touch interactions with less learning efort (e.g., force recognition suggested by
P6 in A13) need to be further investigated.
To make the system a part of the museum facility, the display
and communication methods need to be further examined: (1) Selfserve foor-changing. The participants did not confuse the order ofthe foors because we handed them each foor upon request. We
also instructed the participants to keep the same orientation. To
order and orient the foors in the wild by the users themselves, clear
labels, verbal instructions, and tactile indications need to be tested.
(2) Automated instructions. Our study proved that instructionsdelivered by museum staf (Section 4.2.2 and Section 4.2.3) were
benefcial to understanding the external structure and learning
the system. Automating those instructions is needed to reduce thestaf expertise for system operation. (3) Time and interest-based
instructions for efciency. The tour design task took approximately
45 minutes, which was a considerable amount of time that not every
visitor can aford. Our log data show that the movement styles werelinked to efciency, and the most efcient style was not performed
naturally (Section 5.2.2). Since museum visitors might have varied
tactile skills and needs, customizable instructions that support anefcient exploration need to be investigated.

6.3

Generalizability and Lesson Learned

Many participants hoped that the method would become available in museums. Interestingly, they also suggested applying theWang, et al.

methods to other locations that troubled them, attracted them, or
required their confdence (Section 5.4.4).
Even though the design method is currently implemented in one
particular museum, it could be generalized to a variety of locations,especially buildings with irregular external or internal structuresand complex information. We suggest the following design considerations in applying our method: (1) Categorize the complex
information of a building into three types: structural attractions, exhibits (or informative attractions), and facilities. Simplify the shapeof each type into 3D elements, 2D relief elements, and volumetricsymbols on the foor, respectively. (2) Make foors stackable and ensure the inter-foor 3D elements can be understood by touch whenfoors are stacked. (3) Add audio labels to the points of interest
and ensure they can be recognized by touch. Such a design canbe implemented with relatively inexpensive materials: 3D printers,
touch screens, and conductive ink.
Through our study of introducing an entire museum to frst-time
visitors, another important lesson we learned was that instruction,
guidance, and encouragement could motivate the users and promoteefective information provision when presenting a complex map.
In our study, we instructed the users to travel through inter-foor
structures using fngers and consequently learned they especiallyenjoyed these structures. If we failed to do so, some participants
might overlook them due to the large number of 3D and 2D elements
on each foor. The methods to communicate the map and motivate
the users should be a part of the map design.

6.4

Toward Universal AccessParticipants showed an interest in having the BentoMuseum as an“ask-me-anything” box (A25 and A26). Further investigations (e.g.,conversational agent) should be made to fulfll individual needs
while maintaining simple operation. Some participants also expected it to support the needs of those beyond themselves (A27).
Indeed, it can potentially share information between diferent stakeholders: the blind visitors, museum staf members and service
providers, domestic visitors, foreign visitors, and other visitorswith disabilities to create universal access. For example, the information that is now decoded to audio might be presented in sign
language to support hearing impaired visitors in gaining information access. Diferent services can be connected through the map
to extend access throughout the visit.

7

CONCLUSION

This work investigated how a museum with a massive amountof multidimensional information could provide accessible maps
to blind visitors. We designed 3D and layered museum maps foreach foor of a science museum which can be stacked or placedon a touch screen to learn diferent levels of detail. An authentic
tour design task with 12 blind frst-time museum visitors showed
our system’s efectiveness in obtaining information and building arough mental map. Through user feedback, we learned the potentialof our system to contribute to a positive and inclusive museum
experience. Our next steps include expanding this design method
to other museums and attractions, making it smarter to support
diferent needs, and making it available along with other means ofassistive technologies to support autonomous museum exploration.

BentoMuseum: 3D and Layered Interactive Museum Map for Blind Visitors

ACKNOWLEDGMENTS
We thank all of the participants who took part in our user study. We
also thank Sakiko Tanaka, Bunsuke Kawasaki, and Kotaro Osawa,
the Science Communicators at Miraikan – The National Museumof Emerging Science and Innovation, for their contributions to thesystem design and feld study.

REFERENCES[1] Frances Aldrich, Linda Sheppard, and Yvonne Hindle. 2002. First steps towards
a model of tactile graphicacy. British Journal of Visual Impairment 20, 2 (2002),
62–67.
[2] Vassilios S Argyropoulos and Charikleia Kanari. 2015. Re-imagining the museum through “touch”: refections of individuals with visual disability on theirexperience of museum-visiting in Greece. Alter 9, 2 (2015), 130–143.
[3] Saki Asakawa, João Guerreiro, Dragan Ahmetovic, Kris M. Kitani, and Chieko
Asakawa. 2018. The Present and Future of Museum Accessibility for People with Visual Impairments. In Proceedings of the 20th International ACM
SIGACCESS Conference on Computers and Accessibility (Galway, Ireland) (ASSETS ’18). Association for Computing Machinery, New York, NY, USA, 382–384.
https://doi.org/10.1145/3234695.3240997[4] Japanese Standards Association. 2021. JIS T 0922:2007 Guidelines for older
persons and persons with disabilities – Information content, shapes and display
methods of tactile guide maps [In Japanese]. https://webdesk.jsa.or.jp/books/
W11M0090/index/?bunsyo_id=JIS+T+0922%3A2007[5] Sandra Bardot, Marcos Serrano, Bernard Oriola, and Christophe Joufrais. 2017.
Identifying How Visually Impaired People Explore Raised-Line Diagrams to
Improve the Design of Touch Interfaces. In Proceedings of the 2017 CHI Conferenceon Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17).
Association for Computing Machinery, New York, NY, USA, 550–555. https://doi.org/10.1145/3025453.3025582[6] Graham Black. 2012. The engaging museum: Developing museums for visitor
involvement. Routledge, London.
[7] Anke M Brock, Philippe Truillet, Bernard Oriola, Delphine Picard, and Christophe
Joufrais. 2015. Interactivity improves usability of geographic maps for visually
impaired people. Human–Computer Interaction 30, 2 (2015), 156–194.
[8] Kangwei Chen, Victoria Plaza-Leiva, Byung-Cheol Min, Aaron Steinfeld, andMary Bernardine Dias. 2016. NavCue: Context Immersive Navigation Assistancefor Blind Travelers. In The Eleventh ACM/IEEE International Conference on Human
Robot Interaction (Christchurch, New Zealand) (HRI ’16). IEEE Press, 559.
[9] Anne Chick et al. 2017. Co-creating an accessible, multisensory exhibition
with the National Centre for Craft & Design and blind and partially sighted
participants. REDO Cumulus Conference Proceedings (2017).
[10] Matthew Cock, Molly Bretton, Anna Fineman, Richard France, Claire Madge,
and Melanie Sharpe. 2018. State of Museum Access 2018: does your museum
website welcome and inform disabled visitors? http://vocaleyes.co.uk/state-ofmuseum-access-2018/[11] Eugenia Devile and Elisabeth Kastenholz. 2018. Accessible tourism experiences:the voice of people with visual disabilities. Journal of Policy Research in Tourism,
Leisure and Events 10, 3 (2018), 265–285.
[12] Jocelyn Dodd, Richard Sandell, et al. 2001. Including museums: perspectives on
museums, galleries and social inclusion. University of Leicester, Leicester, UK.
[13] Fatma Faheem and Mohammad Irfan. 2021. Museums for Equality: Diversity andInclusion–A New Concept of Future Museums. IAR Journal of Humanities andCultural Studies 2, 1 (2021), 12–13.
[14] Natália Filová, Lea Rollová, and Zuzana Čerešňová. 2022. Route options ininclusive museums: Case studies from Central Europe. Architecture Papers of theFaculty of Architecture and Design STU 27, 1 (2022), 12–24.
[15] Uttara Ghodke, Lena Yusim, Sowmya Somanath, and Peter Coppin. 2019. The
Cross-Sensory Globe: Participatory Design of a 3D Audio-Tactile Globe Prototype
for Blind and Low-Vision Users to Learn Geography. In Proceedings of the 2019
on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS ’19).
Association for Computing Machinery, New York, NY, USA, 399–412. https://doi.org/10.1145/3322276.3323686[16] Stéphanie Giraud, Anke M Brock, Marc J-M Macé, and Christophe Joufrais. 2017.
Map learning with a 3D printed interactive small-scale model: Improvement of
space and text memorization in visually impaired students. Frontiers in psychology
8 (2017), 930.
[17] Timo Götzelmann. 2016. LucentMaps: 3D Printed Audiovisual Tactile Maps
for Blind and Visually Impaired People. In Proceedings of the 18th International
ACM SIGACCESS Conference on Computers and Accessibility (Reno, Nevada, USA)(ASSETS ’16). Association for Computing Machinery, New York, NY, USA, 81–90.
https://doi.org/10.1145/2982142.2982163[18] Jaume Gual, Marina Puyuelo, and Joaquim Lloveras. 2014. Three-dimensional
tactile symbols produced by 3D Printing: Improving the process of memorizing a

ASSETS ’22, October 23–26, 2022, Athens, Greece

tactile map key. British Journal of Visual Impairment 32, 3 (2014), 263–278.
[19] Jaume Gual, Marina Puyuelo, and Joaquim Lloveras. 2015. The efect of volumetric(3D) tactile symbols within inclusive tactile maps. Applied Ergonomics 48 (2015),
1–10.
[20] Jaume Gual, Marina Puyuelo, Joaquim Lloveras, et al. 2011. Universal design andvisual impairment: Tactile products for heritage access. In Proceedings of the 18th
International Conference on Engineering Design (ICED 11). Lyngby/Copenhagen,Denmark, 155–164.
[21] Jaume Gual, Marina Puyuelo, Joaquim Lloverás, and Lola Merino. 2012. Visual impairment and urban orientation. Pilot study with tactile maps produced through3D printing. Psyecology 3, 2 (2012), 239–250.
[22] João Guerreiro, Dragan Ahmetovic, Kris M. Kitani, and Chieko Asakawa. 2017.
Virtual Navigation for Blind People: Building Sequential Representations of theReal-World. In Proceedings of the 19th International ACM SIGACCESS Conference on
Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS ’17). Association
for Computing Machinery, New York, NY, USA, 280–289. https://doi.org/10.
1145/3132525.3132545[23] Kozue Handa, Hitoshi Dairoku, and Yoshiko Toriyama. 2010. Investigation ofpriority needs in terms of museum service accessibility for visually impaired
visitors. British journal of visual impairment 28, 3 (2010), 221–234.
[24] Kevin Hetherington. 2000. Museums and the visually impaired: the spatial politics
of access. The Sociological Review 48, 3 (2000), 444–463.
[25] Leona Holloway, Kim Marriott, and Matthew Butler. 2018. Accessible Maps for
the Blind: Comparing 3D Printed Models with Tactile Graphics. In Proceedings ofthe 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC,
Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA,
1–13. https://doi.org/10.1145/3173574.3173772[26] Leona Holloway, Kim Marriott, Matthew Butler, and Samuel Reinders. 2019. 3DPrinted Maps and Icons for Inclusion: Testing in the Wild by People Who Are
Blind or Have Low Vision. In The 21st International ACM SIGACCESS Conference
on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS ’19). Association
for Computing Machinery, New York, NY, USA, 183–195. https://doi.org/10.
1145/3308561.3353790[27] Eilean Hooper-Greenhill, Richard Sandell, Theano Moussouri, Helen O’Riain,
et al. 2000. Museums and social inclusion: The GLLAM report. University ofLeicester, Leicester, UK.
[28] Sulaiman Khan, Shah Nazir, and Habib Ullah Khan. 2021. Analysis of Navigation
Assistants for Blind and Visually Impaired People: A Systematic Review. IEEE
Access 9 (2021), 26712–26734. https://doi.org/10.1109/ACCESS.2021.3052415[29] Barbara Leporini, Valentina Rossetti, Francesco Furfari, Susanna Pelagatti, andAndrea Quarta. 2020. Design Guidelines for an Interactive 3D Model as a Supporting Tool for Exploring a Cultural Site by Visually Impaired and Sighted
People. ACM Trans. Access. Comput. 13, 3, Article 9 (aug 2020), 39 pages.
https://doi.org/10.1145/3399679[30] Nina Levent and Christines Reich. 2012. How Can My Museum Help Visitors With
Vision Loss? Museum. American Association of Museums July-August (2012), 21–
22. http://ww2.aam-us.org/docs/default-source/museum/visitors-with-visionloss.pdf?sfvrsn=0[31] Georgia Lindsay. 2020. Contemporary Museum Architecture and Design. Routledge,
New York.
[32] Marianne Loo-Morrey. 2005. Tactile Paving Survey. https://www.hse.gov.uk/
research/hsl_pdf/2005/hsl0507.pdf[33] ICOM The International Council of Museums. 2020. International Museum Day
2020 - Museums for Equality: Diversity and Inclusion. https://imd.icom.museum/past-editions/2020-museums-for-equality-diversity-and-inclusion/[34] ICOM The International Council of Museums. 2022. International Museum Day
2022: The Power of Museums. https://imd.icom.museum/international-museumday-2022-the-power-of-museums/[35] The Braille Authority of North America. 2010. Guidelines and Standards for
Tactile Graphics. http://www.brailleauthority.org/tg/web-manual/index.html[36] Round Table on Information Access for People with Print Disabilities Inc. 2005.
Guidelines on Conveying Visual Information.
https://printdisability.org/
guidelines/guidelines-on-conveying-visual-information-2005/[37] Petros Pistofdis, George Ioannakis, Fotis Arnaoutoglou, Natasa Michailidou,
Melpomeni Karta, Chairi Kiourt, George Pavlidis, Spyridon G Mouroutsos, Despoina Tsiafaki, and Anestis Koutsoudis. 2021. Composing smart museum exhibit
specifcations for the visually impaired. Journal of Cultural Heritage 52 (2021),
1–10.
[38] Jonathan Rowell and Simon Ongar. 2003. The world of touch: an international
survey of tactile maps. Part 2: design. British Journal of Visual Impairment 21, 3(2003), 105–110.
[39] Jonathan Rowell and Simon Ungar. 2003. The world of touch: an international
survey of tactile maps. Part 1: production. British Journal of Visual Impairment
21, 3 (2003), 98–104.
[40] Jonathan Rowell and Simon Ungar. 2005. Feeling our way: tactile map user
requirements-a survey. In International Cartographic Conference, La Coruna. 652–
659.
[41] Richard Sandell. 2002. Museums, society, inequality. Routledge, London.

ASSETS ’22, October 23–26, 2022, Athens, Greece[42] Martin Schmitz, Mohammadreza Khalilbeigi, Matthias Balwierz, Roman Lissermann, Max Mühlhäuser, and Jürgen Steimle. 2015. Capricate: A Fabrication
Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects.
In Proceedings of the 28th Annual ACM Symposium on User Interface Software &Technology (Charlotte, NC, USA) (UIST ’15). Association for Computing Machinery, New York, NY, USA, 253–258. https://doi.org/10.1145/2807442.2807503[43] Jonathan Sweet. 2007. Museum architecture and visitor experience. Museum
Marketing: Competing in the global marketplace (2007), 226–237.
[44] Brandon Taylor, Anind Dey, Dan Siewiorek, and Asim Smailagic. 2016. Customizable 3D Printed Tactile Maps as Interactive Overlays. In Proceedings of the 18th
International ACM SIGACCESS Conference on Computers and Accessibility (Reno,
Nevada, USA) (ASSETS ’16). Association for Computing Machinery, New York,NY, USA, 71–79. https://doi.org/10.1145/2982142.2982167[45] Simon Ungar. 2018. Cognitive mapping without visual experience. In Cognitive
Mapping: Past Present and Future. Routledge, London, 221–248.
[46] Simon Ungar, Mark Blades, and Christopher Spencer. 1993. The role of tactile
maps in mobility training. British Journal of Visual Impairment 11, 2 (1993),
59–61.
[47] Raša Urbas, Matej Pivar, and Urška Stankovič Elesini. 2016. Development oftactile foor plan for the blind and the visually impaired by 3D printing technique.
Journal of graphic engineering and design 7, 1 (2016), 19–26.
[48] Roberto Vaz, Diamantino Freitas, and António Coelho. 2020. Blind and Visually
Impaired Visitors’ Experiences in Museums: Increasing Accessibility throughAssistive Technologies. International Journal of the Inclusive Museum 13, 2 (2020).
[49] Roberto Vaz, Diamantino Freitas, and António Coelho. 2020. Perspectives ofVisually Impaired Visitors on Museums: Towards an Integrative and Multisensory
Framework to Enhance the Museum Experience. In 9th International Conference

Wang, et al.

on Software Development and Technologies for Enhancing Accessibility and Fighting Info-Exclusion (Online, Portugal) (DSAI 2020). Association for Computing
Machinery, New York, NY, USA, 17–21. https://doi.org/10.1145/3439231.3439272[50] Andreas Voigt and Bob Martens. 2006. Development of 3D tactile models forthe partially sighted to facilitate spatial orientation. In 24th eCAADe Conference
Proceedings. CUMINCAD.
[51] Diana Walters. 2009. Approaches in museums towards disability in the United
Kingdom and the United States. Museum management and curatorship 24, 1(2009), 29–46.
[52] Xi Wang, Danny Crookes, Sue-Ann Harding, and David Johnston. 2022. Stories,
journeys and smart maps: an approach to universal access. Universal Access in
the Information Society 21, 2 (2022), 419–435.
[53] Zheshen Wang, Baoxin Li, Terri Hedgpeth, and Teresa Haven. 2009. InstantTactile-Audio Map: Enabling Access to Digital Maps for People with Visual
Impairment. In Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, Pennsylvania, USA) (Assets ’09). Association for Computing Machinery, New York, NY, USA, 43–50.
https://doi.org/10.1145/1639642.1639652[54] Neng-Hao Yu, Sung-Sheng Tsai, I-Chun Hsiao, Dian-Je Tsai, Meng-Han Lee,
Mike Y. Chen, and Yi-Ping Hung. 2011. Clip-on Gadgets: Expanding Multi-Touch
Interaction Area with Unpowered Tactile Controls. In Proceedings of the 24th
Annual ACM Symposium on User Interface Software and Technology (Santa Barbara,
California, USA) (UIST ’11). Association for Computing Machinery, New York,NY, USA, 367–372. https://doi.org/10.1145/2047196.2047243[55] Limin Zeng, Gerhard Weber, et al. 2011. Accessible maps for the visually impaired.
In Proceedings of IFIP INTERACT 2011 Workshop on ADDW, CEUR, Vol. 792. 54–60.