Museum Experience into a Souvenir:
Generating Memorable Postcards from
Guide Device Behavior Log
Yoshiyuki Shoji

Kenro Aihara

College of Science and Engineering
Aoyama Gakuin University
Kanagawa, Japanshoji@it.aoyama.ac.jp

Digital Content and Media Sciences Research Division
National Institute of Informatics
Tokyo, Japankenro.aihara@nii.ac.jp

Noriko Kando

Yuta Nakashima

Hiroaki Ohshima

Information and Society Research Division
National Institute of Informatics
Tokyo, Japankando@nii.ac.jp

Institute for Datability Science
Osaka University
Osaka, Japann-yuta@ids.osaka-u.ac.jp

Graduate School of Information Science
University of Hyogo
Hyogo, Japanohshima@ai.u-hyogo.ac.jp

Shio Takidaira

Masaki Ueta

Takehiro Yamamoto

College of Science and Engineering
Aoyama Gakuin University
Kanagawa, Japan
takidaira@sw.it.aoyama.ac.jp

Graduate School of Applied Informatics
University of Hyogo
Hyogo, Japan
aa20t502@ai.u-hyogo.ac.jp

Graduate School of Information Science
University of Hyogo
Hyogo, Japant.yamamoto@sis.u-hyogo.ac.jp

Yusuke Yamamoto
College of Informatics
Shizuoka University
Shizuoka, Japan
yamamoto@inf.shizuoka.ac.jp

Abstract—This paper proposes a method for automaticallygenerating postcards that reflect each visitor’s museum experience by analyzing the log of our original iPad app that
supports and guides personalized navigation in the National
Museum of Ethnology. Museum experiences have become more
personalized with the evolution of guiding devices. Each visitor
views the different exhibits in a different order. Souvenirs serve
to remind visitors of their museum experience and cement itin their memories; thus, souvenir postcards should be tailoredto each visitor’s museum experience. Such tailored postcards
can effectively remind visitors of their experiences, deepen theirimpressions when they look back at them, and promote postlearning. In this paper, we proposed a system that automaticallygenerates a postcard for each visitor for each visit by selecting thefive most relevant and impressive exhibits based on the searchand navigation logs on our museum guide app. We analyzed
the search logs of the guide devices based on the psychologicaleffects on impressions and memory to estimate which exhibits
had the strongest impact on the visitors. We then conducted anin laboratory controlled user experiment with 16 participants tocheck what exhibits had made an impression on visitors by using
the implemented system. The results showed that the exhibits thatwere seen frequently and the exhibits that participants added totheir favorites were the most memorable.
Index Terms—Search Log Analysis, Ostensive Search Model,
Museum, Memory, Impression, Postcard, Souvenir

I. I NTRODUCTION
Museum experiences are becoming more personalized, so
why not personalize souvenirs in the same way? Museums
are visited by a variety of people for a variety of purposes.
The attributes of visitors vary from kindergarten kids toadults, young to old. Their motivations also vary: for research,hobbies, entertainment, school assignments, to find referencematerial, to pass the time, and so on. In traditional museums,
visitors often follow the same course; they see the same
exhibits in the same order as they follow a prescribed routedue to geographical constraints. As a result, the visitors sharea similar viewing experience.
Museum navigation systems have evolved to be optimized
for such sequential viewing. Audio-guided navigation is commonly used in many museums. With a voice-guided tour, visitors follow the route designated by the museum and enter thenumber attached to the exhibit on the guide device when theyfind something of interest that they want to know more about
it. The guide device provides the visitors with more detailed
information about exhibits by playing the corresponding audio
or video.

New navigation systems that differ from such audio-guided
systems are appearing due to the improved performance ofsmartphones and small devices. Most of the new navigation
systems have an interactive interface, allowing the visitor
to search for an item of interest on the device first, get
the location of the exhibit, and then go straight to it. For
example, the Tokyo National Museum provides a smartphone
application called “Tohaku Navi” 1 . “Tohaku Navi” allows
visitors to select a route from among several candidates thatmatch their viewing time and interests. As another example,
Shigureden, a Waka (Japanese poem) museum in Kyoto,provided a navigation system called “Shigureden Navi” thatwas developed based on the Nintendo DS. Visitors are able tolisten to commentary appropriate to the situation according totheir current location estimated based on wireless signals. In
addition, as visitors operated the device, the exhibits changed
accordingly. Visitors were able to enjoy the museum according
to their own interests. Nowadays, various smartphone applications that provide an interactive museum-viewing experience
have begun to spread.
Academic research on interactive navigation systems is also
active. For example, there is a study to estimate a visitor’scurrent exhibit based on location [1] and camera information [2]. There is also research that enables visitors to view
exhibits according to their individual interests [3], [4], such asrecommending a route based on visitor behavior and exhibit
metadata [5].
These navigation systems add to the personalization of themuseum experience. For example, an excellent searchable
navigation device allows visitors to look up the location of
their favorite exhibits in advance and proceed straight to themwithout regard to the usual route. Devices with a recommendation algorithm recommend different routes based on a visitor’s
individual perspective. Thus, visitors are increasingly viewing
different exhibits in a different order from each other.
A museum is an important place for visitors to learn about
exhibits, see new ones, and experience them. However, it isdifficult to provide sufficient explanations for all exhibits at a
museum site due to the limited exhibition space. Visitors havea limited amount of time to stay at the museum Therefore,
if a visitor sees exhibits in a museum and returns home
without any reference to look back on, the visitor tends
to just remember visiting the museum without any specificexhibit memory and therefore does not acquire any in-depth
knowledge of the exhibits [6].
Museums post information on site to deepen understandingof the exhibits and provide knowledge about the exhibits
viewed; however, for greater information, it is necessary to
do some post-learning by reflection after returning home, forexample, by looking up the exhibits in books or on the internet.
J. H. Falk et al. point out that “Free Choice Learning” andPersonal Contextual Learning are important for meaningfullearning at a museum [7]. Their Context Model states that
1 “Tohaku Navi”: https://www.tnm.jp/modules/r free page/index.php?id=1467&lang=en

Fig. 1. Samples of postcards automatically generated from a visitor’s behavior
log. It consists of five photos of exhibits estimated as impressive and a blankspace for stamping a commemorative seal.

recalling the exhibits from time to time and connecting them
to a visitors’ own experience can help the visitors gain
meaningful knowledge from their museum experience.
Souvenirs have served as one of the triggers for such recall.
Recalling the exhibit over and over again will anchor it in
the visitor’s memory. Past large-scale studies have reported
that memory is one of the major purposes of purchasing
souvenirs [8]. This psychological reason is explained by theStrategic Memory Protection theory [9]; seeing the souvenir
many times creates a long-term impression of the exhibit. As
an example, consider the case where, a few days after visiting a
museum, a visitor sees a replica of an exhibit that they bought
as a souvenir. Seeing the replica reminds them to compare itwith their museum visit and to wonder, for example, “whatwas the meaning of the exhibit’s shape?” Such questions
can induce spontaneous post-learning that encourages deeper
understanding and memory retention.
Conventional museum souvenirs are typically replicas ofthe prime exhibits, catalogs, and items with the museum’s
logo. However, as different individuals see different exhibits,
such uniform souvenirs do not sufficiently induce individual
recall. For example, consider the case where a visitor witha specific purpose searches for an exhibit using a navigation
device and then only views that single object. The visitor may
not pay attention to the museum’s prime exhibits and may not
be interested in the museum’s logo. As a result, it is not easy tomotivate the visitor to investigate the exhibits they have seenin the museum more deeply, even when they happen to havea souvenir at their home. Under these circumstances, a newform of souvenir is needed to induce individualized reflection
and post-learning and to strengthen memory consolidation andimpressions. Such a new souvenir must address the museum
experience in a way that is personalized and includes theexperience itself.

As an example of a souvenir that satisfies these conditions,
we propose a postcard with several images of the exhibit (Seesamples in Figure 1). Postcards that reflect visitor behavior
in the museum will enhance the visitor’s impressions andunderstanding of the museum experience , encouraging themto revisit the museum. Specifically, postcards featuring only
the exhibits that made a strong impression on a visitor may
induce post-learning by reminding them of their questions
when they look back at them later, such as “The museum
didn’t provide a detailed description, so what did this design
on my favorite exhibit mean?”
Various theories and psychological effects about what ismemorable and impressive for individuals have been studied in
the fields of psychology, education, and cognitive science. To
create postcards of memorable exhibits, we use the log datafrom the interactive museum navigation devices to estimate
what the visitors are interested in and then rank the impressive
exhibits.
We conducted an experiment using the operation log of
the “Minpaku Guide” to analyze what kind of visitor action
is related to the impression of the museum visitors. The
Minpaku Guide is an iPad application we originally developed
for museum navigation. It is implemented using the exhibit
data of the National Museum of Ethnology in Osaka, Japan,
and can be used as a guide in the museum site and as apreliminary study. The application is based on the Ostensive
Search model [10]. All search results are shown in respective
panels to provide an extensive view, helping the users occasionally expand and converge their interests. Each area of the
National Museum of Ethnology has an exhibition layout that
corresponds to a geographic region. In this museum, visitors
can move freely between areas without being confined to asequential route. The Minpaku Guide allows users to searchfor exhibits by keywords, navigate using tags assigned tothe exhibits they are interested in, and see recommendations
for exhibits. Therefore, the log data of this iPad app reflects
the behavioral history of individual visitors based on theirinterests.
A sample postcard made based on actual log data is shownin Figure 1. This postcard shows five photos of exhibits froma visitor’s log that can be considered impressive to them. In
order to reinforce the visitors’ sense of ownership, they are
asked to select a stamp and place it on the blank space in theupper right corner themselves. The date and time of the visit
are also shown in the postcard.A souvenir that strongly reflects
these personal experiences can then be viewed later when
the postcard suddenly comes to their attention. We believe
it provides a strong memory of the viewing experience and,at the same time, prods the owner to investigate the exhibitsthat they are interested in.
We implemented several algorithms that automatically generate such postcards from the museum guide app’s operationlog of each user. Each algorithm is designed based on adifferent psychological effect. For validation, we conducted
an experiment in cooperation with the National Museum of
Ethnology. We consider a scenario that a visitor preliminary

makes a route plan by looking for exhibits of interest using
the app. After that, our app navigates the user in the museum
according to the plan, and obtains a postcard based on thelog when making the plan. The postcard was generated accordingly for each participant. The participants were asked toevaluate which algorithm best estimated the degree of visitor
impressions correctly.
This paper contains two contributions:• we proposed and implemented a system to generate
postcards by summarizing museum experiences, and• we demonstrated that the system could accurately summarize the experience by focusing on viewing frequency
through the experiment.
These will serve as a foundation for future advanced research
work that helps visitors learn by using souvenirs.
Our paper is structured as follows. In this section, we
explained the background and objectives of our study. In
Section II, museum experiences, the relationship between
images and impressions, and image summaries are introduced
and discussed in terms of similarity of purpose and technology
used, respectively. In Section III, we describe the method proposed in this study. Section IV sets forth our implementationof an online application for evaluation and an experiment togenerate postcards from actual logs and its results. Section
V discusses the results obtained through the experiments andSection VI summarizes the results and presents our future
prospects.
II. R ELATED W ORKThis study aims to improve the museum experience withsouvenirs by estimating memorable images from behavioral
logs. This research can also be regarded as a summary of many
images along the timeline. We, therefore, discuss and explain
the existing studies from this perspective and the position ofthis study.
A. Enhancing Museum Experience by Information Technology
Several studies have sought to enhance the viewing experience in museums and art galleries using information technology. A study of navigation system logs has been conducted to
improve the accuracy of log collection by attaching near fieldcommunication (NFC) tags to exhibits [11] and to analyze
visitor satisfaction and revisit behavior [12].
We also engaged in some research to convert museum
experience into knowledge through information access technologies. As an approach to promote post-learning, Shoji etal. [13] proposed a method of sending a notification when avisitor approaches the place related to the exhibit that theylike. As another approach, Ueta et al. [14] proposed a system
to automatically generate quizzes that reflect the visitor’sinterests to help them learn in the museum site.
Some research work with such navigation devices involves
souvenirs. Sakkopoulos et al. [15] proposed a mechanism
for purchasing an “e-souvenir.” Their proposed mechanism
allows visitors to purchase an electronic souvenir consisting ofdescriptions of the place. The record of their visiting behavior

from a guide device used a museum or tourism spot, much likebuying paid content for video games. Hornecker et al. [16]also proposed a souvenir based on visiting behavior, named“Digital Backpacking.” Visitors can register an exhibit thatthey would like to recall later by putting their smart card ina reader in front of the exhibit, which allows them to accessa website that displays the registered exhibits. These studies
support active on-the-spot action to recall the experience itself.
On the other hand, it is also important for actual learning
to have a mechanism that encourages people to look up
information when they look back on an exhibit.
Similar to our work, some studies use postcards as partof their systems’ output. For example, Petrelli et al. [17]proposed a system that accumulates a log of visiting activities
by carrying a digital gadget in a museum and placing it close
to an exhibit. This system outputs a postcard with a serial codethat allows visitors to check the exhibits they have visited on
the internet at home based on their activity logs. Visitors canreview their visiting experience at a later date by enteringa serial code into the website. In another example, Not etal. [18] proposed a system that creates a postcard by analyzing
a visitor’s behavior log, documents it as a text, and prints the
text on the back of the postcard.
These existing studies focused on improving the museum’s
on-site satisfaction and assisting in post-learning, but did not
aim to turn the experience itself into a souvenir. Serial codes
are useful in identifying what a visitor saw, but looking at
the code itself does not remind the visitor of the experience.
The purpose of our work is to induce post-learning and deepen
visitor impressions through the souvenir. Therefore, generating
an intuitive postcard that summarizes the visiting experience
itself is a new and important attempt.
B. Estimating Impressive Images
The estimation of images that are retained as impressions
and knowledge is also an important research question. In thefield of computer vision, among others, which features of animage affect human impressions and ease memory retention
have been investigated. As an example, Isola et al. [19] usedmachine learning to predict a memorable image from its
subjects and its image features. By analyzing the featuresthat contributed to the classification, they found that whether
the subject is human influences memory and that predicting
whether the subject remains in one’s memory can be difficult
depending on what the subject is. In our work, it is necessary
to find the images of exhibits that make a strong impression
on visitors and remain in their memory in order to generate
postcards that provoke post-learning.
Dubey et al. [20] estimated the degree to which the subjectof a given photograph is memorable or impressive, taking into
account the situation in which it was viewed. These computer
vision studies focus mainly on the intrinsic features of theimages; however, our study estimates impressionability fromexhibit metadata and visitor behavior logs rather than images
themselves. As another example focusing on external information about images, Khosla et al. [21] proposed a method

to estimate images that become popular on the internet. Theirstudy shows that images that eventually become popular canbe inferred from social cues, such as photographer profiles
and the number of views. The aim of our study differs fromthese; we aim to find images that are capable of inducing
post-learning for a given visitor, rather than the universal
impressiveness or popularity of each image.
C. Photo Stream Summarization
In the field of lifelog summarization, research that summarize streams of multiple photos of daily life into a single
collage are common. Similar to such studies, our systemsummarizes an image stream consisting of exhibits seen byvisitors at a museum or retrieved by a navigation system intoa single postcard. Therefore, it can be regarded as a kind of
lifelog summary for a short span of a museum experience.
Most lifelog summarization aims to faithfully summarize
the events of a period of time. Therefore, the main challenge
is to find representative images of an event in the stream [22],and to provide a more exhaustive and comprehensive summaryof the period of interest by diversifying the images included in
the summary [23]. Exhibits in museums are more informativeand have a more complex background than photos in daily
life. In addition, it is necessary to take into account metadataof high-level semantics and the relationships among exhibits,so these methods cannot be applied directly.
In past research with similar motivation to our study,
Agroudy et al. [24] pointed out the possibility that memory
can be strengthened by reviewing the lifelog. There exist
some research based on a storytelling approach by using
lifelog summarization techniques to organize experiences as
memories [25], [26]. Our research expects to have the same
effect in the limited situation of a museum experience.
III. M ETHOD
In this section, we describe our web application system thatsummarizes a museum experience into postcards.
The proposed system consists of three parts: a museum
guide iPad app, a web application front end, and a set of
ranking modules (See Figure 3). The iPad application collects
logs through the navigation at the museum site to use as acatalogue. Our iPad app also enables to virtually navigate
through the museum, which serves as a catalogue of exhibits
for preliminary studies. The logs include information on what
the visitor searched for and viewed, and when they searchedand viewed them. The web application front end receives thelogs from the iPad app and outputs a postcard image. It uses aranking module to determine which exhibits to include in the
postcards. We test multiple ranking modules, each of which
ranks exhibits based on different hypotheses. The system caneasily switch the ranking module to be used for postcard
generation. This allows the system to create postcards basedon some different psychological effects.
A. Minpaku Guide: A Museum Guide App for iPad
Minpaku Guide is an iPad application for museum navigation that we developed. It contains exhibit data from theLaunch the app

Museum map

Reccomendation screen

Details screen

Menu screen
Search screen

Tag search result screen

CameraOstensive search interface

Handwritten note

Personal screen

Annotation functions

Fig. 2. An example of screen transitions in the ostensive search interface.

behavior log
behavior log

Rank by “Long Time” Modelranking
Rank by “Primacy Eﬀect” Model

postcard

iPad AppWeb Application
Front End

Ranking Modules

Fig. 3. An Overview of the postcard generating system.

National Museum of Ethnology in Japan and makes it possible
to navigate visitors at the museum. We based the design ofthis application on an ostensive search model [10]. As shownin the figure 2, all search results are displayed in an overall
easy-to-read format with thumbnail images laid out as tiles.
In the ostensive search model, the user selects an item ofinterest from a list, browses items related to it, and repeats
the selection and viewing processes alternately. This differsfrom the traditional search model, in which the user changes
the query by reading a snippet. By repeatedly selecting items
of interest from the listed search results, users can eventuallyclarify what they are interested in and get the information
they need. This type of search model is particularly suited tomuseum navigation. When the user has no knowledge of themuseum, it is difficult to create a query because they do notknow what they are looking for. Meanwhile, ostensive searchallows the user to choose what they are interested in fromvarious candidates.

The app has recommendation, search, detailed browsing,and annotation functions. Figure 2 shows the overview ofthe screen transition and functions in our app. All operations,such as browsing search, are logged. Additionally, unconscious
activities such as physical movement are logged using wirelessbeacons.
When the user launches the app, the first screen is displayedwith recommended exhibits. The user can select an exhibit ofinterest from the list and move to its details screen. On thedetails screen, the user can view detailed information about theexhibit (i.e., higher resolution images, descriptions, tags, andvideos). In this case, the user can use the annotation function
for the exhibit they are viewing. Users can add the exhibit
to their favorites list, take a photo of it, or take handwritten
notes. Also, from the exhibit details screen, the user canmove to another exhibit related to the exhibit using tags. Tags
include the geographic region to which the exhibit belongs(e.g., “Oceania”), categories (e.g., “Clothing,” “Agriculture”),
and the location of the exhibit in the museum. The user canreturn to the menu screen from all other screens. In the menu
screen, the user can access four functional screens, such asa map showing the floor plan of the entire museum and theuser’s current location, a search screen to which users caninput queries to find exhibits by title and metadata, a nearby
screen that shows a list of exhibits near the user’s present
location, and a personal screen containing the user’s notes,
photos, and items in the favorites list.
All operations when using these functions are logged. For
example, if a user searches by a keyword query, taps on atag, or views a details screen of an exhibit, the usage history
is recorded along with the timestamp of the operation. As aTABLE I
L IST OF RANKING MODELS BASED ON PSYCHOLOGICAL EFFECTS AND
THEIR AVERAGE RATINGS EVALUATED BY SUBJECTS . * REPRESENTSSTATISTICALLY SIGNIFICANT DIFFERENCES FROM THE BASELINE J USTS EEN (p- VALUE ; * <0.05, ** <0.01).
Categories

Ranking Model# Samples

TimingPrimacy Effect
Recency Effect

102
96

2.83
*3.06

Frequency

Many Times
Long Time

101
102**3.30
2.95

ActivenessTake Photo
Take Note
Favorites

0
7
87

*3.86
**3.32Many Nouns

99

2.84

Just Seen
Not Seen

243
379

2.67
**1.29

Exhibits Feature
Baseline

Ratingcleansing of the logs, all logs are linked to exhibits and shortduration behavioral logs are removed as noise. For example,
if a user views the details of an exhibit in , we can represent
it as (in , view-detail, 20200101-15:00:00). Similarly, items ina list display, such as search results, can be represented as(in , appeared-in-keyword(“query”), 20200101-15:00:00). In
this way, the system can analyze the user’s behavior towards
the exhibit in a centralized manner. As a filter, the system
retains a log of only the cases where the user interacts withthe exhibit for more than one second. This condition is causedby systemic reasons; it is for eliminating the behaviors suchas mistouch or interim progress by using “view next” button.
If the user used the annotation function (e.g., taking notes)
on the details screen, the time spent for annotation is counted
separately from the browsing time.
B. Exhibit Ranking Based on Psychological Effects
From the action log, the system estimates which exhibits
are memorable and ranks them. Various psychological effects
on memory and impressions have been proposed in the fieldsof psychology and cognitive science. In this study, we try theeight ranking models shown in Table I, each of which focuses
on a different psychological effect.
1) Rankings by Timing of Viewing: It is known that the timeand environment in which a person sees an object affect theirmemory. This psychological effect is called the serial-position
effect [27]. In other words, when a visitor sees an object andwhat is going on around them during the time between thestart and the end of the viewing influences the degree ofmemorization of the object. When a person appreciates anobject in chronological order, their memory and impression
are affected by their position in the sequence.
One of the most famous psychological serial-position effects
is the Primacy Effect, which is the phenomenon where only
the first impression of an object remains strong when werecall it later. The idea behind our Primacy Effect ranking
model is that the viewing experience is represented by the
first few exhibits that the visitor sees; therefore, the model

ranks the exhibits in the order in which they are seen. The
implementation is simple; the system sorts the exhibits fromthe logs by timestamp and ranks them in the earliest order.
On the other hand, the phenomenon where only the end
impression remains strong is called the Recency Effect. A
related psychological effect is the “peak end rule [28]. This
rule refers to the fact that the most exciting point in time (peak)
and the end impression of a series of experiences have a strong
influence on the overall impression. As an implementation, thesystem ranks the exhibits displayed on the details screen in thereverse.
2) Rankings on the Frequency of Viewing: It has been
proven that how often a person looks at an object and how longthe person looks at it affect the impression and memorability.
One of the most famous effects is the Mere Exposure effect.
In order to focus on these psychological effects, we implement
two rankings.
The mere exposure effect is also known as the Zajonc
Effect, after Zajonc’s theory [29]. An exhibit that a visitor
sees Many Times is bound to be memorable. Another similar
psychological effect is called Active Recall. This is an effectthat objects that are recalled many times are likely to stay inthe memory without actually seeing them many times. Thiseffect is mainly related to the transition from short-term tolong-term memory. The idea of turning impressive exhibits
into a physical souvenir is based on this theory. Our ranking
model that reflects this psychological effect ranks exhibits by
the number of times that the visitor views their details screens.
How long a person spends watching a thing is related to
how it is retained in memory. The Long Time model ranks
exhibits according to the total time spent looking at the exhibit.
3) Rankings on the Proactivity Towards Exhibits: It is
understood that physical behaviors affect user retention inmemory. When a person behaves actively towards an object,
it becomes more deeply ingrained in their memory. For example, physical repetition, such as writing and speaking, helps
memory retention. This effect is called the “Ikea effect” after
Ikea, an assemble-at-home furniture retailer [30]. People feel
more attached to a piece of furniture when they put their own
hands to work on it (i.e., they assemble it themselves).
Our app provides annotation functions that reflect how
actively a visitor behaves towards the exhibits. The numberof times the visitor uses the note-taking or camera functions
for an exhibit may have a similar effect on their memory.
According to this psychological effect, we design three ranking
models; The Take Photo and Take Note models rank theexhibits by the number of times the user uses the Take-note andTake-photo functionalities, respectively. The Favorite model
uses whether an exhibit is added to the user’s favorite list.
4) Rankings by Exhibits Feature: In some cases, the exhibits themselves have memorable characteristics independentof the visitor’s behavior. Some exhibits have the possibility ofcreating a strong impression just by looking at them.
Concrete things are more memorable than abstract things.
In other words, things whose names and images match eachother are easier to remember. This phenomenon is explained

by a theory in the field of psychology called the “Dual-coding
theory.” When a person remembers something, they remember
an abstract image and a semantic index keyword to recall it.
In some cases, if the indexing keywords are not appropriate,
if they are complex, or if they have little connection to theobject, it is difficult to remember the object.
A similar effect is the verbal encoding of nonverbal information. It is difficult to remember abstract and unnamed
information, but it is easier to remember it when it is verbalized. As a famous example, dance choreography is nonverbal and difficult to remember; however, it has been reported
that the learning efficiency of dance increases when lyrics aretentatively given to the music that are related to the dance
movement. In addition to the dual-coding theory, the name ofthe exhibit may affect its retention in memory.
We consider Many Nouns in its description as the ease ofverbalization of an exhibit. This ranking model considers thememorability of the title and description of exhibits. According
to the Dual-coding Theory, information can be more memorable if it’s associated with more concrete concepts; therefore,
we hypothesize that the appearance of proper nouns, the mostconcrete concepts, involves in our problem. We thus countthe number of proper nouns in the title and description, andrank the exhibits in the descending order of the number. For
implementation, we split the title and the description of theexhibits viewed at least once by the visitor into morphemes
using MeCab, a Japanese morphological analyzer, because
terms in Japanese sentences are not set off by spaces. The
model ranks the exhibits in order of the number of proper
nouns that appeared in the exhibit’s title and description byusing the morpheme information.

rankings generated by our eight models described in Section
III and the two baseline methods (see Table I).
The participants performed two labeling tasks through aseries of experimental sessions. First, they rated the degree
to which each exhibit that appeared in the rankings wasremembered. Next, they sorted the postcards printed on paper
in order of how well they matched their own degree ofmemory.
Due to the influence of COVID-19, we were not able
to conduct the experiment at the museum on-site, so we
conducted the experiment in a preliminary study scenario,
where the participants were asked to make a plan to explore themuseum using our iPad app beforehand. We evaluated whether
the system can correctly estimate what is memorable from thelog. We recruited sixteen participants in two laboratories at
two universities, who manipulated our iPad app to collect their
behavior logs in the respective laboratories. Each experimental
session was performed with four participants and took twohours.
Two additional ranking models were prepared as a baseline.
The Just Seen ranking consists of exhibits that the participant
looked at their details screen at least once but did not conformto any of the psychological effects (i.e., we ignored any behaviors that may be associated with the psychological effects,
such as the number of views, when it is viewed, etc.). The Not
Seen ranking consists of exhibits that the participants neversee their details screen. These two ranking models have no
specific order within the selected exhibits, so we order themrandomly.

C. Generating Postcards from the Behavior Log

The experiment consisted of four phases. First, we explained
an overview of the museum and the basic operation of ouriPad app to each participant. Although the experiment was
conducted in the laboratory, the functions that could only be
activated at the museum (i.e., the map with current position,and the camera function for taking photos of real exhibits)
were briefly explained. The explanations and exercises took
15 minutes.
Next, the participants were given a task and had 30 minutes
to operate the iPad app. They played the role of a visitor who
would be visiting the museum the next day: “You are goingto the museum tomorrow, but you don’t have time enough to
see all the exhibits. Use the iPad app to preview the exhibits
and make a plan of what you will see.” The participants had
30 minutes to use the iPad app freely. Operational constraints
on the task (e.g., list at least n exhibits as favorites) were notset. We recorded a behavioral log of all behaviors.
The participants then played a video game unrelated tothe museum for 30 minutes. This is because, in a memory
experiment, a cognitively demanding task is required between
remembering and recalling. That is, if the labeling process
is performed immediately after the iPad app operation, theexhibits seen later still remain in the brain’s working memory,and rankings related to time cannot be correctly verified. In

Given one of our ranking models, the system generates a
postcard with the images of the top five exhibits. The layoutof the postcards is shown in Figure 1. This postcard has one
large exhibit image, one quarter-size image, and three 1/16size images arranged in a spiral pattern.
The postcards include the date and time of the visit in order
to affirmatively link the visitor’s behavior to the souvenir. A
space for a commemorative stamp is located in the blankspace in the upper right corner. The purpose of this is to givevisitors the opportunity to choose their favorite stamp andplace that stamp on the postcard themselves. This is basedon the idea of the Ikea effect, explained above, to increase
visitors’ attachment to the souvenir.
The size of the postcard is 148-mm wide and 100-mm high.
The sizes of the images are 96 mm × 96 mm, 48 mm × 48
mm, and 24 mm × 24 mm. If the photos of the exhibits are
not square, we crop them to contain their center. The systemgenerates one PNG image for each rank model.
IV. E VALUATION
We conducted a subject experiment to verify the accuracyof memorable exhibit estimation from the behavioral logs andthe quality of the postcards. The experiment compares the

A. User ExperimentTABLE II
S UMMARY OF THE RESULT OF POSTCARD EVALUATION . R ANK MEANS
THE AVERAGE POSITION ON THE PARTICIPANTS ’ POSTCARD RANKING( BETWEEN FIRST AND NINTH ; LOWER IS BETTER ). # BEST AND # WORSTMEANS THE NUMBER OF PARTICIPANTS THAT RANK THE POSTCARD MADE
BY THE MODEL AS BEST OR WORST.

Fig. 4. A screenshot of the evaluation system. The system displays up to 100
exhibit images. Participants rate exhibits as to whether they remember themon a four-point scale.

this experiment, four participants raced against each other ina racing game.
Finally, the output of the system was labeled in both perexhibit and per-postcard manners. Each exhibit was labeledindividually if it was memorable for the participant. The
participants labeled seven exhibits at the top of each ranking.
A specific web application was used for this labeling. Thisapplication generates an evaluation form specifically for thatparticipant when their behavior log is inputted. Figure 4 showsan actual screenshot of the evaluation system. The application
displayed 100 exhibits consist of 76 seen exhibits (i.e., seven
exhibits for eight ranking models and 20 exhibits for Just
Seen model) and 24 Not Seen exhibits in randomly mixedorder. The participants were able to label each exhibit on afour-point scale: from one (do not remember this exhibit) tofour (strongly remember this exhibit). The participants wereable to leave a comment for each exhibit. Several participants
left comments, such as “I think I saw this, but I’m not sure.”
For per-postcard labeling, we printed out hard-copy of
postcards generated from each participant’s behavior log.
The system generated ten postcards per participant based on
the eight proposed methods and two baseline methods. The
postcards contain five photos, but, depending on the operation,
not enough exhibits may be collected (i.e., the participant
only added four exhibits into their favorites). In such cases,
the blank space on the postcard was filled with images ofrandom exhibits. In this experiment, no one used the camera
function. Ultimately, ten postcards were generated, but one
based on the Take Photo ranking model was excluded fromthe evaluation set. The participants physically rearranged thenine postcards and placed the postcard that best matched theirimpressions at the top of the postcards that did not match
their impressions at the bottom. These postcards were given
to participants in a bundle. There was a desk large enough

Ranking Model

Per-exhibit
RatingRank

Primacy Effect
Recency Effect
Many Times
Long Time
Take Note
Favorites
Many Nouns
Just Seen
Not Seen

2.83
3.06
3.30
2.95
3.86
3.32
2.84
2.67
1.29

4.69
3.81
2.94
4.88
7.44
4.25
4.38
5.06
7.56

Per-postcard# Best# Worst
1
2
6
1
0
2
1
2
1

1
0
1
0
5
1
1
1
6

to spread out the postcards, so participants laid them out andpicked up the postcards one by one in the order of degree of
their impressions.
All participants were university students in Japan, between
the ages of 20 and 24. Participants received a book token for
2,000 Japanese yen for a two-hour task. This price is roughly
equivalent to a standard hourly wage for a part-time job.
B. Experimental Result
Table I shows an overview of the results of the per-exhibitlabeling with the average rating by all the participants for the
top exhibits in each ranking.
The Favorite model and Many Time model had significantly higher estimation accuracy than one of our baselines,
the Just Seen model (p < 0.01). The results suggest thathaving seen the exhibit many times and having listed it as afavorite are useful in estimating the exhibits that individuals
remember. Conversely, the Not Seen model was significantly
less accurate than the Just Seen model. It was also foundthat participants were able to distinguish between exhibits
they had seen at least once and those they had not. Take
Note and Recency Effect also had higher estimation accuracy(p < 0.05). However, on Take Note, few participants took
notes. Therefore, the number of exhibits that someone took
note of is only seven. The remaining models also estimate thememorability of the exhibits with higher accuracy than the
Just Seen model, but the differences are small and unreliable.
Table II shows the overview result of the per-postcard
labeling. The results demonstrate that the postcards consistingof exhibits that were viewed with their details screens Many
Times were the most supported by the majority of participants.
Only the postcards made by ranking models Not Seen andTake Note were ranked lower than the Just Seen model.
The postcards made by the Take Note ranking model often
contained random exhibits, as most participants never took
notes. This suggests that postcards consisting of exhibits thatwere not seen did not match the visitors’ viewing experiences
and impressions.

V. D ISCUSSION
The results of the two tasks suggest that the tendency ofparticipants to remember what they saw even 30 minutes laterand the model Many Times, based on the Zajonc effect, may
be the most accurate model to infer impressions from thebehavior logs.
Let us discuss the per-exhibit evaluation result. First, weconsider the models that were able to estimate memorable
exhibits from the behavioral logs. The model with the highest
rating by the participants was the Favorites model. It is
possible that users simply remember what they like and thatthey remembered these exhibits by the operation of pressing
the favorite button. In addition, it is possible that the favorites
function was compatible with this experimental task; that is,there may have been a case where the participants placed
themselves in the task of finding the exhibits they wanted tosee and added them to their favorites. The participants may
have tried to manage the exhibits they wanted to see the next
day by adding them to their favorites list.
Focusing on the individual psychological effects, the models
based on frequency, such as Many Times and Long time,
were rated relatively high. Among these two models, ratings
differed by frequency and time. This is consistent with theresults of the original Zajonc report. In other words, seeing
many times has a greater impact on impression than seeing
over a long time.
Among the models that focused on the timing of viewing,
postcards listing the last five exhibits seen were relatively
highly rated. This may simply be because the last object
seen was still in the participants’ short-term memory. Labelingagain after a longer period of time could change the results.
Postcards consisting of the first five exhibits seen were rated
relatively low. It is possible that the participants saw several
examples of the exhibit during the pre-experimental manipulation and that the exhibits they saw at that time may havemade a stronger impression. It is also possible that they were
distracted by the app’s operations and functions immediately
after the start of the experiment and did not pay attention tothese exhibits.
We next discuss the task of sorting the postcards. For thistask, different participants tended to favor different models.
Different people have different models of what they find
impressive, and there may not be a model that is suitable foreveryone.
In the postcards sorting experiment, the postcards with theexhibits that were viewed more often on the details screen wererated higher. This may simply be because the times of viewing
the detailed screen affect memory, but it is also possible thatthese exhibits had a notable photogenic appearance (i.e., flashy
or vivid colors). Photogenic exhibits are more likely to betapped because they stand out on the tiled ostensive searchinterface. Even when printed on postcards, they may strongly
attract the participants’ attention.
In the evaluation, we conducted the experiment only in alimited situation, such as in a preparatory task and a 30-minuteinterval. For a more substantive evaluation, we need on-siteexperiments in the museum. Furthermore, it is expected thatwhat people remember 30 minutes later will be different fromwhat they remember a week later. In addition, we were notable to evaluate the effect when visitors see the postcards indaily life. Further experiments over a longer period of time
will be necessary.
VI. C ONCLUSION
In this paper, we proposed a system to generate postcards
with memorable exhibits by analyzing the behavior log of
our museum guide app. The contribution of this work is the
proposal of a postcard that summarizes the visitor’s experience
as a souvenir and an experiment that reveals which viewing behaviors are deeply related to memorability. For the evaluation,a web application was implemented to enable the comparisonof multiple rankings. Eight rankings based on psychologicaleffects and two baseline rankings were created. Throughthe in-laboratory controlled user experiments, 16 participants
assessed whether each ranking was able to correctly estimate
the visitors’ impressions. The results suggest that the exhibitsthat were added onto the favorite list and those that wereviewed repeatedly on the detailed information screen weremore memorable.
One remaining issue is that we need on-site evaluation
experiments in a museum. Due to the social situation regarding
COVID-19, it was not possible to conduct the experiment at
the museum site. For this reason, we estimated the impressionability of the museum from the logs during the preliminary
study of exhibits. In order to summarize what visitors see
in a museum and turn it into a souvenir, we must conduct
evaluation experiments on-site. We would like to conduct
larger-scale field experiments in the future.
A more practical application of postcards is another futurechallenge. In this work, the exhibits on the postcards werelimited to those that made a strong impression on the individual visitor. Exhibits that impress a visitor may not encourage
return visits. Rather, it may be important to show exhibitsthat did not create much of an impression or exhibits that thevisitors wanted to view, but were not able to see on the current
visit. Including these on postcards can elicit new interest andencourage a revisit.
In addition, we also want to consider the gamification of
museum visitation behavior using postcards. As an example,
writing a statement on a postcard such as “If you see 5 more
exhibits, you can complete the exhibits in this category” might
motivate people to collect and encourage them to revisit.
ACKNOWLEDGEMENT
This work was supported in part by JSPS KAKENHI
Grant Numbers JP18H03494, JP18H03243, JP18K18161,
JP17H00762, and JP16H01756. It was also supported by
the NII Open Collaborative Research 2020 Program, numbers 20S1001, 20AD03, and 20AD04. We sincerely thankthe Japanese National Museum of Ethnology (Minpaku) for
offering the exhibit meta-data.

R EFERENCES[1] J. Lanir, T. Kuflik, E. Dim, A. J. Wecker, and O. Stock, “The influence of a location-aware mobile guide on museum visitors’ behavior,”
Interacting with Computers, vol. 25, no. 6, pp. 443–460, 2013.
[2] H. Bay, B. Fasel, and L. Van Gool, “Interactive museum guide: Fastand robust recognition of museum objects,” in Proceedings of the firstinternational workshop on mobile vision, 2006.
[3] I. Roes, N. Stash, Y. Wang, and L. Aroyo, “A personalized
walk through the museum: The chip interactive tour guide,” in
CHI ’09 Extended Abstracts on Human Factors in Computing
Systems, ser. CHI EA ’09. New York, NY, USA: Association
for Computing Machinery, 2009, p. 3317–3322. [Online]. Available:
https://doi.org/10.1145/1520340.1520479[4] H. Bay, B. Fasel, and L. Van Gool, “Interactive museum guide,” in The
Seventh International Conference on Ubiquitous Computing UBICOMP,
Workshop on Smart Environments and Their Applications to Cultural
Heritage, 2005.
[5] W. R. Van Hage, N. Stash, Y. Wang, and L. Aroyo, “Finding your way
through the rijksmuseum with an adaptive mobile museum guide,” inExtended Semantic Web Conference. Springer, 2010, pp. 46–59.
[6] K. M. Ellenbogen, “From dioramas to the dinner table: An ethnographic
case study of the role of science museums in family life.” Dissertation
Abstracts International, vol. 64, 2004.
[7] J. H. Falk and L. D. Dierking, Learning from museums. Rowman &Littlefield, 2018.
[8] H. Wilkins, “Souvenirs: What and why we buy,” Journal of Travel
Research, vol. 50, no. 3, pp. 239–247, 2011.
[9] G. Zauberman, R. K. Ratner, and B. K. Kim, “Memories as assets:
Strategic memory protection in choice over time,” Journal of Consumer
Research, vol. 35, no. 5, pp. 715–728, 2008.
[10] I. CAMPBELL and K. VAN RIJSBERGEN, “The ostensive modelof developing information needs,” in CoLIS 2: second international
conference on conceptions of library science: integration in perspective(Copenhague, October 13-16, 1996), 1996, pp. 251–268.
[11] M. Blöckner, S. Danti, J. Forrai, G. Broll, and A. De Luca, “Please touchthe exhibits! using nfc-based interaction for exploring a museum,” inProceedings of the 11th International Conference on Human-Computer
Interaction with Mobile Devices and Services, ser. MobileHCI ’09.
New York, NY, USA: Association for Computing Machinery, 2009.
[Online]. Available: https://doi.org/10.1145/1613858.1613943[12] J. Kang, J. Jang, and C. Jeong, “Understanding museum visitor satisfaction and revisit intentions through mobile guide system: moderatingrole of age in museum mobile guide adoption,” Asia Pacific Journal of
Tourism Research, vol. 23, no. 2, pp. 95–108, 2018.
[13] Y. Shoji, K. Aihara, M. J. Dürst, N. Kando, T. Nakaya, H. Ohshima,
T. Yamamoto, and Y. Yamamoto, “Location-based Reminder for Memorizing What Visitors Learn at a Museum,” in The Third Workshop on
Evaluation of Personalisation in Information Retrieval (WEPIR2021),
Online, 2021, pp. 5 – 8.
[14] M. Ueta, T. Hashiguchi, H.-L. Pham, Y. Shoji, N. Kando, Y. Yamamoto,T. Yamamoto, and H. Ohshima, “Quiz Generation on the Electronic
Guide Application for Improving Learning Experience in the Museum,”in The Third Workshop on Evaluation of Personalisation in Information
Retrieval (WEPIR2021), Online, 2021, pp. 1 – 4.
[15] E. Sakkopoulos, M. Paschou, Y. Panagis, D. Kanellopoulos, G. Eftaxias,
and A. Tsakalidis, “e-souvenir appification: Qos web based mediadelivery for museum apps,” Electronic Commerce Research, vol. 15,
no. 1, pp. 5–24, 2015.
[16] E. Hornecker and M. Stifter, “Digital backpacking in the museum witha smartcard,” in Proceedings of the 7th ACM SIGCHI New Zealandchapter’s international conference on Computer-human interaction:
design centered HCI. ACM, 2006, pp. 99–107.
[17] D. Petrelli, M. T. Marshall, S. O’brien, P. McEntaggart, and I. Gwilt,“Tangible data souvenirs as a bridge between a physical museum visit
and online digital experience,” Personal and Ubiquitous Computing,vol. 21, no. 2, pp. 281–295, 2017.
[18] E. Not, M. Zancanaro, M. T. Marshall, D. Petrelli, and A. Pisetti,“Writing postcards from the museum: composing personalised tangible
souvenirs,” in Proceedings of the 12th Biannual Conference on Italian
SIGCHI Chapter. ACM, 2017, p. 5.
[19] P. Isola, J. Xiao, A. Torralba, and A. Oliva, “What makes an image
memorable?” in CVPR 2011. IEEE, 2011, pp. 145–152.

[20] R. Dubey, J. Peterson, A. Khosla, M.-H. Yang, and B. Ghanem, “Whatmakes an object memorable?” in Proceedings of the ieee international
conference on computer vision, 2015, pp. 1089–1097.
[21] A. Khosla, A. Das Sarma, and R. Hamid, “What makes an image
popular?” in Proceedings of the 23rd international conference on World
wide web. ACM, 2014, pp. 867–876.
[22] A. R. Doherty, D. Byrne, A. F. Smeaton, G. J. Jones, and M. Hughes,“Investigating keyframe selection methods in the novel domain of passively captured visual lifelogs,” in Proceedings of the 2008 international
conference on Content-based image and video retrieval. ACM, 2008,
pp. 259–268.
[23] A. Lidon, M. Bolaños, M. Dimiccoli, P. Radeva, M. Garolera, and
X. Giro-i Nieto, “Semantic summarization of egocentric photo stream
events,” in Proceedings of the 2nd Workshop on Lifelogging Tools andApplications. ACM, 2017, pp. 3–11.
[24] P. E. Agroudy, T. Machulla, R. Rzayev, T. Dingler, M. Funk, A. Schmidt,
G. Ward, and S. Clinch, “Impact of reviewing lifelogging photos
on recalling episodic memories,” in Proceedings of the 2016 ACM
International Joint Conference on Pervasive and Ubiquitous Computing:
Adjunct. ACM, 2016, pp. 1014–1019.
[25] M. Bolanos, M. Dimiccoli, and P. Radeva, “Toward storytelling fromvisual lifelogging: An overview,” IEEE Transactions on Human-Machine
Systems, vol. 47, no. 1, pp. 77–90, 2016.
[26] D. Byrne, A. Kelliher, and G. J. Jones, “Life editing: Third-party
perspectives on lifelog content,” in Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, ser. CHI ’11.
New York, NY, USA: Association for Computing Machinery, 2011,
p. 1501–1510. [Online]. Available: https://doi.org/10.1145/1978942.
1979162[27] J. Deese and R. A. Kaufman, “Serial effects in recall of unorganized
and sequentially organized verbal material.” Journal of experimental
psychology, vol. 54, no. 3, p. 180, 1957.
[28] D. Kahneman et al., “Objective happiness,” Well-being: The foundationsof hedonic psychology, vol. 3, no. 25, pp. 1–23, 1999.
[29] R. B. Zajonc, “Attitudinal effects of mere exposure.” Journal of personality and social psychology, vol. 9, no. 2p2, p. 1, 1968.
[30] M. I. Norton, D. Mochon, and D. Ariely, “The ikea effect: When labor
leads to love,” Journal of consumer psychology, vol. 22, no. 3, pp. 453–
460, 2012.