A System using Tag Cloudfor Recalling Personal Memories
Harumi Murakami and Ryutaro Murakami
Osaka City University, Sugimoto, Sumiyoshi, Osaka 558-8585, Japan
harumi@osaka-cu.ac.jp
http://murakami.media.osaka-cu.ac.jp/

Abstract. The research presented here extends a previous prototype
that supported human recollection with tag clouds created from the useof a personal calendar and Twitter. That system weighted keywords by
combining term frequency and the number of photos taken by users torecall memorable events. The aim in this paper is to improve upon ourprevious work and present a full system that uses tag clouds for recallingpersonal memories. The main differences from our previous work are asfollows. (1) Multiple information sources such as SNSs or instant messengers can be used. (2) To handle multiple information sources, we present
a new unified keyword-weighting algorithm. (3) We implemented newfunctions, such as keyword search, tag search, and photo display, to form
a complete system. Preliminary experiments reveal the usefulness of oursystem in recalling personal memories.
Keywords: Personal memory support, Tag cloud, Calendar, Twitter,LINE, Number of photos

1

Introduction

Memory is crucial for various activities in our daily lives. For example, we may
have to write progress reports about what we have done on particular days orweeks. We may have to plan anniversaries and thus recall what we did last year
or in previous years. Or we might simply want to reminisce about the day wesaw our spouse for the first time. In addition, the amount of information that
we manage is increasing. Consequently, we aim to support human memory.
We have presented a prototype that supports human recollection with tagclouds created from the use of calendar and Twitter [1]. Since we assumed that
we could identify memorable events based on the days when a user took many
photos, we weighted keywords by combining term frequency and the number ofphotos taken by users to generate tag clouds.
However, we found the following problems in the prototype. (1) Information
sources are fixed in a calendar and Twitter. The trends of SNSs are changing,and their preferences are different. We need to easily cope with other information sources. (2) The keyword weighting algorithms for calendar and Twitter are2Harumi Murakami and Ryutaro Murakami

different because they are customized to each source. To cope with new information sources, we need to develop an effective simplified weighting algorithm. (3)There is no function of keyword search, which is obviously useful for recalling
memory. (4) Photos were used for keyword-weighing algorithms, but the photos
themselves were not displayed. Photos are also obviously effective for recalling
the past.
The aim of this research is to improve the system developed in our previous
work to cope with the above problems and to present a new complete systemusing tag cloud for recalling personal memories. The main differences from ourprevious work are as follows. (1) Multiple information sources such as SNSs or
instant messengers can be used. (2) To handle multiple information sources, wepresented a new unified keyword-weighting algorithm. (3) We implemented newfunctions, such as keyword search, tag search, and photo display, to build acomplete system.

2

Tag Browser

Our research generates a tag cloud by extracting keywords from various kinds ofinformation usage and weighting them using term frequency and the number ofphotos. First, we obtain the data written by users and generate history structures[2]. Next, we generate tag clouds from these history structures. We call our newsystem a tag browser. LINE, which is classified as an SNS but is actually themost frequently used instant messenger in Japan, is adopted as well as calendar
and Twitter. The user can set the period and the number of tags (default numberis 30) and the system displays a tag cloud, information logs, and photos.
Several new functions have been developed for the tag browser. (1) Multiple
information sources such as SNSs or instant messengers can be used. (2) Log windows (display position of information sources) are defined automatically basedon the amount of log data. (3) The same keyword-weighting algorithm is used for
all information sources. (4) The user can change the weighting algorithm using aslider interface. (5) Tag and keyword search functions can be made. (6) Photos
are displayed according to date. In what follows, we translated the examples inthis paper from Japanese into English for publication.
2.1

Generating History Structure

The basic components constructing the history structure include time, keywords,and log (original text) sets1 . Nouns and noun phrases are extracted from information logs as keywords. For example, from a LINE message “· · · Yes! In Kyoto
every day is like a festival!” The system extracts “Kyoto,” “every day,” and“festival” as keywords. See Fig. 1 for example of history structures.
1

We have changed from using URI [2] to log in this research. Other attribute information such as URI can be stored optionally. For LINE, the receiver and sender ofthe message are stored as To and From, respectively.

A System using Tag Cloud for Recalling Personal Memories
TimeHistory
Structure Sat Aug 12(Calendar) 00:00:00 JST 2017

Keywords

Log[Hiroshima]

going back to Hiroshima

3…
History
Structure(Twitter)

Time

Keywords

Log

Sat Feb 18
06:15:35 JST 2017

[Randen][Arashiyama][Kyoto]Iʼm at Randen Arashiyama
in Kyoto…

…
History
Structure(LINE)

TimeKeywords

Log

ToFromSat Feb 04
21:58:00 JST 2017

[Kyoto][every day][festival]… Yes! In Kyoto, every day
is like a festival!

IchiRyu…

Fig. 1. Example of history structures2.2

Displaying Tag Clouds

Based on the results of experiments, it was decided to display tag cloud to theleft and information logs to the right. First, we describe how to display a tag
cloud. According to the input (period with or without a keyword), weights ofkeywords are calculated using Eqs. (1) and (2).
First, Wi,tk defines the weight of term tk in each information source i:Wi,tk =

X 1 + Cphoto (Gdate (tk ))Ckeyword (Gdate (tk )),(1)

where the addition occurs for each term tk appearing in each history structure.
Here, Gdate (tk ) is a function that obtains the date of term tk , Cphoto (R) is afunction that obtains the number of photos in range R, and Ckeyword (R) is afunction that obtains the number of keywords (terms) in range R. The denominator works as a normalizing factor for the number of information logs.
Next, Wtk defines the weight of term tk :Wtk =nX

αi Wi,tk ,(2)i=1where n is the number of information sources and the default value for αi is 0.5.
In this research, we assigned the following values to i: calendar: 1, Twitter: 2,and LINE: 3.
The size of tags (keywords) is based on the weights determined by Eq. (2).
Font colors are designed according to the information sources in which theyappear. They reflect the original color image of the applications except for thecalendar. If term tk only appears in the calendar, it is orange, while it is blue inTwitter and green in LINE. If term tk appears in more than one log, it is red.
Tags are sorted by the time of first appearance.

42.3

Harumi Murakami and Ryutaro Murakami

Displaying Information Logs and Photos

According to the amount of data, information logs are displayed from left toright. In each information source, logs are displayed according to time. When
only a period is input, all logs during the period are displayed. In tag or keyword
searches, logs that contain the tag or keyword are displayed. Photos are displayedin ascending order according to date. When only a period is input, all photos
during that period are displayed. In tag or keyword search, the photos taken on
the date matching the date of logs containing the searched tag or keyword aredisplayed.
2.4

ExampleInput
SliderTag CloudKeyword SearchInformation Log

PhotoFig. 2. Screenshot of display for one-month period

Fig. 2 shows an example of the system’s basic usage. The user can set theperiod and number of tags, and a tag cloud is displayed on the left while information logs are displayed on the right. In this case, LINE, Twitter, and calendar
logs are displayed from left to right according to the amount of data. Logs andphotos are displayed according to time order, and the user can scroll them. Theuser can change parameter αi of Eq. (2) (0 ≤ αi ≤ 1) by using the slider.
Fig. 2 shows an example screen of one month (Aug. 2017) for a user. “Fireworks Display” is the largest tag and is shown in red. 10 out of 30 (33%) tags
are related to places. In this month’ case, many tags work as clues to remember
salient events.
When the user clicks a tag in the tag cloud, history structures that containthe keyword are extracted, a new tag cloud is generated, and the information
logs of these history structures are displayed. For example, when the user clicks“fireworks,” information logs containing “fireworks” are searched and displayed.

A System using Tag Cloud for Recalling Personal Memories

5Photos taken on the dates of the retrieved logs are displayed, and thus the user
can recall pleasant memories of “Fireworks Display at Kakogawa.”

Fig. 3. Screenshot of display for keyword search

Fig. 3 shows an example of keyword search. When the user inputs “Kyoto,”
information logs containing “Kyoto” are retrieved. In this case, the period is set
to about half a year. Although the user went to Kyoto just once in February
as in calendar log, many message exchanges using LINE related to Kyoto areretrieved, and various types of information (plan, memory, etc.) about Kyoto
can be recalled. Photos related to Kyoto are also displayed.
The tag browser can be used for business as well. For example, the user canlook for photos of certain presentation slides taken at conferences. Although there
is no manual tag for the photos, they can be found using a date or information
logs searched via tag or keyword search.

3ExperimentWe recruited subjects who satisfied the following conditions: (a) those who takephotos and (b) those using more than one of the following: writing a calendar
schedule on any device, using Twitter, or using LINE. We gathered data for
more than three months, and each experiment used the data of one month.
3.1

Experiment 1: Generating Tag Cloud

We evaluated the usefulness of our algorithm for creating tag clouds. Our subjects were ten males and one female, with an average age of 22.8. We prepared
four tag clouds from four weighting algorithms (our algorithm, raw frequency(number of tk in each history structure), relative frequency (number of tk divided
by number of keywords in each history structure), and a previous algorithm [1])

6Harumi Murakami and Ryutaro Murakami

that display 30 keywords for comparison. The subjects performed the following
task: “Rank the four tag clouds by the ease with which they helped you recall
your memories.” Five subjects selected our algorithm as 1st rank among thefour algorithms. Three subjects selected raw frequency, two selected the previous algorithm, and one selected relative frequency. No subject selected ouralgorithm as 4th (worst) rank. Although the difference was slight, our algorithm
was best among the four algorithms, and we found that it was possible to unify
a keyword-weighting algorithm.
3.2

Experiment 2: Tag Browser

We evaluated the usefulness of our system. Our subjects were five males and onefemale, with average age of 23.2. After recalling a one-month-old memory using
the system, the subjects answered questions at five levels (5 to 1; 5 is best).
Here, we extract some results of the questions in Table 1.
Table 1. Some results of questionnaire
Questions
Mean SD
Q1 Was the system useful for recalling your memories? 4.50 0.50
Q2 Do you want to use this system in the future?
4.33 0.75
Q3 Did you feel fun while using this system?
4.33 0.75

For the question “Which part was the most useful for recalling your memories? - A: Tag cloud, B: Logs, C: Photos,” five subjects answered A: Tag cloudand one subject answered C: Photos. Overall, the results show the usefulness ofour system for recalling personal memories.

4Related Work

This research is a part of our work on personal memory support. Murakami[2] presented the concept of information structure called history structure. The
history structure integrates various kinds of information usage. Murakami et al.
[3] developed a knowledge-space browser that displays a network rather than atag cloud. Matsumoto et al. [1] developed the initial prototype of the tag browser
displaying a tag cloud. This research improved upon the previous work [1] byfurther refining the tag browser to its current form.
Our research uses tag clouds for human memory recall, even though little
research has used them for this purpose. Chen and Jones [4] developed a prototype system called iCLIPS that searched through personal lifelogs for memory
support. In iCLIPS, computer activities and the names of locations and people
were displayed in term clouds, which resemble tag clouds. No detailed algorithms
for generating term clouds and user studies of the prototype have been reported.

A System using Tag Cloud for Recalling Personal Memories

7

We focus on generating better tag clouds rather than accumulating all humanactivities. We also conducted preliminary experiments that demonstrated oursystem’s usefulness.
Mathur et al. [5] presented a prototype system of a tool called LifeView,
which visualizes textual lifelogs for Sentimental Recall and Sharing. In this system, events are manually created by users who manually annotate tags. A tag
cloud (these tags) for one event was displayed. On the other hand, our approach
automatically extracts keywords from information sources and generates tagclouds.
Aires and Goncalves [6] presented Personal Information Dashboard, a web
application that allows users to see, at a glance, various facets of their lives.
In this system, Keywords Cloud is a tag cloud-like visualization that shows themost important words from a set of emails, posts, and/or tweets. To calculate
the important words, they used tf-idf. The presentation (layout) is spacial (notsorting). Keywords Cloud can be configured to show data from a specific timeperiod. Our tag cloud’s algorithm, presentation, and information sources aredifferent from Keywords Cloud.
Many systems and researches have generated tag clouds. The two main purposes are summarizing and navigating certain content. Rivadeneira et al. [7]classified tasks that tag clouds can support into four: (a) search, (b) browsing,(c) impression formation or gisting, and (d) recognition/matching. In general,(a) tag selection algorithms are based on the frequency of the terms or objects,(b) important tags are represented by size or color, and (c) tags are orderedalphabetically.
Venetis et al. [8] evaluated existing algorithms for exploring and understanding a set of objects against tf-idf-based algorithms and presented a maximum
covering algorithm that seems a very good choice for most scenarios in their experiment and a popularity algorithm, which is easier to implement and performs
well in specific contexts. Both algorithms are based on the number of objects
associated with tags. Our research presented a unique algorithm based on thefrequency of terms and photos.
Much research has presented ideas for integrating information in the lightof Personal Information Management (PIM) [9], to overcome information overload (e.g. [10]). History structure is simply generated from existing information
sources. Our research resembles text-based lifelog research. Our approach is simple and does not need special devices to capture information from the world.

5Summary

We improved upon what we developed in our previous work and presented acomplete new system using tag clouds for recalling personal memories. The
main differences from our previous work are as follows. (1) Multiple information sources such as SNSs or instant messengers can be used. (2) To handle
multiple information sources, we presented a new unified keyword-weighting algorithm. (3) We implemented new functions, such as keyword search, tag search,8Harumi Murakami and Ryutaro Murakami

and photo display, to build a complete system. Preliminary experiments reveal
the usefulness of our system for recalling personal memories.

References
1. Matsumoto M., Matsuura S., Mitsuhashi K. Murakami H.: Supporting Human Recollection of the Impressive Events using the Number of Photos. In: Proceedings of
the 6th International Conference on Agents and Artificial Intelligence - Volume 1:
ICAART, pp. 538-543. (2014)
2. Murakami, H.: History Structure for Exploring Desktop Data, In: Proceedings of theSIGIR 2010 Workshop on Desktop Search (Understanding, Supporting and Evaluating Personal Data Search), pp. 25-26. (2010)
3. Murakami H., Mitsuhashi K., Senba K.: Creating User’s Knowledge Space from
Various Information Usages to Support Human Recollection. In: Jiang H., DingW., Ali M., Wu X. (eds.) IEA/AIE 2012. LNCS, vol. 7345, pp. 596-605. Springer,Heidelberg (2012) https://doi.org/10.1007/978-3-319-60045-1 6
4. Chen, Y., Jones, G. F.: Augmenting Human Memory using Personal Lifelogs. In:
Proceedings of the 1st Augmented Human International Conference. ACM, NewYork (2010)
5. Mathur, A., Majumder, A., Datta, S., Menon, S., Malhorta, S., Dahiya, A.: LifeView: a lifelog visualization tool for supporting sentimental recall and sharing. In:OzCHI’12: Proceedings of the 24th Australian Computer-Human Interaction Conference, pp. 371-380. ACM, New York (2012)
6. Aires, J., Goncalves, D.: Personal Information Dashboard - Me, At a Glance. In:
PIM 2012, http://pimworkshop.org/2012/. (2012)
7. Rivadeneira, A. W., Gruen, D. M., Muller, M. J., Millen, D. R.: Getting our head in
the clouds: toward evaluation studies of tagclouds. In: CHI ’07 Proceedings of theSIGCHI Conference on Human Factors in Computing Systems, pp. 995-998. ACM,New York (2007)
8. Venetis, P., Koutrika, G., Garcia-Molina, H.: On the selection of tags for tag clouds.
In: WSDM ’11 Proceedings of the fourth ACM international conference on Web
search and data mining, pp. 835-844. ACM, New York (2011)
9. Jones, W.: Personal Information Management, ARIST 41 Annual Review of Information Science and Technology, 453–504 (2007)
10. Dumais, S., Cutrell, D., Cadiz, J.J., Jancke, G., Sarin, R., Robins, D.C.: Stuff
I’ve Seen: A System for Personal Information Retrieval and Re-Use. In: SIGIR ’03:
Proceedings of the 26th annual international ACM SIGIR conference on Researchand development in information retrieval, pp. 72-79. ACM, New York (2003)