Personal Photo Browser that can Classify Photos
by Participants and Situations
Tomoya ONISHI†, Ryosuke TOKUAMI, Yasuyuki KONO
Department of Informatics,
Kwansei Gakuin University.
2-1 Gakuen, Sanda, Hyogo 669-1337, Japan{rtokuami,kono}@kwansei.ac.jp

ABSTRACTThis paper demonstrates a photo browser which rearranges
photos referring to the persons who were close to the photographer when the photos were taken by consulting Bluetooth
device detection information. Most of Bluetooth devices accompany their owners. Each photo is tagged with Bluetooth
device-IDs which were detected around the moment when itwas taken. Employing the tag information, the system classifies the user’s photo archive into a layered cluster tree interms of tag similarity, and shows its user the photos of herselected cluster on either a map or timelines.

Satoshi NAKAMURA
Center for Promotion of Informatics Education,
Kyoto University.
Yoshida-Honmachi, Sakyo, Kyoto 606-8501, Japan
nakamura@dl.kuis.kyoto-u.ac.jp

1. The system clusters photos by context, i.e., persons whowere close to the user when she took the photos, by referring to Bluetooth device detection log, and2. shows her the clusters and their structures by three viewmodes, i.e., Graph Network, Map, and Timeline View.
3. She browses in her photo archive by switching among
the three views and retrieves/finds photo(s) by exploring
up/down the layered cluster tree and viewing the photos
which belong to the cluster she selected.

Bluetooth equipped devices, e.g., cellphones or PDAs, havebeen ubiquitous in everyday environment, and are identifiACM Classification: H5.2 [Information interfaces and preable using their MAC addresses as unique IDs. In addisentation]: User Interfaces. - Graphical user interfaces.
tion, many of these devices are discoverable from other devices in a communication range. The user takes photos while
General terms: Design, Human Factors
the system logs discoverable wireless devices as the tag for
Keywords: Life log, Bluetooth, tagging, digital photo browser. clustering photos. Our system rearranges photos by participants/situations identified from the detection log in addition
INTRODUCTION
to locations and timeline.
With the wide spread of digital cameras and camera phones,
TAGGING DEVICE DETECTION LOGS
we all have had a huge personal photo archives with tiny
The system tags each photo with Bluetooth devices’ uniquecosts. Methods for browsing/retrieving photos in such a huge
IDs by the moment each device is detected. The photograarchive are desired. A basic method is to sort by timestamps
pher carries not only her camera but also a PDA or a slateof the photos and/or to group photos with selectable time
PC for logging neighboring devices once/twice a minute (descales such as day, week or month. Another method is topends on logger device capabilities). The logger software
place photos on a map where each was taken. These methods
records the information of discoverable wireless devices thatare simple and concrete but inapplicable to huge archives.
consists of the moment when the device was detected, its
Gomi et al.[2] presented a photo browser which was based
MAC address and name (SSID). The logger device is also
on location, time and person detected by face recognition.
configured to be discovered from other photographers. Many
The method, however, requires faces appearing in photos andmobile gadgets such as PC and PDA equip Bluetooth functhus only commemorative photos can be clustered. Althoughtions and many of them are configured as discoverable. Alboth Nair et al. [5] and Monaghan et al. [4] proposed photo
most all mobile phones accompany their owners, so they aresharing among multiple users by aggregating the Bluetooth
good sources for grouping photos which are taken together
presence information collected from them, they did not givewith the owners.
interaction methods but proposed only sharing frameworks.
Hui et al.[3] clusters human relations by sensing Bluetooth
The system correlates obtained log data with photos by theirand Wi-Fi, but they didn’t give a concrete application.
time-stamps, i.e., tags each photo with device-IDs within themoment it was taken before and after 5 minutes. The tag of
Our idea is in the following:a photo is the set of Bluetooth device-IDs correlated to it in
Permission to make digital or hard copies of all or part of this work forother words. The system constructs a huge weighted graphpersonal or classroom use is granted without fee provided that copies arestructure, where each node represents one photo and eachnot made or distributed for profit or commercial advantage and that copies
edge represents correlation between two photos, and clusters
bear this notice and the full citation on the first page. To copy otherwise, tothe graph into clusters of correlated photos by the following
republish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.
process:
AVI’12, May 21-25, 2012, Capri Island, Italy.
Copyright 2012 ACM 978-1-4503-1287-5/12/05...$10.00.

† Currently, JFE Systems, Inc.

Figure 2: Map View (left) and Timeline View (right).
Figure 1: Snapshots of the Graph Network View.

1. The system adds an edge between every pair of nodes (photos) in the archive, if the tags of the nodes has intersection.
Each edge is weighted with Jaccard distance between thetags of end-nodes.
2. The graph is clustered by Girvan-Newman algorithm[6].
The clusters are layered by applying multiple “betweeness” thresholds of the algorithm.
3. The system applies PageRank algorithm[1] to each cluster on each layer. The node with the highest PageRank isselected as the representative image of the cluster.
IMPLEMENTATIONThe system is implemented in Java using Swing library,
JUNG framework for Graph calculation and visualization,
and Google Static Maps API for map-based operations.
Graph Network View

The Graph Network View is the primary view mode of thesystem as shown in Figure 1. In the initial state of the system(Figure 1 left), clusters of the most abstract layer and theircorrelations are displayed. Each thumbnail corresponds to acluster and settled by Fruchterman-Reingold model, a forcebased graph drawing algorithm. The user can 1) switch toanother view by selecting one of radio buttons, 2) jump intoa cluster of more detailed layer by clicking a thumbnail, 3)or climb back layers by manipulating the slide-bar. At thebottom layer each thumbnail corresponds to a photo (Figure1 right) and she can find correlations among the photos.
Map View and Timeline View

Although the Map View displays the same structure with
the Graph View, i.e., each thumbnail corresponds to either
a cluster or a photo depending on the selected cluster, thecoordinates of the thumbnail is the map location where it
is taken (Figure 2 left). Certain community members, e.g.,
SIG-conference participants, tend to meet in many places
and photos which are taken while participating the meetings
tend to settled with certain distance on the conventional mapfaced photo browsers. Using our system, however, she canfind the correlation by seeing the edges in this view.
The Timeline View displays photos of the selected period oftime, i.e., year, month, week, day, in either of the wholearchive or the selected cluster in order of their timestamps.
As shown in Figure 2 (right), each vertical red line corresponds to a photo and the user can see the thumbnail by mov-Table 1: Scale of data and photo archive of subject 1.
Bluetooth ID
Wi-Fi IDYear
# unique# record # unique# record # photo
2009(Apr.-)
34,604 1,078,067
8,523
838,951
4,563
2010
34,063 1,325,123
33,659 2,878,434
6,125
2011
17,356 1,386,228
39,771 1,587,977
7,743

ing the mouse cursor over the red line.
CONCLUDING REMARKS

We have proposed and implemented a photo browser whichrearranges photos referring to the persons who were close
to the photographer when the photos are taken and incorporates conventional browsing interfaces, i.e., map and timeline. Three subjects have been pooling neighboring Bluetooth and Wi-Fi device-IDs in their everyday lives for morethan a year. One of them started pooling in Apr. 2009 andthe scale of the archive is summarized in Table 1.
Table 1 shows the number of detected unique Bluetooth devices are decreasing, because smartphones, which are discoverable only while they are manually set to discovery mode,
are rapidly spreading and conventional cellphones, which aregenerally always discoverable, e.g., Blackberries, are disappearing. On the other hand, Table 1 indicates that mobile WiFi access points including smartphones with tethering functions are quickly increasing. We are currently working on
additionally applying the system to Wi-Fi detection log.
REFERENCES

1. S. Brin and L. Page. The Anatomy of a Large-Scale Hypertextual Web Search Engine, Proc. WWW’98, 1998.
2. A. Gomi and T. Itoh. A Personal Photograph Browser forLife Log Analysis based on Location, Time, and Person,
ACM Symp. Applied Computing, pp. 1250-1257, 2011.
3. P. Hui and J. Crowcroft. Human mobility models and opportunistic communications system design. Philosophical Trans. Royal Society A: Mathematical, Physical andEngineering Sciences, 366(1872), pp. 2005-2016, 2008.
4. F. Monaghan, et al. Generating Useful Photo Context
Metadata for the Semantic Web, Proc. MDM’06, 2006.
5. R. Nair and M. Davis. Bluetooth Pooling to Enrich CoPresense Information, Adjunct Proc. UbiComp’05, 2005.
6. M.E.J. Newman and M. Girvan. Finding and Evaluating Community Structure in Networks, Phys. Rev., E69,026113, 2004.