A Travel Planning System Based on Travel
Trajectories Extracted from a Large Number ofGeotagged Photos on the Web
Kohya Okuyama and Keiji Yanai
The University of Electro-Communications, Tokyo, Japan
Abstract. Due to the recent wide spread of camera devices with GPS,the number of geotagged photos on the Web is increasing rapidly. Some
image retrieval systems and travel recommendation systems which makeuse of geotagged images on the Web have been proposed so far. Whilemost of them handle a large number of geotagged images as a set of
location points, in this paper we handle them as sequences of location
points. We propose a travel route recommendation system which utilizes
actual travel paths extracted from a large number of photos uploaded bymany people on the Web.

1

Introduction

Due to the recent spread of devices having cameras and GPSs such as iPhone,Android phones and some GPS-equipped digital cameras, we can easily record
location information as well as digital photos. In general, photos with location
information are called “geotagged photos”. At the same time, some photo sharing
Web sites which can handle geotagged photos such as Flickr1 and Panoramio2
have become popular, and the number of geotagged photos on these sites has
been increasing rapidly. Since geotagged photos on the photo sharing sites canbe gathered via Web API easily, recently, many researches on geotagged photos
are being carried out in the ﬁeld of multimedia and computer vision.
In this paper, we propose a travel planning system which utilizes a large
number of geotagged photos on the Web and travel paths by many people extracted from them. In general, the places where many photos are taken by many
people means tourist places drawing attention of many tourists such as historical architectures, monuments, and beautiful scenic places. By gathering many
geotagged photos from the Web and analyzing them, we can get to know suchplaces easily. In fact, many works to extract tourist places automatically fromgeotagged photos on the Web have been proposed so far [4, 8, 7, 10].
In addition, if a person travels through several tourist places continuously
within a day or over several days, took geotagged photos at each of all thevisited places, and upload them to the photo sharing sites such as Flickr orPanoramio, we can extract travel traces from a sequence of geotagged photos
taken by the person. Using Web API provided by photo sharing sites, we canobtain user IDs as well as photo IDs and geotag information consisting of a setof values of latitude and longitude as metadata of photos. By obtaining metadata regarding a set of geotagged photos associated with a certain user ID, we
can obtain a sequence of geotag locations which expresses a travel path of theuser. This enables us to handle geotagged photos as not only a set of tourist
places but also a set of travel paths.
1
2

http://www.ﬂickr.com/
http://www.panoramio.com/

2

K. Okuyama and K. Yanai

Then, in this paper, we extract popular tourist places and travel paths ofmany Web users by analyzing a large number of geotagged photos, and propose a
travel route recommendation system using the extracted travel information. Our
proposed system can gather, aggregate and summarize travel route information,and recommends travel routes that many Web user’s preferences reﬂect.
This paper is organized as follows: We describe related work in Section 2.
In Section 3, we explain the overview of the proposed system and its detail. In
Section 4, we show experiments and examples of recommended travel paths. In
Section 5, we conclude this paper.

2

Related Work

Since there are so many geotagged photos on the Web nowadays, several researches have considered the problem of selecting representative or canonicalphotographs for popular locations for tourists. Jaﬀe et al. [4] selected a summary set of photos from a large collection of geotagged photographs based on
keyword tags and geotags. Simon et al. [8] have proposed a method to selectcanonical views for the landmarks by clustering images based on the visual similarity between two views. Kennedy et al. [5] attempted to generate representative
views for the world’s landmarks based on the clustering and on the generated
link structure. Zheng et al. [10] built a large-scale landmark image database including 5314 landmarks using about one million geotagged photos and twenty
million Web images.
Y. Zheng et al. [9] analyzed GPS log data recorded by handy GPS devices,and estimated popular human travel routes and places. Since their work used
GPS log data, they can analyze precise human traces. However, the problem isthat such data is very expensive to obtain, and large-scale analysis on GPS trace
data is impossible, since carrying handy GPS during travel is not very common.
Although travel paths extracted from geotagged photos are much coarser thanGPS traces, we can obtain them much more than GPS traces instead.
As works on travel planning using geotagged photos on the Web, Cao et
al. [2] proposed a travel planning system which recommends travel places basedon user’s preferences. This system recommends not routes and just only places.
On the other hand, X. Lu [6] proposed using travel traces extracted from geotagged photos on the Web for travel route recommendation. To generate travel
paths, they used all the places extracted from geotagged photos. On the other
hand, in our system, we use only geo-location information on popular tourist
places and the order of traveling among them, which are represented as “tripmodels” in this paper. Arase et al. [1] also proposed a travel recommendation
system which utilizes travel routes extracted from geotagged photos. While theirobjective is to recommend travel plans for the users who have not decided evenareas to visit, the objective of our system is to recommend travel routes withina given area for the uses who have decided the area to visit but have not decided
the tourist spots to visit within the area.

3

Proposed System

3.1

Overview

In this paper, we propose a travel route planning system, which recommends
several eﬃcient travel routes so as to visit user’s favorite places. We assume thatA Travel Planning System Based on Travel Trajectories

3

users use this system before starting trips after deciding which area they aregoing to travel to. Here, an “area” means a regional range over which tourist
places we usually visit during one trip are distributed. For example, New York,Paris, London, Kyoto and Tokyo are examples of “areas” in this paper. Our
system helps us to decide the places to visit and the order of the places to visit
within the given area. How to use the system is as follows:
1. The system presents popular places in the given area with representative
tags and photos, and then the user selects some favorite places.
2. The system presents several recommended routes which travel through thegiven places, and the user selects a favorite route.
3. The system shows the selected route on the online map.
To realize this system, as oﬄine processing, we extract representative tourist
places and common travel paths within the given area from a large number ofgeotagged photos collected from Web photo sharing sites in advance. This oﬄine
processing consists of the following four steps: (1) data collection (2) extractionof common tourist places (3) selection of representative photos and textural tagsfor each tourist place and (4) extraction and modeling of travel routes,
In this section, we explain the oﬄine processing ﬁrst, and the online processing of the proposed system next.
3.2

Data Collection and Selection

As Web photo sharing sites, we use Flickr 3 since Flickr has more than one billion
geotagged photos which can be searched and obtained easily via Flickr API 4 .
First of all, we obtain metadata of geotagged photos using the “ﬂickr.photos.search”
method of Flickr API. We obtain the following metadata items for each geotagged photo:¶³
Metadata used in the proposed system
id unique ID of the photoowner user ID who uploaded the photodate-taken date and time of taking the phototags text tags for the photo
latitude，longitude geotag information
accuracy accuracy of the given geotag. Higher value means high accuracy.
µ´
Before downloading photos, we remove noise photos and select only geotagged
photos suitable for extracting travel trajectories. First, we remove geotagged
photos the geotag accuracy of which are less than 11. Next, to extract travel
trajectories eﬀectively, we select only the photos uploaded by the users who
uploaded more than two photos taken in the same day.
Moreover, geotags of some photos are attached not by GPS but by clicking
online maps, and some of them sometimes have exactly the same geotags. Such
photos are also not appropriate for our work. Thus, we exclude all the pairs of
the photos whose geotag locations are exactly identical but whose taken time
are diﬀerent by more than ﬁve minutes. As an example, we show the number ofphotos before and after noise removal for “user A” and “user B” in Table 1. In
3
4

http://ﬂickr.com/
http://www.ﬂickr.com/services/api/

4

K. Okuyama and K. Yanai

this example, user A is estimated to be inaccurate regarding location information, while user B is estimated to be highly accurate, and probably used GPS
to attach geotags to his/her photos. For this work, user A is much more usefulthan user B.
Table 1. The number of photos before and after noise removal
user A
user B

3.3

number of original images number of images after cleaning
25189
192
4793
4766

Tourist Place Detection

To detect tourist places from geotagged photos, we apply clustering for geotagsof the collected photos. As clustering methods, you use a hierarchical clustering
as described as follows:
1. Initially, all the geotag locations are regarded as being cluster centers.
2. Aggregate two clusters the distance between which is the closest into one
new cluster. The location of the new cluster is deﬁned as being the average
location of two points.
3. If the closest distance between any two clusters become less than the predeﬁned threshold, clustering will be ﬁnished. Otherwise, repeat from Step2.
To compute distance D between two locations, we use the spherical distance ascomputed in the following equation:
ρ = R cos−1 {sin δA sin δB + cos δA cos δB cos(λA − λB )}(1)

where R represents the radius of the earth, and δA , λA , δB , λB represent latitude
and longitude of place A and place B, respectively.
In Figure 1, we show two geotag clustering results in case that the thresholds
are set as 100 meters and 400 meters. The left of the ﬁgure shows about 400
clusters in case of the 100-meter threshold , and the right shows about 100
clusters in case of the 400-meter threshold.

Fig. 1. (Left) Place clusters for the 100-meter threshold. (Right) Place clusters for the
400-meter threshold.

A Travel Planning System Based on Travel Trajectories

3.4

5

Extraction of Representative Tags and Photos

In this subsection, we describe a method to assign representative text tags andphotos to each of the common tourist clusters. Representative tags and photos
are helpful to explain the places both semantically and visually.
In general, text tags attached to Web photos are very various and diverse.
For example, the photos of the Kinkaku Temple typically have “Kinkakuji, Kyoto, Japan, temple” as tags. Among these tags, “Japan” and “ Kyoto” are notappropriate to explain the place since they means broad area. ”Kinkakuji” is thebest tag to explain the place of ”Kinkaku temple”, since it is a unique tag forthe place.
To select unique tags for the places, we compute a evaluation score of uniqueness of tags scorec,t to the places in the following equation:
Nc (t)c∈C Nc (t)scorec,t = ∑where C, c, t, and Nc (T ) represents all the clusters, a cluster, a tag, and thenumber of photos having tag t in cluster c, respectively. If scorec,t is more thanthe pre-deﬁned threshold, the tag t is regarded as being one of representative
tags of the cluster c.
In Table 2 and Table 3, we show two examples before and after the tag
selection on the clusters including the Kinkaku temple and the Kiyomizu temple.
In these examples, some tags which means broad areas such as “japan” and“kansai” are eliminated successfully.
Table 2. Frequent tags before the selection, and selected tags on the cluster including
the Kinkaku temple.
kinkakuji japanesebuses japan kansai
before selection
kyoto ofriceandzen
after selection
kinkakuji japanesebuses
Table 3. Frequent tags before the selection, and selected tags on the cluster including
the Kiyomizu temple.
october japan october kyoto kiyomizudera
before selection
2009 kiyomizutemple
after selection
kiyomizudera kiyomizutemple

Next, we select some representative photos for each cluster by employing
local-feature-based image analysis. We decide representative photos based on
the number of matched local feature points. As local features, we use SURF
（Speeded-Up Robust Features) [3]. We extract SURF descriptors from each of
the photos, and search for matched local point pairs within the same cluster.
We select the top ﬁve photos as representative ones in terms of the numberof matched points in each cluster. Two ﬁgures in Figure 2 show local feature
matching between two photos of the Kinkaku temple taken from the diﬀerent
angles and between two photos of the Kinkaku temple and the Kiyomizu temple.
In general, the pair of the photos of the same landmark bring much more matchedpoints than the pair of the diﬀerent landmarks. As a result, the landmark taken
in many photos gathers many matched points and it is selected as representative
one.

6

K. Okuyama and K. Yanai

Fig. 2. SURF point matching. (Left) between two photos of the Kinaku Temple.
(Right) between the Kinkaku and the Kiyomizu temple.

3.5

Modeling Travel Trajectories

In the proposed system, after selecting tourist place clusters and their tags andphotos, we extract travel trajectories of many people from geotagged photos ,and generate “trip models” from them which represent canonical move sequencesamong tourist place clusters.
At ﬁrst, we extract geotag sequences from geotagged photos taken by one
user within a certain day in the time order, and gather them regarding manyusers and many days as a set of one-day travel trajectories. We call an oneday travel trajectory as “a trip”. We show the places where a certain user tookgeotagged photos within the same day in the left of Figure 3, and their travel
path in the right of Figure 3.

Fig. 3. (Left) Photo places of a certain user in a day. (Right) Travel sequences.

Some “trips” include many geotag places densely, while some “trips” include
some places sparsely most of which are popular tourist places. In addition, some”trips” might include many geotag locations around the same landmarks. In thisway, the resolution of geotags varies depending on users greatly. Then, to makethe resolution of “trips” even and to remove redundant places, we convert a“trip” trajectory, which is a sequence of geotagged places, into a “trip model”,
which is a sequence of the moves between common tourist places detected in

A Travel Planning System Based on Travel Trajectories

7

Fig. 4. An example of generating a new “trip model”.

the previous section. In this paper, all the travel paths are represented as “tripmodels”.
Although we can extract “trip models” from geotagged photos in this way,
the number of “trip models” is not sometimes enough. Then, in the next step, we
generate new “trip models” by combining several “trip models”. The proposed
method to generete new “trip models” shown in Figure 4 is as follows:
1. Search for the two “trip models” both of which include the same tourist
places.
2. Concatenate the ﬁrst part of one trip model before the place and the latterpart of the other trip model after the place.
3. If the generated “trip model” does not include duplicate places, the “tripmodel” is regarded as being valid. Otherwise, it will be discarded.
Note that to prevent loops from being made in trip models, we limit the sameplace to being included once in one “trip model”.
In Figure 4, a new “trip model” is generated by linking the ﬁrst part of “tripmodel A” with the latter part of “trip model B”. In addition, we can concatenate
the ﬁrst part of “trip model B” and the latter part of “trip model A”.
3.6

Online System

In this subsection, we explain online processing on a route recommendation system which makes use of common tourist places and “trip models” extracted fromgeotagged photos. The online processing of the system consists of the following
three steps:
1. Selection of tourist places where a user like to visit
2. Presenting travel route candidates and selection from them
3. Presenting the selected travel route on the map
Each of three ﬁgures in Figure 5 corresponds to each of the above three steps.
Selection of places In the ﬁrst step, the system shows common tourist places
with their tags and representative photos, and asks a user to select some places
where he/she like to visit. As help for a user to select places, the system shows

8

K. Okuyama and K. Yanai

Fig. 5. Three steps in the online systemsome information on each place which are obtained via Yahoo! Local Search API
API 5 as well.
Showing and selection of “trip models” The system searches the “tripmodel” database for the routes including as many tourist places the user selected
as possible. To search for “trip models” quickly, we prepare an search index on“trip models” regarding each of common tourist places in advance.
Presenting a travel route on the map In the last step, the system shows theselected “trip model” on the map. “Trip models” do not contain road information
between tourist places, but contain only information on sequences of tourist
places. Then, we obtain common road routes between the tourist places by using
the Directions Service of Google Maps API 6 . Using this service as well as basic
function of Google Maps API, we can present the selected “trip model” on the
map as shown in the right ﬁgure of Figure 5.

4

Experiments

To gather a large number of geotagged images within areas where people travel
around in one day, popular tourist areas are appropriate. As a target area in theexperiment, we selected “Kyoto” which is one of the most popular tourist areasin Japan.
We gathered twenty thousand geotagged photos taken in the Kyoto areafrom Flickr via Flickr API. After noise removal described in Section 3.2, wecollected 1805 geotagged photos uploaded by 162 unique users. After carrying
out hierarchical clustering with the 400-meter threshold in terms of the radiusof clusters, we obtained 154 tourist place clusters and 18,742 “trip models”, partof which are shown in Table 6.
Figure 7 shows a part of common tourist place candidates with representative
tags and photos, and address information on the locations obtained from Yahoo!
Local Search API. The tourist places are shown in the descending order of thenumber of the cluster members of geotagged photos, that is, in the order of
popularity.
As a case study, we selected four places, “Kyoto Station”, “Fushimi temple”,“Uji bridge” and “Nijo castle”. As route candidates that go through the given5

http://developer.yahoo.co.jp/webapi/map/localsearch/
v1/localsearch.html
6
http://code.google.com/intl/ja/apis/maps/

A Travel Planning System Based on Travel Trajectories

9Fig. 6. Extracted common tourist places and “trip models” in “Kyoto”.
Table 4. The number of visiting tourist places, and the total and average distances of
the three routes.
Route candidate num. of places Total distance (km) Avg. dist. between places (km)Trip 19
29.8
3.73
Trip 215
52.7
3.76
Trip 3
11
54.1
5.41

four places, three route, “Trip 1”, “Trip 2” and “Trip 3”, are presented in theascending order of the total moving distance. When clicking each of route candidates, each of the three routes are displayed on the map as shown in Figure
8, Figure 9 and Figure 10, respectively. Table 4 shows the number of common
tourist places included in the route candidates, the total moving distances andthe average moving distances between the common tourist places.
From Table 4, a user who wants to move quickly between selected places will
select “Trip 1” or “Trip 2”, while a user who wants to visit as many tourist
places as possible might select “Trip 3”. In this way, selection from the route
candidates depends on the user’s preference greatly. Thus, the important thing ispresenting as many route candidates and their additional information as possible
which suit user’s preference conditions.

5

Conclusions

In this paper, we proposed a travel route recommendation system based onsequences of geotagged photos on the Web, which presents several travel route
candidates with representative tags and photos by selecting tourist places wherea user like to visit. To gather information on travel routes, we proposed “tripmodels” represented by the order sequences of tourist places. In the experiments,
we build a “trip model” database on the Kyoto area as a case study.
As future work, we plan to apply many other areas over the world than Kyoto
and extend the system to recommend travel routes taking account of travel time.

References
1. Arase, Y., Xie, X., Hara, T., Nishio, S.: Mining people’s trips from large scale
geo-tagged photos. In: Proc. of ACM International Conference Multimedia (2010)
2. Cao, L., Luo, J., Gallagher, A., Jin, X., Han, J., Huang, T.: A worldwide tourism
recommendation system based on geotagged web photos. In: Proc. of IEEE International Conference on Acoustics, Spech and Signal Processing (2010)

10

K. Okuyama and K. Yanai

Fig. 7. Part of place candidates.

Fig. 8. The route of “Trip1”. B: Uji
bridge. C: Kyoto St. D: Nijo castle.
I:Fushimi temple.

Fig. 9. The route of “Trip2”. C: Nijo castle. E: Kyoto St. H: Fushimi temple. O:
Uji bridge.

Fig. 10. The route of “Trip3”. A:
Fushimi temple. H: Uji bridge. I: Kyoto
St. J: Nijo castle.

3. Herbert, B., Andreas, E., Tinne, T., Luc, G.: Surf: Speeded up robust features.
Computer Vision and Image Understanding pp. 346–359 (2008)
4. Jaﬀe, A., Naaman, M., Tassa, T., Davis, M.: Generating summaries and visualization for large collections of geo-referenced photographs. In: Proc. of ACM SIGMM
International Workshop on Multimedia Information Retrieval. pp. 89–98 (2006)
5. Kennedy, L., Naaman, M.: Generating diverse and representative image search
results for landmarks. In: Proc. of the ACM International World Wide Web Conference. pp. 297–306 (2008)
6. Lu, X., Wang, C., Yang, J., Pang, Y., Zhang, L.: Photo2Trip: Generating travel
routes from geo-tagged photos for trip planning. In: Proc. of ACM International
Conference Multimedia (2010)
7. Quack, T., Leibe, B., Gool, L.V.: World-scale mining of objects and events from
community photo collections. In: Proc. of ACM International Conference on Imageand Video Retrieval. pp. 47–56 (2008)
8. Simon, I., Snavely, N., Seitz, S.M.: Scene summarization for online image collections. In: Proc. of IEEE International Conference on Computer Vision (2007)
9. Zheng, Y., Zhang, L., Xie, X., Ma, W.Y.: Mining interesting locations and travel
sequences from gps trajectories. In: Proc. of ACM World Wide Web Conference(2009)
10. Zheng, Y., Zhao, M., Song, Y., Adam, H., Buddemeier, U., Bissacco, A., Brucher,F., Chua, T., Neven, H.: Tour the world: building a web-scale landmark recognitionengine. In: Proc. of IEEE Computer Vision and Pattern Recognition (2009)