DISSERTATION ABSTRACT

Geo-Temporal Mining and Searching of Eventsfrom Web-based Image Collections
Massimiliano Ruocco
Norwegian University of Science and Technology
ruocco@idi.ntnu.noAbstract
The proliferation of Web- and social media-based photo-sharing applications have not
only opened many possibilities but also resulted in new needs and challenges. They haveresulted in a large amount of personal photos being available for public access. One of
the most interesting characteristics of these data is that they are surrounded by 1) textualannotations, also called tags, which are intended to describe and categorize, by collective
user efforts, the uploaded resources 2) temporal information referring to when a pictures has
been taken and often by 3) a locational information describing where the picture has beentaken. Despite the recent developments and technological advances in Web-based mediasharing applications, the continuously increasing amount of available information has madethe access to the photos a demanding task. In general, we can address this challenge byallowing photo collections to be organized and browsed through the concept of events. We
also believe most users are familiar with searching photo collections using events as starting
points.
Aiming at supporting the detection and search of event-related photos, this thesis proposes a novel framework for extracting pictures related to real-life events from a collectionof Web-images by leveraging on their temporal geographical and textual annotations andcomparing the proposed approach with existing related state-of-the-art approaches. Second,a set of geographical features is proposed describing the characteristics of the geographicalprofile of query terms deriving concepts from exploratory analysis. Third, the thesis provides
two different tag-based search framework to improve the effectiveness of searching images related to events. The first framework is based on temporal and geographical proximity ofquery terms to the temporal neighbours of a given timestamped query, while the second
framework is based on a novel machine-learning based query expansion method combining
the heterogeneous textual, temporal and geographical similarity between query terms andcandidate expansion terms for the selection of the expansion terms given a free text textual
query.
All the proposed methods have been evaluated by performing extensive experiments onreal data gathered from media-sharing applications on the Web. Where possible, comparison with related techniques has been performed to reinforce the validity of the presented
approaches. The proposed methods have shown promising results in both the extraction andclustering of event-related images and searching event-related pictures by using metrics fromACM SIGIR Forum

119

Vol. 48 No. 2 December 2014

the state of the art.
This doctoral work has been performed at the Department of Computer and Information Science at the Norwegian University of Science and Technology (NTNU) under adviseof Associate Professor Heri Ramampiaro and with co-supervisors Associate Professor Roger
Midtstraum and Associate Professor Randi Karlsen. Professor Ramesh Jain (University ofCalifornia Irvine), Assistant Professor Claudia Hauff (TU Delft) and Associate Professor
Trond Aalberg (NTNU) served as dissertation committee members.
Available online at: http://www.idi.ntnu.no/research/doctor_theses/ruocco.pdf

ACM SIGIR Forum120Vol. 48 No. 2 December 2014