Announcing the Final Examination of Kai Li for the degree of Doctor of Philosophy Time & Location: June 6, 2017 at 10:00 AM in HEC 450 Title: Hashing for Multimedia Similarity Modeling and Large-scale Retrieval

In recent years, the amount of multimedia data such as images, texts, and videos have been growing rapidly on the Internet. Motivated by such trends, this thesis is dedicated to exploiting hashingâ€"based solutions to reveal multimedia data correlations and support intraâ€"media and interâ€"media similarity search among huge volumes of multimedia data. We start by investigating a hashingâ€"based solution for audioâ€"visual similarity modeling and apply it to the audioâ€"visual sound source localization problem. We show that synchronized signals in audio and visual modalities demonstrate similar temporal changing patterns in certain feature spaces. We propose to use a permutationâ€"based random hashing technique to capture the temporal order dynamics of audio and visual features by hashing them along the temporal axis into a common Hamming space. In this way, the audioâ€"visual correlation problem is transformed into a similarity search problem in the Hamming space. Our hashingâ€"based audioâ€"visual similarity modeling has shown superior performances in the localization and segmentation of sounding objects in videos. The success of the permutationâ€"based hashing method motivates us to generalize and formally define the supervised rankingâ€"based hashing problem, and study its application to largeâ€"scale image retrieval. Specifically, we propose an effective supervised learning procedure to learn optimized rankingâ€"based hash functions that can be used for largeâ€"scale similarity search. Compared with the randomized version, the optimized rankingâ€"based hash codes are much more compact and discriminative. Moreover, it can be easily extended to kernel space to discover more complex ranking structures that cannot be revealed in linear subspaces. Experiments on large image datasets demonstrate the effectiveness of the proposed method for image retrieval. We further studied the rankingâ€"based hashing method for the crossâ€"media similarity search problem. Specifically, we propose two optimization methods to jointly learn two groups of linear subspaces, one for each media type so that featuresâ€TM ranking orders in different linear subspaces maximally preserve the crossâ€"media similarities. Additionally, we develop this rankingâ€"based hashing method in the crossâ€"media context into a flexible hashing framework with a more general solution. We have demonstrated through extensive experiments on several realâ€"world datasets that the proposed crossâ€"media hashing method can achieve superior crossâ€"media retrieval performances against several stateâ€"ofâ€"theâ€"art algorithms. Lastly, to make better use of the supervisory label information, as well as to further improve the efficiency and accuracy of supervised hashing, we propose a novel multimedia discrete hashing framework that optimizes an instanceâ€"wise loss objective, as compared to the pairwise losses, using an efficient discrete optimization method. In addition, the proposed method decouples the binary codes learning and hash function learning into two separate stages, thus making the proposed method equally applicable for both singleâ€"media and crossâ€"media search. Extensive experiments on both singleâ€"media and crossâ€"media retrieval tasks demonstrate the effectiveness of the proposed method.