Online Hashing with Similarity Learning

Online hashing methods usually learn the hash functions online, aiming to efficiently adapt to the data variations in the streaming environment. However, when the hash functions are updated, the binary codes for the whole database have to be updated to be consistent with the hash functions, resulting in the inefficiency in the online image retrieval process. In this paper, we propose a novel online hashing framework without updating binary codes. In the proposed framework, the hash functions are fixed and a parametric similarity function for the binary codes is learnt online to adapt to the streaming data. Specifically, a parametric similarity function that has a bilinear form is adopted and a metric learning algorithm is proposed to learn the similarity function online based on the characteristics of the hashing methods. The experiments on two multi-label image datasets show that our method is competitive or outperforms the state-of-the-art online hashing methods in terms of both accuracy and efficiency for multilabel image retrieval. Introduction The large amount of visual data (e.g. images and videos) that is produced everyday leads to the development of efficient index and retrieval methods. Among these methods, hashing methods that perform approximate nearest neighbor search by mapping high-dimensional data to binary codes have attracted increasing attention [Wang et al. 2016; Wang et al. 2018]. Locality Sensitive Hashing (LSH) [Andoni and Indyk 2006] is a representative of the hashing methods. By generating as hash functions random projection vectors sampled from a multidimensional Gaussian distribution with zero mean and identity covariance matrix, LSH can hash similar data into similar binary codes in a high probability. However, LSH is a data-independent hashing method and usually needs long binary codes to achieve a good precision. To obtain compact binary codes for the data, data-dependent hashing methods [Weiss, Torralba, and Fergus 2009; Weiss, Fergus, and Torralba 2012; He, Wang, and Cheng 2019; Weng and Zhu 2020a] employ machine learning techniques to learn the hash functions according to the data. For example, some hashing methods [Ge, He, and Sun Copyright © 2021, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. 2014; Liu et al. 2014; Li, Hu, and Nie 2017] build graphs to describe the relationship between the data and learn the hash functions from the graphs. Some hashing methods [Gong et al. 2013; He, Wen, and Sun 2013] adopt dimensionality reduction techniques to project the data into a low-dimensional space and quantize each dimension into a bit. Due to the development of deep learning methods, deep hashing methods [Shen et al. 2018; Do, Doan, and Cheung 2016; Cao et al. 2018; Wu et al. 2019] learn the neural networks as the hash functions. Since the memory space is limited, deep hashing methods learn the hash functions by using minibatch-based stochastic optimization which requires multiple passes over a given database. Although these methods can achieve good search performance, they are designed for the static database and have poor scalability for the ever-growing database. When new data arrives, the hash functions have to be re-learnt by accumulating all the training data, which is time-consuming. To address this challenge, online hashing methods [Lin et al. 2019; Cakir et al. 2017; Lin et al. 2020b, 2018a; Weng et al. 2019] employ online learning techniques to learn the hash functions online from the streaming data. For example, Online Sketching Hashing (OSH) [Leng et al. 2015] maintains a small-size data sketch from the streaming data and learns hash functions from the data sketch. Online Kernel Hashing (OKH) [Huang, Yang, and Zheng 2018] constructs a loss function by using the label information and updates the hash functions according to the loss function. Balanced Similarity for Online Discrete Hashing (BSODH) [Lin et al. 2019] uses an asymmetric graph regularization to preserve the similarity between the streaming data and the existing dataset and update the hash function according to the graph. Online Supervised Hashing (OSupH) [Cakir, Bargal, and Sclaroff 2017] and Hadamard Matrix Guided Online Hashing (HMOH) [Lin et al. 2020a] explore how to generate the target codes according to the label information and update the hash functions according to the target codes. Although the current online hashing methods can achieve good search performance and are efficient in learning the hash functions online, they have to recompute the binary codes for the database when the hash functions are updated. It is inefficient since recomputing the binary codes needs to accumulate the whole database as shown in Fig. 1(a). Mutual Information Hashing (MIH) [Cakir et al.