Original: R EVISITING L OCALITY -S ENSITIVE B INARY C ODES FROM R ANDOM F OURIER F EATURES
Revised: R EVISITING L OCALITY -S ENSITIVE B INARY C ODES FROM R ANDOM F OURIER F EATURES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The method of Random Fourier Feature (RFF) has been popular for large-scale learning, which generates non-linear random features of the data.
Revised: The method of Random Fourier Feature (RFF) has been popular for large-scale learning, which generates non-linear random features of the data.

Original: It has also been used to construct binary codes via stochastic quantization for efﬁcient information retrieval.
Revised: It has also been used to construct binary codes via stochastic quantization for efﬁcient information retrieval.

Original: In this paper, we revisit binary hashing from RFF, and propose SignRFF, a new and simple strategy to extract RFF-based binary codes.
Revised: In this paper, we revisit binary hashing from RFF, and propose SignRFF, a new and simple strategy to extract RFF-based binary codes.

Original: We show the locality-sensitivity of SignRFF, and propose a new measure called ranking efﬁciency to theoretically compare different Locality-Sensitive Hashing (LSH) methods, which provides the ﬁrst systematic and uniﬁed framework for LSH comparison.
Revised: We show the locality-sensitivity of SignRFF and propose a new measure called ranking efﬁciency to theoretically compare different Locality-Sensitive Hashing (LSH) methods, which provides the a new systematic and uniﬁed framework for LSH comparison.

Original: It suggests that SignRFF should be preferred in the high similarity region.
Revised: It suggests that SignRFF should be preferred in the high similarity region.

Original: Experiments are conducted to show that SignRFF is consistently better than the previous RFF-based method, and also outperforms other data-dependent and deep learning based hashing methods with sufﬁcient number of hash bits.
Revised: Experiments are conducted to show that SignRFF is consistently better than the previous RFF-based method and also outperforms other data-dependent and deep learning based hashing methods with sufﬁcient number of hash bits.

Original: Moreover, the proposed ranking efﬁciency aligns well with the empirical search performance.
Revised: Moreover, the proposed ranking efﬁciency aligns well with the empirical search performance.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Developing efﬁcient machine learning algorithms in large-scale problems has been an important research topic to deal with massive data.
Revised: Developing efﬁcient machine learning algorithms in large-scale problems has been an important research topic to deal with massive data.

Original: In this paper, we focus on efﬁcient retrieval/search methods, speciﬁcally by designing similarity-preserving binary representations of the data.
Revised: In this paper, we focus on efﬁcient retrieval/search methods, speciﬁcally, by designing similarity-preserving binary representations of the data.

Original: That is, for each data vector x ∈ R d , we hash it into a length- b binary 0/1 vector h ( x ) ∈ { 0 , 1 } b , where the geometry of the data should be well preserved in the Hamming space.
Revised: That is, for each data vector x ∈ R d , we hash it into a length- b binary 0/1 vector h ( x ) ∈ { 0 , 1 } b , where the geometry of the data should be well preserved in the Hamming space.

Original: Searching with binary codes has been widely applied in many applications, such as large-scale image retrieval (Weiss et al., 2008; Gong &
Revised: Searching with binary codes has been widely applied in many applications, such as large-scale image retrieval (Weiss et al., 2008; Gong &

Original: Lazebnik, 2011; Kulis & Darrell, 2009; He et al., 2013; Lin et al., 2015; Liu et al., 2016; 2017; Song et al., 2018; Yu et al., 2018; Yan et al., 2021).
Revised: Lazebnik, 2011; Kulis & Darrell, 2009; He et al., 2013; Lin et al., 2015; Liu et al., 2016; 2017; Song et al., 2018; Yu et al., 2018; Yan et al., 2021).

Original: The beneﬁts are two-fold.
Revised: The beneﬁts are two-fold.

Original: Firstly, it may largely reduce the memory cost for storing massive datasets, especially with high-dimensional data.
Revised: Firstly, it may largely reduce the memory cost for storing massive datasets, especially with high-dimensional data.

Original: Secondly, it can signiﬁcantly speedup the retrieval process.
Revised: Secondly, it can signiﬁcantly speedup the retrieval process.

Original: For instance, the binary codes can be used to build hash tables (e.g., Indyk & Motwani (1998)) for sub-linear time approximate nearest neighbor (ANN) search.
Revised: For instance, the binary codes can be used to build hash tables (e.g., Indyk & Motwani (1998)) for sub-linear time approximate nearest neighbor (ANN) search.

Original: Moreover, in the Hamming space, we can also apply exhaustive search, which is much faster than computing the Euclidean distances, plus the technique of multi-index hashing (Norouzi et al., 2012) can further accelerate the exact Hamming search by orders of magnitude.
Revised: Moreover, in the Hamming space, we can also apply exhaustive search, which is much faster than computing the Euclidean distances, plus the technique of multi-index hashing (Norouzi et al., 2012) can further accelerate the exact Hamming search by orders of magnitude.

Original: In general, binary hashing methods can be categorized into supervised and unsupervised approaches.
Revised: In general, binary hashing methods can be categorized into supervised and unsupervised approaches.

Original: In this work, we focus on the unsupervised setting.
Revised: In this work, we focus on the unsupervised setting.

Original: Locality-Sensitive Hashing (LSH) (Indyk & Motwani, 1998) is one of the early hashing methods leading to binary embedding.
Revised: Locality-Sensitive Hashing (LSH) (Indyk & Motwani, 1998) is one of the early hashing methods leading to binary embedding.

Original: The LSH targeting at cosine similarity (Charikar, 2002), also known as SimHash, generates binary hash codes using random hyperplanes by taking the sign of data-independent random Gaussian projections.
Revised: The LSH targeting at cosine similarity (Charikar, 2002), also known as SimHash, generates binary hash codes using random hyperplanes by taking the sign of data-independent random Gaussian projections.

Original: For the cosine similarity, LSH has strict theoretical guarantee on the approximation error and search efﬁciency, but typically requires relatively long codes to achieve good performance.
Revised: For the cosine similarity, LSH has strict theoretical guarantee on the approximation error and search efﬁciency, but typically requires relatively long codes to achieve good performance.

Original: Hence, many works have focused on learning data-dependent short binary hash codes, through different objective functions.
Revised: Hence, many works have focused on learning data-dependent short binary hash codes, through different objective functions.

Original: Examples include Iterative Quantization (ITQ) (Gong & Lazebnik, 2011), Spectral Hashing (SpecH) (Weiss et al., 2008) and Binary Reconstruction Embedding (BRE) (Kulis & Darrell, 2009).
Revised: Examples include Iterative Quantization (ITQ) (Gong & Lazebnik, 2011), Spectral Hashing (SpecH) (Weiss et al., 2008) and Binary Reconstruction Embedding (BRE) (Kulis & Darrell, 2009).

Original: Recently, some unsupervised deep learning based methods have been proposed in the computer vision community, many of which are, to some extent, “task-speciﬁc” for cross-modal/video/image retrieval, implemented based on some deep models like the autoencoder and VGG-16 (Liu et al., 2016;
Revised: Recently, some unsupervised deep learning based methods have been proposed in the computer vision community, many of which are, to some extent, “task-speciﬁc” for cross-modal/video/image retrieval, implemented based on some deep models like the autoencoder and VGG-16 (Liu et al., 2016;

Original: Do et al., 2017; Chen et al., 2018;
Revised: Do et al., 2017; Chen et al., 2018;

Original: Li et al.,2019; Yang et al., 2019; Hansen et al., 2020; Liu et al., 2020; Qiu et al., 2021).
Revised: Li et al., et al., 2020; Qiu et al., 2021).

Original: By taking the advantage of the complicated model structures (e.g.,
Revised: By taking the advantage of the complicated model structures (e.g.,

Original: CNN layers), these deep methods show promising performance in many image retrieval tasks.
Revised: CNN layers), these deep methods show promising performance in many image retrieval tasks.

Original: Compared with the data-dependent methods (including deep methods), LSH has three advantages:
Revised: Compared with the data-dependent methods (including deep methods), LSH has three advantages:

Original: • Although the data-dependent procedures can provide improved performance with fewer binary codes, a known undesirable effect of many of these mechanisms is the performance degradation when we increase the code length b , as reported in prior literature (Raginsky & Lazebnik, 2009; Joly & Buisson, 2011).
Revised: • Although the data-dependent procedures can provide improved performance with fewer binary codes, a known undesirable effect of many of these mechanisms is the performance degradation when we increase the code length b , as reported in prior literature (Raginsky & Lazebnik, 2009; Joly & Buisson, 2011).

Original: In our experiments, we will show that this may also be an issue for deep learning based methods.
Revised: In our experiments, we will show that this may also be an issue for deep learning based methods.

Original: On the other hand, the performance (i
Revised: On the other hand, the performance (i

Original: e., search accuracy) of the data-independent LSH would keep boosting with larger b .
Revised: e., search accuracy) of the data-independent LSH would keep boosting with larger b .

Original: In many scenarios, it is often the case that only using short codes (e
Revised: In many scenarios, it is often the case that only using short codes (e

Original: g., ≤ 128 bits) cannot achieve a desirable level of search accuracy for practical purposes.
Revised: g., ≤ 128 bits) cannot achieve a desirable level of search accuracy for practical purposes.

Original: In these cases, one may need to use longer codes anyway, where LSH could be more favorable.
Revised: In these cases, one may need to use longer codes anyway, where LSH could be more favorable.

Original: • LSH is very simple to implement (only with random projections), while data-dependent methods require additional optimization/training which might be costly.
Revised: • LSH is very simple to implement (only with random projections), while data-dependent methods require additional optimization/training which might be costly.

Original: Moreover, large deep learning models would typically need longer inference time, which might be infeasible in practice where the query speed is important.
Revised: Moreover, large deep learning models would typically need longer inference time, which might be infeasible in practice where the query speed is important.

Original: • It is difﬁcult to characterize the properties of the data-dependent methods theoretically, while LSH enjoys rigorous guarantees on the retrieval/approximation performance.
Revised: • It is difﬁcult to characterize the properties of the data-dependent methods theoretically, while LSH enjoys rigorous guarantees on the retrieval/approximation performance.

Original: Therefore, LSH is still a popular hashing method with great research interest and many practical applications (Shrivastava & Li, 2014;
Revised: Therefore, LSH is still a popular hashing method with great research interest and many practical applications (Shrivastava & Li, 2014;

Original: Qi et al., 2017; Driemel & Silvestri, 2017; Chen et al., 2019;
Revised: Qi et al., 2017; Driemel & Silvestri, 2017; Chen et al., 2019;

Original: Zandieh et al., 2020;
Revised: Zandieh et al., 2020;

Original: Lei et al., 2020; Daghaghi et al., 2021).
Revised: Lei et al., 2020; Daghaghi et al., 2021).

Original: 1. L OCALITY -S ENSITIVE H ASHING FROM R ANDOM F OURIER F EATURES
Revised: 1. L OCALITY -S ENSITIVE H ASHING FROM R ANDOM F OURIER F EATURES

Original: Kernel methods have gained great success in many machine learning tasks (Schölkopf & Smola, 2002; Zhu & Hastie, 2001; Avron et al., 2017; Sun et al., 2018).
Revised: Kernel methods have gained great success in many machine learning tasks (Schölkopf & Smola, 2002; Zhu & Hastie, 2001; Avron et al., 2017; Sun et al., 2018).

Original: However, standard kernel methods require the n × n kernel matrix, which becomes computationally expensive on large-scale datasets with massive data points.
Revised: However, standard kernel methods require the n × n kernel matrix, which becomes computationally expensive on large-scale datasets with massive data points.

Original: To this end, Rahimi & Recht (2007) proposes the method of random
Revised: To this end, Rahimi & Recht (2007) proposes the method of random

Original: Fourier feature (RFF), which deﬁnes a feature map that approximates shift-invariant kernels by the linear inner products.
Revised: Fourier feature (RFF), which deﬁnes a feature map that approximates shift-invariant kernels by the linear inner products.

Original: As such, the RFF preserves the “non-linear locality structure” of the data.
Revised: As such, the RFF preserves the “non-linear locality structure” of the data.

Original: This leads to numerous applications in large-scale learning where one trains linear models on RFF to approximate training non-linear kernel machines (Yang et al., 2012;
Revised: This leads to numerous applications in large-scale learning where one trains linear models on RFF to approximate training non-linear kernel machines (Yang et al., 2012;

Original: Chwialkowski et al., 2015;
Revised: Chwialkowski et al., 2015;

Original: Sutherland & Schneider, 2015; Avron et al., 2017; Sun et al., 2018; Tompkins & Ramos, 2018).
Revised: Sutherland & Schneider, 2015; Avron et al., 2017; Sun et al., 2018; Tompkins & Ramos, 2018).

Original: Given the popularity of RFF, one natural step is to apply it to search problems.
Revised: Given the popularity of RFF, one natural step is to apply it to search problems.

Original: Raginsky & Lazebnik (2009) proposes to construct binary codes from the RFF using stochastic binary quantization.
Revised: Raginsky & Lazebnik (2009) proposes to construct binary codes from the RFF using stochastic binary quantization.

Original: We call this method “SQ-RFF”, whose convergence (as b → ∞ ) and concentration can be theoretically characterized.
Revised: We call this method “SQ-RFF”, whose convergence (as b → ∞ ) and concentration can be theoretically characterized.

Original: The author showed that SQ-RFF can achieve competitive search accuracy with sufﬁcient b , e.g., b ≥ 512 .
Revised: The author showed that SQ-RFF can achieve competitive search accuracy with sufﬁcient b , e.g., b ≥ 512 .

Original: Since RFF itself is a widely used tool for large-scale kernel learning, SQ-RFF could be a convenient/useful tool in practical scenarios where RFF has been generated for training kernel machines, and the data scientist wants to further use it for efﬁcient near neighbor retrieval.
Revised: Since RFF itself is a widely used tool for large-scale kernel learning, SQ-RFF could be a convenient/useful tool in practical scenarios where RFF has been generated for training kernel machines, and the data scientist wants to further use it for efﬁcient near neighbor retrieval.

Original: 1. O UR C ONTRIBUTIONS
Revised: 1. O UR C ONTRIBUTIONS

Original: Given the beneﬁts of LSH and RFF, we revisit hashing methods for non-linear kernels, and improve the prior RFF based hashing method.
Revised: Given the beneﬁts of LSH and RFF, we revisit hashing methods for non-linear kernels, and improve the prior RFF based hashing method.

Original: Our approach is named as “SignRFF”.
Revised: Our approach is named as “SignRFF”.

Original: Speciﬁcally,
Revised: Speciﬁcally,

Original: • We theoretically compare several linear and non-linear LSH methods in terms of a novel measure called the ranking efﬁciency , which is deﬁned based on the probability of retrieving a wrong/reversed similarity ranking.
Revised: • We theoretically compare several linear and non-linear LSH methods in terms of a novel measure called the ranking efﬁciency , which is deﬁned based on the probability of retrieving a wrong/reversed similarity ranking.

Original: Under this uniﬁed metric, the proposed SignRFF is uniformly better than SQ-RFF.
Revised: Under this uniﬁed metric, the proposed SignRFF is uniformly better than SQ-RFF.

Original: Moreover, the ranking efﬁciency also indicates that typically one should prefer SignRFF over linear LSH when the near neighbors are close to each other, which is the ﬁrst systematic comparison of linear vs. non-linear LSH in literature.
Revised: Moreover, the ranking efﬁciency also indicates that typically one should prefer SignRFF over linear LSH when the near neighbors are close to each other, which is the ﬁrst systematic comparison of linear vs. non-linear LSH in literature.

Original: • Empirically, we conduct near neighbor search experiments on benchmark image datasets to compare the proposed SignRFF with other popular hashing methods, which veriﬁes the superiority of SignRFF over SQ-RFF.
Revised: • Empirically, we conduct near neighbor search experiments on benchmark image datasets to compare the proposed SignRFF with other popular hashing methods, which veriﬁes the superiority of SignRFF over SQ-RFF.

Original: We validate that the ranking efﬁciency metric aligns well with the empirical results.
Revised: We validate that the ranking efﬁciency metric aligns well with the empirical results.

Original: Moreover, SignRFF outperforms various competing methods (including deep methods) with moderately large number of hash bits, indicating its advantage in applications to achieve a high search recall/precision.
Revised: Moreover, SignRFF outperforms various competing methods (including deep methods) with moderately large number of hash bits, indicating its advantage in applications to achieve a high search recall/precision.

Original:  P RELIMINARIES 2. L OCALITY -S ENSITIVE H ASHING (LSH)
Revised:  P RELIMINARIES 2. L OCALITY -S ENSITIVE H ASHING (LSH)

Original: In large-scale information retrieval, exhaustive search of the exact nearest neighbors is usually too expensive.
Revised: In large-scale information retrieval, exhaustive search of the exact nearest neighbors is usually too expensive.

Original: A common relaxation in this setting is the Approximate Nearest Neighbor (ANN) search, where we return the “good” neighbors of a query with high probability.
Revised: A common relaxation in this setting is the Approximate Nearest Neighbor (ANN) search, where we return the “good” neighbors of a query with high probability.

Original: In this paper, we will consider the search problem with data points in R d .
Revised: In this paper, we will consider the search problem with data points in R d .

Original: X denotes the database consisting of n data points, and q is a query point.
Revised: X denotes the database consisting of n data points, and q is a query point.

Original: x , y are two data points with ρ = cos( x , y ) .
Revised: x , y are two data points with ρ = cos( x , y ) .

Original: Deﬁnition 2.1 ( ˜ S -neighbor) .
Revised: Deﬁnition 2.1 ( ˜ S -neighbor) .

Original: For a similarity measure S : R d × R d (cid:55)→ R , the ˜ S -neighbor set of q is deﬁned as { x ∈ X : S ( x , q ) > ˜ S } .
Revised: For a similarity measure S : R d × R d (cid:55)→ R , the ˜ S -neighbor set of q is deﬁned as { x ∈ X : S ( x , q ) > ˜ S } .

Original: Deﬁnition 2.2 ( ( c, ˜ S ) -ANN) .
Revised: Deﬁnition 2.2 ( ( c, ˜ S ) -ANN) .

Original: Assume δ > 0 is a parameter.
Revised: Assume δ > 0 is a parameter.

Original: An algorithm A is a ( c, ˜ S ) -ANN method provided the following: with probability at least 1 − δ , for 0 < c < 1 , if there exists an ˜ S -neighbor of q in X , A returns a c ˜ S -neighbor of q .
Revised: An algorithm A is a ( c, ˜ S ) -ANN method provided the following: with probability at least 1 − δ , for 0 < c < 1 , if there exists an ˜ S -neighbor of q in X , A returns a c ˜ S -neighbor of q .

Original: One popular method satisfying Deﬁnition 2.2 is the Locality-Sensitive Hashing (LSH).
Revised: One popular method satisfying Deﬁnition 2.2 is the Locality-Sensitive Hashing (LSH).

Original: Deﬁnition 2.3 (Indyk & Motwani (1998)) .
Revised: Deﬁnition 2.3 (Indyk & Motwani (1998)) .

Original: A family of hash functions H is called ( ˜ S, c ˜ S, p 1 , p 2 ) - locality-sensitive for similarity measure S and 0 < c < 1 , if for ∀ x , y ∈ R d and hash function h uniformly chosen from H , it holds that: 1) If S ( x , y ) ≥ ˜ S , then P ( h ( x ) =
Revised: A family of hash functions H is called ( ˜ S, c ˜ S, p 1 , p 2 ) - locality-sensitive for similarity measure S and 0 < c < 1 , if for ∀ x , y ∈ R d and hash function h uniformly chosen from H , it holds that: 1) If S ( x , y ) ≥ ˜ S , then P ( h ( x ) =

Original: h ( y )) ≥ p 1 ; 2)
Revised: h ( y )) ≥ p 1 ; 2)

Original: If
Revised: If

Original: S ( x , y ) ≤ c ˜ S , then P ( h ( x ) = h ( y )) ≤ p 2 , with p 2 < p 1 .
Revised: S ( x , y ) ≤ c ˜ S , then P ( h ( x ) = h ( y )) ≤ p 2 , with p 2 < p 1 .

Original: A key intuition of LSH is that, similar data points with have higher chance of hash collision in the Hamming space.
Revised: A key intuition of LSH is that, similar data points with have higher chance of hash collision in the Hamming space.

Original: In this paper, we will speciﬁcally consider the LSH for the cosine similarity (Charikar, 2002).
Revised: In this paper, we will speciﬁcally consider the LSH for the cosine similarity (Charikar, 2002).

Original: For a data vector x ∈ R d , the LSH binary code is given by
Revised: For a data vector x ∈ R d , the LSH binary code is given by

Original: [Equation]
Revised: [Equation]

Original: where w T is a standard Gaussian random vector.
Revised: where w T is a standard Gaussian random vector.

Original: We use b i.i.d.
Revised: We use b i.i.d.

Original: w 1 , ..., w b to generate b LSH binary codes.
Revised: w 1 , ..., w b to generate b LSH binary codes.

Original: The collision probability between the codes of two data points x , y is
Revised: The collision probability between the codes of two data points x , y is

Original: [Equation]
Revised: [Equation]

Original: where ρ = cos( x , y ) is their cosine similarity.
Revised: where ρ = cos( x , y ) is their cosine similarity.

Original: Note that P ( h LSH ( x ) = h LSH
Revised: Note that P ( h LSH ( x ) = h LSH

Original: ( y )) is increasing in ρ , which, By Deﬁnition 2.3, is the key to ensure the locality sensitivity.
Revised: ( y )) is increasing in ρ , which, By Deﬁnition 2.3, is the key to ensure the locality sensitivity.

Original: 2. K ERNELIZED L OCALITY -S ENSITIVE H ASHING (KLSH)
Revised: 2. K ERNELIZED L OCALITY -S ENSITIVE H ASHING (KLSH)

Original: In this paper, we will consider the popular Gaussian kernel function deﬁned for x , y ∈ R d as
Revised: In this paper, we will consider the popular Gaussian kernel function deﬁned for x , y ∈ R d as

Original: [Equation]
Revised: [Equation]

Original: where γ is a hyper-parameter.
Revised: where γ is a hyper-parameter.

Original: Let Ψ : R d (cid:55)→ F be the feature map with the kernel induced feature space F .
Revised: Let Ψ : R d (cid:55)→ F be the feature map with the kernel induced feature space F .

Original: To incorporate non-linearity in LSH, Kulis & Grauman (2009) proposes Kernelized
Revised: To incorporate non-linearity in LSH, Kulis & Grauman (2009) proposes Kernelized

Original: Locality-Sensitive Hashing (KLSH), by applying LSH (1) in the kernel induced feature space F , i.e., h ( x ) = sign ( w T Ψ( x )) .
Revised: Locality-Sensitive Hashing (KLSH), by applying LSH (1) in the kernel induced feature space F , i.e., h ( x ) = sign ( w T Ψ( x )) .

Original: However, as in many cases (e.g., for the Gaussian kernel) the map Ψ cannot be explicitly identiﬁed, constructing the random Gaussian projection vector w needs some careful design.
Revised: However, as in many cases (e.g., for the Gaussian kernel) the map Ψ cannot be explicitly identiﬁed, constructing the random Gaussian projection vector w needs some careful design.

Original: KLSH approximates the random Gaussian distribution using a sufﬁcient number of data points, by the Central Limit Theorem (CLT) in the Reproducing Kernel Hilbert Space.
Revised: KLSH approximates the random Gaussian distribution using a sufﬁcient number of data points, by the Central Limit Theorem (CLT) in the Reproducing Kernel Hilbert Space.

Original: One ﬁrst samples m data points from X to form a kernel matrix K , then uniformly picks t points from [1 , ..., m ] at random to approximate the Gaussian distribution.
Revised: One ﬁrst samples m data points from X to form a kernel matrix K , then uniformly picks t points from [1 , ..., m ] at random to approximate the Gaussian distribution.

Original: The hash code has the form
Revised: The hash code has the form

Revised: KLSH

Original: [Equation]
Revised: [Equation]

Original: where w = K − 1 / 2 e t , and e t ∈ { 0 , 1 } m has ones in the entries with indices of the t selected points.
Revised: where w = K − 1 / 2 e t , and e t ∈ { 0 , 1 } m has ones in the entries with indices of the t selected points.

Original: We see that the codes h 1 , ..., h b are actually dependent, and the quality of using CLT to approximate Gaussian distribution is not guaranteed, especially in high-dimensional kernel feature space.
Revised: We see that the codes h 1 , ..., h b are actually dependent, and the quality of using CLT to approximate Gaussian distribution is not guaranteed, especially in high-dimensional kernel feature space.

Original: Jiang et al.
Revised: Jiang et al.

Original: re-formulates KLSH as applying LSH on the kernel principle components in the kernel induced feature space, resolving a theoretical concern of KLSH.
Revised: re-formulates KLSH as applying LSH on the kernel principle components in the kernel induced feature space, resolving a theoretical concern of KLSH.

Original: Notably, since KLSH uses a pool of data samples to approximate the Gaussian distribution, the hash codes are in fact dependent in practical implementation.
Revised: Notably, since KLSH uses a pool of data samples to approximate the Gaussian distribution, the hash codes are in fact dependent in practical implementation.

Original: It is observed that the performance of KLSH also drops as b increases (Joly & Buisson, 2011), similar to the behavior of many data-dependent methods.
Revised: It is observed that the performance of KLSH also drops as b increases (Joly & Buisson, 2011), similar to the behavior of many data-dependent methods.

Original: We provide more detailed explanation in Appendix C.
Revised: We provide more detailed explanation in Appendix C.

Original:  L OCALITY -S ENSITIVE B INARY C ODES F ROM R ANDOM F EATURES
Revised:  L OCALITY -S ENSITIVE B INARY C ODES F ROM R ANDOM F EATURES

Original: Rahimi & Recht (2007) proposes random Fourier feature (RFF) to alleviate the computational bottleneck of standard kernel methods.
Revised: Rahimi & Recht (2007) proposes random Fourier feature (RFF) to alleviate the computational bottleneck of standard kernel methods.

Original: For a data vector x , the RFF for the Gaussian kernel is deﬁned as
Revised: For a data vector x , the RFF for the Gaussian kernel is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where w ∼ N (0 , γ 2 I d ) and τ ∼ Unif (0 , 2 π ) where I d is the identity matrix
Revised: where w ∼ N (0 , γ 2 I d ) and τ ∼ Unif (0 , 2 π ) where I d is the identity matrix

Original: It holds that
Revised: It holds that

Original: [Equation]
Revised: [Equation]

Original: That is, the non-linear kernel similarity can be preserved in expectation by the linear inner product of RFF.
Revised: That is, the non-linear kernel similarity can be preserved in expectation by the linear inner product of RFF.

Original: To obtain good approximation, we generate b independent RFFs for each data point using i.i.d.
Revised: To obtain good approximation, we generate b independent RFFs for each data point using i.i.d.

Original: w 1 , ..., w b and τ 1 , ..., τ b , which can be subsequently used for various learning tasks.
Revised: w 1 , ..., w b and τ 1 , ..., τ b , which can be subsequently used for various learning tasks.

Original: From now on, we will assume the data vectors are normalized to have unit l 2 norm: 1) this is a standard pre-processing step in practice; and 2) the theoretical guarantees of LSH (1) is only valid for the cosine similarity (i.e., normalized data).
Revised: From now on, we will assume the data vectors are normalized to have unit l 2 norm: 1) this is a standard pre-processing step in practice; and 2) the theoretical guarantees of LSH (1) is only valid for the cosine similarity (i.e., normalized data).

Original: Nevertheless, our analysis can be easily extended to the general unnormalized case.
Revised: Nevertheless, our analysis can be easily extended to the general unnormalized case.

Original: With normalized data, the Gaussian kernel can be written as a function of ρ = cos( x , y ) :
Revised: With normalized data, the Gaussian kernel can be written as a function of ρ = cos( x , y ) :

Original: [Equation]
Revised: [Equation]

Original: To extend RFF to efﬁcient search problems, Raginsky & Lazebnik (2009) designs a mapping [0 , 1] (cid:55)→ { 0 , 1 } for each RFF.
Revised: To extend RFF to efﬁcient search problems, Raginsky & Lazebnik (2009) designs a mapping [0 , 1] (cid:55)→ { 0 , 1 } for each RFF.

Original: For x ∈ R d , the code is produced by stochastic quantization (SQ):
Revised: For x ∈ R d , the code is produced by stochastic quantization (SQ):

Original: [Equation]
Revised: [Equation]

Original: where ξ is a random perturbation from Unif ( − 1 , 1) , with two possible choices on the dependency.
Revised: where ξ is a random perturbation from Unif ( − 1 , 1) , with two possible choices on the dependency.

Original: In Raginsky & Lazebnik (2009), the same ξ is used for all data points.
Revised: In Raginsky & Lazebnik (2009), the same ξ is used for all data points.

Original: Another option in literature is to use i.i.d.
Revised: Another option in literature is to use i.i.d.

Original: ξ for each data point, such that the binary codes admit E
Revised: ξ for each data point, such that the binary codes admit E

Original: [ h ( x ) h ( y )]
Revised: [ h ( x ) h ( y )]

Original: = k ( x, y ) .
Revised: = k ( x, y ) .

Original: For example, it is used in Zhang et al.
Revised: For example, it is used in Zhang et al.

Original: ; Li & Li (2021) for large-scale low-precision training.
Revised: ; Li & Li (2021) for large-scale low-precision training.

Original: We also tested this strategy in our experiments.
Revised: We also tested this strategy in our experiments.

Original: The search performance is however very poor, because more variation is introduced.
Revised: The search performance is however very poor, because more variation is introduced.

Original: On the other hand, we actually do not need the exactly estimation/recovery of the kernel values in search problems.
Revised: On the other hand, we actually do not need the exactly estimation/recovery of the kernel values in search problems.

Original: Hence, in this paper we will consider the baseline using the same ξ for all data samples as in Raginsky & Lazebnik (2009).
Revised: Hence, in this paper we will consider the baseline using the same ξ for all data samples as in Raginsky & Lazebnik (2009).

Original: The collision probability of the so-called “SQ-RFF” is given as follows.
Revised: The collision probability of the so-called “SQ-RFF” is given as follows.

Original: Theorem 3.1 (Raginsky & Lazebnik (2009)) .
Revised: Theorem 3.1 (Raginsky & Lazebnik (2009)) .

Original: For the SQ-RFF (5), for ∀ x , y ∈ R d , it holds that
Revised: For the SQ-RFF (5), for ∀ x , y ∈ R d , it holds that

Original: [Equation]
Revised: [Equation]

Original: This method is locality-sensitive w.r.t.
Revised: This method is locality-sensitive w.r.t.

Original: ρ because of the fact that (6) is an increasing function of ρ .
Revised: ρ because of the fact that (6) is an increasing function of ρ .

Original: Hence, it is also locality-sensitive w.r.t.
Revised: Hence, it is also locality-sensitive w.r.t.

Original: the kernel k ( ρ ) by the monotonicity of k ( ρ ) .
Revised: the kernel k ( ρ ) by the monotonicity of k ( ρ ) .

Original: Proposition 3.2.
Revised: Proposition 3.2.

Original: The SQ-RFF (5) is (˜ k, c ˜ k, P SQ ( ρ 1 ) , P SQ ( ρ 2 ))
Revised: The SQ-RFF (5) is (˜ k, c ˜ k, P SQ ( ρ 1 ) , P SQ ( ρ 2 ))

Original: -locality sensitive w.r.t.
Revised: -locality sensitive w.r.t.

Original: similarity measure k ( · ) , where ρ 1 = log(˜ k ) /γ 

Revised: similarity measure k ( · ) , where ρ 1 = log(˜ k ) /γ

Original: + 1 and ρ 2 = log( c ˜ k ) /γ 2 + 1 .
Revised: + 1 and ρ 2 = log( c ˜ k ) /γ 2 + 1 .

Original: In the kernel approximation problem, stochastic rounding introduces higher variance due to the noise ξ , which hurts the kernel estimation accuracy (Li & Li, 2021).
Revised: In the kernel approximation problem, stochastic rounding introduces higher variance due to the noise ξ , which hurts the kernel estimation accuracy (Li & Li, 2021).

Original: Later, we will see that a similar effect also presents in search problems.
Revised: Later, we will see that a similar effect also presents in search problems.

Original: To this end, we propose our simple strategy—directly taking the sign of the RFF, i.e., applying deterministic quantization.
Revised: To this end, we propose our simple strategy—directly taking the sign of the RFF, i.e., applying deterministic quantization.

Original: Formally, the so-called “SignRFF” approach constructs the binary codes for x by
Revised: Formally, the so-called “SignRFF” approach constructs the binary codes for x by

Original: [Equation]
Revised: [Equation]

Original: Operationally, SignRFF is extremely convenient.
Revised: Operationally, SignRFF is extremely convenient.

Original: At the ﬁrst glance, it may appear a bit surprising that this simple scheme has not been studied in literature.
Revised: At the ﬁrst glance, it may appear a bit surprising that this simple scheme has not been studied in literature.

Original: We believe one of the reasons is that the theoretical correctness of SignRFF is hard to justify.
Revised: We believe one of the reasons is that the theoretical correctness of SignRFF is hard to justify.

Original: Indeed, with some recent progress on RFF, we can show that SignRFF is locality-sensitive.
Revised: Indeed, with some recent progress on RFF, we can show that SignRFF is locality-sensitive.

Original: Lemma 3.3 (Li & Li (2021)) .
Revised: Lemma 3.3 (Li & Li (2021)) .

Original: For two normalized data points x , y with cosine ρ , let F ( · ) be the
Revised: For two normalized data points x , y with cosine ρ , let F ( · ) be the

Original: RFF deﬁned as (4).
Revised: RFF deﬁned as (4).

Original: The joint distribution of z x = F ( x ) and z y = F ( y ) is
Revised: The joint distribution of z x = F ( x ) and z y = F ( y ) is

Original: [Equation]
Revised: [Equation]

Original: where a ∗ x = cos − 1 ( z x ) , a ∗ y = cos − 1 ( z y ) , and φ σ ( · ) is the p.d.f.
Revised: where a ∗ x = cos − 1 ( z x ) , a ∗ y = cos − 1 ( z y ) , and φ σ ( · ) is the p.d.f.

Original: of N (0 ,
Revised: of N (0 ,

Original: σ 2 ) with σ = (cid:112) 2(1 − ρ ) γ .
Revised: σ 2 ) with σ = (cid:112) 2(1 − ρ ) γ .

Original: Furthermore, E [ sign ( F ( x )) sign ( F ( y ))] is an increasing function of ρ .
Revised: Furthermore, E [ sign ( F ( x )) sign ( F ( y ))] is an increasing function of ρ .

Original: Proposition 3.4.
Revised: Proposition 3.4.

Original: The SignRFF (7) is (˜ k, c ˜ k, P sign ( ρ 1 ) , P sign ( ρ 2 ))
Revised: The SignRFF (7) is (˜ k, c ˜ k, P sign ( ρ 1 ) , P sign ( ρ 2 ))

Original: -locality sensitive w.r.t.
Revised: -locality sensitive w.r.t.

Original: to k ( · ) , with ρ 1 = log(˜ k ) /γ 2 + 1 and ρ 2 = log( c ˜ k ) /γ 2 + 1 , with collision probability
Revised: to k ( · ) , with ρ 1 = log(˜ k ) /γ 2 + 1 and ρ 2 = log( c ˜ k ) /γ 2 + 1 , with collision probability

Original: [Equation]
Revised: [Equation]

Original: where f ( z x , z y | ρ ) is the density function (8).
Revised: where f ( z x , z y | ρ ) is the density function (8).

Original: Proof.
Revised: Proof.

Original: By Deﬁnition 2.3 and the monotonicity of k ( ρ ) , it sufﬁces to show that the collision probability, P sign ( ρ ) , is increasing in ρ .
Revised: By Deﬁnition 2.3 and the monotonicity of k ( ρ ) , it sufﬁces to show that the collision probability, P sign ( ρ ) , is increasing in ρ .

Original: This immediately follows from Lemma 3.3 that E
Revised: This immediately follows from Lemma 3.3 that E

Original: [ sign ( F ( x )) sign ( F ( y ))]
Revised: [ sign ( F ( x )) sign ( F ( y ))]

Original: = P sign [Equation] ρ [Equation]
Revised: = P sign ( ρ ) − (1 − P sign ( ρ ))

Original: −

Original: [Equation]

Original: =
Revised: =

Original: [Equation] 1 is increasing in ρ .
Revised: 2 P sign ( ρ ) − 1 is increasing in ρ .

Original:  T HEORETICAL C OMPARISON OF LSH M ETHODS : A N EW S CHEME
Revised:  T HEORETICAL C OMPARISON OF LSH M ETHODS : A N EW S CHEME

Original: In our theoretical analysis, we assume for KLSH (Kulis & Grauman, 2009) that the Gaussian projection in the kernel induced feature space F is truly random.
Revised: In our theoretical analysis, we assume for KLSH (Kulis & Grauman, 2009) that the Gaussian projection in the kernel induced feature space F is truly random.

Original: That is, KLSH “ideally” performs
Revised: That is, KLSH “ideally” performs

Original: LSH in the kernel space: h KLSH ( x ) =
Revised: LSH in the kernel space: h KLSH ( x ) =

Original: sign ( w T Ψ( x )) where w is a random Gaussian vector with proper dimensionality.
Revised: sign ( w T Ψ( x )) where w is a random Gaussian vector with proper dimensionality.

Original: Since Ψ( x ) T Ψ( y ) =
Revised: Since Ψ( x ) T Ψ( y ) =

Original: k ( x , y ) , applying (2) in F we obtain 
Revised: k ( x , y ) , applying (2) in F we obtain P

Original: [Equation]
Revised: [Equation]

Original: Proposition 4.1.
Revised: Proposition 4.1.

Original: KLSH is (˜ k, c ˜ k, 1 − cos − 1 (˜ k [Equation] π [Equation]
Revised: KLSH is (˜ k, c ˜ k, 1 − cos − 1 (˜ k ) π , 1 − cos − 1 ( c ˜ k ) π ) -locality sensitive w.r.t.

Original: [Equation]

Original: ) -locality sensitive w.r.t.

Original: k ( · ) .
Revised: k ( · ) .

Original: Recall Deﬁnition 2.3 of the (˜ k, c ˜ k, p 1 , p 2 ) -LSH.
Revised: Recall Deﬁnition 2.3 of the ( ˜ k, c ˜ k, p 1 , p 2 ) -LSH.

Original: It is known (Indyk & Motwani, 1998) that one can construct an LSH data structure with the worst case query time O ( n R ) , where R :=
Revised: It is known (Indyk & Motwani, 1998) that one can construct an LSH data structure with the worst case query time O ( n R ) , where R :=

Original: log p 1 log p 2 is called the LSH efﬁciency , which has been used in literature to theoretically compare different LSH methods, e.g. SimHash vs. MinHash (Shrivastava & Li, 2014).
Revised: log p 1 log p 2 is called the LSH efﬁciency , which has been used in literature to theoretically compare different LSH methods, e.g. SimHash vs. MinHash (Shrivastava & Li, 2014).

Original: However, we should note that the
Revised: However, we should note that the

Original: LSH efﬁciency is only based on a worst case analytical bound.
Revised: LSH efﬁciency is only based on a worst case analytical bound.

Original: Therefore, it may not well explain or predict the practical search performance (See Appendix A.1 for related results).
Revised: Therefore, it may not well explain or predict the practical search performance (See Appendix A.1 for related results).

Original: We propose a novel measure for evaluating the search accuracy of LSH schemes, namely, the ranking efﬁciency , that can better compare LSHs in practice.
Revised: We propose a novel measure for evaluating the search accuracy of LSH schemes, namely, the ranking efﬁciency , that can better compare LSHs in practice.

Original: The deﬁnition is motivated by the observation that regardless of the concrete approach (e.g., building hash tables or exhaustive search), the nearest neighbor retrieval, to a large extent, essentially boils down to estimating the Euclidean (cosine) similarity rankings in the Hamming space.
Revised: The deﬁnition is motivated by the observation that regardless of the concrete approach (e.g., building hash tables or exhaustive search), the nearest neighbor retrieval, to a large extent, essentially boils down to estimating the Euclidean (cosine) similarity rankings in the Hamming space.

Original: Suppose x and y are two points in the database, and q is a query with ρ x = cos( q , x ) , ρ y = cos( q , y ) .
Revised: Suppose x and y are two points in the database, and q is a query with ρ x = cos( q , x ) , ρ y = cos( q , y ) .

Original: Assume x is the true nearest neighbor of q , implying ρ x > ρ y .
Revised: Assume x is the true nearest neighbor of q , implying ρ x > ρ y .

Original: By the property of LSH, we know that the hash collision probability p x > p y .
Revised: By the property of LSH, we know that the hash collision probability p x > p y .

Original: For an LSH hash function h , deﬁne the corresponding collision probability estimators as
Revised: For an LSH hash function h , deﬁne the corresponding collision probability estimators as

Original: [Equation]
Revised: [Equation]

Original: Now the problem becomes comparing ˆ p x and ˆ p y to estimate the true ranking of p x and p y .
Revised: Now the problem becomes comparing ˆ p x and ˆ p y to estimate the true ranking of p x and p y .

Original: We consider the event of obtaining a wrong similarity comparison from our estimation, i.e. ˆ p x ≤ ˆ p y .
Revised: We consider the event of obtaining a wrong similarity comparison from our estimation, i.e. ˆ p x ≤ ˆ p y .

Original: Obviously, a higher probability implies worse search accuracy, as we are more likely to retrieve the wrong nearest neighbor y .
Revised: Obviously, a higher probability implies worse search accuracy, as we are more likely to retrieve the wrong nearest neighbor y .

Original: Denote
Revised: Denote

Original: [Equation]
Revised: [Equation]

Original: By the CLT, we know that as b → ∞ ,
Revised: By the CLT, we know that as b → ∞ ,

Original: [Equation]
Revised: [Equation]

Original: Σ

Original: [Equation]

Original: b ) .
Revised: b ) .

Original: This approximation would be good with a sufﬁciently large number of b , e.g., b ≥ 30 .
Revised: This approximation would be good with a sufﬁciently large number of b , e.g., b ≥ 30 .

Original: In this regime, we can compute
Revised: In this regime, we can compute

Original: [Equation]
Revised: [Equation]

Original: where Φ( · ) is the c.d.f.
Revised: where Φ( · ) is the c.d.f.

Original: of standard normal distribution.
Revised: of standard normal distribution.

Original: For all the LSH methods studied in this paper, the E x and E y are simply the collision probabilities, and V x = E x
Revised: For all the LSH methods studied in this paper, the E x and E y are simply the collision probabilities, and V x = E x

Original: (1 − E x ) , V y =
Revised: (1 − E x ) , V y =

Original: E y (1 − E
Revised: E y (1 − E

Original: y
Revised: y

Original: from the binomial distribution.
Revised: from the binomial distribution.

Original: Yet, the covariance V xy is difﬁcult to be analytically computed.
Revised: Yet, the covariance V xy is difﬁcult to be analytically computed.

Original: For simplicity, we assume that this term has same relative inﬂuence on the estimators of all approaches, thus dropping it in the formal deﬁnition of the ranking efﬁciency , which is given as below.
Revised: For simplicity, we assume that this term has same relative inﬂuence on the estimators of all approaches, thus dropping it in the formal deﬁnition of the ranking efﬁciency , which is given as below.

Original: Deﬁnition 4.1 ( ( ρ, c ) -Ranking Efﬁciency) .
Revised: Deﬁnition 4.1 ( ( ρ, c ) -Ranking Efﬁciency) .

Original: For a LSH method, let the hash collision probability at cosine ρ and cρ be E and E c , respectively.
Revised: For a LSH method, let the hash collision probability at cosine ρ and cρ be E and E c , respectively.

Original: Let V =
Revised: Let V =

Original: E
Revised: E

Original: (1 − E ) , V c =
Revised: (1 − E ) , V c =

Original: E c
Revised: E c

Original: (1 − E c ) .
Revised: (1 − E c ) .

Original: The ( ρ, c ) -ranking efﬁciency is deﬁned as RE =
Revised: The ( ρ, c ) -ranking efﬁciency is deﬁned as RE =

Original: E − E c √ V + V c .
Revised: E − E c √ V + V c .

Original: Remark 4.1.
Revised: Remark 4.1.

Original: The RE can also be deﬁned in terms of the non-linear kernel k ( · ) .
Revised: The RE can also be deﬁned in terms of the non-linear kernel k ( · ) .

Original: We use cosine ρ here to address both linear and non-linear LSH for generality.
Revised: We use cosine ρ here to address both linear and non-linear LSH for generality.

Original: In most retrieval applications, we may concern more about high c (e.g., c = 0 . 95 ) which corresponds to similar data points.
Revised: In most retrieval applications, we may concern more about high c (e.g., c = 0 . 95 ) which corresponds to similar data points.

Original: In general, higher RE implies smaller probability of the reversed estimated ranking in (12), which is more favorable.
Revised: In general, higher RE implies smaller probability of the reversed estimated ranking in (12), which is more favorable.

Original: In Figure [Figure] we provide the comparison of ranking efﬁciency at different ρ , which covers the cases in our empirical study (Section 5).
Revised: In Figure 1, we provide the comparison of ranking efﬁciency at different ρ , which covers the cases in our empirical study (Section 5).

Original: We observe some consistent patterns:
Revised: We observe some consistent patterns:

Original: • Non-linear LSH.
Revised: • Non-linear LSH.

Original: Firstly, compared with the other RFF-based approach SQ-RFF, the proposed SignRFF is uniformly more efﬁcient at all ρ .
Revised: Firstly, compared with the other RFF-based approach SQ-RFF, the proposed SignRFF is uniformly more efﬁcient at all ρ .

Original: In other words, in search tasks, the stochastic quantization of SQ-RFF introduces more “variation” that lowers the ranking efﬁciency.
Revised: In other words, in search tasks, the stochastic quantization of SQ-RFF introduces more “variation” that lowers the ranking efﬁciency.

Original: Compared with KLSH, SignRFF is more efﬁcient when γ is large (e.g., γ > 2 ) in all plots, while KLSH tends to be more efﬁcient with small γ . In general, SignRFF is the best in terms of ranking efﬁciency. • When should we prefer non-linear LSH? The ranking efﬁciency gives a novel theoretical guidance on the appropriate choice between linear and non-linear LSH methods.
Revised: Compared with KLSH, SignRFF is more efﬁcient when γ is large (e

Original: We see that, when the target ρ is high (e.g., ρ > 0 . 8 ), with proper tuning, kernel methods (SignRFF and KLSH) can be more efﬁcient than linear LSH.

Original: However, if the target ρ is small (e.g., ρ < 0 . 7 ), linear LSH might be more favorable. In other words, SignRFF could be better than LSH on datasets where the near neighbors are close to each other with high similarity/short distance, which is usually the case for, e [Figure] g., image data.
Revised: g., γ > 2 ) in

Original: [Figure]
Revised: [Figure]

Original: r
Revised: r

Original: [Figure]
Revised: [Figure]

Original: k
Revised: k

Original: i
Revised: i

Original: [Figure]
Revised: ng

Original:  i ci
Revised: e ff i c

Original: [Figure]

Original: i
Revised: i

Revised: en cy

Original: [Figure]
Revised: [Figure]

Original: i c i
Revised: all plots, while KLSH tends to be more efﬁcient with small γ . In general, SignRFF is the best in terms of ranking efﬁciency. • When should we prefer non-linear LSH? The ranking efﬁciency gives a novel theoretical guidance on the appropriate choice between linear and non-linear LSH methods.

Original: [Figure]
Revised: We see that, when the target ρ is high (e.g., ρ > 0 . 8 ), with proper tuning, kernel methods (SignRFF and KLSH) can be more efﬁcient than linear LSH. However, if the target ρ is small (e.g., ρ < 0 . 7 ), linear LSH might be more favorable. In other words, SignRFF could be better than LSH on datasets where the near neighbors are close to each other with high similarity/short distance, which is usually the case for, e g., image data.

Original: The ranking efﬁciency measures the search accuracy for given target ρ value.
Revised: The ranking efﬁciency measures the search accuracy for given target ρ value.

Original: In practice, a dataset contains many different pairs (with different ρ ).
Revised: In practice, a dataset contains many different pairs (with different ρ ).

Original: Our experiments on real datasets in the next section show that the performance is largely consistent with the prediction based on the “average ρ ” value.
Revised: Our experiments on real datasets in the next section show that the performance is largely consistent with the prediction based on the “average ρ ” value.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we conduct image retrieval experiments on benchmark datasets to demonstrate the effectiveness of our approach and justify the theoretical comparison of ranking efﬁciency.
Revised: In this section, we conduct image retrieval experiments on benchmark datasets to demonstrate the effectiveness of our approach and justify the theoretical comparison of ranking efﬁciency.

Original: 5. D ATASETS , M ETHODS AND E VALUATION
Revised: 5. D ATASETS , M ETHODS AND E VALUATION

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: Datasets.

Original: We use three popular benchmark datasets in image retrieval literature.
Revised: We use three popular benchmark datasets in image retrieval literature.

Original: The SIFT1M dataset (Jégou et al., 2011) contains 1M 128-dimensional SIFT image features, and 1000 query samples.
Revised: The SIFT1M dataset (Jégou et al., 2011) contains 1M 128-dimensional SIFT image features, and 1000 query samples.

Original: The MNIST dataset (LeCun, 1998) contains 60000 hand-written digits.
Revised: The MNIST dataset (LeCun, 1998) contains 60000 hand-written digits.

Original: We randomly choose 1000 samples from the test set as the query points.
Revised: We randomly choose 1000 samples from the test set as the query points.

Original: The CIFAR dataset (Krizhevsky, 2009) contains 50000 natural images.
Revised: The CIFAR dataset (Krizhevsky, 2009) contains 50000 natural images.

Original: We use the gray-scale images in our experiments, and randomly select 1000 test samples as the queries.
Revised: We use the gray-scale images in our experiments, and randomly select 1000 test samples as the queries.

Original: In addition, when comparing our methods with VGG-16 (Simonyan & Zisserman, 2015) based deep methods, following prior literature (e.g.,Yang et al.
Revised: In addition, when comparing our methods with VGG-16 (Simonyan & Zisserman, 2015) based deep methods, following prior literature (e.g.,Yang et al.

Original: ; Qiu et al.
Revised: ; Qiu et al.

Original: , we use the 4096-d features from the last fully connected layer of the pre-trained
Revised: , we use the 4096-d features from the last fully connected layer of the pre-trained

Original: VGG-16 network as the input data for shallow methods for fairness.
Revised: VGG-16 network as the input data for shallow methods for fairness.

Original: This dataset is called “CIFARVGG”.
Revised: This dataset is called “CIFARVGG”.

Original: On all datasets, the data points are normalized to have unit norm.
Revised: On all datasets, the data points are normalized to have unit norm.

Original: In Figure 2, we report the average cosine similarity between queries to their N -nearest neighbor, from N = 1 to 1000 .
Revised: In Figure 2, we report the average cosine similarity between queries to their N -nearest neighbor, from N = 1 to 1000 .

Original: Methods.
Revised: Methods.

Original: We compare the following unsupervised hashing methods: 1) LSH (Charikar, 2002); 2)
Revised: We compare the following unsupervised hashing methods: 1) LSH (Charikar, 2002); 2)

Original: Iterative Qauntization (ITQ) (Gong & Lazebnik, 2011); 3) Spectral Hashing (SpecH) (Weiss et al., 2008); 4) Binary Reconstruction Embedding (BRE) with 1000 random samples for model training as suggested by Kulis & Darrell (2009); 5) KLSH with m = 500 random samples for formulating the kernel matrix and t = 50 samples for the CLT Gaussian approximation, which should be more accurate than (300 , 30) as recommended in Kulis & Grauman (2009); 6) SQ-RFF (Raginsky &
Revised: Iterative Qauntization (ITQ) (Gong & Lazebnik, 2011); 3) Spectral Hashing (SpecH) (Weiss et al., 2008); 4) Binary Reconstruction Embedding (BRE) with 1000 random samples for model training as suggested by Kulis & Darrell (2009); 5) KLSH with m = 500 random samples for formulating the kernel matrix and t = 50 samples for the CLT Gaussian approximation, which should be more accurate than (300 , 30) as recommended in Kulis & Grauman (2009); 6) SQ-RFF (Raginsky &

Original: Lazebnik, 2009); 7) Our proposed SignRFF.
Revised: Lazebnik, 2009); 7) Our proposed SignRFF.

Original: For kernel LSH methods 5) - 7), we tune the parameter γ over 0 .
Revised: For kernel LSH methods 5) - 7), we tune the parameter γ over 0 .

Original: 1 ∼ 5 and report the best γ with highest average recall over multiple b = 32 ∼ 1024 .
Revised: 1 ∼ 5 and report the best γ with highest average recall over multiple b = 32 ∼ 1024 .

Original: For each tested method, we generate binary codes and perform exhaustive search in the Hamming space.
Revised: For each tested method, we generate binary codes and perform exhaustive search in the Hamming space.

Original: Evaluation.
Revised: Evaluation.

Original: For each query point, the ground truth nearest neighbors are set by ranking the top N = 100 smallest Euclidean distance (top-100 largest cosine similarity).
Revised: For each query point, the ground truth nearest neighbors are set by ranking the top N = 100 smallest Euclidean distance (top-100 largest cosine similarity).

Original: After running each algorithm, the Hamming ranking returns R estimated nearest neighbors to each query.
Revised: After running each algorithm, the Hamming ranking returns R estimated nearest neighbors to each query.

Original: We report the average recall over 1000 queries.
Revised: We report the average recall over 1000 queries.

Original: Note that, recall@N is equivalent to precision@N in our setting.
Revised: Note that, recall@N is equivalent to precision@N in our setting.

Original: See
Revised: See

Original: Appendix A.2 for more detailed explanation and additional results on the retrieval precision.
Revised: Appendix A.2 for more detailed explanation and additional results on the retrieval precision.

Original: 5. R ESULTS
Revised: 5. R ESULTS

Original: In Figure 3, we report the recall@N (precision@N) against the number of binary codes b :
Revised: In Figure 3, we report the recall@N (precision@N) against the number of binary codes b :

Original: • On SIFT and MNIST, data-dependent methods (ITQ, SpecH, BRE) perform well with low bits, but the recall does not improve much after b > 100 ∼ 200 .
Revised: • On SIFT and MNIST, data-dependent methods (ITQ, SpecH, BRE) perform well with low bits, but the recall does not improve much after b > 100 ∼ 200 .

Original: Yet, this low bit region does not help much in practice, since the recall level is too low (e.g. < 0 .
Revised: Yet, this low bit region does not help much in practice, since the recall level is too low (e.g. < 0 .

Original: 3 on SIFT and CIFAR) to be satisfactory for real-world tasks.
Revised: 3 on SIFT and CIFAR) to be satisfactory for real-world tasks.

Original: As we mentioned, this is a known limitation of these methods.
Revised: As we mentioned, this is a known limitation of these methods.

Original: When b ≥ 256 , LSH-type methods start to dominate, as expected.
Revised: When b ≥ 256 , LSH-type methods start to dominate, as expected.

Original: On CIFAR, SignRFF and KLSH outperform the data-dependent methods even with low bits.
Revised: On CIFAR, SignRFF and KLSH outperform the data-dependent methods even with low bits.

Original: • On all three datasets, SignRFF is substantially better than SQ-RFF with all b .
Revised: • On all three datasets, SignRFF is substantially better than SQ-RFF with all b .

Original: Due to dependence, KLSH has higher recall than SignRFF when b is small (e.g., ≤ 256 ), but is beaten by SignRFF with more bits.
Revised: Due to dependence, KLSH has higher recall than SignRFF when b is small (e.g., ≤ 256 ), but is beaten by SignRFF with more bits.

Original: The gap is signiﬁcant and consistent when b is as large as 512 , where SignRFF achieves the highest recall on all three datasets.
Revised: The gap is signiﬁcant and consistent when b is as large as 512 , where SignRFF achieves the highest recall on all three datasets

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: LSHITQSpecHBRESQ -
Revised: In Figure

Original: [Figure]
Revised: [Figure]

Original: KLSHSignRFF

Original: In Figure 4, we present the recall versus the number of retrieved neighbors, with b = 512 and b = 1024 .
Revised: we present the recall versus the number of retrieved neighbors [Figure] with b = 512 and b = 1024 .

Original: SignRFF performs the best among all competing methods on SIFT and CIFAR.
Revised: SignRFF performs the best among all competing methods on SIFT and CIFAR.

Original: On
Revised: On

Original: MNIST, SignRFF performs similarly to LSH and better than the other methods with sufﬁcient b .
Revised: MNIST, SignRFF performs similarly to LSH and better than the other methods with sufﬁcient b .

Original: Ranking efﬁciency.
Revised: Ranking efﬁciency.

Original: In Figure 5, we additionally provide the recall against the Gaussian parameter γ , to justify the effect of ranking efﬁciency discussed in Section 4.
Revised: In Figure 5, we additionally provide the recall against the Gaussian parameter γ , to justify the effect of ranking efﬁciency discussed in Section 4.

Original: In Figure 1, the theoretical comparison suggests that KLSH would perform better with small γ , while SignRFF is more efﬁcient larger γ , which matches the empirical evidence (e.g., 1 vs. 2.5 on SIFT).
Revised: In Figure 1, the theoretical comparison suggests that KLSH would perform better with small γ , while SignRFF is more efﬁcient larger γ , which matches the empirical evidence (e.g., 1 vs. 2.5 on SIFT).

Original: Moreover, as shown in Figure 2, the average ρ between each query and its neighbors is around 0.8, 0.9 and 0.95 for
Revised: Moreover, as shown in Figure 2, the average ρ between each query and its neighbors is around 0.8, 0.9 and 0.95 for

Original: MNIST, SIFT and CIFAR, respectively.
Revised: MNIST, SIFT and CIFAR, respectively.

Original: Figure 1 predicts that compared with SQ-RFF (blue), LSH (green) would perform better on MNIST, similarly on SIFT, and worse on CIFAR, which aligns with[Figure] c a ll [Figure]
Revised: Figure 1 predicts that compared with SQ-RFF (blue), LSH (green) would perform better on MNIST, similarly on SIFT, and worse on CIFAR, which aligns with

Original: Figure 3 and Figure 4.
Revised: Figure 3 and Figure 4.

Original: Therefore, although the behavior of KLSH slightly deviates from the theory due to the dependence among hash codes, our experiments in general verify the effectiveness of the ranking efﬁciency as an informative measure to compare different LSH methods.
Revised: Therefore, although the behavior of KLSH slightly deviates from the theory due to the dependence among hash codes, our experiments in general verify the effectiveness of the ranking efﬁciency as an informative measure to compare different LSH methods.

Original: 5. C OMPARISON WITH D EEP H ASHING
Revised: 5. C OMPARISON WITH D EEP H ASHING

Original: We provide additional experiments on the CIFAR-VGG dataset with a recent unsupervised deep hashing method for image retrieval, the Contrastive Information Bottleneck (CIB) (Qiu et al., 2021), which uses VGG-16 pre-trained model as the backbone.
Revised: We provide additional experiments on the CIFAR-VGG dataset with a recent unsupervised deep hashing method for image retrieval, the Contrastive Information Bottleneck (CIB) (Qiu et al., 2021), which uses VGG-16 pre-trained model as the backbone.

Original: We apply the same training setting as in Qiu et al.
Revised: We apply the same training setting as in Qiu et al.

Original: (2021): 60 epochs, 64 mini-batch size and Adam optimizer with 0.001 learning rate.
Revised: (2021): 60 epochs, 64 mini-batch size and Adam optimizer with 0.001 learning rate.

Original: We note that, CIB (as well as many other deep methods) is not designed to ﬁnd the most similar data points.
Revised: We note that, CIB (as well as many other deep methods) is not designed to ﬁnd the most similar data points.

Original: Instead, by using techniques like cropping and rotation in training CNN, these methods aim at robustly ﬁnding data points with same label as the query, which does not necessarily imply high similarity with the query when represented by the fc7 layer of VGG-16.
Revised: Instead, by using techniques like cropping and rotation in training CNN, these methods aim at robustly ﬁnding data points with same label as the query, which does not necessarily imply high similarity with the query when represented by the fc7 layer of VGG-16.

Original: Hence, to favor CIB, in this experiment we expand the range of true neighbors of a query to the top-1000 similar points in the database.
Revised: Hence, to favor CIB, in this experiment we expand the range of true neighbors of a query to the top-1000 similar points in the database.

Original: See Appendix B for examples of the retrieved images and more discussion.
Revised: See Appendix B for examples of the retrieved images and more discussion.

Original: • From Figure 6, we see that CIB performs the best with 32 and 64 bits than LSH methods, which illustrates the beneﬁt of deep hashing with short codes.
Revised: • From Figure 6, we see that CIB performs the best with 32 and 64 bits than LSH methods, which illustrates the beneﬁt of deep hashing with short codes.

Original: However, the recall is only 0.3 ∼ 0.4, and does not improve with more bits, since simply increasing the size of last layer in the deep net typically would not elevate the model performance signiﬁcantly.
Revised: However, the recall is only 0.3 ∼ 0.4, and does not improve with more bits, since simply increasing the size of last layer in the deep net typically would not elevate the model performance signiﬁcantly.

Original: When b ≥ 256 , LSH, SignRFF and KLSH provide much higher recall than CIB.
Revised: When b ≥ 256 , LSH, SignRFF and KLSH provide much higher recall than CIB.

Original: • Among LSHs, KLSH performs the best on this task.
Revised: • Among LSHs, KLSH performs the best on this task.

Original: SignRFF again consistently improves SQ-RFF, and matches the recall of KLSH with b = 1024 .
Revised: SignRFF again consistently improves SQ-RFF, and matches the recall of KLSH with b = 1024 .

Original: In addition, linear LSH slightly outperforms SignRFF, which is consistent with the theoretical comparison in Figure 1 as expected (the target ρ is ∼ 0.7 for this task, where LSH is more efﬁcient than SignRFF). [Figure]
Revised: In addition, linear LSH slightly outperforms SignRFF, which is consistent with the theoretical comparison in Figure 1 as expected (the target ρ is ∼ 0.7 for this task, where LSH is more efﬁcient than SignRFF

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Random Fourier Feature (RFF) is a popular method in large-scale kernel learning, which has also been used for constructing binary codes for fast non-linear neighbor search (Raginsky & Lazebnik, 2009).
Revised: Random Fourier Feature (RFF) is a popular method in large-scale kernel learning, which has also been used for constructing binary codes for fast non-linear neighbor search (Raginsky & Lazebnik, 2009).

Original: In this work, we revisit RFF-based binary hashing, and propose a new strategy named SignRFF.
Revised: In this work, we revisit RFF-based binary hashing, and propose a new strategy named SignRFF.

Original: We analyze its locality-sensitive property, and provide a new uniﬁed theoretical framework to compare different LSH methods, based on a measure called ranking efﬁciency .
Revised: We analyze its locality-sensitive property, and provide a new uniﬁed theoretical framework to compare different LSH methods, based on a measure called ranking efﬁciency .

Original: The metric implies that, we should prefer SignRFF over the linear LSH in the high similarity region when data points are close to each other.
Revised: The metric implies that, we should prefer SignRFF over the linear LSH in the high similarity region when data points are close to each other.

Original: Empirically, we demonstrate that SignRFF is signiﬁcantly better than the previous RFF-based approach, SQ-RFF, and outperforms data-dependent and deep hashing methods when the number of hash bits is moderately large.
Revised: Empirically, we demonstrate that SignRFF is signiﬁcantly better than the previous RFF-based approach, SQ-RFF, and outperforms data-dependent and deep hashing methods when the number of hash bits is moderately large.

Original: Moreover, the theoretical comparisons of the proposed ranking efﬁciency is consistent with the empirical results, suggesting that it could serve as a good theoretical tool for comparing the search performance of different LSH methods in practice. 
Revised: Moreover, the theoretical comparisons of the proposed ranking efﬁciency is consistent with the empirical results, suggesting that it could serve as a good theoretical tool for comparing the search performance of different LSH methods in practice. [Figure] [Figure]

Original: A M ORE A NALYTICAL F IGURES AND E XPERIMENTS A. C OMPARISON OF LSH E FFICIENCY
Revised: A M ORE A NALYTICAL F IGURES AND E XPERIMENTS A. C OMPARISON OF LSH E FFICIENCY

Original: [Figure]
Revised: [Figure]

Original: Recall Deﬁnition 2.3 of the (˜ k, c ˜ k, p 1 , p 2 ) -LSH.
Revised: Recall Deﬁnition 2.3 of the (˜ k, c ˜ k, p 1 , p 2 ) -LSH.

Original: It is known (Indyk & Motwani, 1998) that one can construct an LSH data structure with the worst case query time O ( n R ) , where
Revised: It is known (Indyk & Motwani, 1998) that one can construct an LSH data structure with the worst case query time O ( n R ) , where

Original: [Equation]
Revised: [Equation]

Original: is called the LSH efﬁciency , which has been used in literature to theoretically compare different
Revised: is called the LSH efﬁciency , which has been used in literature to theoretically compare different

Original: LSH methods, e.g. SimHash vs. MinHash (Shrivastava & Li, 2014).
Revised: LSH methods, e.g. SimHash vs. MinHash (Shrivastava & Li, 2014).

Original: However, we found that in our case, the LSH efﬁciency does not provide much informative comparison of different LSH methods of interest.
Revised: However, we found that in our case, the LSH efﬁciency does not provide much informative comparison of different LSH methods of interest.

Original: In Figure 7, we provide the LSH efﬁciency at multiple γ .
Revised: In Figure 7, we provide the LSH efﬁciency at multiple γ .

Original: Firstly, we see that the differences among the curves are very small.
Revised: Firstly, we see that the differences among the curves are very small.

Original: Basically, the ﬁgures tells us that SignRFF is always better than
Revised: Basically, the ﬁgures tells us that SignRFF is always better than

Original: KLSH and SQ-RFF, but do not provide the comparison of SignRFF and KLSH regarding γ , as the ranking efﬁciency does.
Revised: KLSH and SQ-RFF, but do not provide the comparison of SignRFF and KLSH regarding γ , as the ranking efﬁciency does.

Original: Also, the ﬁgures seem to suggest that SignRFF could (roughly) be better than LSH with large γ and large ρ , but it does not give a good threshold at around ρ = 0 .
Revised: Also, the ﬁgures seem to suggest that SignRFF could (roughly) be better than LSH with large γ and large ρ , but it does not give a good threshold at around ρ = 0 .

Original: 7 (validated by the experiments) as suggested by the ranking efﬁciency.
Revised: 7 (validated by the experiments) as suggested by the ranking efﬁciency.

Original: Thus, LSH efﬁciency is insufﬁcient to well predict the practical search performance.
Revised: Thus, LSH efﬁciency is insufﬁcient to well predict the practical search performance.

Original: A. M ORE E XPERIMENTS ON S EARCH P RECISION
Revised: A. M ORE E XPERIMENTS ON S EARCH P RECISION

Original: Deﬁne N as the number of ground truth neighbors.
Revised: Deﬁne N as the number of ground truth neighbors.

Original: The search recall and precision are deﬁned as
Revised: The search recall and precision are deﬁned as

Original: [Equation]
Revised: [Equation]

Original: Consequently, by deﬁnition, recall@N is equivalent to precision@N. Therefore, in our experiments, i.e., Figure 3, Figure 5 and Figure 6, the curves are also “precision@N”.
Revised: Consequently, by deﬁnition, recall@N is equivalent to precision@N. Therefore, in our experiments, i.e., Figure 3, Figure 5 and Figure 6, the curves are also “precision@N”.

Original: In Figure 8, we report more results on the search precision@R. As we can see, on SIFT, CIFAR and MNIST, the SignRFF method again performs the best as long as b ≥ 256 .
Revised: In Figure 8, we report more results on the search precision@R. As we can see, on SIFT, CIFAR and MNIST, the SignRFF method again performs the best as long as b ≥ 256 .

Original: The comparison is fairly consistent with the recall (precision) curves presented in Section 5.
Revised: The comparison is fairly consistent with the recall (precision) curves presented in Section 5.

Original: [Figure]
Revised: [Figure]

Original: A. D ATA P ROCESSING T IME
Revised: A. D ATA P ROCESSING T IME

Original: Another beneﬁt of SignRFF is its simplicity in implementation.
Revised: Another beneﬁt of SignRFF is its simplicity in implementation.

Original: This can be reﬂected in the data processing time.
Revised: This can be reﬂected in the data processing time.

Original: Note that, the query time consists of two parts: the processing time (to generate query binary codes)
Revised: Note that, the query time consists of two parts: the processing time (to generate query binary codes)

Original: [Figure]
Revised: [Figure]

Original: and the search time (to compare the query codes to the database).
Revised: and the search time (to compare the query codes to the database).

Original: In our experiments, since we adopt the standard Hamming ranking method, the search time to return a ﬁxed number of retrieved points are the same for all algorithms.
Revised: In our experiments, since we adopt the standard Hamming ranking method, the search time to return a ﬁxed number of retrieved points are the same for all algorithms.

Original: Hence, we plot the comparison of processing time (for 1000 queries), as shown in Figure 9.
Revised: Hence, we plot the comparison of processing time (for 1000 queries), as shown in Figure 9.

Original: We observe that SignRFF and LSH are the most efﬁcient methods.
Revised: We observe that SignRFF and LSH are the most efﬁcient methods.

Original: KLSH is notably slower than SignRFF.
Revised: KLSH is notably slower than SignRFF.

Original: The two data-dependent methods,
Revised: The two data-dependent methods,

Original: SpecH and BRE, are signiﬁcantly slower than SignRFF.
Revised: SpecH and BRE, are signiﬁcantly slower than SignRFF.

Original: This reveals a potential advantage of the simplicity of SignRFF in practical retrieval systems.
Revised: This reveals a potential advantage of the simplicity of SignRFF in practical retrieval systems.

Original: [Figure]
Revised: [Figure]

Original: B M ORE I MAGE R ETRIEVAL R ESULTS ON CIFAR-VGG
Revised: B M ORE I MAGE R ETRIEVAL R ESULTS ON CIFAR-VGG

Original: In Section 5.3, we compare the locality-sensitive hashing methods with a deep learning based method, the Contrastive Information Bottleneck (CIB) (Qiu et al., 2021).
Revised: In Section 5.3, we compare the locality-sensitive hashing methods with a deep learning based method, the Contrastive Information Bottleneck (CIB) (Qiu et al., 2021).

Original: CIB is built upon the pre-trained VGG-16 CNN model, whose objective is to generate binary representations such that images from the same class are close.
Revised: CIB is built upon the pre-trained VGG-16 CNN model, whose objective is to generate binary representations such that images from the same class are close.

Original: Thus, same as many other deep hashing methods, the objective of CIB is slightly different from our setting (we ﬁnd the most similar data points, without any label information), as being in the same class does not necessarily implies high similarity.
Revised: Thus, same as many other deep hashing methods, the objective of CIB is slightly different from our setting (we ﬁnd the most similar data points, without any label information), as being in the same class does not necessarily implies high similarity.

Original: In fact, in the empirical evaluation of many of these papers (e.g., Yang et al.
Revised: In fact, in the empirical evaluation of many of these papers (e.g., Yang et al.

Original: ; Qiu et al.
Revised: ; Qiu et al.

Original: , the true neighbors are simply set as those data points with same label as the query.
Revised: , the true neighbors are simply set as those data points with same label as the query.

Original: In our setting, we strictly follow the ranking to ﬁnd the true neighbors with highest similarities.
Revised: In our setting, we strictly follow the ranking to ﬁnd the true neighbors with highest similarities.

Original: To better illustrate the difference, we present top-10 retrieval result of two queries (automobile and cat) in Figure 10.
Revised: To better illustrate the difference, we present top-10 retrieval result of two queries (automobile and cat) in Figure 10.

Original: For
Revised: For

Original: SignRFF and KLSH, the retrieved images mostly have high similarity with the query, but may include some data points from other classes (e.g., truck vs. automobile, dog vs. cat).
Revised: SignRFF and KLSH, the retrieved images mostly have high similarity with the query, but may include some data points from other classes (e.g., truck vs. automobile, dog vs. cat).

Original: On the contrary, the retrieved images of CIB clearly have lower similarity with the query, but contain fewer other classes.
Revised: On the contrary, the retrieved images of CIB clearly have lower similarity with the query, but contain fewer other classes.

Original: This is largely because the VGG-16 is pre-trained by classiﬁcation.
Revised: This is largely because the VGG-16 is pre-trained by classiﬁcation.

Original: Hence, in this experiment we set the true neighbors as the top-1000 similar data points, where CIB could be evaluated properly since most images from the same class would, at least, have not too low similarity.
Revised: Hence, in this experiment we set the true neighbors as the top-1000 similar data points, where CIB could be evaluated properly since most images from the same class would, at least, have not too low similarity.

Original: [Figure]
Revised: [Figure]

Original: C I MPACT OF THE D EPENDENCE IN KLSH
Revised: C I MPACT OF THE D EPENDENCE IN KLSH

Original: In practical implementation, the hash codes of KLSH are dependent.
Revised: In practical implementation, the hash codes of KLSH are dependent.

Original: We give an intuitive explanation on how this dependence affect the search performance.
Revised: We give an intuitive explanation on how this dependence affect the search performance.

Original: Firstly, let us review the reason why data-independent methods with independent codes (LSH, SQRFF, SignRFF) can boost search accuracy with increasing b .
Revised: Firstly, let us review the reason why data-independent methods with independent codes (LSH, SQRFF, SignRFF) can boost search accuracy with increasing b .

Original: Essentially, same as the intuition of the rank efﬁciency, comparing Hamming distance is equivalent to searching for the data points with highest estimated hash collision probability using b codes.
Revised: Essentially, same as the intuition of the rank efﬁciency, comparing Hamming distance is equivalent to searching for the data points with highest estimated hash collision probability using b codes.

Original: Let x , y be two database points and q be the query, with ρ x > ρ y .
Revised: Let x , y be two database points and q be the query, with ρ x > ρ y .

Original: This means the hash collision probability p x > p y .
Revised: This means the hash collision probability p x > p y .

Original: We estimate the probability by averaging the collision indicators:
Revised: We estimate the probability by averaging the collision indicators:

Original: [Equation]
Revised: [Equation]

Original: For data-dependent methods, with sufﬁcient b , (12) gives the probability of wrong raking (i.e., ˆ p x < ˆ p y ).
Revised: For data-dependent methods, with sufﬁcient b , (12) gives the probability of wrong raking (i.e., ˆ p x < ˆ p y ).

Original: We see that it strictly decreases with larger b .
Revised: We see that it strictly decreases with larger b .

Original: Recall the steps of KLSH.
Revised: Recall the steps of KLSH.

Original: We ﬁrst sample m data points from database X , denoted as ˜ X , to form a kernel matrix K .
Revised: We ﬁrst sample m data points from database X , denoted as ˜ X , to form a kernel matrix K .

Original: Then we uniformly pick t points, denoted as ˜ X (cid:48) , from [1 , ..., m ] at random to approximate the Gaussian distribution.
Revised: Then we uniformly pick t points, denoted as ˜ X (cid:48) , from [1 , ..., m ] at random to approximate the Gaussian distribution.

Original: After some algebra, the hash code has the form
Revised: After some algebra, the hash code has the form

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where w = K − 1 / 2 e t , and e t ∈ { 0 , 1 } m has ones in the entries with indices of the t selected points.
Revised: where w = K − 1 / 2 e t , and e t ∈ { 0 , 1 } m has ones in the entries with indices of the t selected points.

Original: To generate multiple codes, we use the same pool of points ˜ X , but with different choice of e t , i.e., ˜
Revised: To generate multiple codes, we use the same pool of points ˜ X , but with different choice of e t , i.e., ˜

Original: X (cid:48) to approximate the Gaussian distribution.
Revised: X (cid:48) to approximate the Gaussian distribution.

Original: Boosted performance with small b .
Revised: Boosted performance with small b .

Original: For two hash codes h k and h k − 1 of x , the k ( x , x i ) terms in (14) are the same.
Revised: For two hash codes h k and h k − 1 of x , the k ( x , x i ) terms in (14) are the same.

Original: The w ’s are dependent since the points used for Gaussian approximation (i.e. ˜ X (cid:48) ) may overlap.
Revised: The w ’s are dependent since the points used for Gaussian approximation (i.e. ˜ X (cid:48) ) may overlap.

Original: Thus, the conditional hash collision probability is usually larger than the unconditional one,
Revised: Thus, the conditional hash collision probability is usually larger than the unconditional one,

Original: [Equation]
Revised: [Equation]

Original: and it may increase as k get larger (intuitively, by dependence, more previous collisions implies higher chance of collision later on).
Revised: and it may increase as k get larger (intuitively, by dependence, more previous collisions implies higher chance of collision later on).

Original: Similarly,
Revised: Similarly,

Original: [Equation]
Revised: [Equation]

Original: Thus, at the beginning, the more similar (dissimilar) the data pair is, the more the estimator ˆ p is upward (downward) biased.
Revised: Thus, at the beginning, the more similar (dissimilar) the data pair is, the more the estimator ˆ p is upward (downward) biased.

Original: In other words, similar points would have further increased hash collision probability, while dissimilar points would have even lower chance of collision.
Revised: In other words, similar points would have further increased hash collision probability, while dissimilar points would have even lower chance of collision.

Original: This is the main reason that the empirical performance of KLSH is higher than theoretical prediction (where we assume independence) with short binary codes.
Revised: This is the main reason that the empirical performance of KLSH is higher than theoretical prediction (where we assume independence) with short binary codes.

Original: Slow improvement with large b .
Revised: Slow improvement with large b .

Original: However, this is not the whole story.
Revised: However, this is not the whole story.

Original: As more bits are used, there is less and less marginal information left.
Revised: As more bits are used, there is less and less marginal information left.

Original: That is, the later generated codes would be more and more dependent on the previous ones.
Revised: That is, the later generated codes would be more and more dependent on the previous ones.

Original: In an extreme case, at the point when all (cid:0) mt (cid:1) combinations of the t points in ˜ X (cid:48) have been used, the hash codes produced afterwards would all be the same as some previously generated ones, which would hardly improve the distance estimation anymore.
Revised: In an extreme case, at the point when all (cid:0) mt (cid:1) combinations of the t points in ˜ X (cid:48) have been used, the hash codes produced afterwards would all be the same as some previously generated ones, which would hardly improve the distance estimation anymore.

Original: This is why for KLSH, the recall curve becomes ﬂat as b increases.
Revised: This is why for KLSH, the recall curve becomes ﬂat as b increases.