Original: A Lower Bound of Hash Codes’ Performance
Revised: A Lower Bound of Hash Codes’ Performance

Original: Abstract
Revised: Abstract

Original: As a crucial approach for compact representation learning, hashing has achievedgreat success in effectiveness and efficiency.
Revised: As a crucial approach for compact representation learning, hashing has achieved great success in effectiveness and efficiency.

Original: Numerous heuristic Hamming spacemetric learning objectives are designed to obtain high quality hash codes.
Revised: Numerous heuristic Hamming space metric learning objectives are designed to obtain high-quality hash codes.

Original: Nevertheless, a theoretical analysison criteria of  learning good hash codes remains largelyunexploited.
Revised: Nevertheless, a theoretical analysis of criteria for learning good hash codes remains largely unexploited.

Original: In this paper, we prove that inter-class distinctiveness and intra-classcompactness among hash codes determine the lower bound of hash codes’ performance.
Revised: In this paper, we prove that inter-class distinctiveness and intra-class compactness among hash codes determine the lower bound of hash codes’ performance.

Original: Promoting these two characteristics could lift the bound and improvehash learning.
Revised: Promoting these two characteristics could lift the bound and improve hash learning.

Original: We  propose a surrogate model to fully exploit such objective byestimating posterior of hash codes and further control it, which results in low-biasoptimization.
Revised: We then propose a surrogate model to fully exploit the above objective by estimating the posterior of hash codes and controlling it, which results in a low-bias optimization.

Original: Extensive experiments reveal  effectiveness of the proposed method.
Revised: Extensive experiments reveal the effectiveness of the proposed method.

Original: By testing on a series of hashing methods, we obtain performance improvementsamong all of them, with an up to 26 .
Revised: By testing on a series of hash-models, we obtain performance improvements among all of them, with an up to 26 .

Original: 5% increase in mean Average Precision andan up to 20 .
Revised: 5% increase in mean Average Precision and an up to 20 .

Original: 5% increase in accuracy.
Revised: 5% increase in accuracy.

Original:  Introduction
Revised:  Introduction

Original: Explosive increase of multimedia digital contents requires people to develop large-scale processingtechniques for effective and efficient multimedia understanding [36, 49, 52, 11, 32, 29].
Revised: The explosive increase of multimedia digital content requires people to develop large-scale processing techniques for effective and efficient multimedia understanding [47, 50, 52, 53].

Original: Meanwhile,growing focus on carbon neutrality initiatives urge researchers to handle above issues with lowpower and memory consumption.
Revised: Meanwhile, growing focus on carbon neutrality initiatives urges researchers to handle above issues with low power and memory consumption [31, 30].

Original: Fortunately, hashing, as a key role in compact representationlearning, has been demonstrating its efficiency and effectiveness over  past few decades towards abovedemands [18, 22, 43, 15, 27, 25, 21].
Revised: Fortunately, hashing, as a key role in compact representation learning, has demonstrated efficiency and effectiveness over the past few decades towards above demands [22].

Original: By digesting multimedia contents into short binary codes,hashing is able to significantly reduce storage, boost speed, and preserve key information.
Revised: By digesting multimedia contents into short binary codes, hashing is able to significantly reduce storage, boost speed and preserve key information [15, 56, 27, 25, 21].

Original: Recent years,compared to hand-crafted hashing functions, learning to hash shows its advantagesto alleviate information loss by performing data-dependent Hamming space projection with a nonlinear model [33, 48, 46, 47, 27, 25, 21, 24].
Revised: In recent years, learning to hash has shown its advantages in alleviating information loss by performing data-dependent Hamming space projection with a non-linear model [32, 46, 24] compared to handcrafted hashing functions.

Original: The purpose of optimizing hash models is to generatesemantically meaningful hash codes for downstream tasks.
Revised: The purpose of optimizing hash-models is to generate semantically meaningful hash codes for downstream tasks.

Original: A practical convention is to increasesimilarities among codes of  same category, otherwise the opposite.
Revised: A practical convention is to increase similarities among codes of the same category, otherwise the opposite.

Original: Such convention is validated tobe effective in many applications [8,42, 26, 41, 50, 34].
Revised: Such convention is validated to be effective in many applications [8, 41, 26, 40, 48, 33].

Original: Practices in current advances still leave two questions which remain largely unexploited: Howcould we determine a hash model could produce good codes, and by what means to achieve thisgoal?
Revised: Practices in current advances still leave two questions which remain primarily unexploited: When should we determine a hash-model for producing good codes, and by what means to achieve this goal?

Original: Taking  comprehensive study on  above questions is essential, since it not only guides us todevelop high performance algorithms in typical hash-based applications, but also provides potentialpossibilities to inspire future works in hash learning.
Revised: Taking a comprehensive study on the above questions is essential since it not only guides us to develop high-performance algorithms in typical hash-based applications but also provides potential possibilities to inspire future works in hash learning.

Original: Although some works [48, 1] try to answer  above questions by building connections between hashcodes and common descriptors with domain knowledge, we still seeks for theoretical guarantees forhash codes’ performance.
Revised: Although some works [46, 1] try to answer the above questions by building connections between hash codes and common descriptors with domain knowledge, we still seek theoretical guarantees for hash codes’ performance.

Original: Similar situations occur in related works [47, 46, 31], where objectives tooptimize hashing models are designed by intuitions or heuristics.
Revised: Similar situations occur in related works [45, 44, 30], where objectives to optimize hash-models are designed by intuitions or heuristics.

Original: By formulating correlations among hash codes as inter-class distinctiveness and intra-class compactness, we try to address  above issues and provide a lower bound of hash codes’ performance.
Revised: By formulating correlations among hash codes as inter-class distinctiveness and intra-class compactness, we try to address the above issues and provide a lower bound of hash codes’ performance.

Original: Based on this, we could further derive from them as objectives to lift  lower bound.
Revised: Based on this, we could further derive from them as objectives to lift the lower bound.

Original: Moreover, toeffectively train model towards such targets, estimating and further controlling  posterior of hash codesis necessary, since bit-level dependence exists in posterior distribution.
Revised: Moreover, to effectively train model towards such targets, estimating and further controlling the posterior of hash codes is necessary since bit-level dependence exists in posterior distribution.

Original: If we neglect this, bias will beintroduced during hashing model optimization and therefore hinders performance.
Revised: If we neglect this, bias will be introduced during hash-model optimization, hindering performance.

Original: Summarizing bothof them, our contribution is presented below:
Revised: Summarizing both of them, Our main contributions are summarized as follows:

Original: 1) To the best of our knowledge, we are the first to formulate  lower bound of hash codes’ performance,which is directly proportional to inter-class distinctiveness and intra-class compactness among codes.
Revised: 1) To the best of our knowledge, we are the first to formulate a lower bound of hash codes’ performance, which is directly proportional to inter-class distinctiveness and intra-class compactness among codes.

Original: Base on this, an objective is further proposed to lift this lower bound and thus enhance performance.
Revised: Based on this, an objective is further proposed to lift this lower bound and thus enhance performance.

Original: 2) To perform low-bias optimization towards above objective, a novel posterior estimation over hashcodes is proposed.
Revised: A novel posterior estimation over hash codes is proposed to perform low-bias optimization towards the above objective.

Original: As a non-linear function, it could be utilized for optimizing hashing models viagradient descent.
Revised: As a non-linear function, it could be utilized for optimizing hash-models via gradient descent.

Original: Such plug-and-play component is able to integrate into current hashing models toboost performance.3)
Revised: Such plug-and-play component is able to integrate into current hash-models to boost performance.

Original:  Extensive experiments on three benchmark datasets reveal  effectiveness of  proposed method in two typical tasks.
Revised: 3) Extensive experiments on three benchmark datasets reveal the effectiveness of the proposed method in two typical tasks.

Original: Specifically we obtain an up to 26 .
Revised: Specifically, we obtain an up to 26 .

Original:  increase on mean Average Precision for fast retrieval.
Revised: 5% increase on mean Average Precision for fast retrieval.

Original: We also observe an up to 20 .
Revised: We also observe an up to 20 .

Original: 5% increase on accuracy for hash-based recognition.
Revised: 5% increase on accuracy for hash-based recognition.

Original: Rest of paper is organized as follows: We first give a short reviewon current hashing methodsand preliminaries (Secs. 2 and 3).
Revised: The rest of paper is organized as follows: We first briefly review current hashing methods and give preliminaries (Secs. 2 and 3).

Original: Then, lower bound on hash codes’ performance by inter-classdistinctiveness and intra-class compactness is proposed (Sec. 4).
Revised: Then, lower bound on hash codes’ performance by inter-class distinctiveness and intra-class compactness is proposed (Sec. 4).

Original: To fully exploit such guidance,posterior estimation over multivariate Bernoulli distribution is designed for code control (Sec. 5).
Revised: To fully exploit such guidance, posterior estimation over multivariate Bernoulli distribution is designed for code control (Sec. 5).

Original: Experiments (Sec. 6) are then conducted for evaluation.
Revised: Experiments (Sec. 6) are then conducted for evaluation.

Original:  Related Works
Revised:  Related Works

Original: This paper will focus on data-dependent learning-to-hash methods.
Revised: This paper will focus on data-dependent learning-to-hash methods.

Original: Recent advances on buildingthese models are validated to be effective on hash-based recognition and fast retrieval [18, 1, 26, 41,17].
Revised: Recent advances in building these models are validated to be effective in hash-based recognition and fast retrieval [18, 1, 26, 40, 17].

Original: The typical approach utilizes a projection function to convert raw inputs into compact binarycodes and conduct classification or approximate nearest neighbor search on codes for downstreamtasks [8, 42, 50].
Revised: The typical approach utilizes a projection function to convert raw inputs into compact binary codes and conduct classification or approximate nearest neighbour search on codes for downstream tasks [8, 41, 48].

Original: As aforementioned in introduction, to obtain such  projector, current practicessuggest us to formulate it as a Hamming space metric learning problem under binary constraints.
Revised: As mentioned in introduction, to obtain such a projector, current practices suggest that we formulate it as a Hamming space metric learning problem under binary constraints.

Original: Pointwise [51, 21, 30], pairwise [26, 55, 4, 25, 5], triplet [28, 10], listwise [54, 45] and prototypebased [6, 53, 17, 2] objectives are all studied for model training, where the core idea is to adjustsimilarities among samples from same / different categories.
Revised: Pointwise [49, 21, 29], pairwise [26, 55, 4, 25, 5, 37], triplet [28, 10], listwise [54, 43] and prototype-based [6, 51, 17, 2] objectives are all studied for model training, where the core idea is to adjust similarities among samples from same / different categories.

Original: Besides  above objectives, we should also notice that hashing model optimization is NP-hard [1, 48,30, 41].
Revised: Besides the above objectives, we should also note that hash-model optimization is NP-hard [1, 46, 29, 40].

Original: This is caused by the discrete, binarized hash codes formulation.
Revised: That is caused by the discrete, binarized hash code formulation.

Original: To tackle this, previousworks propose a lot of methods that target on following purposes: a) Surrogate functions thatapproximate hashing by continuous functions, such as tanh [5], Stochastic neuron [9], affine [39] or
Revised: To tackle the issue, previous works propose many methods that target on following purposes: a) Surrogate functions that approximate hashing by continuous functions, such as tanh [5], Stochastic neuron [9], affine [38] or

Original: Straight-Through [3].
Revised: Straight-Through [3].

Original: By adopting these, the discrete hashing model optimization is transformed intoapproximated but easier one via gradient descent.
Revised: By adopting these, the discrete hash-model optimization is transformed into approximated but easier one via gradient descent.

Original: b) Non-differentiable optimization algorithm design,which focus on solutions that directly handle discrete inputs, including Coordinate descent [38],
Revised: b) Non-differentiable optimization algorithm design, which focuses on solutions that directly handle discrete inputs, including Coordinate descent [36],

Original: Discrete gradient estimation [13], etc .
Revised: Discrete gradient estimation [13], etc .

Original: There are also bag of tricks which help for stable training.
Revised: There is also bag of tricks which help for stable training.

Original: Forexample, minimizing quantization error reduces  gap between raw outputs of  model and final hashcodes [26], and code-balancing alleviates the problem of producing trivial codes [12].
Revised: For example, minimizing quantization error reduces the gap between raw outputs of the model and final hash codes [26], and code-balancing alleviates the problem of producing trivial codes [12].

Original: Please refer to literature reviews on above works [47, 46, 31] for comprehensive studies on learningto hash.
Revised: Please refer to literature reviews on above works [45, 44, 30] for comprehensive studies on learning to hash.

Original: We could see two issues exist and remain largely unexploited in above works.
Revised: We could see two issues exist and remain primarily unexploited in above works.

Original: The firstis objective design, which is commonly proposed by intuitive and with few theoretical guarantees.
Revised: The first is objective design, which is commonly proposed by intuitive and with few theoretical guarantees.

Original: The second is model optimization, where bit-level dependence is important and involved in codecontrol but mostof works neglect it.
Revised: The second is model optimization, where bit-level dependence is essential in code control but most works neglect it.

Original: To tackle these, we would firstly give a lower bound of hashcodes’ performance and utilize it as a guidance for hashing model training.
Revised: To tackle these, we would firstly give a lower bound of hash codes’ performance and utilize it as a guidance for hash-model training.

Original: Then, a novel modeloptimization approach is proposed in a multivariate perspective for low-bias code control.
Revised: Then, a novel model optimization approach is proposed in a multivariate perspective for low-bias code control.

Original:  Preliminaries
Revised:  Preliminaries

Original: Learning to hash performs data-dependent compact representation learning.
Revised: Learning to hash performs data-dependent compact representation learning.

Original: Inputs x ∈ X ⊆ R dare firstly transformed into h -dim real-valued vector ℓ by F θ , a projection parameterized by θ :
Revised: Inputs x ∈ X ⊆ R d are firstly transformed into h -dim real-valued vector ℓ by F θ , a projection parameterized by θ :

Original: R d F θ −−→ R h .
Revised: R d F θ −−→ R h .

Original: And ℓ is then binarized to h bits binary code b ∈ B ⊆
Revised: And ℓ is then binarized to h bits binary code b ∈ B ⊆

Original: H h :x F θ −−→ ℓ bin −−→ b , where bin ( · ) = (cid:26) +1 , ( · ) ≥ 0 , − 1 , ( · ) < 0 .
Revised: H h : x F θ −−→ ℓ bin −−→ b , where bin ( · ) = (cid:26) +1 , ( · ) ≥ 0 , − 1 , ( · ) < 0 .

Original: Typical usage of hash codes.
Revised: Typical usage of hash codes.

Original: Common hashing-based tasks include fast retrieval [19, 40, 27, 2] andrecognition [26, 38, 41, 33].
Revised: Common hashing-based tasks include fast retrieval [19, 39, 27, 2] and recognition [26, 36, 40, 32].

Original: For fast retrieval, given a query under  same distribution of X , we convertit into query hash code q and conduct fast approximate nearest neighbor search in B to produce ranklist R .
Revised: For fast retrieval, given a query under the same distribution of X , we convert it into query hash code q and conduct fast approximate nearest neighbour search in B to produce rank list R .

Original: In R , samples are organized from nearest to furthest to query.
Revised: In R , samples are organized from nearest to furthest to query.

Original: Correspondingly among allresults in R , true positives tp i ∈ TP indicate samples that match with query while others are falsepositives fp i ∈ FP ( i indicates rank).
Revised: Correspondingly among all results in R , true positives tp i ∈ TP indicate samples that match with query while others are false positives fp i ∈ FP ( i indicates rank).

Original: As a criterion, Average Precision (AP) is commonly adoptedto determine how well the retrieved results are.
Revised: As a criterion, Average Precision (AP) is commonly adopted to determine how well the retrieved results are.

Original: It encourages true positives to have higher ranks thanfalse positives: AP = 1 | TP | (cid:80) ∀
Revised: It encourages true positives to have higher ranks than false positives: AP = 1 | TP | (cid:80) ∀

Original: tp i,m ∈ TP P @
Revised: tp i,m ∈ TP P @

Original: i , where P @
Revised: i , where P @

Original: i is the rank- i precision.
Revised: i is the rank- i precision.

Original: As for hash-basedrecognition, since current works formulate it as a variant of retrieval, we could still adopt AP as acriterion to indicate performance.
Revised: As for hash-based recognition, since current works formulate it as a variant of retrieval, we could still adopt AP as a criterion to indicate performance.

Original: Detailed explanation is placed in Supp.
Revised: The detailed explanation is placed in Supp.

Original: Sec. C.
Revised: Sec. C.

Original:  Lower Bound by Inter-Class Distinctiveness and Intra-Class Compactness
Revised:  Lower Bound by Inter-Class Distinctiveness and Intra-Class Compactness

Original: We start at an arbitrary rank list for studying  characteristics of codes in  above scenarios.
Revised: We start at an arbitrary rank list for studying the characteristics of codes in the above scenarios.

Original: To calculate AP easily, we first introduce mis-rank that indicates how many false positives take higher places thantrue positives.
Revised: To calculate AP easily, we first introduce mis-rank that indicates how many false positives take higher places than true positives.

Original: We refer readers to Supp.
Revised: We refer readers to Supp.

Original: Secs.
Revised: Secs.

Original: A and B for detailed description and proof.
Revised: A and B for detailed description and proof.

Original: Definition.
Revised: Definition.

Original: Mis-rank m indicates how many false positives have higher ranks than a true positive i.e .,tp i,m is at rank i , meanwhile m = | (cid:8) d ( q , fp ) <
Revised: Mis-rank m indicates how many false positives have higher ranks than a true positive, e.g ., tp i,m is at rank i , meanwhile m = | (cid:8) d ( q , fp ) <

Original: d (cid:0) q , tp i,m (cid:1)(cid:9) | , where d ( q , · ) is distance between qand a sample, and | · | is number of elements in set. 
Revised: d (cid:0) q , tp i,m (cid:1)(cid:9) | , where d ( q , · ) is distance between q and a sample, and | · | is number of elements in set. Remark.

Revised: The rank- i precision P @

Revised: i is derived to be ( i − m ) / i .

Original: Then, average precision could be derived as:
Revised: Then, average precision could be derived as:

Original: [Equation]
Revised: [Equation]

Original: for all true positives in rank list.
Revised: for all true positives in rank list.

Original: This is because the rank- i precision P @
Revised: This is because the rank- i precision P @

Original: i for any true positive 
Revised: i for any true positive tp

Original: tp i,mequals to ( i − m ) / i .
Revised: i,m equals to ( i − m ) / i .

Original: From the derivation, we could immediately obtain that AP increases iff m decreases.
Revised: From the derivation, we could immediately obtain that AP increases iff m decreases.

Original: Therefore,now we could focus on how to reduce m to thereby increase average precision.
Revised: Therefore, now we could focus on how to reduce m to thereby increase average precision.

Original: Noticed that  value ofm is highly related to two distances, d ( q , fp ) and d ( q , tp ) ,  following proposition is raised:
Revised: Noticed that the value of m is highly related to two distances, d ( q , fp ) and d ( q , tp ) , the following proposition is raised:

Original: Proposition.
Revised: Proposition.

Original: [Equation]
Revised: [Equation]

Original: where · denotes upper bound.
Revised: where · denotes upper bound.

Original: Correspondingly,
Revised: Correspondingly,

Original: [Equation]
Revised: [Equation]

Original: where · denotes lower bound.
Revised: where · denotes lower bound.

Original: If inputs x j have associated labels y j ,  above studies are able to be further generalized.
Revised: If inputs x j have associated labels y j , the above studies are able to be further generalized.

Original: Class-wisecenters, inter-class distinctiveness and intra-class compactness are introduced to formulate our finallower-bound of hash codes’ performance.
Revised: Class-wise centers, inter-class distinctiveness and intra-class compactness are introduced to formulate our final lower-bound of hash codes’ performance.

Original: Definition.
Revised: Definition.

Original: Center c c ∈ C is a representative hash code of a specific class c :
Revised: Center c c ∈ C is a representative hash code of a specific class c :

Original: [Equation]
Revised: [Equation]

Original: where b j is the j th sample in set B and y j is the label of b j .
Revised: where b j is the j -th sample in set B and y j is the label of b j .

Original: Definition.
Revised: Definition.

Original: Inter-class distinctiveness is min D inter where D inter is a set  measures distances betweenall centers over C : (cid:8) d (cid:0) c 
Revised: Inter-class distinctiveness is min D inter where D inter is a set that measures distances between all centers over C : (cid:8) d (cid:0) c j , c k (cid:1)

Original: j , c k (cid:1) | ∀ c j , c k ∈ C (cid:9) .
Revised: | ∀ c j , c k ∈ C (cid:9) .

Original: In contrast, intra-class compactness is 1 / max D intrawhere D intra = (cid:8) d (cid:0) b j , c c (cid:1) | ∀ y j = c (cid:9) measures  distances among samples of  same class to theircorresponding center.
Revised: In contrast, intra-class compactness is 1 / max D intra , where D intra = (cid:8) d (cid:0) b j , c c (cid:1) | ∀ y j = c (cid:9) measures the distances among samples of the same class to their corresponding center.

Original: Combining above propositions and definitions, we could reach:
Revised: Combining above propositions and definitions, we could reach:

Original: Proposition.
Revised: Proposition.

Original: [Equation]
Revised: [Equation]

Original: This proposition reveals how AP is affected by  above two inter- and intra- factors.
Revised: This proposition reveals how AP is affected by the above two inter- and intra- factors.

Original: Actually, it couldcover common scenarios as a criterion of hash codes’ performance, e.g ., fast retrieval and recognition,which will be explained in supplementary materials.
Revised: Actually, it could cover common scenarios as a criterion of hash codes’ performance, e.g ., fast retrieval and recognition, which will be explained in supplementary materials.

Original: Intuitively, lifting this lower bound wouldenhance performance, formulating the following objective:
Revised: Intuitively, lifting this lower bound would enhance performance, formulating the following objective:

Original: [Equation]
Revised: [Equation]

Original: Inspired by [53], we give a specific solution to implement Eqs.
Revised: Inspired by [51], we give a specific solution to implement Eqs.

Original: (2) and (3) under supervised hashingby firstly defining distance maximized class-specific centers and then shrinking hash codes to theircorresponding centers.
Revised: (2) and (3) under supervised hashing by firstly defining distance maximized class-specific centers and then shrinking hash codes to their corresponding centers.

Original: We will validate  effectiveness with such  solution in experiments.
Revised: We will validate the effectiveness of such a solution in experiments.

Original: It is worthnoting that achieving above goal is an open topic, including unsupervised or other complex scenarios,which we leave for future study.
Revised: It is worth noting that achieving above goal is an open topic, including unsupervised or other complex scenarios, which we leave for future study.

Original: Next, to optimize hashing models with  above supervision, weintroduce posterior estimation over hash codes and then control it.
Revised: Next, to optimize hash-models with the above supervision, we introduce posterior estimation over hash codes and then control it.

Original:  Posterior Estimation for Code Control
Revised:  Posterior Estimation for Code Control

Original: It is essentially hard to minimize  distance between a hash code and a target to achieve Eq.
Revised: It is essentially hard to minimize the distance between a hash code and a target to achieve Eq.

Original: (3) due totwo reasons.
Revised: (3) for two reasons.

Original: Firstly,  posterior distribution of B given inputs X is formulated to be under multivariate Bernoulli distribution: p ( B | X ) ∼ MVB .
Revised: Firstly, the posterior distribution of B given inputs X is formulated to be under multivariate Bernoulli distribution: p ( B | X ) ∼ MVB .

Original: If we do not take correlations among different variablesinto account, optimization on B is biased.
Revised: If we do not take correlations among different variables into account, optimization on B is biased.

Original: Secondly, dealing with hash codes is a {− 1 , +1 } discreteoptimization, which is not trivial to handle.
Revised: Secondly, dealing with hash codes is a {− 1 , +1 } discrete optimization, which is not trivial to handle.

Original: Therefore, we try to organize Eq.
Revised: Therefore, we try to organize Eq.

Original: (3) as aMaximum
Revised: (3) as a

Original:  Likelihood Estimation (MLE) to tackle  above issues.
Revised: Maximum Likelihood Estimation (MLE) to tackle the above issues.

Original: Considering  definition of Hamming distancebetween an arbitrary hash code b and its target t :
Revised: Considering the definition of Hamming distance between an arbitrary hash code b and its target t :

Original: [Equation]
Revised: [Equation]

Original: < 0 } where 1 is the characteristic function and ( · ) i indicates value on the i th dimension.
Revised: where 1 is the characteristic function and ( · ) i indicates value on the i -th dimension.

Original: Since Hammingdistance measures how many bits are different between two codes,  probability of d ( b , t ) 
Revised: Since Hamming distance measures how many bits are different between two codes, the probability of d ( b , t ) =

Original: = δ can beformulated as:
Revised: δ can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: Therefore, b and t have distance δ iff p ( d ( b , t ) = δ ) is maximized.
Revised: Therefore, b and t have distance δ iff p ( d ( b , t ) = δ ) is maximized.

Original: However, to precisely calculateit is difficult, since it involves  joint probability of b .
Revised: However, to precisely calculate it is difficult, since it involves the joint probability of b .

Original: Therefore, we try to estimate all joint probabilitiesby adopting a surrogate model P π parameterized by π to perform estimation:
Revised: Therefore, we try to estimate all joint probabilities by adopting a surrogate model P π parameterized by π to perform estimation:

Original: [Equation]
Revised: [Equation]

Original: where o is  probabilities of a Categorical distribution p ( O | X ) (cid:98)
Revised: where o is the probabilities of a Categorical distribution p ( O | X ) (cid:98)

Original: = p ( B | X ) , which contains 2 hentries.
Revised: = p ( B | X ) , which contains 2 h entries.

Original: Each entry of o estimates one of a specific joint probability by feeding ℓ into P π :
Revised: Each entry of o estimates one of a specific joint probability by feeding ℓ into P π :

Original: [Equation]
Revised: [Equation]

Original: where ι is the h bits binary representation of i .
Revised: where ι is the h bits binary representation of i .

Original: For example, o 3 = p ( b = ( -1-1+1+1 )) when h = 4 .
Revised: For example, o 3 = p ( b = ( -1-1+1+1 )) when h = 4 .

Original: To train P π , we perform Maximum Likelihood Estimation (MLE) over p ( O | X ) :
Revised: To train P π , we perform Maximum Likelihood Estimation (MLE) over p ( O | X ) :

Original: [Equation]
Revised: [Equation]

Original: where i = 
Revised: where i = h

Original: [Equation]
Revised: [Equation]

Original: If P π is ready for estimation, we could directly adopt it as a nice non-linear function to maximizep ( d ( b , t ) = k ) .
Revised: If P π is ready for estimation, we could directly adopt it as a nice non-linear function to maximize p ( d ( b , t ) = k ) .

Original: Note that to realize Eq.
Revised: Note that to realize Eq.

Original: (3), we want b and its corresponding center c are closest i.e .maximize p ( d ( b , c ) = 0) .
Revised: (3), we want b and its corresponding center c are closest, i.e . maximize p ( d ( b , c ) = 0) .

Original: So, we could reformulate this maximization as another MLE:
Revised: So, we could reformulate this maximization as another MLE:

Original: [Equation]
Revised: [Equation]

Original: where p ( d ( b , c ) = 0) =
Revised: where p ( d ( b , c ) = 0) =

Original: p ( b i = c 
Revised: p ( b i = c i

Revised: [Equation]

Original: i , 1 ≤ i ≤ h ) ,
Revised: 1 ≤ i ≤ h ) ,

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: 6) which could be optimized by calculating surrogate gradients of Eq.
Revised: which could be optimized by calculating surrogate gradients of Eq.

Original: (6) through P π :
Revised: (6) through P π :

Original: [Equation]
Revised: [Equation]

Original: Unfortunately, such joint probability estimation on MVB requires O (cid:0) 2 h (cid:1) complexity. This is notflexible when h is large.
Revised: Unfortunately, such joint probability estimation on MVB requires O (cid:0) 2 h (cid:1) complexity, which is not flexible when h is large.

Original: By adopting the idea of block code [44], above estimation is able to performon long hash bits by separating hash codes into a series of blocks.
Revised: By adopting the idea of block code [42], the above estimation is able to perform on long hash bits by separating hash codes into a series of blocks.

Original: Specifically, any h bits hashcodes can be split into u blocks while each block consumes h / u bits.
Revised: Specifically, any h bits hash codes can be split into u blocks while each block consumes h / u bits.

Original: Correspondingly, u independentsurrogate networks P π i , 1 ≤ i ≤ u are adopted to perform  above estimation and back-propagationsimultaneously.
Revised: Correspondingly, u independent surrogate networks P π i , 1 ≤ i ≤ u are adopted to perform the above estimation and back-propagation simultaneously.

Original: With such decomposition, the whole problem is transformed into u sub-problemswith a reduced complexity O (cid:0) 2 h (cid:1) → O (cid:0) u · 2 h / u (cid:1) .
Revised: With such decomposition, the whole problem is transformed into u sub-problems with a reduced complexity O (cid:0) 2 h (cid:1) → O (cid:0) u · 2 h / u (cid:1) .

Original: Summarizing all of things, overall optimization via gradient descent is placed in Alg.
Revised: As a summarization all of things, overall optimization via gradient descent is placed in Alg.

Original: We firstperform Eq.
Revised: We first perform Eq.

Original: (2) step by using  pre-defined centers and then perform Eq.
Revised: (2) step by using the pre-defined centers and then perform Eq.

Original: (3) step by back-propagationvia Eq.
Revised: (3) step by backpropagation via Eq.

Original: Two models F θ , P π are optimized with learning rate η 1 , η 2 respectively.
Revised: Two models F θ , P π are optimized with learning rate η 1 , η 2 respectively.

Original: Algorithm 1 One of 1: procedure T RAIN ( F θ , P π ) ▷ Training procedure of two models F θ , P π . Generate class-specific centers c ∈ C , | C | =
Revised: Algorithm 1 One of implementations under supervised circumstance.

Original: Class-num; ▷ (Eq.
Revised: ▷ (

Original: 3: repeat ▷ Main training loop. Sample x from X with label y ; 5: ℓ = F θ ( x ) ; 6: o = P π ( ℓ ) ; 7: π ← π − η 1 ∂L π ∂ π ; ▷ (Eq. 8: θ ← θ − η 2 ˆ ∂L θ ∂ θ with corresponding center c ; ▷ Eq.
Revised: Eq.

Original: (3), Eq.
Revised: (3), return end

Original: 9: until Total epoch exceeds; 10: return F θ ; ▷ Optimized hashing model F θ 11: end procedure

Original:  Experiments
Revised:  Experiments

Original: We conduct extensive experiments on three benchmark datasets to confirm  effectiveness of ourproposed method.
Revised: We conduct extensive experiments on three benchmark datasets to confirm the effectiveness of our proposed method.

Original: To make fair comparisons, we first provide experiments setup. 
Revised: To make fair comparisons, we first provide experiments setup. [Figure] [Figure]

Original: 6.1 Setup
Revised: 6.1 Setup

Original: Our experiments focus on performance evaluation on two typical hash-based tasks, fast retrieval andrecognition.
Revised: Our experiments focus on performance comparisons on two typical hash-based tasks, fast retrieval and recognition, with and without integrating our proposed method.

Original: The  pipeline is tofirstly train models on training split, then hash all samples in basesplit.
Revised: The general evaluation pipeline is to train models in the training split, then hash all samples in the base split.

Original: Then, hashed queries in query split  get rank lists from base splitfor retrieval, while queries’predictions from classifier are used for recognition.
Revised: Then for retrieval, any hashed queries in query split are used to get rank lists of the base split from nearest to farthest according to Hamming distance.

Revised: As for recognition, we adopt a k NN classifier or a linear model to produce queries’ predictions.

Original: 6.1.1 Datasets
Revised: 6.1.1 Datasets

Revised: [Table]

Original: Our experiments are conducted on three datasets, varying from scale, variety and perplexity to validatemethods’ performance in different scenarios.
Revised: Our experiments are conducted on three datasets, varying in scale, variety and perplexity to validate methods’ performance in different scenarios.

Original: Both single-label and multi-label datasets are included.
Revised: Both single-label and multi-label datasets are included.

Original: We follow previous works to generate three splits, which is detailed below:
Revised: We follow previous works to generate three splits, which are detailed below:

Original: CIFAR-10 [20] is a single-label 10 -class dataset.
Revised: CIFAR-10 [20] is a single-label 10 -class dataset.

Original: The whole dataset contains 6 , 000 images for eachclass.
Revised: The whole dataset contains 6 , 000 images for each class.

Original: Following [5], we form the dataset by 500 | 5 , 400 | 100 for each class randomly as train,base, query splits, respectively.[Figure] [Figure] [Table]
Revised: Following [5], we split the dataset into 500 | 5 , 400 | 100 for each class randomly as train, base and query splits, respectively.

Original: NUS-WIDE [7] consists of 81 labels and images may have one or more labels.
Revised: NUS-WIDE [7] consists of 81 labels and images may have one or more labels.

Original: We followprevious works [5] to pick the most frequent21 labels and their associated images ( 195 , 834 )for experiments.
Revised: We follow previous works [5] to pick the most frequent 21 labels and their associated images ( 195 , 834 ) for experiments.

Original: Specifically, 193 , 734 imagesare randomly picked to form  base split whileremaining 2 , 100 images are adopted for queries.10 , 500 images are randomly sampled from basesplit for training models.
Revised: Specifically, 193 , 734 images are randomly picked to form the base split while remaining 2 , 100 images are adopted for queries. 10 , 500 images are randomly sampled from the base split for training models.

Original: ImageNet [11] is a large-scale dataset consistsof 1 , 000 classes.
Revised: ImageNet [11] is a large-scale dataset consists of 1 , 000 classes.

Original: To conduct experiments, wefollow [5] to pick a subset of 100 classes where all images of these classes in  training set / validationset are as base split / query split respectively ( 128 , 503 | 4 , 983 images).
Revised: To conduct experiments, we follow [5] to pick a subset of 100 classes where all images of these classes in the training set / validation set are as base split / query split respectively ( 128 , 503 | 4 , 983 images).

Original: We then randomly sample100 images per class in  base split for training.
Revised: We then randomly sample 100 images per class in the base split for training.

Original: 6.1.2 Implementation Details
Revised: 6.1.2 Implementation Details

Original: Our method is able to integrate into common hashing models.
Revised: Our method is able to integrate into common hash-models.

Original: Due to  limitation of computationresources, we choose a series of representative deep hashing methods for comparison, including
Revised: Due to the limitation of computation resources, we choose a series of representative deep hashing methods for comparison, including

Original: HashNet [5], DBDH [55], DSDH [26], DCH [4], GreedHash [41] and CSQ [53] and denotemethods with our integration as ( · ) D .
Revised: HashNet [5], DBDH [55], DSDH [26], DCH [4], GreedHash [40] and CSQ [51]. When our method is integrated, we append a subscript ( · ) D to the original methods’ name.

Original: All methods adopted for experiments are  from a publicbenchmark implemented with PyTorch [35].
Revised: All methods adopted for experiments are implemented from a public benchmark with PyTorch [34].

Original: For fair comparisons, we conduct experiments withsame backbone (ResNet-50 [16]) and hyper-parameters for all tested methods.
Revised: For fair comparisons, we conduct experiments with the same backbone (ResNet-50 [16]) and hyper-parameters for all tested methods.

Original: We adopt Adam [37]optimizer with default configuration and learning rate of our method η 1 = η 2 = 1 e − 3 for training.
Revised: We adopt Adam [35] optimizer with default configuration and learning rate of our method η 1 = η 2 = 1 e − 3 for training.

Original: For multi-label datasets, we simply modify Alg.
Revised: For multi-label datasets, we simply modify Alg.

Original: 1 Line 8 with sum of multiple losses.
Revised: 1 Linewith the sum of multiple losses.

Original: Blocknumber u of P is set to bits / 8 .
Revised: Block number u of P is set to bits / 8 .

Original: For example, if  hash codelength is 64 , there will be 8 sub models
Revised: For example, if the length of hash code is 64 , there will be 8 sub-models P

Original: P π 1 ∼ P π 8 trained in parallel.
Revised: π 1 ∼ P π 8 trained in parallel.

Original: To mitigate randomness, we report average performance for 5 runson all experiments.
Revised: To mitigate randomness, we report average performance for 5 runs on all experiments.

Original: Evaluation metric adopted for retrieval is mean Average Precision (mAP@ K )where K = 54 , 000 | 5 , 000 | 1 , 000 on CIFAR-10, NUS-WIDE, ImageNet, respectively.
Revised: The evaluation metric adopted for retrieval is mean Average Precision (mAP@ R ) where R = 54 , 000 | 5 , 000 | 1 , 000 on CIFAR-10, NUS-WIDE, ImageNet, respectively.

Original: Forrecognition performance, if methods have  auxiliary classification branch, we directly use predictionsfrom it.
Revised: For recognition performance, if methods have an auxiliary classification branch, we directly use its predictions from it.

Original: Otherwise, we adopt a k NN classifier built on base split and vote with 100 nearest neighbors.
Revised: Otherwise, we adopt a k NN classifier built on base split and vote with 100 nearest neighbours.

Original: 6.2 Ablation Study
Revised: 6.2 Ablation Study

Revised: [Figure]

Original: In  ablation study, we try to reveal correctness of our proposed objectives Eqs.
Revised: In the ablation study, we try to reveal thecorrectness of our proposed objectives Eqs.

Original: (2) and (3), andeffectiveness of our proposed posterior estimation model P π .
Revised: (2) and (3), and the effectiveness of our proposed posterior estimation model P π .

Original: We answer  following questions byconducting experiments on ImageNet to demystify  above concerns.
Revised: We answer the following questions by conducting experiments on ImageNet to demystify the above concerns.

Original: We focus on retrieval performancein ablation study, while we observe similar results on recognition.
Revised: We focus on retrieval performance in the ablation study, while we observe similar results on recognition.

Original: KL ( Est ., Real )

Original: = 7 . 88 e −0 .0 .1 .1 .2 .2 .3 .3 .p

Original: × 10 −

Original: Real Est.

Original: Method

Original: KL real

Original: Naïve 0.

Original: Ours 0.

Original: Is hash model ’s performance lower-boundedby Eq.
Revised: Is hash-model’s performance lower-bounded by Eq.

Original: Validating correctness of Eq.
Revised: Validating correctness of Eq.

Original: (1) isimportant, but it is essentially hard to conductexperiments to confirm it.
Revised: (1) is important, but it is essentially hard to conduct experiments to confirm it.

Original: Nevertheless, we stillreveal correlation between model performanceand inter-class distinctiveness / intra-class compactness by tracking mAP w.r.t .
Revised: Nevertheless, we still reveal the correlation between model performance and inter-class distinctiveness / intra-class compactness by tracking mAP w.r.t .

Original: min D inter max D intra during training.
Revised: min D inter max D intra during training.

Original: To calculate D inter and D intra ,we first hash all samples from  base split andcalculate centers of them over all classes.
Revised: To calculate D inter and D intra , we first hash all samples from the base split and calculate their centers of them over all classes.

Original: The99 . 9 percentile of min D inter and max D intrais picked to avoid outliers.
Revised: The 99 . 9 percentile of min D inter and max D intra is picked to avoid outliers.

Original: We conduct trackingon CSQ D with 16 , 32 , 64 bits per epoch to draw
Revised: We conduct tracking on CSQ D with 16 , 32 , 64 bits per epoch to draw

Original: Fig.
Revised: Fig.

Original: As  figure shows, lines goes from lowerleft to upper right with small fluctuations, whichindicates linear relationship between mAP andmin D inter max D intra on all bit-lengths.
Revised: As the figure shows, lines go from lower left to upper right with small fluctuations, which indicates the linear relationship between mAP and min D inter max D intra on all bit-lengths.

Original: This may partiallyconfirm our theoretical analysis in Sec. 4.
Revised: This may partially confirm our theoretical analysis in Sec. 4.

Original: Convergence speed and efficiency of P π .
Revised: Convergence speed and efficiency of P π .

Original: Toverify convergence speed and efficiency of ourposterior estimation model P π , we test it bysubstituting with two variants: BCE from [53] and Cauchy loss from [4].
Revised: To verify the convergence speed and efficiency of our posterior estimation model P π , we test it by substituting with two variants: BCE from [51] and Cauchy loss from [4].

Original: mAP is evaluated perepoch for three variants with three bit-lengths 16 , 32 , 64 . The convergence curve is plotted in Fig.
Revised: mAP is evaluated per epoch for three variants with three bit-lengths 16 , 32 , 64 and the convergence curve is plotted in Fig.

Original: From the figure, we could see convergence speed of ours and BCE are similar, while Cauchy isslightly slow.
Revised: From the figure, we could see that ours and BCE’s convergence speed are similar, while Cauchy is slightly slow.

Original: Meanwhile, our method has continuously higher performance than other two after ∼ 10epochs for all bit-lengths, which is potentially due to our method could perform low-bias optimization.
Revised: Meanwhile, our method has continuously higher performance than other two after ∼ 10 epochs for all bit-lengths, which is potentially due to our method could perform low-bias optimization.

Original: Note that when training with Cauchy loss, performance is instead lower when codes are longer.
Revised: Note that when training with Cauchy loss, performance is instead lower when codes are longer.

Original: This may because inner-product based Cauchy loss could not handle long bits.
Revised: This may be caused by the inner-product based Cauchy loss could not handle long bits.

Original: We also measureaveraged training time per epoch of all variants on a single NVIDIA RTX 3090, which is placed in Tab. 1.
Revised: We also measure averaged training time per epoch of all variants on a single NVIDIA RTX 3090, which is placed in Tab. 1.

Original: In this table, our model does not consume significant longer time than others to train.
Revised: In this table, our model does not consume significant longer time than others to train.

Original: Aboveobservations reveal  efficiency of our proposed method.
Revised: The above observations reveal the efficiency of our proposed method.

Original: It could be a drop-in replacement withoutintroducing significant overhead.
Revised: It could be a drop-in replacement without introducing significant overhead.

Original: Moreover, convergence speed and efficiency will not drop whenbit-length becomes long, which reveals  flexibility of the blocked  design.
Revised: Moreover, convergence speed and efficiency will not drop when the bit-length becomes long, which reveals the flexibility of the blocked code design.

Original: Could P π estimate multivariate Bernoulli distribution?
Revised: Could P π estimate multivariate Bernoulli distribution?

Original: We design an extra toy experimentsto validate the ability of estimating multivariate Bernoulli distribution by our surrogate model P π . 
Revised: We design an extra toy experiments to validate the ability of estimating multivariate Bernoulli distribution by our surrogate model P π . [Table]

Original: Specifically, 8 bits MVB is generated with randomly 256 joint probabilities.
Revised: Specifically, 8 bits MVB is generated with randomly 256 joint probabilities.

Original: We then take 10 , 000samples from it as inputs to train P π .
Revised: We then take 10 ,samples from it as inputs to train P π .

Original: The model will further estimate distribution by feeding another100 samples and taking  mean of all o s (Eq.
Revised: The model will further estimate distribution by feeding another 100 samples and taking the mean of all o s (Eq.

Original: (4)) as result.
Revised: (4)) as result.

Original: The estimated distribution is evaluated by Kullback–Leibler divergence [23] ( KL ) with real distribution, as well as bar plot, demonstrated in
Revised: The estimated distribution is evaluated by Kullback–Leibler divergence [23] ( KL ) with real distribution, as well as bar plot, demonstrated in

Original: Fig.
Revised: Fig.

Original: As  figure shows, our predicted joint probabilities almost cover real probabilities and KL islow.
Revised: As the figure shows, our predicted joint probabilities almost cover real probabilities and KL is low.

Original: As a comparison, if we estimate real distribution with  product of edge probabilities directly( w / o correlations between variables), KL will be significantly increased (row 1 in table).
Revised: As a comparison, if we estimate real distribution with the product of edge probabilities directly ( w / o correlations between variables), KL will be significantly increased (row 1 in table).

Original: Such toyexperiment reveals  our method is better to estimate MVB s than the naïve one.
Revised: Such toy experiment reveals that our method is better for estimating MVB s than the naïve one.

Original: 6.3 Performance Enhancements when Integrating into State-of-the-Art
Revised: 6.3 Performance Enhancements when Integrating into State-of-the-Art

Revised: [Figure]

Revised: [Figure]

Revised: for different methods.

Revised: By integrating our auxiliary objective, we could observe significant performance and min D inter max D intra increase in most cases (from solid dots to translucent dots).

Revised: Regression line is also plotted with 95% confidence interval to reveal linear relationship between two metrics.

Original: To evaluate performance gain when integrating our proposed method into hashing models, we conductexperiments on three datasets.
Revised: To evaluate performance gain when integrating our proposed method into hash-models, we conduct experiments on three datasets.

Original: Specifically, we first report  original performance of tested modelsand then re-run with our method incorporated to make  comparison.
Revised: Specifically, we first report the original performance of tested models and then re-run with our method incorporated to make a comparison.

Original: Both 16 , 32 , 64 bits results arereported to show performance from short codes to long codes.
Revised: Both 16 , 32 ,bits results are reported to show performance from short codes to long codes.

Original: Retrieval Performance.
Revised: Retrieval Performance.

Original: Retrieval performance comparisons under mAP is shown in Tab. 2.
Revised: Retrieval performance comparisons under mAP is shown in Tab. 2.

Original: Withour method integrated, allof tested hashing methods have performance enhancement.
Revised: With our integrated method, all tested hashing methods have performance enhancement.

Original: We observean up to 26 .
Revised: We observe an up to 26 .

Original: 5% increase and 5 .
Revised: 5% increase and 5 .

Original: 02% in average, which is a significant improvement.
Revised: 02% on average, which is a significant improvement.

Original: Specifically,performance increase on HashNet, DBDH and DSDH is higher than other methods (first 3 rows),especially on ImageNet dataset.
Revised: Specifically, performance increase on HashNet, DBDH and DSDH is higher than other methods (first 3 rows), especially on ImageNet dataset.

Original: A potential reason is  all of these three methods use pairwise metriclearning objectives and inner-product to approximate Hamming distance, which may not handleoptimization well when dataset size and class number is large.
Revised: A potential reason is that all of these three methods use pairwise metric learning objectives and inner-product to approximate Hamming distance, which may not handle optimization well when dataset size and class number is large.

Original: Meanwhile, our method works on[Table] multi-label datasets where all methods also obtain performance gain, indicating  multi-label datamay also benefit from our objective.
Revised: Meanwhile, our method works on multi-label datasets where all methods also obtain performance gain, indicating that multi-label data may also benefit from our objective.

Original: Furthermore, we also observe 3 .
Revised: Furthermore, we [Figure]

Original:  46% average mAP increase on CSQ.
Revised: also observe a 3 . 46% average mAP increase on CSQ.

Original: It adopts  similar objective but  different optimization approach with ours.
Revised: It adopts a similar objective but a different optimization approach with ours.

Original: This increase showseffectiveness of our proposed posterior estimation approach, which could perform better optimizationthan theirs.
Revised: This increase shows the effectiveness of our proposed posterior estimation approach, which could perform better optimization than theirs

Original: [Table]
Revised: [Table]

Original: CSQ 87 .89 .89 .

Original: CSQ D 87 .

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: Recognition Performance.
Revised: Recognition Performance.

Original: Similarly, we report  recognition performance of these methodsby measuring classification accuracy on ImageNet with 16 , 32 , 64 bits codes.
Revised: Similarly, we report the recognition performance of these methods by measuring classification accuracy on ImageNet with 16 , 32 , 64 bits codes.

Original: Results arereported in Tab. 3.
Revised: Results are reported in Tab. 3.

Original: Generally, we observe similar performance gain.
Revised: Generally, we observe similar performance gains.

Original: When methods haveour integration, their accuracy is significantlyincreased, with an up to 20 .
Revised: When methods have our integration, their accuracy is significantly increased, up to 20 .

Original: 5% and 5 .
Revised: 5% and 5 .

Original: 3% inaverage.
Revised: 3% on average.

Original: Specifically, some of them utilize anauxiliary classification branch to produce predictions, i.e ., GreedyHash and DSDH.
Revised: Specifically, some of them utilize an auxiliary classification branch to produce predictions, i.e ., GreedyHash and DSDH.

Original: Whilefor others, we adopt a k NN classifier to classifysamples by voting with  base split.
Revised: While for others, we adopt a k NN classifier to classify samples by voting with the base split.

Original: We obtainincrease for both approaches.
Revised: We obtain increase for both approaches.

Original: This indicates ourapproach also helps for hash-based recognitionin different scenarios, by leveraging inter-classdistinctiveness and intra-class compactness.
Revised: This indicates that our approach also helps for hash-based recognition in different scenarios, by leveraging inter-class distinctiveness and intra-class compactness.

Original: From  above comparisons, we confirm  effectiveness of our proposed method.
Revised: From the above comparisons, we confirm the effectiveness of our proposed method.

Original: Overall, ourmethod is validated to be flexible and promising to deploy in  above scenarios with various bits.
Revised: Overall, our method is validated to be flexible and promising to deploy in the above scenarios with various bits.

Original: 6.4 Qualitative Comparisons
Revised: 6.4 Qualitative Comparisons

Original: t-SNE visualization.
Revised: t-SNE visualization.

Original: To further analyze how our objective affects hash codes by adjusting D interand D intra , we conduct codes visualization by t-SNE [14] on HashNet and DBDH.
Revised: To further analyze how our objective affects hash codes by adjusting D inter and D intra , we conduct codes visualization by t-SNE [14] on HashNet and DBDH.

Original: Specifically,we randomly sample 5 , 000 64 bits codes from CIFAR-10 base split.
Revised: Specifically, we randomly sample 5 , 000 64 bits codes from CIFAR-10 base split.

Original: t-SNE is then performed onthese codes as shown in Fig.
Revised: t-SNE is then performed on these codes as shown in Fig.

Original: 4, where points’ color indicates class.
Revised: 5, where points’ color indicates class.

Original: From these figures, we give ourhumble explanation.
Revised: From these figures, we give our humble explanation.

Original: Firstly, please look at Fig. 4(a), codes extracted from  original HashNet aremixed among different classes.
Revised: Firstly, please look at Fig. 5(a), codes extracted from the original HashNet are mixed among different classes.

Original: While in Fig. 4(c), clusters are loose.
Revised: While in Fig. 5(c), clusters are loose.

Original: These representations resultin low min D inter and high max D intra .
Revised: These representations result in low min D inter and high max D intra .

Original: When they are trained along with our method (Figs. 4(b)and 4(d)), we get more separable and distinct clusters compared to  original ones.
Revised: When they are trained along with our method (Figs. 5(b) and 5(d)), we get more separable and distinct clusters compared to the original ones.

Original: Meanwhile, codes 
Revised: Meanwhile, codes in a cluster are more compact than the original methods.

Original: [Figure] [Figure]
Revised: Quantitative results of min D inter max D intra placed under figures also reveal this phenomenon.

Original: different methods,
Revised: mAP w.r.t .

Original: i
Revised: min D inter max D intra visualization. To further confirm the relationship between hashing performance and min D inter max D intra , as in Sec. 6.2, we plot mAP w.r.t .

Original: e .
Revised: min D inter max D intra for all tested methods on 64 bits ImageNet base split (Fig. 4). As the figure shows, most of the methods obtain higher mAP with larger min D inter max D intra and move from lower-left region to upper-right region (solid dots to translucent dots indicate original methods to integrated methods). Meanwhile, according to the positions of these dots, mAP and min D inter max D intra are supposed to be under linear relationship for different methods, i.e .

Original: higher min D inter max D intra leads tohigher mAP.
Revised: higher min D inter max D intra leads to higher mAP.

Original: Therefore, we give  regression linewith 95% confidence interval on  plot to confirmour observations.
Revised: Therefore, we give a regression line with 95% confidence interval on the plot to confirm our observations.

Original: Extended from this, our proposed lower bound could also be a criterion ofhash codes’ performance.
Revised: Extended from this, our proposed lower bound could also be a criterion of hash codes’ performance.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we conduct  comprehensive study on  characteristics of hash codes.
Revised: In this paper, we conduct a comprehensive study on the characteristics of hash codes.

Original: As a result, we provehash codes’ performance is lower-bounded by inter-class distinctiveness and intra-class compactness.
Revised: As a result, we prove that hash codes’ performance is lower-bounded by inter-class distinctiveness and intra-class compactness.

Original: Formulating such  lower bound as an objective, we could further lift it in hash model training.
Revised: Formulating such a lower bound as an objective, we could further lift it in hash-model training.

Original: Meanwhile, our proposed surrogate model for posterior estimation over hash codes’ fully exploitsabove guidance to perform low-bias model optimization and finally produce good codes.
Revised: Meanwhile, our proposed surrogate model for posterior estimation over hash codes’ fully exploits the above guidance to perform low-bias model optimization and finally produce good codes.

Original: Extensiveexperiments conducted on three benchmark datasets confirm  effectiveness of our proposed method.
Revised: Extensive experiments conducted on three benchmark datasets confirm the effectiveness of our proposed method.

Original: We are able to boost current hashing models’ performance with a flexible integration.
Revised: We are able to boost current hash-models’ performance with a flexible integration.

Original: Limitation and Broader Impacts
Revised: Limitation and Broader Impacts

Original: Hashing and related compact representation learning are able to significantly reduce computationalrequirements while improving memory efficiency when deploying to real scenarios.
Revised: Hashing and related compact representation learning are able to significantly reduce computational requirements while improving memory efficiency when deploying to real scenarios.

Original: However, amain challenge for these techniques is information loss which results in  performance drop.
Revised: However, the main challenge for these techniques is information loss which results in a performance drop.

Original: To tacklethis, our work studies on hash codes’ performance gives a lower-bound, which is validated in twospecific tasks.
Revised: To tackle this, our work on on hash codes’ performance gives a lower bound, validated in two specific tasks.

Original: However, generalizing such lower-bounds to various scenarios, e.g . semi-supervised,unsupervised hash learning, are leaved for future study.
Revised: However, generalizing such lower bounds to various scenarios, e.g . semi-supervised, unsupervised hash learning, are left for future study.

Original: Meanwhile, our posterior estimation approachis not verified for extremely long bits.
Revised: Meanwhile, our posterior estimation approach is not verified for extremely long bits.

Original: Nevertheless, our work may still provide potentials of inspiringresearchers to improve not only hash learning, but also broader areas that adopt metric learning.
Revised: Nevertheless, our work may still provide the potential to inspire researchers to improve not only hash learning, but also broader areas that adopt metric learning.

Original: Ourstudy on posterior estimation may also help for precise discrete optimizations.
Revised: Our study on posterior estimation may also help for precise discrete optimizations.

Revised: Acknowledgments and Disclosure of Funding

Revised: This work is supported by National Key Research and Development Program of China (No.

Revised: 2018AAA0102200), the National Natural Science Foundation of China (Grant No. 62122018,

Revised: No. 61772116, No. 62020106008, No. 61872064).

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Sec.7.(c)
Revised: [Yes] See Sec.

Original:  Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] No explicit negative impacts.
Revised: [No] No explicit negative impacts.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Sec. 4.(b)
Revised: [Yes] See Sec. 4.

Original:  Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Sec. 4.
Revised: [Yes] See Sec. 4.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Sec. 6.
Revised: [Yes] See Sec. 6.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] See Sec. 6.
Revised: [Yes] See Sec. 6.

Original: (d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Sec. 6. 
Revised: [Yes] See Sec. 6. 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Sec. 6.
Revised: [Yes] See Sec. 6.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See Sec. 6.
Revised: [Yes] See Sec. 6.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] Codes appear in supplementary materials.
Revised: [Yes] Codes appear in supplementary materials.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re us- ing/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... 
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: (a) Did you include the full text of
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [Table]
Revised: [N/A]