Title: Locality Sensitive Hashing in Fourier Frequency Domain For Soft Set Containment Search
Venue: NeurIPS 2023
Accepted: yes
Keywords: Locality sensitive hashing

Fourier transform

Order embeddings
Abstract: In many search applications related to passage retrieval, text entailment, and subgraph search, the query and each 'document' is a set of elements, with a document being relevant if it contains the query. These elements are not represented by atomic IDs, but by  embedded representations, thereby extending set containment to *soft* set containment. Recent applications address soft set containment by encoding sets into fixed-size vectors and checking for elementwise *vector* *dominance*. This 0/1 property can be relaxed to an asymmetric *hinge* *distance* for scoring and ranking candidate documents. Here we focus on data-sensitive, trainable indices for fast retrieval of relevant documents. Existing LSH methods are designed for mostly symmetric or few  simple asymmetric distance functions, which are not suitable for hinge distance. Instead, we transform hinge distance into a proposed *dominance* *similarity* measure, to which we then apply a Fourier transform, thereby expressing dominance similarity as an expectation of inner products of functions in the frequency domain. Next, we approximate the expectation with an importance-sampled estimate. The overall consequence is that now we can use a traditional LSH, but in the frequency domain. To ensure that the LSH uses hash bits efficiently, we learn hash functions that are sensitive to both corpus and query distributions, mapped to the frequency domain. Our experiments show that the proposed asymmetric dominance similarity is critical to the targeted applications, and that our LSH, which we call FourierHashNet, provides a better query time vs. retrieval quality trade-off, compared to several baselines. Both the Fourier transform and the trainable hash codes contribute to performance gains.

Locality Sensitive Hashing in Fourier Frequency Domain For Soft Set Containment Search

Abstract

In many search applications related to passage retrieval, text entailment, and subgraph search, the query and each 'document' is a set of elements, with a document being relevant if it contains the query. These elements are not represented by atomic IDs, but by embedded representations, thereby extending set containment to soft set containment. Recent applications address soft set containment by encoding sets into fixed-size vectors and checking for elementwise vector dominance . This 0/1 property can be relaxed to an asymmetric hinge distance for scoring and ranking candidate documents. Here we focus on data-sensitive, trainable indices for fast retrieval of relevant documents. Existing LSH methods are designed for mostly symmetric or few simple asymmetric distance functions, which are not suitable for hinge distance. Instead, we transform hinge distance into a proposed dominance similarity measure, to which we then apply a Fourier transform, thereby expressing dominance similarity as an expectation of inner products of functions in the frequency domain. Next, we approximate the expectation with an importance-sampled estimate. The overall consequence is that now we can use a traditional LSH, but in the frequency domain. To ensure that the LSH uses hash bits efficiently, we learn hash functions that are sensitive to both corpus and query distributions, mapped to the frequency domain. Our experiments show that the proposed asymmetric dominance similarity is critical to the targeted applications, and that our LSH, which we call FOURIERHASHNET, provides a better query time vs. retrieval quality trade-off, compared to several baselines. Both the Fourier transform and the trainable hash codes contribute to performance gains.

1 Introduction

Consider a corpus X of sets x (which we call 'documents') over some universe of discrete items, and let q be a query which is also a subset of this universe. We wish to retrieve those x ∈ X which satisfy q ⊆ x . In most real-world applications, the items in the universe are not just opaque IDs, but are embedded in a rich feature space, demanding that the definition of ' q ⊆ x ' be generalized suitably.

We formalize the notion of soft set containment by writing q = { q i } and x = { x i } and the corresponding sets of item embeddings as { ⃗ q i } and { ⃗ x i } . If q, x are sentences, ⃗ q i , ⃗ x i may be perword contextual embeddings output from a transformer. If q, x are graphs, ⃗ q i , ⃗ x i may be contextual node embeddings, such as those output by a Graph Neural Network (GNN). These set-of-vector representations of q and x are generally of variable sizes. A suitable set encoding gadget, such as simple pooling [41, 31] or a trainable Deep Set [60] or Set Transformer [27] network, converts them to fixed-size vectors given by q = SetEnc( { ⃗ q i } ) and x = SetEnc( { ⃗ x i } ) , with x , q ∈ R K . Several applications [52, 26, 10, 31] then use the test ' q ≤ x ' (elementwise vector dominance) as a surrogate for testing if q ⊆ x .

37th Conference on Neural Information Processing Systems (NeurIPS 2023).

To convert the Boolean test for vector dominance, q ≤ x , into a graded score suitable for ranking (and backpropagation), these applications [52, 26, 10, 31] use a form of (asymmetric) hinge distance d ( q, x ) = ∥ ∥ [ q -x ] + ∥ ∥ 1 = ∑ k max { 0 , q [ k ] -x [ k ] } . (1) d ( q, x ) = 0 when q ≤ x holds elementwise, and measures the extent of the constraint violation otherwise. A search system must retrieve the topτ documents x with the smallest d ( q, x ) , given query q . Several example applications that fit into this framework are elaborated in Appendix B. Even if an application does not fit (1) exactly, our technique may help address other asymmetric distances.

Our goal When corpus X is large, it is impractical to evaluate (1) for each document x . Our goal is to retrieve these τ documents without explicitly evaluating d ( q, x ) for all x ∈ X , within query time that scales slowly with | X | . To achieve this, we design an asymmetric Locality Sensitive Hashing (ALSH) method tailored for hinge distance (1), which then immediately addresses soft set-containment based search.

Prior work and their limitations When set elements are represented by atomic IDs, Bloom filters [36] and maximum inner product search (MIPS) can be used to find the best τ corpus items that are closest to being supersets [46, 59, 45, 2]. However, these techniques are designed specifically for items with opaque IDs, rather than contextual embeddings. LSH [7, 53, 17, 19, 1] has been established as a standard technique for fast approximate near-neighbor search (e.g., FAISS, DPR) in the space of contextual embeddings. However, they predominantly work for symmetric notions of relevance, such as Jaccard similarity, dot product, cosine similarity, or Hamming distance, rather than asymmetric distances like (1). Neyshabur and Srebro [35] propose a LSH suited for asymmetric relevance (ALSH), but it does not provide a satisfactory solution for (1), as our experiments show.

1.1 Our contributions

Responding to the above motivations, we present FOURIERHASHNET, a new LSH for hinge distancebased asymmetric distance measures. Specifically, we make the following contributions.

Scalable hinge distance search for soft set containment From several applications, we distil the strongly-motivated problem of fast topτ retrieval using hinge distance (1), to capture soft set containment. To our knowledge, (A)LSH for hinge distance has not been explored till date.

Transformation of hinge distance to enable ALSH design One could leverage its shift-invariant property to apply a Fourier transform on the negative distance, express it as the dot product similarity between the corresponding Fourier features and then use Asymmetric LSH (ALSH) [35]. However, as we show in Section 3.1, using the negative distance leads to singularities of the underlying Fourier transform at some points. This in turn does not allow us to design an LSH for such measure. We circumvent this problem by a suitable transformation of hinge distance to a dominance similarity , whose Fourier transform is absolutely convergent.

Design of Fourier features Next, we propose a novel method of lifting the dense vectors to frequency domain, such that the dominance similarity in the original space can be expressed as the cosine similarity between the infinite dimensional Fourier features. However, our dominance similarity function is not a positive definite kernel. Hence, unlike Rahimi and Recht [40], we cannot apply Bochner theorem [44] to obtain finite dimensional Fourier features. Instead, we first scale the Fourier features with a sinc function and then obtain finite dimensional Fourier features via importance sampling.

Trainable hashcode design The cosine similarity between the sampled Fourier features is the unbiased estimate of our dominance similarity measure. This allows the use of conventional random hyperplane LSH. However, such an LSH is not guided by the underlying data distribution. To mitigate this limitation, we compute the hashcodes by feeding the Fourier features into a trainable neural hashing network. Prior approaches [54, 15] to trainable hashing encourage bucket balance over the entire corpus, regardless of the query workload. However, this approach is not optimal if most corpus items are irrelevant for most queries, as is usually the case. We propose a new loss function that encourages the best-match hash bucket for a query to include relevant documents and exclude irrelevant documents.

Experiments We show, through extensive experiments, that FOURIERHASHNET is more effective than existing LSH schemes, and that both frequency domain representations and the new trainable hashcode contribute to our gains.

2 Preliminaries

Notation Throughout, we will use [ K ] to mean { 1 , . . . , K } or { 0 , . . . , K -1 } as convenient. We use q to indicate a query and x to indicate a corpus 'document'. Their (possibly learnt) representations are denoted by x , q ∈ R K . For supervision, ( q, x ) may come with a binary relevance judgment rel( q, x ) ∈ { 0 , 1 } . Wehave defined a potentially learnable distance d ( q, x ) -acomputable surrogate for rel( q, x ) -above in Eqn.(1). One can define a similarity measure sim( q, x ) by applying a monotonically decreasing function on the distance d ( q, x ) . We define ι = √ -1 and denote the set of corpus items as X = { x 1 , x 2 , ..., x N } . We indicate the domain of query and corpus items as Q and X respectively. Given a function s ( t ) , its Fourier transform is the function S : C → C which satisfies s ( t ) = ∫ ∞ -∞ S ( ιω ) e ιωt dω , where ω is the frequency variable and S ( ιω ) = 1 2 π ∫ ∞ -∞ s ( t ) e -ιωt dt . For a vector q or x ∈ R K , the Fourier transform is synthesized using a frequency vector ω ∈ R K of same dimension as x or q . Here, a function s ( x ) can be expanded as s ( x ) = ∫ ω ∈ R K S ( ι ω ) e -ι ω ⊤ x d ω .

2.1 Locality sensitive hashing

Indexing corpus items Given a set of corpus items X = { x 1 , x 2 , ..., x N } , an LSH will hash each item x i , L times, which is called the number of trials . For each trial ℓ ∈ [ L ] , it prepares B buckets , which are indexed as the pair ( ℓ, b ) with ℓ ∈ [ L ] and b ∈ [ B ] . In the context of LSH, we draw L independent samples of hash functions h ( ℓ ) from a single hash family H , such that h ( ℓ ) : R K → [ B ] . A corpus item x is inserted in the bucket indexed ( ℓ, h ( ℓ ) ( x )) , for each ℓ ∈ [ L ] .

Symmetric LSH Given a query q , a symmetric LSH computes bucket indices ( ℓ, h ( ℓ ) ( q )) for all L using the same hash functions h ( ℓ ) used for indexing the corpus. Only those items x that are in bucket ( ℓ, h ( ℓ ) ( q )) are considered as candidates ; overall, the candidates are in the union of these buckets. In the rest of the paper, we will describe retrieval for one bucket under one trial, with the understanding that L buckets will contribute candidates. An LSH exists if the query and corpus items are hashed in the same bucket with high (low) probability as long as their similarities are high (low). Formally, we define symmetric LSH as follows.

Definition 2.1 (Symmetric Locality Sensitive Hashing (LSH)) . Given a domain of queries Q and corpus X with Q , X ⊂ Z and a similarity measure sim : Z × Z → R . A distribution over mappings H : Z → N is said to be a ( S 0 , cS 0 , p 1 , p 2 ) -LSH for the similarity function sim( · ) if for all q ∈ Q and x ∈ X we have, with p 1 > p 2 and c < 1 ,

if sim( q, x ) ≤ cS 0 , then Pr h ∼H [ h ( q ) = h ( x )] ≤ p 2 .

if sim( q, x ) ≥ S 0 , then Pr h ∼H [ h ( q ) = h ( x )] ≥ p 1

The hash family H is tailored to the specific choice of similarity function sim( q, x ) (equivalently, the distance d ( q, x ) ). When q , x ∈ R K and sim( q, x ) = cos( q , x ) , the choice of H corresponds to the uncountable set of all hyperplanes in K dimensions passing through the origin [9]. When sim( q, x ) is the Jaccard similarity | q ∩ x | / | q ∪ x | , H is the space of minwise independent hash functions [7].

̸

Asymmetric LSH (ALSH) In many applications, like the current setup (1), we have asymmetric similarity where sim( q, x ) = sim( x, q ) . In such cases, we employ two different hash families G and H to determine the bucket of query and corpus respectively. Formally, we define ALSH as follows:

Definition 2.2 (Asymmetric Locality Sensitive Hashing (ALSH) [35]) . An asymmetric LSH is ( S 0 , cS 0 , p 1 , p 2 ) -ALSH for a similarity function sim( · , · ) over Q , X if we have two different distributions over mappings G and H such that, with p 1 > p 2 and c < 1 ,

if sim( q, x ) ≤ cS 0 then Pr g ∼G ,h ∼H [ g ( q ) = h ( x )] ≤ p 2 .

if sim( q, x ) ≥ S 0 then Pr g ∼G ,h ∼H [ g ( q ) = h ( x )] ≥ p 1

As an example, given ∥ x ∥ ≤ 1 , consider sim( q, x ) = q ⊤ x / || q || 2 , which can be re-written as cos( α ( q ) , β ( x )) , where α ( q ) = [0; q / ∥ q ∥ 2 ] , β ( x ) = [ √ 1 -∥ x ∥ 2 2 ; x ] . Thus, we can apply random hyperplane hash on both α ( x ) and β ( x ) to construct g ( q ) = sign( w · α ( q )) and h ( x ) = sign( w · β ( x )) with w ∼ N ( 0 , I ) . If ∥ x ∥ is unbounded, no ALSH exists for sim( q, x ) = q ⊤ x / || q || 2 [35]. In ( S 0 , cS 0 , p 1 , p 2 ) -ALSH, retrieval of items with similarity score more than S 0 out of a database of items having a similarity score less than cS 0 will admit time-complexity O ( n ρ log n ) and space complexity O ( n 1+ ρ ) where ρ = log p 1 / log p 2 [35].

<latexi sh

1_b

64="OV

5p

UL9+j

IT

rRAuWd0o

>

B7n

c

NSwM

EJ2

X

C

k

/Q

Z

v

F

g

H

8

f

G

D

m

Y

y

K

3

P

<latexi sh

1_b

64="I+u2oO

Uc

p

zM

>

B7n dV

Sw

E

3Wr q/

vg

T

0

X

YF

R

PC

8

Z

D

L

G

f

Jk

Q

5

H

K

<latexi sh

1_b

64="5Cq

9UFuKO

E

yH

f2

W8A

>

B

X

c

VDLSgN

J

r

Y

oP

I

jm

k

d

Z

/

vn

w

Q

R

z3

0

G

T

p

M

7

+

<latexi sh

1_b

64="U/7LSGvBZ

pnDN

IgHT235u+

E

>A

9X

cdV

kM

r

f

w

0

F

Q

Wq

z

8

m

O

j

J

Y

o

R

P

y

C

K

<latexi sh

1_b

64="2vz

IHYKBZLwA7

qg+rT

j

m

0

>

C

c

V

S

N

FJ3

G

M

n

y

o

QE

W

p

/

f

d

k

U

D

9

8

u

P

O

R

X

5

<latexi sh

1_b

64="c

Y

UyPd

XW

ZL0

FA

NQ

>

C

H

VD

SgM

3

2vqk

Bo

Gr

R

5

mn

T

I

j

/

E

uf

7

p

9

z

O

Jw

8

+

K

<latexi sh

1_b

64="N/B

zF

Yg8n

5

+c3

Pw

C

>A

7X

V

S

M

EJ2

T

R

U9k

L

r

o

D

p

f

vQ

j

dm

K

yu

W

O

q

0

GZ

I

H

<latexi sh

1_b

64="

0/HF

O

q3

BC

nMjy

w

IUZ

c

>A

7X

dVDLSg

2v

k

K

G

J

W

9o

5

mT

+

z

u8

Q

N

Y

r

R

p

f

E

P

<latexi sh

1_b

64="Kz+c

C

ZY

yWgv

A

>

X

VDLS

N

EJ

r

fU

9

oP

k

uQ

I5

H

MT

n

w

R

7

O

3

F

p

0

2

<latexi sh

1_b

64="yjR

POQ3

g

9

GXu/

E

n

k

>A

B

cdVDJS

N

L

vT

0

I

F

+

U

p

r

f

YM

q

K

o

W

z2

7

Z

H

8

C

w

5

m

<latexi sh

1_b

64="o8

PW7

5p

2LD

ZT

B

uX

g

>A

C

H

cdV

S

N

EJz

F

I

Ym vG

0

9

/

QR

f

3

q

OM

k

K

n

j

wr

U

+

y

<latexi sh

1_b

64="p

+

RDd j/

c

OuA2M

JT5U

>

B8H

V

NSw

E

3

W

X

Y

9F

0

rGC

Z

K

k

m

n

Q

v

g

P

L

z

7

f

I

y

q

o

<latexi sh

1_b

64="I

5E

pd

F

X

+SZw7UL2WN

k

>A

B8H

c

VD

gM

3

rP

m

R

Q0

j

K9

J

O

G

fo

/

n

T

q

Y

u

z

Cy

v

<latexi sh

1_b

64="

0/HF

O

q3

BC

nMjy

w

IUZ

c

>A

7X

dVDLSg

2v

k

K

G

J

W

9o

5

mT

+

z

u8

Q

N

Y

r

R

p

f

E

P

<latexi sh

1_b

64="WfP

Zjv

Q

2

SGIo

M

>A

8X

cdV

N

w

EJ

+

CR

05

q/

9OK

g

3

n

D

k

y

p

r

F

H

m

Y

u

U

z

<latexi sh

1_b

64="N/B

zF

Yg8n

5

+c3

Pw

C

>A

7X

V

S

M

EJ2

T

R

U9k

L

r

o

D

p

f

vQ

j

dm

K

yu

W

O

q

0

GZ

I

H

<latexi sh

1_b

64="

FS5TwZO8

uR

G

E

>A

B

X

c

V

N

/

L

0

WP

K

D

m7

vH

2+

Q

M

n

z

j

o

I

f

py

C

k

Y

<latexi sh

1_b

64="pzKMXS9ER/

L

PfH

J

0D3

A

>

B7n cdV

g

F

U

2vqk

w

Gj o+

u

Z

N

Y

Q

C

5

O

ym rW

T

I

8

<latexi sh

1_b

64="pzKMXS9ER/

L

PfH

J

0D3

A

>

B7n cdV

g

F

U

2vqk

w

Gj o+

u

Z

N

Y

Q

C

5

O

ym rW

T

I

8

<latexi sh

1_b

64="jI

DQR

3

BA/Y

u

>

+

cdV

S

N

FJ

U

2vW

o

MZ

f

pT

On zE

7

P

w

m

K

rg

9

k

X

C

5G

0

8

y

<latexi sh

1_b

64="2grj

v

S

T

c

dL

>A

B+3

DJ

O

W

UG

z

u

9

k

y

p0

f8

/

R

H

Y

q

F

C

7

5

m

K

Z

o

Q

X

n

<latexi sh

1_b

64="2grj

v

S

T

c

dL

>A

B+3

DJ

O

W

UG

z

u

9

k

y

p0

f8

/

R

H

Y

q

F

C

7

5

m

K

Z

o

Q

X

n

<latexi sh

1_b

64="QfAKB7

0LE

HTN2c

w

G

g

>

C

X

dVDJS

O

W

z

q

M

Ik

Y

y

3r

m

Z

p9

v

j

/

nU

o

8

u

P

F

+

R

5

<latexi sh

1_b

64="

vGEzHRS0k

d

UnKIgY

u

>A

B8X

c

V

N

wM

2

7

+

T

C

q

o

F

5

m

y

Z

W

9

/

P

p

3

r

Q

O

f

j

<latexi sh

1_b

64="kFOpowYnP9fQdV

A38

mUv0

>

B

X

c

DLSgM

2

q

zZ

R

y

5

I

C/

u

Er

j

7

W

K

J

G

H

T

N

+

<latexi sh

1_b

64="PWJSC

V

rkv qZ

E

2

8

>A

B+n cd

DL

N

F

3

Y

U

j

R

p

0

Mw

z

u

Q

5

G

I

7

o

X

y

O/

T

f

H

m

x

q

Fourier

Fourier

F

x

(

ιω

1

..M

)

F

q

(

ιω

1

..M

)

CE loss

h

(

x

)

g

(

q

)

z z

q

W W

W W

·

)

·

)

loss(

Q,X

|

W

)

d

(

q,x

)

̂

W

φ

q

φ

x

̂

̂

(a) Fourier transform

(b) Data driven hashcode training

φ

q

φ

x

cos

z z z

x

q,x

)

Given the set of training queries Q and corpus X , with supervised relevance scores rel( q, x ) ∈ { 0 , 1 } and the surrogate score d ( q, x ) defined in Eq. (1), we aim to design an LSH of the distance d ( q, x ) which can efficiently retrieve topτ corpus items for any new query q ′ .

Why are existing methods not suitable? As we discussed in Section 2.1, relevance metrics for popular LSHs are mostly symmetric, e.g. , cosine, dot-product, and Jaccard similarity. In particular, Jaccard similarity, although commonly used in set-related applications, is not suitable for our problem, where we define rel( q, x ) = 1 when q ⊆ x and 0 otherwise - it is possible that there exists a higher overlap between q and x when q ̸⊆ x , and a lower overlap when q ⊆ x . E.g., suppose q = { a, b } , x 1 = { a, b, c, d, e } , and x 2 = { b } . Here, rel( q, x 1 ) = 1 and rel( q, x 2 ) = 0 . However, the Jaccard similarity J ( q, x ) is not able to reflect the order of rel( q, x ) since J ( q, x 1 ) = 2 / 5 < J ( q, x 2 ) = 1 / 2 .

As discovered by Charikar [9, Lemma 1], the similarity functions in symmetric LSH are inversely related to a metric , which must satisfy symmetry and triangle inequality. Although a query normalized dot product similarity appears asymmetric, it can be expressed using cosine similarity. This readily allows us to use a random hyperplane based (asymmetric) LSH. In contrast, it is not immediately apparent how to find such a connection for our asymmetric hinge distance (1).

3 FOURIERHASHNET: A new ALSH for hinge distance search

Overview of our approach We design an ALSH for d ( q, x ) in three steps. In the first step, we construct a suitable dominance similarity function sim( q, x ) from d ( q, x ) in such a way that there exists a probability distribution p : R K → [0 , 1] and bounded Fourier representations F q ( ι ω ) and F x ( ι ω ) of both query q and corpus items x such that

In the second step, we approximate the expected value of the F q ( ι ω ) ⊤ F x ( ι ω ) using a finite sample of Fourier features. This allows us to apply random hyperplane LSH , similar to asymmetric dot product LSH. However, these hyperplanes are drawn from an isotropic Gaussian distribution in a data-oblivious manner, which results in suboptimal bucket distribution in terms of accuracy-efficiency trade off. To tackle this issue, in the third step, we train the random hyperplanes W which takes the Fourier features as input and give (soft) binary hashcodes, which are optimized to effectively trade off between accuracy and efficiency. Next, we provide the details of the above three steps.

3.1 Design of dominance similarity function sim( q, x ) from hinge distance

Limitations of simple choices of dominance similarity function sim( q, x ) Adominance similarity function sim( q, x ) is inversely related to the hinge distance d ( q, x ) . Chierichetti and Kumar [11] characterized that, any function of a similarity measure is LSHable, if and only if this function is a probability generating function. However, this characterization applies only to symmetric LSH and no such guiding principle is available for an ALSH. In this context, one can experiment with simple designs of sim that are inversely related to d . An immediate choice is sim( q, x ) = -d ( q, x ) .

However, if we allow q and x to be any vector from R K , then sim( q, x ) is not bounded. Finding ALSH for unbounded similarity measures is extremely difficult if not impossible. For example, no ALSH exists even for dot product similarity in two or more dimensions [35]. Moreover, suppose we express sim( q, x ) using the Fourier expansion

Then, S ( ιω ) , i.e. , the Fourier transform of the function s ( t ) = -[ t ] + used in each dimension, has a singularity at ω = 0 . In particular, we have S ( ιω ) = -ιδ ′ ( ω ) / 2 + 1 / 2 πω 2 . Here, δ ′ ( ω ) is the derivative of Dirac delta functional. Thus, S ( ιω ) becomes unbounded as ω → 0 . These issues eventually prevent us from designing bounded Fourier features F q ( ι ω ) and F x ( ι ω ) for Eq. (2).

sim( q, x ) with bounded Fourier transform The key reason for which S ( ιω ) becomes unbounded as ω → 0 is that the function s ( t ) = -[ t ] + is unbounded at t → ∞ . However, in practice, the embeddings are bounded and we have a bounded difference | q [ k ] -x [ k ] | ≤ T . Thus, it is reasonable to ignore the effect of [ q [ k ] -x [ k ]] + when | q [ k ] -x [ k ] | > T . To this end, we compute sim( q, x ) as

In practice, we choose T as a hyperparameter greater than max k | q [ k ] -x [ k ] | . Upon restricting the computation within this domain, one can immediately show that sim( q, x ) = KT -d ( q, x ) .

3.2 Computation of finite dimensional Fourier features for dominance similarity sim( q, x )

Fourier transform of s ( t ) We next compute the Fourier representation S ( ιω ) of s : R → R (4). Proposition 3.1. (Proven in Appendix D) s ( t ) specified in Eq. (4) has Fourier transform

Now, we define S q ( ι ω ) and S x ( ι ω ) , the query and corpus specific Fourier representations:

Here, u k = sign(Re( S ( ιω k ))) , v k = sign(Im( S ( ιω k ))) . They ensure that the dot-product S q ( ιω k ) ⊤ S x ( ιω k ) equals to the real part of the integrand in the RHS of Eq. (6). Since the dominance similarity sim( q, x ) is a real quantity, the imaginary part of the RHS integrates to zero. Therefore, using the dot product of the vectors S q ( ιω k ) and S x ( ιω k ) , which are purely real, we can express

Here, S · ( ι ω ) = [ S · ( ιω 1 ) , .., S · ( ιω K )] , ω = [ ω 1 , .., ω K ] . Note that, the expression of S q ( ιω k ) is different from S x ( ιω k ) in Eq. (7). This maintains the asymmetry in the final dot product in Eq. (8).

Inspired by the seminal work of Rahimi and Recht [40], several works [50, 37] have exploited Fourier transformations to approximate various functions using inner product between the feature maps. However, the functions that Rahimi and Recht [40] considered are shift invariant positive definite kernels. This allowed them to leverage Bochner's theorem [44] which establishes that the Fourier

transformation of these kernels are probability distributions. However, in Eq. (8), there is no such readily available probability distribution. In response, we attempt to find out a probability distribution p ( ω ) which allows us to draw samples using an importance sampling like procedure, as follows:

Let { ω j } j M =1 ∼ p ( ω ) be M i.i.d random samples. We compute the Monte Carlo estimate as follows:

Here, F · ( ι ω 1 ..M ) = [ F · ( ι ω 1 ) , .., F · ( ι ω M )] . Note that, as suggested by Eqs. (7) and (9), || F q ( ι ω 1 ...M ) || 2 = || F x ( ι ω 1 ..M ) || 2 = ∑ M j =1 ∑ K k =1 | Re ( S ( ιω j k ) ) | + | Im ( S ( ιω j k ) ) | p ( ω j k ) . Thus, the value is independent of the query or corpus, which leads to the proportionality relation. We choose the probability distribution p ( ω ) guided the proportionality constant || F · ( ι ω 1 ..M ) || and set p ( ω ) = ∏ k ∈ [ K ] p ( ω ) , where p ( ω ) ∝ | Re( S ( ιω )) | + | Im( S ( ιω )) | . However, the integral of these terms may not be bounded. Therefore, we set the support of p ( ω ) between [ -ω max , ω max ] , thus eliminating the higher frequency terms. The effect on the overall score is small. Attenuation of the higher frequency signals can be seen as a multiplication with a low pass filter in the frequency domain, which affects a convolution in the time domain. Its impact on the similarity score is proportional to 1 /ω max . This still allows us for ALSH despite frequency truncation.

Theorem 3.2. (Proven in Appendix D) Let q , x ∈ R K , cos -1 be Lipschitz with Lipschitz constant L cos ; the hyperparameter T in Eq. (4) be chosen such that T > || q -x || ∞ ; the frequency sampling distribution p ( ω j k ) ∝ [ | Re( S ( ω j k )) | + | Im( S ( ω j k )) | ] with the support set ω j k ∈ [ -ω max , ω max ] and the proportionality constant I ( ω max ) = ∫ ω max -ω max [ | Re( S ( ω ) | + | Im( S ( ω )) | ] dω . Then, the mapping g ( q )[ i ] = sign( w ⊤ i F q ( ω 1 ...M )) and h ( x )[ i ] = sign( w ⊤ i F x ( ω 1 ...M )) where w i ∼ N (0 , I ) , constitutes a ( S 0 , cS 0 , p 1 , p 2 ) -ALSH for some p 1 and p 2 if we choose the support set [ -ω max , ω max ] and the number of samples M as follows:

Based on the outlined assumptions, the above Theorem guarantees that FOURIERHASHNET is an ( S 0 , cS 0 , p 1 , p 2 ) -ALSH for the asymmetric dominance similarity score, subject to appropriate choices of ω max to bound the effect of frequency truncation and M to bound the variance of the Monte Carlo sample estimate.

3.3 Trainable hashing network

Random hyperplane LSH Eq. (10) provides an asymmetric transformation on the input querycorpus pair, which maps it into the cosine similarity space, thus allowing for Random Hyperplanes hashing. We sample H spherically symmetrically distributed normal vectors { w i } H i =1 , i.e. , w i ∼ N (0 , I ) , each perpendicular to a random hyperplane passing though the origin. For each query q and the corpus x , we can generate H -bit hashcodes g ( q ) , h ( x ) ∈ {± 1 } H from the Fourier features (10) as follows: g ( q )[ i ] = sign( w ⊤ i F q ( ι ω 1 ...M )) and h ( x )[ i ] = sign( w ⊤ i F x ( ι ω 1 ...M )) . Consequently, we can index the given corpus with N items, into a hash table with 2 H buckets. For each query q , we restrict our search within bucket b = g ( q ) . If the corpus items are uniformly distributed across all buckets, then it enables sub-quadratic time retrieval with N/ 2 H comparisons (per trial).

Data driven hashcode generation The above random hyperplane LSH approach suffers from two distinct limitations: (1) the quality of Monte Carlo approximation obtained in Eq. (10), depends on the suitability of p ( ω ) , and (2) the hyperplanes are data oblivious. Data oblivious hyperplanes provide the best efficiency if the corpus embeddings are uniformly spread over the K dimensional sphere, which allows the random hyperplanes to evenly allocate the corpus items across different hashcodes. However, in practice, the spatial distribution of the embeddings is not uniform. This results in a skewed distribution of the corpus items across the hash buckets.

To tackle the first problem, we improve the quality of the Fourier features through a trainable nonlinear transformation. Here, we use two networks ϕ q and ϕ x which takes the Fourier features for the query and corpus, i.e. , F q ( ι ω 1 ..M ) and F x ( ι ω 1 ..M ) as input and outputs corresponding

transformed Fourier representations z q = ϕ q ( F q ( ι ω 1 ..M )) and z x = ϕ x ( F x ( ι ω 1 ..M )) . We train ϕ q and ϕ x by minimizing a BCE loss on { cos( z q , z x ) , rel( q, x ) } pairs for q ∈ Q and x ∈ X as follows:

Next, we train the random hyperplanes W = [ w 1 , w 2 , .. ] using the transformed Fourier features { z q } and { z x } . The final hashcodes g ( q ) and h ( x ) are obtained as g ( q ) = sign( ̂ Wz q ) , h ( x ) = sign( ̂ Wz x ) , where ̂ W are the final trained random hyperplanes. For training purposes, we use tanh( W · ) as a smooth surrogate of sign( W · ) . The loss function loss ( Q,X | W ) used to train W consists of three components.

(1) Collision minimizer For any query q , our goal is to ensure that assigned bucket contains only positive items. Assuming corpus items are uniformly distributed across buckets, we ensure that for any query q , the N/ 2 H most relevant items X q ✓ measured in terms of d ( q, x ) will have higher amount of bit overlap than rest of the items X q ✗ . Here, X q ✓ and X q ✗ indicate positive and negative silver instances (not gold instances) indicating top N/ 2 H items in terms of the (possibly

Algorithm 1 FOURIERHASHNET

1: function Train( X , Q , { rel( q, x ) } q ∈ Q,x ∈ X )

2: Draw ω 1 ...M ∼ p ( ω )

3: Compute F q ( ι ω 1 ..M ) , F x ( ι ω 1 ..M ) (Eq. (9))

4: Train ϕ q , ϕ x from rel( q, x ) , F · ( ι ω 1 ..M ) (Eq. (12))

5: Train W by minimizing the loss (14)

6: Return ̂ ϕ x , ̂ ϕ q , ̂ W

1: function Index ( { F x ( ι ω 1 ..M ) } x ∈ X )

2: Require: Trained networks ̂ ϕ x , ̂ W

3: h ( x ) ← sign( ̂ W ̂ ϕ x ( F x ( ι ω 1 ..M ))) ∀ x ∈ X

4: for x ∈ X do

5: hash x to bucket b = h ( x )

6: Return the bucket sets B

1: function Retrieve( q ′ )

2: Require: Trained networks ̂ ϕ q , ̂ W

3: Compute F q ( ι ω 1 ...M ) based on q ′

4: g ( q ′ ) ← sign( ̂ W ̂ ϕ q ( F q ( ι ω 1 ...M )))

5: Rank all x in the bucket b = g ( q ′ ) based on the distance d ( q ′ , x ) to obtain the list List q ′ .

6: Return List q ′

trained) hinge distance d ( q, x ) . We encode this by minimizing the following ranking loss.

This loss encourages that tanh( Wz q ) ⊤ tanh( Wz x ) > tanh( Wz q ) ⊤ tanh( Wz x ′ ) + 1 , i.e. , the number of common bits between q and x ∈ X q ✓ is atleast one more than the same between q and x ′ .

(2) Fence Sitting We set fence sitting loss as ∆ 2 = ∑ x ∈ X ||| tanh( Wz x ) | -1 ∥ 1 . This prevents the optimizer from arriving at a trivial solution by setting all hashcodes to zero.

Algorithm 1 summarizes the overall procedure.

(3) Bit Balance Weset the bit balance loss as ∆ 3 = ∑ i ∈ [ H ] | ∑ x ∈ X tanh( Wz x )[ i ] | . This enforces that each position should have an equal number of +1 and -1 , thus ensuring that each random hyperplane evenly splits the set of points. Finally, we estimate W by minimizing the loss, with λ · as hyperparameters such that ∑ i λ i = 1 , which is given as follows: loss ( Q,X | W ) = λ 1 ∆ 1 + λ 2 ∆ 2 + λ 3 ∆ 3 , (14)

Difference from existing trainable LSH LSH training has been extensively studied [54, 15, 43], with Fence Sitting and Bit Balance losses being well known. However, the Collision Minimizer loss differs significantly from existing approaches. Current techniques seek to ensure load balance across hash buckets for all corpus items, including the ones that may not be relevant to most queries. This is unnecessary for query workloads which touch upon only a small subset of the corpus to generate the best responses. In contrast, our Collision Minimizer loss ensures that only the top-most bucket for any given query allows relevant items and explicitly denies irrelevant items. Thus, it is informed by the query workload, rather than assuming load balance for all items in the corpus. Such an approach may result in balanced bucket loads, but not necessarily.

4 Experiments

In this section, we provide a comprehensive evaluation of our method against several baselines and ablations on four datasets. Appendix F describes additional experiments. Our code is in https://github.com/structlearning/fhashnet .

4.1 Experimental setup

Datasets We experiment on datasets sampled from anonymized real-world Web log data, viz , MSWEB and MSNBC. MSWEB [5] is generated using logs from www.microsoft.com , containing

(a) MSWEB-1

(b) MSWEB-2

(c) MSNBC-1

(d) MSNBC-2

0.8

↑

0.6

FourierHashNet

P

A

0.4

ODP-RH

RH

M

0.2

WMH

0.0

0.0

0.1

0.2

Time

(sec)

0.8

↑0.6

FourierHashNet

P

A

0.4

ODP-RH

RH

M

0.2

OWMH

0.0

0.0

0.1

0.2

Time

(sec)

0.8

0.6

FourierHashNet

0.4

RH

●DP-RH

N

0.2

WMH

0.0

0

1

2

Time

(sec)

0.8

0.6

FourierHashNet

0.4

RH

A

ODP-RH

M

0.2

WMH

0.0

0

1

2

Time

(sec)

records of the areas of the website visited by the users. MSNBC [8] is a collection of logs of user page requests from msnbc.com . In both cases, a record (either q or x ), is a passage that is regarded as a bag of words. Given a collection V of such word bags, ( | V | = 11234 for MSWEB and | V | = 111290 for MSNBC), we sample | Q | = 500 bags from V , designating them as queries, and designate the rest as corpus items X = V \ Q . Consistent with typical information retrieval application scenarios [51], we generate gold relevance labels based on (multi)set containment for MSWEB (MSNBC). (Additional methods for evaluation are explored in Appendix F.) We build the query set Q , such that the number of relevant items N q ⊕ = |{ x ∈ X : rel( q, x ) = +1 }| ∈ [5 , 500] for each query q . We create four datasets by changing average relevance counts per query, N q ⊕ . They are: (1) MSWEB-1 where N q ⊕ = 35 . 624 . (2) MSWEB-2 where N q ⊕ = 20 . 392 . (3) MSNBC-1 where N q ⊕ = 24 . 09 (4) MSNBC-2 where N q ⊕ = 19 . 78 The set of queries Q is partitioned into 20% training set Q tr, 20% validation set Q dev and 60% test set Q test .

Design of query and corpus embeddings q , x We begin with a pre-trained sentence transformer model [41] to obtain 768 dimensional dense contextual representations feature q and feature x for the each word in bags q and x . Embeddings of words belonging to a bag are fed into a deep set [60] network to obtain a bag representation q , x ∈ R K , with K = 294 (chosen via hyperparameter sweep). To train the parameters inside the deep set network, we use q , x to compute the proposed asymmetric hinge distance d ( q, x ) (1), feed it into a trainable sigmoid layer σ and minimize

̂ ̂ ̂ Evaluation Given a test query q ∈ Q test and a set of N ′ q candidate corpus items, we rank them in increasing order of their hinge distances d ( q, x ) . Then we evaluate the average precision (AP) for the query and average over queries to report mean average precision (MAP) - see Appendix E.6.

∑ q,x BCE ( rel( q, x ) , σ ( -d ( q, x )) ) (15) which uses a BCE loss on the gold relevance labels. Once we obtain q and x , we use Algorithm 1 to obtain trained ϕ q , ϕ x and W (Train ( · ) ), which are then used for indexing (Index ( · ) ).

4.2 Effect of different similarity measures on LSH

Setup Here, we compare FOURIERHASHNET against the three LSH baselines, viz , Random hyperlane (RH) [9], Dot product LSH (DP-RH) [35] and Weighted MinHash (WMH) [12], that are tailored towards cosine similarity, dot product similarity and Weighted Jaccard Similarity, respectively. For each LSH method, we train the embeddings q , x and the final hashcodes g ( q ) and h ( x ) using the same networks, as in our method. Furthermore, we set the final relevance measure for ranking to be the similarity score for which the LSH is designed.

Results We vary the mixing hyperparameters λ 1 and λ 2 in our loss (14) and the number of buckets B to explore the tradeoff between accuracy (MAP) and average query time. In Figure 2, we summarize the results. We observe that: (1) FOURIERHASHNET outperforms all the baselines by providing significantly better time-vs.-MAP trade-off across all datasets. In MSWEB datasets, all the baselines except DP-RH show poor performance. All baselines perform poorly for the MSNBC dataset. We remark that cosine similarity, dot product or weighted Jaccard similarity are not suited for vector dominance search. Therefore, the maximum possible MAP obtained by them are severely constrained. (2) In MSWEB datasets, DP-RH performs moderately, by achieving a MAP value around 0.4-0.42 within 0.03 seconds (average query time). This is because dot product can be computed significantly faster than all the other distance/similarity measures. In particular, it is ∼ 7 . 5 × faster than our hinge distance (1), ∼ 10 . 3 × faster than cosine, and ∼ 5 . 1 × faster than Jaccard similarity.

(a) MSWEB-1

(b) MSWEB-2

(c) MSNBC-1

(d) MSNBC-2

FourierHashNet

0.6

0.6

ODP-RH+Hinge

RH+Hinge

↑0.4

↑0.4

0.6

0.6

AP

P

P

0.4

P

0.4

WMH+Hinge

M

0.2

M

A

0.2

M

A

0.2

M

A

0.2

0.0

0.0

0.0

0.0

0.00

0.01

0.02

0.00

0.01

0.02

0.0

0.1

0.2

0.0

0.1

0.2

Time

(sec)

Time

(sec)

个

Time (sec)→

Time

e(sec）→

4.3 Comparison against other efficient indexing techniques

In Section 4.2, we used the similarity score corresponding to each LSH method for final candidate ranking. The baselines performed poorly, which may result from a poor choice of final similarity score or the indexing method. Here, we evaluate FOURIERHASHNET against baseline indexing methods applied on hinge distance guided embeddings. We consider LSH and Inverted File Indexing (IVF) based indexing variants in this section, and discuss graph-based indexing methods in Appendix F.

Comparison with LSH based indexing . Shrivastava and Li [47] showed that an LSH not tailored to the final scoring function may still provide an effective filter. Accordingly, we set the final similarity function to be dominance similarity, and compare against four possible LSH baselines.

Given the embeddings q , x trained (15) using hinge distance, we feed them into the four baselines, each of which trains a hashing network in a different way. (1) RH+Hinge: We train a set of random hyperplanes represented by W and compute the hashcodes as h ( q ) = sign( Wq ) and h ( x ) = sign( Wx ) . (2) DP-RH+Hinge: We train random hyperplanes W for these embeddings to compute the hashcodes as g ( q ) = sign( W [0 , q / || q || ]) and h ( x ) = sign( W [ √ T 2 -|| x || 2 , x ]) . (3) WMH+Hinge: We use the best performing WMH implementation from DrHash toolkit [58] to obtain the hashcodes. (4) FLORA[15]: We train asymmetric hash networks (net 1 , net 2 ) using an end-to-end data-driven approach, which minimizes bit balance and decorrelation loss, along with a consistency loss which predicts the final similarity score using cos( net 1 ( q ) , net 2 ( x )) .

Figure 3 compares the performance of FOURIERHASHNET, RH+Hinge, DP-RH+Hinge, WMH+Hinge and FLORA in terms of MAP for MSWEB and MSNBC datasets. Here we analyze the section of the trade-off curve which provides ≥ 10X speedup compared to exhaustive search. The complete tradeoff curve is provided in Appendix F. (1) The newly designed baselines are now seen to perform significantly better than those used in the previous experiments with Figure 2. However FOURIERHASHNET still outperforms all the baselines. (2) RH+Hinge, despite achieving the second highest scores in many cases, is seen to suffer from a large variance in performance within any given time budget. This would make it difficult to tune the hyperparameters to achieve the requisite performance v/s retrieval speed trade-off. (3) DP-RH+Hinge is seen to have a significantly worse performance than FOURIERHASHNET everywhere. This indicates that DP-RH is ill-suited to asymmetric hinge distance based retrieval. (4) We observe that for the same amount of query time invested, FLORA's MAP can lag ours by over 10%, particularly when faster average query times are required. FLORA's hyperparameter tuning is also more delicate, with there being unsuccessful settings (where MAP grows very slowly with query time) very close to relatively successful ones.

Comparison with IVF indexing We use the widely used FAISS-IVF [21] library, which supports IVF indexing based on L2 distance (IVF-L2) and Inner Product similarity (IVF-IP). Additionally, we propose an alternative Fourier+IVF+IP, where we apply Fourier transformation on the input embeddings, before using IVF-IP. We provide embeddings q , x , trained using hinge distance to all the methods, and use hinge distance to rank retrieved items.

Figure 4 compares the performances in terms of MAP, across all datasets. We observe that: (1) FOURIERHASHNET outperforms both IVF-L2 and IVF-IP across all datasets. FAISS-IVF retrieval suffers because its quantizers, that assign vectors to the V oronoi cells, rely on a metric like L2 or IP, which are unsuitable for asymmetric hinge distance. (2) Fourier transformation provides a significant boost in performance across all datasets, as seen while comparing Fourier+IVF+IP against IVF+IP. However, FOURIERHASHNET still outperforms Fourier+IVF+IP, most noticeably in MSWEB-2.

0

50

100

No. of Corpus items

0

.

0

0

.

2

0

.

4

0

.

6

MAP

(

×

10

)

FourierHashNet

Fourier+IVF+IP

IVF+L2

IVF+IP

(a) MSWEB-1

0

50

100

No. of Corpus items

0

.

0

0

.

2

0

.

4

0

.

6

MAP

(

×

10

)

(b) MSWEB-2

0

50

100

No. of Corpus items

0

.

0

0

.

2

0

.

4

0

.

6

MAP

(

×

10 2

)

(c) MSNBC-1

0

50

100

No. of Corpus items

0

.

0

0

.

2

0

.

4

0

.

6

MAP

(

×

10 2

)

(d) MSNBC-2

4.4 Ablation study

Data driven vs data oblivious LSH To perform ablation study on our proposed hashcode training method, we propose an alternative FHASH (UNTRAINED). This applies our Fourier features followed by a data oblivious random hyperplane LSH, without any data driven hashcode training.

In Figure 5, we compare the complete design of our method, i.e. , FOURIERHASHNET and FHASH (UNTRAINED) against the untrained versions of

(a) MSWEB-1

(b) MSWEB-2

Figure 5: Effect of untrained RH

FourierHashNet

RH+Hinge (Untrained)

FHash (Untrained)

DP-RH+Hinge(Untrained)

0.6

0.6

0.4

0.4

IAP

AP

M

0.2

M

0.2

0.0

0.0

0.00

0.01

0.02

0.00

0.01

0.02

Time

(sec）→→

Time

(sec）→

RH+Hinge and DP-RH+Hinge. We make the following observations: (1) Benefit of Fourier transformation: The MAP vs time trade-off curve of FHASH (UNTRAINED), consistently dominates all the baselines across both datasets. (2) Benefit of hashcode training: Compared to FHASH (UNTRAINED), we observe that FOURIERHASHNET allows for significantly more choices of trade-off points, where higher MAP is required.

̸

Figure 6 compares the performance of the two variations of the losses in terms of MAP, for MSWEB datasets. We observe that: (1) Our loss containing the collision minimizer term performs better than its variant which uses the decorrelation loss. In MSWEB-2, latter provides a MAP of 0 . 4 in 0 . 014 secs, which our loss achieves in 50% of the time. (2) Our method allows for greater freedom in navigating the performance vs average query time trade-off, as seen in MSWEB-2, as it is more spread out across the time axis.

5 Conclusion

We have presented FOURIERHASHNET, an ALSH for asymmetric hinge distance, strongly motivated by text, image and graph retrieval applications. By converting hinge distance to a proposed dominance similarity and applying a suitable Fourier transform to the dominance similarity, we can estimate the distance as an inner product over an importance-sampled spectrum, which further enables the use of a trainable LSH in the frequency domain. Experiments show that FOURIERHASHNET dramatically speeds up queries while preserving or improving retrieval accuracy. Our approach can be extended to any shift invariant functions including Chamfer distance, box embeddings, etc. Box embeddings are known to model more complex set operations like set overlap and set difference [42, 13], making them an interesting avenue for future research. One limitation of FOURIERHASHNET compared to simple symmetric LSHs is the increase in computational cost to compute the Fourier transform. One can explore other types of transformations to mitigate this cost.

Ablation on collision minimizer Here, we replace the collision minimizer in loss ( Q,X | W ) (14) with decorrelation loss which encourages hashcodes to be dissimilar: ∆ 1 = ∑ x = y | tanh( Wz x ) ⊤ tanh( Wz y ) | , a commonly used loss in prior work [54, 15].

(a) MSWEB-1

(b) MSWEB-2

↑0.5

A

0.3

MA

P

0.3

M

0.1

Decorrelation

0.1

Decorrelation

0.00

0.01

0.02

0.00

0.01

0.02

Time

(sec)

↑

Time

(sec)

↑

References

Alexandr Andoni, Piotr Indyk, and Ilya Razenshteyn. Approximate nearest neighbor search in high dimensions. In Proceedings of the International Congress of Mathematicians: Rio de Janeiro 2018 , pages 3287-3318. World Scientific, 2018.

Alex Auvolat, Sarath Chandar, Pascal Vincent, Hugo Larochelle, and Yoshua Bengio. Clustering is efficient for approximate maximum inner product search. arXiv preprint arXiv:1507.05910 , 2015.

Haim Avron, Huy Nguyen, and David Woodruff. Subspace embeddings for the polynomial kernel. Advances in neural information processing systems , 27, 2014.

Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326 , 2015.

John S Breese, David Heckerman, and Carl Kadie. Empirical analysis of predictive algorithms for collaborative filtering. arXiv preprint arXiv:1301.7363 , 2013.

Andrei Z Broder. On the resemblance and containment of documents. In Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No. 97TB100171) , pages 21-29. IEEE, 1997.

Andrei Z Broder, Moses Charikar, Alan M Frieze, and Michael Mitzenmacher. Min-wise independent permutations. In Proceedings of the thirtieth annual ACM symposium on Theory of computing , pages 327-336, 1998.

Igor Cadez, David Heckerman, Christopher Meek, Padhraic Smyth, and Steven White. Visualization of navigation patterns on a web site using model-based clustering. In Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining , pages 280-284, 2000.

Moses S Charikar. Similarity estimation techniques from rounding algorithms. In Proceedings of the thiry-fourth annual ACM symposium on Theory of computing , pages 380-388, 2002.

Tejas Chheda, Purujit Goyal, Trang Tran, Dhruvesh Patel, Michael Boratko, Shib Sankar Dasgupta, and Andrew McCallum. Box embeddings: An open-source library for representation learning using geometric structures. arXiv preprint arXiv:2109.04997 , 2021. URL https: //www.iesl.cs.umass.edu/box-embeddings/main/index.html .

Flavio Chierichetti and Ravi Kumar. Lsh-preserving functions and their applications. Journal of the ACM (JACM) , 62(5):1-25, 2015.

Ondrej Chum, James Philbin, Andrew Zisserman, et al. Near duplicate image detection: Min-hash and tf-idf weighting. In Bmvc , volume 810, pages 812-815, 2008.

Shib Sankar Dasgupta, Michael Boratko, Siddhartha Mishra, Shriya Atmakuri, Dhruvesh Patel, Xiang Lorraine Li, and Andrew McCallum. Word2box: Capturing set-theoretic semantics of words using box embeddings. arXiv preprint arXiv:2106.14361 , 2021.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL Conference , 2019. URL https://www.aclweb.org/anthology/N19-1423.pdf .

Khoa Doan, Shulong Tan, Weijie Zhao, and Ping Li. Asymmetric hashing for fast ranking via neural network measures. arXiv preprint arXiv:2211.00619 , 2022.

Sreenivas Gollapudi and Rina Panigrahy. Exploiting asymmetry in hierarchical topic extraction. In Proceedings of the 15th ACM international conference on Information and knowledge management , pages 475-482, 2006.

E. Grant, C. Hegde, and P. Indyk. Nearly optimal linear embeddings into very low dimensions. In Global Conference on Signal and Information Processing (GlobalSIP) , 2013.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems , 30, 2017.

Piotr Indyk, Ilya Razenshteyn, and Tal Wagner. Practical data-dependent metric compression with provable guarantees. Advances in Neural Information Processing Systems , 30, 2017.

Sergey Ioffe. Improved consistent sampling, weighted minhash and l1 sketching. In 2010 IEEE international conference on data mining , pages 246-255. IEEE, 2010.

Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data , 7(3):535-547, 2019.

Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei. Image retrieval using scene graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3668-3678, 2015.

Purushottam Kar and Harish Karnick. Random feature maps for dot product kernels. In Artificial intelligence and statistics , pages 583-591. PMLR, 2012.

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 6769-6781, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.550. URL https://www.aclweb.org/anthology/ 2020.emnlp-main.550 .

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016.

Alice Lai and Julia Hockenmaier. Learning to predict denotational probabilities for modeling entailment. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers , pages 721-730, Valencia, Spain, April 2017. Association for Computational Linguistics. URL https://aclanthology.org/ E17-1068 .

Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning , pages 3744-3753. PMLR, 2019.

Ping Li. 0-bit consistent weighted sampling. In Proceedings of the 21th ACM SIGKDD International conference on knowledge discovery and data mining , pages 665-674, 2015.

Jimmy J. Lin, Rodrigo Nogueira, and Andrew Yates. Pretrained transformers for text ranking: Bert and beyond. WSDM Conference , 2021. URL https://dl.acm.org/doi/pdf/10. 1145/3437963.3441667 .

Fanghui Liu, Xiaolin Huang, Yudong Chen, and Johan AK Suykens. Random features for kernel approximation: A survey on algorithms, theory, and beyond. arXiv preprint arXiv:2004.11154 , 2020.

Zhaoyu Lou, Jiaxuan You, Chengtao Wen, Arquimedes Canedo, Jure Leskovec, et al. Neural subgraph matching. arXiv preprint arXiv:2007.03092 , 2020.

Qin Lv, William Josephson, Zhe Wang, Moses Charikar, and Kai Li. Multi-probe lsh: efficient indexing for high-dimensional similarity search. In Proceedings of the 33rd international conference on Very large data bases , pages 950-961, 2007.

Yu A Malkov and Dmitry A Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence , 42(4):824-836, 2018.

Bhaskar Mitra and Nick Craswell. An introduction to neural information retrieval. Foundations and Trends in Information Retrieval , 13:1-126, 2018. URL https://www.nowpublishers. com/article/Details/INR-061 .

Behnam Neyshabur and Nathan Srebro. On symmetric and asymmetric lshs for inner product search. In International Conference on Machine Learning , pages 1926-1934. PMLR, 2015.

Anna Pagh, Rasmus Pagh, and S Srinivasa Rao. An optimal bloom filter replacement. In Proceedings of the sixteenth annual ACM-SIAM symposium on Discrete algorithms , pages 823-829, 2005.

Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A Smith, and Lingpeng Kong. Random feature attention. arXiv preprint arXiv:2103.02143 , 2021.

Ninh Pham and Rasmus Pagh. Fast and scalable polynomial kernels via explicit feature maps. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 239-247, 2013.

Maxim Raginsky and Svetlana Lazebnik. Locality-sensitive binary codes from shift-invariant kernels. Advances in neural information processing systems , 22, 2009.

Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in neural information processing systems , 20, 2007.

Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bertnetworks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, 11 2019. URL https://arxiv.org/ abs/1908.10084 .

Hongyu Ren, Weihua Hu, and Jure Leskovec. Query2box: Reasoning over knowledge graphs in vector space using box embeddings. arXiv preprint arXiv:2002.05969 , 2020.

Indradyumna Roy, Abir De, and Soumen Chakrabarti. Adversarial permutation guided node representations for link prediction. In AAAI Conference , 2021. URL https://arxiv.org/ abs/2012.08974 .

Walter Rudin. Fourier analysis on groups . Courier Dover Publications, 2017.

Fumin Shen, Wei Liu, Shaoting Zhang, Yang Yang, and Heng Tao Shen. Learning binary codes for maximum inner product search. In Proceedings of the IEEE International Conference on Computer Vision , pages 4148-4156, 2015.

Anshumali Shrivastava and Ping Li. Asymmetric LSH (ALSH) for sublinear time maximum inner product search (MIPS). NeurIPS , abs/1405.5869, 2014. URL https://arxiv.org/ pdf/1405.5869.pdf .

Anshumali Shrivastava and Ping Li. In defense of minhash over simhash. In Artificial Intelligence and Statistics , pages 886-894. PMLR, 2014.

Konstantinos Skianis, Giannis Nikolentzos, Stratis Limnios, and Michalis Vazirgiannis. Rep the set: Neural networks for learning set representations. In Silvia Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , volume 108 of Proceedings of Machine Learning Research , pages 1410-1420. PMLR, 26-28 Aug 2020. URL https://proceedings.mlr.press/v108/skianis20a.html .

Zhao Song, David Woodruff, Zheng Yu, and Lichen Zhang. Fast sketching of polynomial kernels of polynomial degree. In International Conference on Machine Learning , pages 9812-9823. PMLR, 2021.

Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems , 33:7537-7547, 2020.

Manolis Terrovitis, Panagiotis Bouros, Panos Vassiliadis, Timos Sellis, and Nikos Mamoulis. Efficient answering of set containment queries for skewed item distributions. In Proceedings of the 14th International Conference on Extending Database Technology , pages 225-236, 2011.

Ivan Vendrov, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. Order-embeddings of images and language. arXiv preprint arXiv:1511.06361 , 2015. URL https://arxiv.org/pdf/1511. 06361 .

Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, and Josh Attenberg. Feature hashing for large scale multitask learning. In ICML , pages 1113-1120, 2009. URL https: //arxiv.org/pdf/0902.2206.pdf .

Yair Weiss, Antonio Torralba, and Rob Fergus. Spectral hashing. Advances in neural information processing systems , 21, 2008.

Wei Wu, Bin Li, Ling Chen, and Chengqi Zhang. Canonical consistent weighted sampling for real-value weighted min-hash. In 2016 IEEE 16th International Conference on Data Mining (ICDM) , pages 1287-1292. IEEE, 2016.

Wei Wu, Bin Li, Ling Chen, and Chengqi Zhang. Consistent weighted sampling made more practical. In Proceedings of the 26th International Conference on World Wide Web , pages 1035-1043, 2017.

Wei Wu, Bin Li, Ling Chen, Chengqi Zhang, and S Yu Philip. Improved consistent weighted sampling revisited. IEEE Transactions on Knowledge and Data Engineering , 31(12):2332-2345, 2018.

Wei Wu, Bin Li, Ling Chen, Junbin Gao, and Chengqi Zhang. A review for weighted minhash algorithms. IEEE Transactions on Knowledge and Data Engineering , 34(6):2553-2573, 2022.

Xiao Yan, Jinfeng Li, Xinyan Dai, Hongzhi Chen, and James Cheng. Norm-ranging lsh for maximum inner product search. 31, 2018.

Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems , 30, 2017.

Locality Sensitive Hashing in Fourier Frequency Domain For Soft Set Containment Search (Appendix)

A Limitations of our work

(1) In Eq. (9), the probability distribution p ( ω ) is determined based on the proportionality constant || F · ( ι ω 1 ..M ) || , and we set p ( ω ) = ∏ k ∈ [ K ] p ( ω ) , where p ( ω ) ∝ | Re( S ( ιω )) | + | Im( S ( ιω )) | . We note that this choice of distribution is not informed by the data distribution. Doing so may further improve FOURIERHASHNET.

(2) In our approach, the dominance similarity function is represented as an expectation of inner products of functions in the frequency domain, as described in Eq. (10). The accuracy of this representation relies on the quality of Monte Carlo approximations, which is influenced by the number of ω samples used. Our experimental findings, presented in Figure13, suggest that it may be necessary to generate up to 100 samples per dimension to reduce the approximation error to acceptable levels. A better choice of p ( ω ) may reduce the number of samples needed.

(3) During our experimental investigations, we discovered that the computation of our proposed dominance similarity score is approximately 7.5X slower, compared to the dot product similarity on our datasets. This aligns with earlier observations where matrix subtraction operations have been known to be significantly slower than dot product. This disparity in computation speed represents another potential limitation of our approach, which could be addressed by exploring alternative designs for the dominance similarity function.

B Example applications of soft set containment

Natural Language Inference (NLI) In (NLI) [4], q and x are sentences, regarded as sequences of words as items. A transformer network [14, 41] converts each sentence to an embedding vector. We infer x = ⇒ q if q ≤ x [26]. Consider now a claim verification application that, given a claim as a query and a Web-scale corpus, needs to quickly retrieve passages that best support the given claim. This application exactly motivates FOURIERHASHNET.

Market basket Given a basket of supermarket items, we may query purchase logs for frequent supersets to make recommendations. The corpus contains itemsets purchased in the past, each is a 'document' x . The query q is the current basket. Hard set containment tests for q ⊂ x . However, we would like to 'soften' items in q (say, from one toothpaste brand to another. Each item has a short textual description, which is passed into BERT [41] and the [CLS] embedding read out as the item representation. An itemset is then embedded using a (suitably fine-tuned) set encoder (such as Deep Set), giving us q and x . To the extent q ≤ x , we regard the query basket as 'soft-contained' in the document basket.

Knowledge graph (KG) completion Vendrov et al. [52] embedded types and entities in a KG to vectors such that if two types are related via t 1 is-subtype-of t 2 then ⃗ t 1 ≤ ⃗ t 2 was encouraged, and if entity e is-instance-of t , then ⃗ e ≤ ⃗ t was encouraged in suitable loss functions. Later, these order embeddings were generalized to box embeddings where is-subtype-of and is-instance-of were modeled as boxes in high dimensions contained in other boxes [10]. These models naturally motivate fast retrieval using dominance similarity.

Subgraph isomorphism search Here we expect a corpus graph x will be relevant if query graph q is a subgraph of x , i.e., that x has a subgraph that is isomorphic to q . In reality, we want to score highly corpus graphs that have subgraphs almost isomorphic to the query graph. A graph neural network (GNN) [25, 18] can build suitable contextual embeddings q and x for the entire graphs, which can be used to test for approximate subgraph isomorphism, i.e. , q ≤ x . There are several applications of subgraph isomorphism search. In material and drug design, there are large molecule databases. A researcher wishes to predict properties of a new query molecule by retrieving similar molecules in the database. Each molecule is modeled as a modest-sized graph with nodes (atom, DNA bases) and edges (valence, etc.). In image search [22], the query q may be a graph fragment, e.g., ⟨ person, feeding, pet ⟩ , and the goal is to find corpus graphs x where q is approximately a subgraph [31], e.g. , x can contain ⟨ man, feeding, dog ⟩ .

C Further discussion on related work

In this section, we discuss existing work related to each of the three major components of our workset embeddings, use of frequency domain for computing representations, and locality sensitive hashing.

Neural set embeddings Motivated by various machine learning questions that are better formalized by using a set of items as primitive, there has also been a recent line work on set embeddings. Zaheer et al. [60] consider models that act on sets and characterize the structure of such permutation-invariant models, but do not consider asymmetric query based measures e.g., containment. Skianis et al. [48] casts the similarity measurement between sets as a combinatorial flow problem, which in turn is approximated by a linear program. Lee et al. [27] propose the Set Transformer, a model that uses self-attention to model interactions among the input set elements. Our model is different from the existing line of work in focusing on asymmetric metrics that can measure containment, as well as in using the frequency domain representation of the metric to build a scalable LSH.

Application of frequency domain transformation in machine learning One of the most celebrated uses of the frequency domain representation was by [40], who proposed using random Fourier features for shift-invariant kernels. Since dot-product kernels are not shift-invariant, Bochner's theorem, a key tool in creating random Fourier features, does not apply. Hence, alternative random feature-based approximations have been proposed, primarily focusing on polynomial kernels [23, 38, 3, 49, 30]. All of the above work is on symmetric kernels. For our asymmetric dominance similarity function, we design a sampling distribution by taking into account the individual frequency-level coefficients of the Fourier representation. Raginsky and Lazebnik [39] provided a random Fourier based approach for LSH. However they only focused on similarity of the Mercer Kernel class, which is symmetric in nature.

Locality sensitive hashing The third main pillar of our work is locality sensitive hashing (LSH) which enables efficient retrieval. Answering queries using sketches or hashes in order to measure the similarity or containment of documents has a long history, pioneered by Broder [6]. In more recent years, semantic search or vector search, sometimes called 'dense passage retrieval' [24] employing scalable near-neighbor search engines, has emerged as a credible, often more powerful, alternative [34, 29] to standard information retrieval schemes, as the vector embeddings can capture more nuanced contextualization and semantics. While there are a number of variants of LSH, including multi-probe [32], our presentation and experimentation focus on a single-probe setting in which the hashing hyperplanes are learned from the data. In particular, we build upon the asymmetric hash constructions in Shrivastava and Li [46] and Neyshabur and Srebro [35].

D Proofs of the technical results

D.1 Proof of Proposition 3.1

Proof. Consider two functions BOX a,b (for arbitrary positive constants a, b ) and RELU T defined as follows:

Observe that s can be written in terms of these new functions as follows: s ( t ) = BOX T,T ( t ) -RELU T ( t ) . By linearity of Fourier transform,

where, F f ( ιω ) denotes Fourier transform of f ( t ) for any function f . Now let us compute F RELU T ( ιω ) .

Since BOX T,T ( t ) is a rectangular pulse, its Fourier transform is

Substituting the above Eqs. (22) and (23) into Eq. (18), we get S ( ιω ) as follows.

D.2 Proof of Theorem 3.2

Theorem 3.2. Let q , x ∈ R K , cos -1 be Lipschitz with Lipschitz constant L cos ; the hyperparameter T in Eq. (4) be chosen such that T > || q -x || ∞ ; the frequency sampling distribution p ( ω j k ) ∝ [ | Re( S ( ω j k )) | + | Im( S ( ω j k )) | ] with the support set ω j k ∈ [ -ω max , ω max ] . Furthermore, we define ∆ sim := T T -max x , q || x -q || ∞ and I ( ω max ) = ∫ ω max -ω max [ | Re( S ( ω ) | + | Im( S ( ω )) | ] dω . Then, the mapping g ( q )[ i ] = sign( w ⊤ i F q ( ω 1 ...M )) and h ( x )[ i ] = sign( w ⊤ i F x ( ω 1 ...M )) where w i ∼ N (0 , I ) , constitutes a ( S 0 , cS 0 , p 1 , p 2 ) -ALSH for some p 1 and p 2 if

Proof Define Ω = [ -ω max , ω max ] K . For ω ∈ Ω , we have the following relationships:

We also define that ̂ sim ω max ( q, x ) = ∫ ω ∈ Ω F q ( ι ω ) ⊤ F x ( ι ω ) p ( ω ) d ω . This gives us the following result:

The last inequality is due to Lemma D.1. Next we bound Term-2 in Eq. (30).

≤ L cos πKI ∣ ∣ ∣ ∣ F q ( ω 1 ...M ) ⊤ F x ( ω 1 ...M ) /M -∫ ω ∈ Ω F q ( ι ω ) ⊤ F x ( ι ω ) p ( ω ) d ω ∣ ∣ ∣ ∣ (34) = L cos πKI ( ω max ) ∣ ∣ ∣ ∣ ∑ j ∈ [ M ] F q ( ω j ) ⊤ F x ( ω j ) /M -∫ ω ∈ Ω F q ( ι ω ) ⊤ F x ( ι ω ) p ( ω ) d ω ∣ ∣ ∣ ∣ (35) Taking expectation with respect to ω , we have:

The last inequality follows from bound on the variance due the following:

Putting Eqs. (32) (40) into Eq. (30), we will have the upper bound for sim( q, x ) ≤ cS 0 :

Similarly, we have the lower bound for sim( q, x ) > S :

Now, note that:

The last inequality (i) is due to the conditions that:

This leads us to the following relationship:

Thus, we have:

D.3 Effect of truncating frequencies

Then, if T > max x , q || x -q || ∞ , then we have:

Proof Let S ( ιω ) be the Fourier transform of s ( t ) , F -1 is the inverse Fourier transform, ⋆ is the convolution operator. Further, we define the LOWPASS ( ω | ω max ) is defined as

Let us define the filtered or frequency truncated signal of s ( t ) as defined in Eq. (4) as:

In this context, sim ω max ( q, x ) can be expressed as follows:

Equality (i) is due to the fact that the newly added imaginary integrand is an odd function (since, Im( S ( ιω )) = -Im( S ( -ιω )) and Re( S ( ιω )) = Re( S ( -ιω )) ) and thus, the integral from -ω max and ω max is zero. Equality (ii) is due to Eq. (53). Next we focus on ̂ s ω max ( t ) . Since F -1 [ S ( ιω )] =

s ( t ) and -1 [ LOWPASS ( ω ω max ) = sin ω max t

Bounding the error | ̂ s ω max ( t ) -s ( t ) | when -T < t ≤ 0 We separately bound the four integrals as follows:

Thus, from Proposition D.2, we have:

Bound on the second integral ∫ ω max t ω max ( t -T ) ∆ s ( t -τ ω max ) sin τ πτ dτ : Since τ ∈ ( ω max ( t -T ) , ω max t ] , we have 0 ≤ t -τ/ω max < T . Thus, s ( t -τ ω max ) = T -t + τ ω max . We compute the bound on the second term as:

Thus, from Proposition D.2, we have:

Bound on the third integral ∫ ω max ( t + T ) ω max t ∆ s ( t -τ ω max ) sin τ πτ dτ : When τ ∈ ( ω max t, ω max ( t + T )] , we have -T ≤ t -τ/ω max < 0 . Moreover, -T ≤ t < 0 . Thus, s ( t ) = s ( t -τ/ω max ) = T , resulting in ∆ s ( t -τ ω max ) = 0 and the third integral is zero.

Bound on the fourth integral ∫ ∞ ω max ( t + T ) ∆ s ( t -τ ω max ) sin τ πτ dτ : When τ ∈ ( ω max ( t + T ) , ∞ ) , we have -∞ < t -τ/ω max < -T . Then, s ( t -τ ω max ) = 0 . We compute the bound on the first term as:

(80)

Thus, from Proposition D.2, we have:

Bounding the error | ̂ s ω max ( t ) -s ( t ) | when 0 ≤ t < T ̂ s ω max ( t ) -s ( t ) = ∫ ω max ( t -T ) -∞ ∆ s ( t -τ ω max ) sin τ πτ dτ + ∫ ω max t ω max ( t -T ) ∆ s ( t -τ ω max ) sin τ πτ dτ (82) + ∫ ∞ ω max t ∆ s ( t -τ ω max ) sin τ πτ dτ (83)

Now, since 0 ≤ t < T , we have s ( t ) = T -t , throughout the next part of the proof.

Bound on the first integral ∫ ω max ( t -T ) -∞ ∆ s ( t -τ ω max ) sin τ πτ dτ : Since τ ∈ ( -∞ , ω max ( t -T )] , we have T ≤ t -τ/ω max < ∞ . Thus, s ( t -τ ω max ) = 0 and ∆ s ( t -τ ω max ) = s ( t -τ ω max ) -s ( t ) = 0 -( T -t ) .

Inequality (i) is due to Proposition D.2

Bound on the second integral ∫ ω max t ω max ( t -T ) ∆ s ( t -τ ω max ) sin τ πτ dτ : Since τ ∈ ( ω max ( t -T ) , ω max t ] , we have 0 ≤ t -τ/ω max < T . Thus, s t -τ ω max = T -t + τ ω max .

Bound on the fourth integral ∫ ∞ ω max ( t + T ) ∆ s ( t -τ ω max ) sin τ πτ dτ : When τ ∈ ( ω max ( t + T ) , ∞ ) , we have -∞ < t -τ/ω max < -T . Then, s ( t -τ ω max ) = 0 .

Inequality (i) is due to Proposition D.2

Inequality (i) is due to Proposition D.2 Hence,

̂ The above result, together with sim( q, x ) = ∑ k ∈ [ K ] s ( q [ k ] -x [ k ]) and ̂ sim ω max ( q, x ) = ∑ k ∈ [ K ] ̂ s ω max ( q [ k ] -x [ k ]) prove the Lemma. Q.E.D. Proposition D.2. If α and β have the same sign and α < β , then we have:

Proof Note that d dt cos t t = -sin t t -cos t t 2 . Then we have:

This gives us the following inequality:

E Additional details about the experimental setup

E.1 Dataset Generation

We obtain the MSWEB 1 and MSNBC 2 datasets from the UCI Machine Learning repository. Both of the datasets contain anonymized logs of real world user web activity. Each data item in MSWEB is a set of text snippets denoting areas of the website www.microsoft.com visited by an user within a specified time frame. Similarly, MSWEB consists of multi-sets denoting user page requests under various news categories at www.msnbc.com . Overall, we regard each data set as a collection of items, each item being a bag of words. For each data set, we sample a subset of items and designate them as queries, and the remaining items are designated as corpus items. The (binary) query-corpus relevance for MSWEB-1 and MSWEB-2 are governed by set containment, while for MSNBC-1 and MSNBC-2 we use multi-set (bag) containment. I.e., x is relevant for q iff q ⊆ x . To test the ability of FOURIERHASHNET to retrieve semantically similar items close to the gold items, we report not only on MAP based on gold labels but also scores of the top-10 candidates (Figure 3). The dataset characteristics are summarized in Table 7. We create datasets which differ greatly in terms of corpus size (10734 for MSWEB-1 and MSWEB, 110790 for MSNBC-1 and MSNBC-2), as well as span a range of average query selectivity between 1 . 7 × 10 -4 and 3 . 3 × 10 -3 . We set aside 100 query graphs each for training and validation, and use the remaining 300 for testing.

E.2 Learning Representations for q and x for the baselines

During experiments for Section 4.2, in each of the baseline method (Cosine similarity, Dot product and Weighted Jaccard), we use the respective similarity scoring functions, and minimize a pairwise ranking loss on the gold relevance labels to learn the deep set network. We observed that the pairwise ranking loss performs better than the BCE loss for the baselines. The margin enabled pairwise ranking loss is specified as

where sim is the choice of similarity scoring baseline, X q ✓ , X q ✗ are the set of relevant and irrelevant corpus items for the query q . We use the best performing margin among { 1 , 0 . 1 } .

E.3 Sampling from arbitrary distribution

One key component of FOURIERHASHNET is sampling ω 1 ...M ∼ p ( ω ) . We have chosen to set p ( ω ) ∝ | Re( S ( ιω )) | + | Im( S ( ιω )) | , with the support set between p ( ω ) between [ -100 , 100] . The samples are drawn using Inverse Transform Sampling.

E.4 Details about fourier transformation network

In our experiments, we generate M = 10 samples for ω . The neural networks ϕ q and ϕ x are linear layers which output 10 dimensional transformed Fourier representations z q = ϕ q ( F q ( ι ω 1 ..M )) and z x = ϕ x ( F x ( ι ω 1 ..M )) . These are trained using the BCE loss specified in Eq. (12).

E.5 Details about hashcode generation network

We use the same hashcode training procedure for FOURIERHASHNET, as well as the DP-RH and RH baselines. In all three cases, we generate 64 dimensional hashcodes. For FOURIERHASHNET, the random hyperplanes W are trained on 10 dimensional trained Fourier representations z q , z x . For RH we use the original embeddings q and x . For DP-RH, we use the augmented embeddings g ( q ) = sign( W [0 , q / || q || ]) and h ( x ) = sign( W [ √ T 2 -|| x || 2 , x ]) .

E.6 AP and MAP measurements

Suppose a query q is associated with N q ⊕ relevant corpus items (as judged by humans). Suppose the system provides a ranking over all N corpus items, and the relevant items occur at ranks r 1 , . . . , r N q ⊕ . Then AP for query q is defined as (1 /N q ⊕ ) ∑ N q ⊕ i =1 ( i/r i ) . This is because, up to position r i , we have seen i relevant items, which means we can shorthand i/r i as prec @ i (precision at i ). We can rewrite the sum as 1 N q ⊕ ∑ N r =1 prec @ r × rel@ r , where N is the size of the whole corpus, and rel@ r is the (0/1) relevance of the item at rank r . In case the retrieval algorithm does not assess all N corpus items, but stops with the best L hash buckets, which contain, say, N ′ items, we should use the following formula for AP: 1 N q ⊕ ∑ N ′ q r =1 prec @ r × rel@ r . Note that we should still divide by N q ⊕ , otherwise an algorithm that maps the query to a densely relevant but small bin, which fails to retrieve most relevant items, might be rewarded in an unfair manner.

E.7 Top-10 score measurement

In Appendix F, we provide additional experiments where we compare FOURIERHASHNET with all the baselines not only in terms of MAP, but also in terms of the Top-10 score. We use the sum of Top-10 scores normalized in [0 , 1] via the sigmoid transformation used in Eq. (15): Top-10 ( q ) = ∑ x ∈ Top-10 ( N ′ q ) σ ( -d ( q, x )) . Any hashing protocol is expected to retrieve the corpus items, which have the highest similarity scores with respect to any given query. The Top-10 score evaluates it independently of how the retrieved items match with true relevant items. Therefore, the Top-10 scores provide an evaluation mechanism that is independent of the gold relevance labels and solely relies on the scores dictated by the trained embeddings. This offers a valuable means of assessing performance without being influenced by subjective human judgments.

E.8 Licenses

We utilize a publicly available pre-trained sentence transformer model [41], which is licensed under the Apache License 2.0. Additionally, we employ the DrHash toolkit [58] for various implementations of the baseline Weighted Minhash (WMH) algorithms. The DrHash toolkit is publicly available under the MIT License. We duly acknowledge the original authors of the baseline methods in our citations.

q

F Additional experiments

F.1 Applying baseline LSHs on hinge distance guided embeddings

In continuation of the results reported in Figure 3, in Figure 8, we present the complete view as well as the zoomed versions of the trade-off curves for all datasets.

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(a) MSWEB-1

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(b) MSWEB-2

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(c) MSNBC-1

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

2

4

6

8

10

Top-10 Score

→

(d) MSNBC-2

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(a) MSWEB-1

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(b) MSNBC-1

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(c) MSWEB-2

0

.

0

0

.

1

0

.

2

Time (sec)

→

2

4

6

8

10

Top-10 Score

→

(d) MSNBC-2

Figure 8: Trade-off between average query time and accuracy (MAP and Top-10 scores) for MSWEB and MSNBC datasets (First two rows: complete view across full time axis, last two rows: Zoomed version of first two rows until the average query time there is ≥ 10X speedup compared to exhaustive search). We apply different LSH methods on hinge distance guided embeddings similar to Figure 3, then use the hinge distance to finally rank the retrieved items.

Beyond the observations made in Figure 3, we make the following additional observations. (1) The complete view for both Top-10 score and MAP score, clearly demonstrates FLORA's high sensitivity to hyperparameter tuning. FLORA is seen to have the highest variance in scores for any given time budget across all the baselines. In terms of Top-10 score, while FLORA is marginally ahead of FOURIERHASHNET in a few instances in the MSWEB datasets, it is significatly outperformed by FOURIERHASHNET in the MSNBC datasets. This may be due to the significantly higher average query selectivity in the MSNBC datasets.

(2) In terms of Top-10 score, FOURIERHASHNET achieves the maximum possible value 4 × faster than the nearest competitor RH+Hinge, in MSWEB-1 and MSWEB-2. In terms of MAP score,

RH+Hinge and WMH+Hinge achieve a maximum MAP of 0.5 in MSWEB-1 and MSWEB-2, and 0.62 in MSNBC-1 and MSNBC-2. However, FOURIERHASHNET achieves the same MAP values 1.33 × faster in MSWEB-1 and MSWEB-2, and 2 × faster in MSNBC-1 and MSNBC-2. (3) Interestingly, the gap between FOURIERHASHNET and RH+Hinge seems to widen for MSWEB-2, when compared to MSWEB-1. This is possibly due to the presence of several queries in MSWEB-1, which have ≥ 300 relevant corpus items. This affords RH+Hinge a greater opportunity to fetch high scoring items, which is not the case in MSWEB-2.

F.2 Ablation study on hashcode training

In continuation of the results reported in Figure 5, in Figure 9, we present the complete view as well as the zoomed versions of the trade-off curves for all datasets.

0

.

00 0

.

05 0

.

10 0

.

15 0

.

20

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

00 0

.

05 0

.

10 0

.

15 0

.

20

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

00 0

.

05 0

.

10 0

.

15 0

.

20

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(a) MSWEB-1

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(b) MSWEB-2

0

.

00 0

.

05 0

.

10 0

.

15 0

.

20

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(c) MSNBC-1

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

2

4

6

8

10

Top-10 Score

→

(d) MSNBC-2

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(a) MSWEB-1

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(b) MSNBC-1

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(c) MSWEB-2

0

.

0

0

.

1

0

.

2

Time (sec)

→

2

4

6

8

10

Top-10 Score

→

(d) MSNBC-2

Beyond the observations made in Figure 5, we make the following additional observations. (1) In terms of both Top-10 score and MAP score, FHASH (UNTRAINED) clearly outperforms both RH+Hinge and DP-RH+Hinge, across all four datasets. This strongly highlights the advantage of our

Fourier feature generation method.

(2) In every setup, FOURIERHASHNET enables a wider range of options for accuracy score vs retrieval time trade-off.

F.3 Ablation study on collision minimizer (14)

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

Collision Minimizer

Decorrelation

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

Collision Minimizer

Decorrelation

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

Collision Minimizer

Decorrelation

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

MAP

→

Collision Minimizer

Decorrelation

0

.

00 0

.

05 0

.

10 0

.

15 0

.

20

Time (sec)

→

7

.

5

8

.

0

8

.

5

9

.

0

9

.

5

Top-10 Score

→

Collision Minimizer

Decorrelation

(a) MSWEB-1

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

8

.

0

8

.

5

9

.

0

9

.

5

Top-10 Score

→

Collision Minimizer

Decorrelation

(b) MSNBC-1

0

.

00 0

.

05 0

.

10 0

.

15 0

.

20

Time (sec)

→

5

6

7

8

9

10

Top-10 Score

→

Collision Minimizer

Decorrelation

(c) MSWEB-2

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

8

.

25

8

.

50

8

.

75

9

.

00

9

.

25

9

.

50

9

.

75

Top-10 Score

→

Collision Minimizer

Decorrelation

(d) MSNBC-2

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

Collision Minimizer

Decorrelation

0

.

000

.

050

.

100

.

150

.

200

.

25

Time (sec)

→

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

Collision Minimizer

Decorrelation

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

Collision Minimizer

Decorrelation

0

.

000

.

050

.

100

.

150

.

200

.

25

Time (sec)

→

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

MAP

→

Collision Minimizer

Decorrelation

0

.

00

0

.

01

0

.

02

Time (sec)

→

7

.

5

8

.

0

8

.

5

9

.

0

9

.

5

Top-10 Score

→

Collision Minimizer

Decorrelation

(a) MSWEB-1

0

.

000

.

050

.

100

.

150

.

200

.

25

Time (sec)

→

8

.

0

8

.

5

9

.

0

9

.

5

Top-10 Score

→

Collision Minimizer

Decorrelation

(b) MSNBC-1

0

.

00

0

.

01

0

.

02

Time (sec)

→

5

6

7

8

9

10

Top-10 Score

→

Collision Minimizer

Decorrelation

(c) MSWEB-2

0

.

000

.

050

.

100

.

150

.

200

.

25

Time (sec)

→

8

.

25

8

.

50

8

.

75

9

.

00

9

.

25

9

.

50

9

.

75

Top-10 Score

→

Collision Minimizer

Decorrelation

(d) MSNBC-2

̸

In continuation of the results reported in Figure 6, we present the complete view as well as the zoomed versions of the trade-off curves for all datasets. Beyond the observations made in Figure 6, we make the following additional observations.

(1) In MSWEB-2, the alternative variant shows a sudden plunge in MAP performance trade-off at around 0.1 seconds. This type of discontinuous drop is not observed in any of our cases.

(2) In MSWEB-1, there is a variation of 0.1 MAP at around 0.05 seconds. Such high variability is not observed for any of our trade-off curves.

F.4 Identifying best performing Weighted Minhash algorithm for our datasets

For implementation of baseline Weighted Minhash (WMH) algorithm, we use the best performing WMHimplementation available in the DrHash toolkit [58] to obtain the hashcodes. We comapare

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

chum

licws

pcws

icws

ccws

i2cws

gollapudi2

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

chum

licws

pcws

icws

ccws

i2cws

gollapudi2

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

chum

licws

pcws

icws

ccws

i2cws

gollapudi2

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

chum

licws

pcws

icws

ccws

i2cws

gollapudi2

(a) MSWEB-1

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

chum

licws

pcws

icws

ccws

i2cws

gollapudi2

(b) MSNBC-1

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

chum

licws

pcws

icws

ccws

i2cws

gollapudi2

(c) MSWEB-2

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

chum

licws

pcws

icws

ccws

i2cws

gollapudi2

(d) MSNBC-2

across the 8 available baselines in the toolkit: minhash [7], chum [12], icws [20], pcws [56], licws [28], ccws [55], i2cws [57] and gollapudi2 [16].

We make the following observations.

(1) Across all four of our datasets, for both MAP and Top-10 score, the top 3 performers are minhash, gollapudi2 and chum. The remaining algorithms are often significantly worse in performance, as can be seen for MAP in all 4 datasets and for Top-10 in MSNBC-2.

(2) Among the top 3 performers, chum is seen to be the best perform in terms for both MAP and Top-10 score, in MSWEB-2, MSNBC-1 and MSNBC-2. In MSWEB-1, chum is tied with minhash and gollapudi2 for the top position.

Driven by these observations, we choose chum as the representative baseline for WMH in our experiments.

F.5 Effect of M , number of samples of ω on FOURIERHASHNET performance

(a) MSWEB-1

(b) MSWEB-1

(a) MSWEB-1

(b) MSWEB-2

10

0.8

0.8

↑

Score

Score

6

↑0.6

↑0.6

8

IAP

AP

8

M

0.4

M

0.4

Top-10

Top-10

7

FourierHashNet(10)

FourierHashNet(10)

FourierHashNet(10)

6

FourierHashNet(10)

0.2

FourierHashNet(50)

0.2

FourierHashNet(50)

FourierHashNet(50)

FourierHashNet(50)

FourierHashNet(100)

FourierHashNet(100)

FourierHashNet(100)

5

FourierHashNet(100)

6

0.00

Time

0.05

(sec）→

0.10

0.15

0.20

0.00

Time(

0.05

(sec）→

0.10

0.15

0.20

0.00

Time (sec）→

0.05

0.10

0.15

0.20

0.00

Time

0.05

(sec）→→

0.10

0.15

0.20

Here we check the impact of varying the number of samples ( M ) for ω . We consider three different values of M , i.e. , 10, 50 and 100, for generating the fourier features F q ( ι ω 1 ..M ) and F x ( ι ω 1 ..M ) which are then fed into the neural networks ϕ q and ϕ x for learning the transformed Fourier representations z q and z x , using the BCE loss specified in Eq. (12). Finally, we train the random hyperplanes W and check the MAP and Top-10 score performances for the three variations - FOURIERHASHNET(10), FOURIERHASHNET(50) and FOURIERHASHNET(100). We observe that the final performance tradeoff of both MAP and Top-10 scores, remains roughly the same across all three variants. This shows that trainable Fourier transformation is able to compensate for the quality of Monte Carlo approximations affected by the number of ω samples M .

Next, we investigate how well the MC estimates of the Fourier features approximate the dominance similarity function sim( q, x ) . Here, we set the dimension of q and x as K = 1 . We set T = 20 and we sample q , x ∼ Unif [ -20 , 20] . Finally, we compute ̂ sim M ( q, x ) = || F · ( ι ω 1 ..M ) || 2 M cos( F q ( ι ω 1 ..M ) , F x ( ι ω 1 ..M )) and measure the variation of ϵ sim = E q , x ∼ Unif [ -20 , 20] [ || sim( q, x ) -̂ sim M ( q, x ) || ] with M , the number of samples of ω . Figure 13 summarizes the results, which shows as M increases, ϵ sim decreases.

0

200

400

600

800

1000

#  samples (M)

0

5

10

15

20

25

30

35

sim

F.6 Applying baseline LSHs on hinge distance guided embeddings with noisy labels

In certain applications, the accuracy of ground truth labels can be compromised by noise or subjective human judgments of relevance. We evaluate the performance of FOURIERHASHNET and the baselines in a noisy label setup to test its robustness.

Starting with the hinge distance guided embeddings, we initially rank the corpus items based on their dominance similarity scores. Subsequently, we intentionally flip the labels of the bottom-ranking 10% of positive labels to negative, and an equal number of highest ranked negatively labeled items to positive. This simulation reflects a plausible scenario since the lowest ranked positive items and the highest ranked negative items are particularly susceptible to misclassification in real-world settings. Furthermore, this approach ensures that the average query selectivity for each dataset remains unchanged.

As before, we apply different the LSH methods on hinge distance guided embeddings, viz , RH+Hinge, DP-RH+Hinge, WMH+Hinge, FLORA and FOURIERHASHNET; and, then use the hinge distance to finally rank the retrieved items.

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

MAP

→

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(a) MSWEB-1

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(b) MSWEB-2

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(c) MSNBC-1

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Time (sec)

→

2

4

6

8

10

Top-10 Score

→

(d) MSNBC-2

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

.

0

0

.

2

0

.

4

0

.

6

MAP

→

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(a) MSWEB-1

0

.

0

0

.

1

0

.

2

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(b) MSNBC-1

0

.

00

0

.

01

0

.

02

Time (sec)

→

0

2

4

6

8

10

Top-10 Score

→

(c) MSWEB-2

0

.

0

0

.

1

0

.

2

Time (sec)

→

2

4

6

8

10

Top-10 Score

→

(d) MSNBC-2

We make the following observations.

(1) In terms of MAP score, FOURIERHASHNET continues to outperform all other baselines across all four datasets. Furthermore, when comparing the performance in the presence of noise, as depicted in Figure 14, to the corresponding results obtained in the noiseless setting illustrated in Figure 8, we observe that FOURIERHASHNET outperforms all its competitors by a significantly higher margin in the presence of noise.

(2) In terms of Top-10 score, we note that the results in Figure 14 for the noisy setup are identical to the results presented in the noiseless setting shown in Figure 8. This observation indicates that the evaluation based on Top-10 score is unaffected by label noise. This supports the argument made in Appendix E.7 that Top-10 score evaluation enables a more subjective assessment of performance, focusing on the quality of the embeddings themselves.

F.7 Comparison with graph based indexing methods

Here, we compare the performance of FOURIERHASHNET with a representative graph based indexing method. We use the Hierarchical Navigable Small Worlds (HNSW) implementation of Hnswlib [33]. We extend the SpaceInterface class of nmslib/Hnswlib to implement HNSW for hinge distance. In order to track the number of distance computations performed by HNSW during retrieval, we used a counter inside fstdistfunc_ . We count the number of distance computations as a surrogate for real time, to avoid non-determinism in measurements and low-level implementation differences. We search across different values of M, ef and ef_construction, and track the number of distance computations against corresponding MAP values.

Table presents our study on MSWEB-1dataset with 10734 corpus items. We observe that FOURIERHASHNET LSH has an edge over HNSW in the regime of fewer distance computations, with a MAP of 0.53 using 1162 distance computations. However, when allowed more distance computations, HNSW outperforms FOURIERHASHNET with a MAP of 0.71 in 3528 computations, and a MAP of 0.77 in 4773 computations.

Peer reviews:
comments: SUMMARY:
This paper presents a novel approach, called FourierHashNet, for fast soft set containment search. The key idea is to extend set containment to soft set containment by representing query and document elements as embedded representations, instead of atomic IDs. The authors propose a dominance similarity measure based on hinge distance and transform it into the frequency domain using a Fourier transform. This allows for the efficient use of traditional LSH techniques.

STRENGTHS:
Strength:

1. The significance of the proposed asymmetric dominance similarity measure is emphasized, indicating its critical role in the targeted applications. By transforming the dominance similarity measure into the frequency domain, the authors enable the utilization of traditional LSH methods. This approach not only enhances retrieval efficiency but also demonstrates a better trade-off between query time and retrieval quality.

2. This paper is well-written. It effectively communicates the technical aspects of the proposed method, including the use of the Fourier transform and the learning of data-sensitive hash codes.

WEAKNESSES:
NA

QUESTIONS:
Is it possible to discuss the potential of FourierHashNet in the similarity search of general kernel measure?

LIMITATIONS:
Yes, the authors adequately addressed the limitations
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nLocality-sensitive hash (LSH) functions are mappings from sets of queries and documents to \"buckets,\" in such a way that similar queries and documents are assigned to the same bucket. This paper proposes an asymmetric LSH where the notion of similarity is related to the hinge distance (called dominance similarity, based on the element-wise inequalities between components of two vectors). The method conceptually follows three steps: truncate, transform, and sample. First, the dominance similarity is truncated to a bounded region of the input domain; this is done so that the problem is tractable. Then, the similarity is Fourier-transformed into a form where the contributions from different vectors separate into an inner product. This form is used to sample Fourier representations that are used to compute the LSH in practice.\n\nThe paper also contains experiments comparing the proposed ALSH with other LSH algorithms on two web log datasets, where the task is to perform vector search under the hinge distance. FourierHashNet outperforms these methods by a sizable margin. To further improve practical performance, the paper proposes a learnable version of the algorithm and conducts ablations to investigate its effect.\n\n**Post rebuttal:** The authors provide a general framework for ALSH, of which the hinge distance LSH is a specific instance. This substantially extends the utility and impact of the method over the original proposal.\n\nSTRENGTHS:\n**S1: Important topic.** The problem of designing an asymmetric LSH with an exotic collision probability is useful and interesting. There is not much work in this area, but there are several emerging LSH applications (not just search) where this capability will be helpful. For example, the [LGD sampler [NeurIPS 2019]](https://proceedings.neurips.cc/paper_files/paper/2019/file/a1e865a9b1065392ed6035d8ccd072d9-Paper.pdf) uses LSH functions that are sensitive to classification loss functions. Some of these are non-symmetric, so new ways to construct asymmetric LSH can correspond to faster optimization routines for more problems. \nThe efficient kernel-matrix multiplication algorithm of Backurs et. al. [[ICML 2021]](https://arxiv.org/abs/2102.08341) is based on LSH and currently only works for symmetric kernels - asymmetric LSH could substantially extend this framework.\nAn asymmetric LSH was recently developed for the linear regression/classification loss and was used to learn differentially-private classifiers [[CCS 2021]](https://dl.acm.org/doi/abs/10.1145/3460120.3485255); new ALSH functions will also directly expand the usefulness of this framework.\nTherefore, ALSH is an increasingly important topic with the potential to impact search, ranking, optimization, numerical linear algebra, differential privacy, and likely other areas.\n\n**S1: Novelty.** This paper attacks a technically difficult problem (ALSH for the hinge distance) and introduces some new ALSH techniques to do the design. I checked the math and the results seem to be correct. Note that I did not check details (e.g. the algebra involved in going from (7) to (8), or the derivation of (5) in the appendix).\n\nWEAKNESSES:\n**W1: Presentation.** The presentation and flow of the argument can be improved. Here are some specific examples:\n- The \"i\" used to denote sqrt(-1) should probably be the standard italic i (with a dot) rather than the symbol \\i. \n- It may be more correct to refer to the \"hinge distance\" as the \"hinge quasimetric\" or the \"hinge divergence\" because it is not strictly speaking a metric (it obeys all properties except symmetry).\n- On line 177, consider mentioning that Equation (3) is just the Fourier Transform of -d(q,x) from Equation (1). It took me a moment to determine why (3) was written as the sum rather than as K iterated integrals.\n- Equation (4) could be introduced as a \"clamped\" or \"truncated\" version of -d(q,x), to make its definition more clear to the reader.\n- The \"Fence Sitting\" and \"bit balance\" losses don't seem particularly well-known (the only reference I could find was in \"Adversarial Permutation Guided Node Representations for Link Prediction\" in [AAAI 2021](https://arxiv.org/abs/2012.08974) - this paper should probably be cited). Therefore, these losses could possibly use some more explanation.\n- The results figures for the experiments would benefit from better layout to make the graphics larger (e.g. using subplots with tight_layout or similar).\n- Several components seem out of place. For example, the formal definitions of LSH and ALSH are introduced but are not used to prove that the hash function is indeed an ALSH. Observations about symmetric LSH (e.g. the paper by Chierichetti and Kumar) are mentioned throughout the development of the asymmetric LSH in Section 3, and it might be clearer to collect all these arguments in Section 2. The sampling algorithm is fairly involved and probably merits its own section. \n\n**W2: Impact.** The main contribution of the paper is an asymmetric LSH function for a very specific similarity function. This paper could have an impact on the LSH algorithm area and on any area that uses the hinge distance for set search. However, I have some concerns about the applicability of the ideas outside of what seems to be a fairly niche search problem.\n\n- **On LSH.** As mentioned above, there are several problems that can benefit from asymmetric LSH. However, it is hard to see which contributions from this paper might be more broadly applicable. It would be much more interesting (and appeal to a wider audience) if this algorithm can be generalized into a broader \"truncate - transform - sample\" template. Such a template might apply directly to other applications. It is also fairly essential to demonstrate that this is, indeed, an ALSH (that provably satisfies Definition 2.2).\n- **On set-search.** The community seems to be moving away from representing sets as a single vector and toward approaches that examine the pairwise similarity between all elements in the set (e.g. [ColBERT](https://arxiv.org/pdf/2004.12832.pdf), [ColBERT v2](https://arxiv.org/abs/2112.01488), [PLAID](https://arxiv.org/abs/2205.09707), several follow-ups). This approach was originally motivated by the comparatively high cost of performing transformer inference but is now [doing very well](https://arxiv.org/abs/2212.01340) on tasks like passage ranking. Many of the papers cited as applications are more than five years old (pre-2019), so it is not clear whether search with the hinge distance is still a pressing problem.\n\n**W3: Experiments.** The baselines in the experiments are not very competitive, leading to a limited evaluation. LSH is a classical, standard technique but it is no longer state-of-the-art for most search/ranking problems. Other approaches (like FAISS-IVF or HNSW) have taken over as the search algorithm of choice. These methods are also compatible with the hinge distance (though, this does require use of the C++ APIs rather than the simpler Python API). They also tend to be an order of magnitude faster than LSH, according to well-established benchmarks ([ann-benchmarks](https://ann-benchmarks.com), see also the NeurIPS 21 competition [big-ann-benchmarks](https://big-ann-benchmarks.com)).\n\nQUESTIONS:\nI like this line of research and would be willing to raise my score if some of the following items can be addressed. In particular, I'm looking for evidence that the hinge distance is an important bottleneck and for evidence that the new techniques developed in this paper could be useful more broadly.\n\n- Is it possible to generalize the sampling process (bottom of page 5) into a generic algorithm with theoretical guarantees (e.g. on the expectation and possibly variance of the estimate)? This would go a long way towards addressing W2, as it would be a result that other works could build off of.\n\n- Is element-wise vector inequality the current state-of-the-art for the applications mentioned in the paper? ColBERT (which examines all pairwise relationships between the two sets) and follow-ups have shown very good results in natural language applications, and determinental point processes are SOTA in many market basket analysis applications (e.g. \"Learning Nonsymmetric Determinantal Point Processes\" at [NeurIPS 2019](https://proceedings.neurips.cc/paper_files/paper/2019/file/cae82d4350cc23aca7fc9ae38dab38ab-Paper.pdf)). I am less familiar with the knowledge graphs and subgraph isomorphism applications mentioned in the appendix so it might be possible that the problem is very strongly motivated by these areas. I did take a quick look at the references but I did not find this search problem mentioned.\n\n- Can this algorithm handle box embeddings (perhaps with some modifications to the algorithm / the embeddings)? Search over box embeddings is a known bottleneck and is one reason why, despite modeling advantages, they have not replaced angular-similarity embeddings in major industrial recommendation pipelines.\n\n- Is it possible to prove that this scheme is an ALSH (i.e. satisfies Defintion 2.2) for the hinge distance? Due to the truncation in Equation (4), it might be necessary to restrict the domain over which we expect Definition 2.2 to hold (but this would be fine and would not affect the quality of the theoretical result).\n\n- Just to clarify - what are the \"gold\" and \"silver\" instances on page 6? Is this the same as the gold relevance labels from the datasets (page 7)?\n\nLIMITATIONS:\nYes - no issues.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper studies a new search problem called vector dominance (or set containment), where the authors provide strong motivation from various real-world applications. They present a new approach named FourierHashNet along with a fresh asymmetric vector dominance distance to address this problem. Through extensive experimentation, the study confirms the effectiveness of the proposed distance measure and demonstrates the superior performance of FourierHashNet compared to established LSH and ALSH baselines.

STRENGTHS:
- The authors study a new yet significant problem, which contains many well-motivated applications regarding text, image, and graph retrieval.

- They propose an effective distance measure customized for the problem they looked at.

- The FourierHashNet is interesting, and it is reasonable to combine learning to hash together with Fourier features.

- They conduct extensive experiments to confirm the effectiveness of FourierHashNet.

WEAKNESSES:
Overall, in my opinion, this is a well-written paper in various aspects.

QUESTIONS:
Please further proofread and polish the paper.

LIMITATIONS:
This work does not appear to have any negative social impact.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07