Title: An Efficient and Robust Framework for Approximate Nearest Neighbor Search with Attribute Constraint
Venue: NeurIPS 2023
Accepted: yes
Keywords: approximate nearest neighbor search

attribute filtering

high-dimensional vector

proximity graph
Abstract: This paper introduces an efficient and robust framework for hybrid query (HQ) processing, which combines approximate nearest neighbor search (ANNS) with attribute constraint. HQ aims to find objects that are similar to a feature vector and match some structured attributes. Existing methods handle ANNS and attribute filtering separately, leading to inefficiency and inaccuracy. Our framework, called native hybrid query (NHQ), builds a composite index based on proximity graph (PG) and applies joint pruning for HQ. We can easily adapt existing PGs to this framework for efficient HQ processing. We also propose two new navigable PGs (NPGs) with optimized edge selection and routing, which improve the overall ANNS performance. We implement five HQ methods based on the proposed NPGs and existing PGs in NHQ, and show that they outperform the state-of-the-art methods on 10 real-world datasets (up to 315$\times$ faster with the same accuracy).

An Efficient and Robust Framework for Approximate Nearest Neighbor Search with Attribute Constraint

Abstract

This paper introduces an efficient and robust framework for hybrid query (HQ) processing, which combines approximate nearest neighbor search (ANNS) with attribute constraint. HQ aims to find objects that are similar to a feature vector and match some structured attributes. Existing methods handle ANNS and attribute filtering separately, leading to inefficiency and inaccuracy. Our framework, called native hybrid query (NHQ), builds a composite index based on proximity graph (PG) and applies joint pruning for HQ. We can easily adapt existing PGs to this framework for efficient HQ processing. We also propose two new navigable PGs (NPGs) with optimized edge selection and routing, which improve the overall ANNS performance. We implement five HQ methods based on the proposed NPGs and existing PGs in NHQ, and show that they outperform the state-of-the-art methods on 10 real-world datasets (up to 315 × faster with the same accuracy).

1 Introduction

Approximate nearest neighbor search (ANNS) is a crucial problem in data science and AI applications [27, 43, 9, 47]. For example, in a paper-retrieval system based on ANNS, a user wants to find papers that are most similar to her query text. As Fig. 1(a) shows, the system converts each paper's unstructured text and the query text into feature vectors in a high-dimensional space. Then it uses a vector index to perform ANNS and obtain papers with similar content. Many effective ANNS methods have been developed to balance query efficiency and accuracy [18, 19, 36, 40, 11].

However, ANNS does not support many real-world scenarios where users want to find objects with similar feature vectors to the query object and meets the given attribute constraint (e.g., topic , venue , and publish_year of a paper) [51, 41]. We call this a hybrid query (HQ) [51, 54, 60, 32, 50, 41, 42]. For instance, in Fig. 1(b), a user wants to find some recent papers from top-tier conferences related to her research interest. She can form a HQ by providing a descriptive text of her interest and two paper attributes (e.g., NeurIPS and 2022 ). However, the traditional ANNS shown in Fig. 1(a) only finds papers with similar content, and it cannot guarantee matching attribute constraint. Therefore, many have tried to add attribute filtering (AF) on top of ANNS to answer a HQ [54, 51, 60].

Vearch [29] uses ANNS to find candidates that match the feature vector, and then applies AF to get the final results [32]. This strategy can work with other ANNS libraries, such as SPTAG [37], NGT [59], and Faiss [17]. On the other hand, Alibaba AnalyticDB-V (ADBV) [54] employs product

Xiaoliang Xu ∗

I prefer NeurIPS 2022

NeurIPS 2022

User

Input

Unstructured data

Structured attributes

Input

Publication

NeurIPS

Year

2022

An image retrieval processing that support attribute filtering

beyond vector similarity search, it is to only search vectors that

satisfy a given filtering condition, e.g., finding the T-shirts similar

to a given image vector that also cost less than $100.

Embedding

CVPR 2021

Vector Index

Index construction

Search

Papers

Unstructured text

Embedding

Feature vectors

Feature vector

Embedding

Query text

Results

Query text

(b) Query input

Feature vector

Attribute

constraints

Attribute

constraints

Merging

Results

(c) Existing HQ processing

Feature vector

+

Attribute constraints

Feature vector

+

Attribute constraints

Composite Index

Results

NeurIPS 2022

(d) Our HQ processing

Candidates

Candidates

Limited efficiency and accuracy

High efficiency and accuracy

'

'

Attribute

Index

Vector

Index

quantization (PQ)[28] to create multiple query plans. One of them is to do AF first and then ANNS to return the final results. This method is scalable and suitable for most use cases [51]. Milvus [38] partitions the dataset into subsets based on common attributes. This way, it can quickly select the subsets that meet the query's attribute constraints, and only perform ANNS on them instead of the whole dataset. This improves the search efficiency [51].

These solutions have a common idea: they do AF and ANNS separately in different orders (see Appendix A for details). No matter what order they use, the whole query pipeline can be simplified as Fig. 1(c), where two subquery systems (for AF and ANNS) are built independently. They can answer a HQ, but they are not designed for it, which affects their efficiency and effectiveness.

1.1 Limitations of Existing Methods

L1: Two indexes need to be kept at the same time. Fig. 1(c) shows that existing methods need both the attribute index and vector index for two separate subquery systems. This takes more memory space and adds more logic (e.g., merging ) to make sure two indexes are consistent and query results are correct [54]. Updating different indexes at the same time is hard when the data changes often.

L2: Extra computational cost from two separate pruning strategies. For existing methods, pruning is done based on attribute index and vector index separately, which increases computational cost. For example, suppose that the vector index in Fig. 1(c) is a proximity graph (PG, we define it in Def. 4 ) [52], where each vertex is an object's feature vector. Then, some vertices near the query object's feature vector but not matching the attributes can make the search follow wrong paths and return wrong answers to a HQ. We prune these wrong answers in the attribute-filtering phase, but the computational cost for returning them in the ANNS phase is unnecessary and inefficient.

L3: Query results rely on the merging of the candidates from both subquery systems. As Fig. 1(c) shows, we need to merge the topk candidates C 1 and C 2 from both subquery systems to get the final results R = C 1 ∩ C 2 . But usually we have | R |≪ k . This is because existing methods do ANNS and AF separately, so each system can only return candidates that meet one type of constraint. To make | R | bigger, we need to make C 1 and C 2 bigger, which makes both systems slower [60].

L4: Existing methods are not friendly for PGs. Recent methods, like ADBV [54] and Milvus [51], use a 'first AF, then ANNS' strategy to answer a HQ; they only use PQ [28] for ANNS. But many studies have shown PG is 10 × faster than PQ for ANNS [5, 33], so we need to find a way to use PG in HQ. A simple way is to replace PQ-based vector index with PG-based one, but it has some problems: (1) In the 'first AF, then ANNS' strategy, we want to find similar vectors from a dynamic space of the vectors filtered by attributes. It is not efficient to build a vector index for these filtered vectors in runtime. (2) Even worse, if we list all possible combinations of attribute values and prebuild a vector index for each one offline (like [56]), it would take too much memory space, e.g., m n different indexes for n attributes where each attribute has m values.

To sum up, most of these limitations happen because existing methods answer HQ in a 'decompositionassembly' model, where a HQ is split into two subqueries that process separately and then combine the results. This motivates us to present a new general framework for HQ that works well with existing PGs. It has the well-designed composite index and joint pruning modules to support ANNS with attribute constraint, instead of keeping two separate indexes and doing pruning separately.

1.2 Our Solution and Contributions

To the best of our knowledge, we are the first to present a N ative H ybrid Q uery framework (NHQ) for ANNS with attribute constraint in a fused way, not the 'decomposition-assembly' model used by

existing methods. As Fig. 1(d) shows, our framework first combines feature vectors and attributes in a well-designed composite index (for L1 ); then it prunes the search space on the composite index by considering both the feature vector and attribute constraints (for L2 ); and finally, it gets the final topk results directly without a merging operation (for L3 ). Our NHQ is a general framework, where the composite index is based on PG, so existing PGs can easily work with NHQ (for L4 ) and it also supports custom-optimized PGs. Our main contributions are:

We introduce NHQ, a general framework for ANNS with attribute constraint ( §3 ). NHQ works well with existing PGs, and can make them handle HQ only by a lightweight modification.

We design two navigable PGs (NPGs) with optimized edge selection and routing strategies ( §4 ), which have better efficiency-vs-accuracy trade-off than basal PGs.

We use the proposed NPGs and existing PGs in our NHQ framework, and get several new HQ methods that beat the state-of-the-art competitors on all datasets (up to 315 × faster, §5.3 ).

We show our methods' better effectiveness, efficiency, and scalability on 10 real-world datasets, compared with state-of-the-art HQ frameworks and methods ( §5 ).

2 Preliminaries

Definition 1 Object Set. An object set is defined as a set S = { e 0 , . . . , e n -1 } of size n . For each object e ∈ S , (1) its features are represented as a high-dimensional vector, denoted by ν ( e ) , (2) e has a set of attributes denoted by { a 0 , . . . , a m -1 } and e.a i indicates the value of attribute a i of e . Moreover, we define the feature vector set of all objects in S as X = { ν ( e ) | e ∈ S} .

Example 1 An object set could refer to different types of data, e.g., images, papers. When we specify each object e as a paper, it carries two types of information: one is the implicit semantics behind the text, which is usually represented as a feature vector ν ( e ) by neural network, e.g., BERT [13]; and another is the explicit attributes, e.g., { venue , topic } , such as e. venue = ' NeurIPS '.

Given an object set S and a user-input query object q , many approaches have been studied to retrieve the most similar objects to q from S by considering feature vectors' distance alone [15, 36, 16]. In the following, we first introduce ANNS, then formally define the HQ that we study.

For an object e ∈ S and its feature vector ν ( e ) ∈ X , we write ν ( e ) = [ ν ( e ) 0 , ν ( e ) 1 , . . . , ν ( e ) d -1 ] , where ν ( e ) i is the value of ν ( e ) on the i -th dimension. We focus on the high-dimensional case where d is from hundreds to thousands. For any two objects e, o ∈ S with feature vectors ν ( e ) , ν ( o ) ∈ X , we can measure their similarity with different methods, such as Euclidean distance [19] and Cosine similarity [36]. Euclidean distance is the most popular method [33], which is in Eq. 1.

Definition 2 NNS [18, 20, 45]. Given an object set S and a query object q with the feature vector ν ( q ) , the NNS aims at obtaining the topk objects from S whose feature vectors are closest to ν ( q ) .

In Def. 2, the exact topk objects (denoted by T ) hold that

Exact NNS on a large S is not feasible because it takes too much computation [52]. So, an ANNS is more realistic, as it balances accuracy and efficiency with a vector index [52]. Let T be the exact topk objects from Def. 2 and R be the approximate topk objects from an ANNS method. We can use recall rate Recall @ k to measure the search accuracy of the approximate method:

A bigger Recall @ k means more accurate results from ANNS.

A query object

feature vector

Attribute constraints

0

u

1

u

q

2

u

3

u

4

u

5

u

x

x

x

An object set

...

Feature vector space

{

(

) |

}

e

e

ν

=

∈

X

S

Attribute vector space

{ (

) |

}

e

e

=

∈

ℓ

Y

S

Online: joint pruning

Fusion distance

1.0

0

(

,

)

Γ q u

1

(

,

)

Γ q u

2

(

,

)

Γ q u

3

(

,

)

Γ q u

4

(

,

)

Γ q u

1.2

1.3

0.5

0.6

0.1

5

(

,

)

Γ q u

Offline: building composite index

To return the query results fast, ANNS makes a vector index based on the feature vector. We can classify ANNS methods into four types by how they make the index: quantization [28, 23]; tree [46, 2]; hashing [22, 24]; and proximity graph (PG)[36, 19]. Many works [33, 19] have shown that the PG-based methods have better efficiency vs accuracy trade-off.

Definition 3 Hybrid Queries (HQ). Given an object set S and a query object q with a feature vector ν ( q ) and a set of attributes { a 0 , . . . , a m -1 } of size m , the HQ returns the topk objects from S , denoted by T . The objects in T satisfy two conditions: 1 ⃝ attribute filtering (AF): they have the same attributes as q -that is, for any e ∈ T , ∀ i = 0 , 1 , · · · , m -1 , e.a i = q.a i ; and 2 ⃝ ANNS: their feature vectors are the closest to ν ( q ) among those that meet 1 ⃝ .

It is also too slow to answer an exact HQ on a large S . So, current research works on an approximate HQ processing [51, 54] and the query accuracy is measured by Eq. 3 for a returned R from the approximate method. HQ is an extended ANNS with attribute constraint [51]. But it is a challenging problem, because all existing methods have accuracy and efficiency issues ( L1-L4 in §1.1 ).

3 NHQFramework

We begin by defining proximity graph (PG) and then introduce the NHQ framework based on PG.

Definition 4 PG. Given an object set S , we define the PG of S w.r.t. an distance threshold B as a graph G = ( V, E ) with the vertex set V and edge set E . 1 ⃝ For each vertex u ∈ V , it corresponds to an object e ∈ S . 2 ⃝ For any two vertices u i and u j from V , if u i u j ∈ E , we have δ ( ν ( u i ) , ν ( u j )) ≤ B , where ν ( u i ) and ν ( u j ) is the feature vectors of objects u i and u j , respectively.

Let N ( u i ) be the neighbor set of u i . Our idea is to build a PG that uses both feature vectors and attributes as a composite index. In the index, the neighbors of u may have the same attributes as u . We can use Eq. 1 to measure the feature vector distance, but we need to quantify the attributes and define a distance metric function [55]. Then, we can fuse the feature vector distance and the attribute distance into a fusion distance to compare the objects. Therefore, we can directly prune the vertices that have either dissimilar feature vectors or different attributes with the fusion distance.

Fusion distance. Given an object set S with feature vectors X (see Fig. 2, bottom left), we can use different encoding methods [3, 44, 48] to encode the attributes of each object e ∈ S . Ordinal encoding works well for structured attributes [53]. We use ordinal encoding ℓ ( . ) to get an attribute vector ℓ ( e ) = [ ℓ ( e ) 0 , · · · , ℓ ( e ) m -1 ] for each object e , where ℓ ( e ) i is e.a i 's encoded value (e.g., { NeurIPS , NLP } can be encoded as [1 , 3] by ℓ ( . ) ). Then, we get attribute vectors Y = { ℓ ( e ) | e ∈ S} .

We can measure the distance between two feature vectors ν ( e i ) and ν ( e j ) in X by using Eq. 1. We can measure the distance between two attribute vectors ℓ ( e i ) and ℓ ( e j ) in Y by using this formula:

̸

where ϕ ( ℓ ( e i ) k , ℓ ( e j ) k ) is

In Eq. 4-5, m is the number of dimensions in ℓ ( e i ) , and ℓ ( e i ) k is the value on the k -th dimension. The smaller this distance is, the more similar the attribute vectors are.

Wecan fuse the feature vector distance δ ( ν ( e i ) , ν ( e j )) and the attribute vector distance χ ( ℓ ( e i ) , ℓ ( e j )) into a single distance Γ ( e i , e j ) for objects e i and e j by using this formula:

where w ν and w ℓ are distance weights. The smaller this distance is, the more similar the objects are in both feature vectors and attributes. Eq. 6 is a simple and practical way to combine two different distances. It also makes it easy to build a composite index on top of existing PGs, because we just need to change the distance measure from Eq. 1 to Eq. 6. For example, if we set w ν = 1 and w ℓ = 0 , we get Γ ( e i , e j ) = δ ( ν ( e i ) , ν ( e j )) , which is the same as building a PG based on feature vector. If we set w ν = 0 and w ℓ = 1 , we get Γ ( e i , e j ) = χ ( ℓ ( e i ) , ℓ ( e j )) , which is a PG based on attribute. So, we can find the best balance between two distances by adjusting w ν and w ℓ .

Weight configuration. We found that w ν = 1 and w ℓ = δ ( ν ( e i ) , ν ( e j )) /m give the best query performance for most datasets and algorithms (see Appendix U). These weights do not depend on the dataset, but only on the feature vector distance δ ( ν ( e i ) , ν ( e j )) of two objects and the attribute vector dimension m . The idea is to fine-tune the feature vector distance with the attribute distance. For example, if e i and e j have the same attributes, that is, χ ( ℓ ( e i ) , ℓ ( e j )) = 0 , we keep the feature vector distance as it is (i.e., Γ ( e i , e j ) = δ ( ν ( e i ) , ν ( e j )) ). If e i and e j have completely different attributes, that is, χ ( ℓ ( e i ) , ℓ ( e j )) = m , we double the feature vector distance to get Γ ( e i , e j ) = 2 · δ ( ν ( e i ) , ν ( e j )) . In general, we have δ ( ν ( e i ) , ν ( e j )) ≤ Γ ( e i , e j ) ≤ 2 · δ ( ν ( e i ) , ν ( e j )) .

Composite index. We build a composite index based on Algorithm 1 (see Fig. 2, bottom right). We start with V = S , so each object e ∈ S is a vertex u ∈ V in G , and E = ∅ . Then, we add an edge u i u j ∈ E between two vertices u i , u j ∈ V if Γ ( u i , u j ) ≤ B ′ , where B ′ is a fusion distance threshold. Note that our composite index does not incur extra space cost, as shown in Lemma 1.

Lemma 1 The composite index and the ordinary PG have the same index size for an object set S .

Theorem 1 Let Ω min and Ω max be the minimum and maximum distances between feature vectors on S . Suppose we have at least | N ( e ) | objects with the same attributes as e for any object e ∈ S . Then, 1 ⃝ for any vertex u and its neighbor o in the composite index, we have Ω min ≤ Γ ( u, o ) ≤ Ω max ; 2 ⃝ for any vertex u and its neighbor o in the ordinary PG, we have Ω min ≤ Γ ( u, o ) ≤ 2 · Ω max .

Theorem 1 shows that our composite index has a smaller fusion distance bound.

Joint pruning. Given a query object q , we use a composite index G and a seed set P (usually chosen randomly from V [52]) to find the approximate topk objects based on Algorithm 2. We follow these steps (see Fig. 2, top right): 1 ⃝ Initialization . We use a visited vertex set C to store the vertices for search expansion and a result set R of size k to store the current query results. We set both sets to P at first. 2 ⃝ Search expansion . We take out the vertex u i with the smallest Γ ( q, u i ) from C as the next visited vertex, and then add N ( u i ) to C . 3 ⃝ Query results update . We update R with the better vertices in N ( u i ) . For any vertex u j ∈ N ( u i ) and a vertex u r ∈ R that is the farthest from q , we replace u r with u j in R if Γ ( q, u j ) < Γ ( q, u r ) . This means u j is more similar to q in both feature vectors and attributes than u r . We repeat 2 ⃝ and 3 ⃝ until R does not change, then we return R as the approximate topk objects.

Example 2 We show an example of joint pruning for finding the nearest object (i.e., top-1) to the query object q in Fig. 2 (top right). We start with a random vertex as the seed, i.e., P = { u 0 } ,

P1

P2

P3

P4

P1

P2

P3

P4

lune

(a) Naïve edge selection

(b) RNG's edge selection

1

u

0

u

2

u

i

u

3

u

4

u

9

u

8

u

7

u

6

u

5

u

5

u

6

u

7

u

8

u

9

u

i

u

4

u

3

u

2

u

0

u

1

u

P1

P2

P3

P4

(c) Our edge selection

2

u

0

u

1

u

3

u

4

u

5

u

6

u

7

u

8

u

9

u

i

u

and set C = { u 0 } and R = { u 0 } . Then, we expand the search space with u 0 's neighbors and get C = { u 1 , u 2 , u 3 , u 4 } . We update R with u 4 because Γ ( q, u 4 ) = 0 . 5 is smaller than Γ ( q, u 0 ) = 1 . 0 . We continue to expand the search space with u 4 's and u 5 's neighbors and get R = { u 5 } . Since no vertex u j ∈ N ( u 5 ) has a smaller distance than Γ ( q, u 5 ) , we return u 5 as the approximate top-1 nearest object. We prune the vertices u 1 , u 2 , and u 3 from the search space.

Theorem 2 Given a query object q , the composite index guarantees at least the same Recall @ k of HQ as the ordinary PG. (see Appendix H for the proof)

Theorem 2 states that our composite index has a higher or equal Recall @ k than the ordinary PG.

4 Navigable PG Algorithm

To form the composite index, we can deploy a specific PG in NHQ and change its original distance measure to the fusion distance (Eq. 6). This gives the PG the ability to handle HQ effectively. However, current PGs have limitations that affect NHQ's performance with the PG-based composite index. Therefore, we present two new navigable PGs (NPGs) in this section. We optimize the edge selection and routing strategies for building and searching on a PG, respectively.

4.1 Edge Selection

Edge selection is a crucial step for building a PG. It determines the neighbors of each object e in an object set S . Different strategies build different structures for a PG, which affect its search performance [19]. Existing PGs use two factors for edge selection: distance between two vertices (D1) and distribution of all vertices (D2) [52]. Early PGs like NSW [35] and KGraph [14] only use D1 and connect each vertex with some of its nearest neighbors (Fig. 3(a)). However, this can cause redundant computations and reduce search efficiency [34]. Recent PGs also use D2 with RNG's edge selection, which diversifies the neighbors' direction [33]. However, RNG's edge selection still fails to diversify the neighbors well (Fig. 3(b)). Due to the space limitation, we explain this in Appendix I.

Our edge selection. We design a new edge selection strategy that uses D1 and D2. It connects u i with one nearest neighbor in each area of u i (P1-P4 in Fig. 3(c)). We define the landing zone of u i and one of its neighbors u j ∈ N ( u i ) , which is the area where only the vertices in it can be added to N ( u i ) . We then describe our edge selection strategy based on the landing zone.

Definition 5 Landing zone. The landing zone L ( u i , u j ) of a vertex u i and one of its neighbors u j ∈ N ( u i ) is an area defined by H ( u i , u j ) \ B ( u i , δ ( ν ( u i ) , ν ( u j ))) . Here, H ( u i , u j ) is the half space with u i that is split by the perpendicular bisector U ( u i , u j ) of the line between u i and u j . And B ( u i , δ ( ν ( u i ) , ν ( u j ))) is the hypersphere centered at u i with radius δ ( ν ( u i ) , ν ( u j )) .

Example 3 As Fig. 3(c) shows, in a two dimensional space, U ( u i , u 0 ) is a perpendicular bisector of the line connecting u i and u 0 (i.e., the red line), H ( u i , u 0 ) is located on the upper side of U ( u i , u 0 ) , and B ( u i , δ ( ν ( u i ) , ν ( u 0 ))) is the area enclosed by the green circle. Therefore, the landing zone L ( u i , u 0 ) formed by u i and u 0 is the green shaded region (i.e., H ( u i , u 0 ) \ B ( u i , δ ( ν ( u i ) , ν ( u 0 ))) ).

To find the areas without any neighbor of u i , we intersect the landing zones of all its existing neighbors (i.e., ⋂ u j ∈ N ( u i ) L ( u i , u j ) ). Then, we add the closest vertex (w.r.t. u i ) within this area into N ( u i ) , making the neighbors more diverse. We get N ( u i ) for each vertex u i ∈ V as follows (Algorithm 3).

1 ⃝ Candidates acquisition . We obtain a set of l candidate neighbors for u i from ( V \ { u i } ) , denoted by C ( u i ) . We can use random sampling or an extra index to get C ( u i ) (e.g., we get C ( u i ) based

on NSW [35] and KGraph [14] in §4.3). We ensure that l ≥ R , where R is the maximum number neighbors for u i (i.e., | N ( u i ) | ≤ R ). 2 ⃝ Neighbors initialization . We sort the elements in C ( u i ) in ascending order of distance to u i . We initialize N ( u i ) with u i 's nearest candidate neighbor u t from C ( u i ) and remove u t from C ( u i ) . 3 ⃝ Neighbors update . We select the nearest vertex u p of u i from C ( u i ) , and add it to N ( u i ) if it is in the intersection area (i.e., ⋂ u j ∈ N ( u i ) L ( u i , u j ) ) of the landing zones of u i and its existing neighbors. We repeat this process until C ( u i ) = ∅ or | N ( u i ) | = R .

Our strategy ensures that u i 's neighbors are diverse in terms of the areas where C ( u i ) is located. This is because we add u p to N ( u i ) from a different area each time. We have the following theorem.

Theorem 3 The angle between any two neighbors is not less than π/ 3 . (See Appendix J for the proof)

Example 4 In Fig. 3(c), assuming N ( u i ) = { u 0 } and C ( u i ) = { u 3 , u 4 , · · · , u 9 } , we include u 3 in N ( u i ) because it lies within L ( u i , u 0 ) (the green region). Next, we add u 5 to N ( u i ) since it belongs to L ( u i , u 0 ) ∩ L ( u i , u 3 ) . We continue this process until each new area

Algorithm 3: Edge Selection ( u i , l , R )

̸

has one neighbor of u i . Consequently, in the search procedure, we can efficiently route to the query object q by utilizing the neighbors of u i present in the same area as q .

Complexity. To obtain C ( u i ) , the time complexity depends on the methods used, so we skip this term in our analysis. The time complexity of sorting C ( u i ) is O ( l · log( l )) . For each u p ∈ C ( u i ) , we check at most | N ( u i ) | times ( | N ( u i ) | ≤ R ) to see if u p is in L ( u i , u j ) for each u j ∈ N ( u i ) . So, our edge selection gets the final N ( u i ) with no more than l · R checks. Thus, the time complexity of our edge selection on the vertex set V is O ( l · ( R +log( l )) · | V | ) .

4.2 Routing

Routing is a crucial process for searching on a PG. It determines a path from the start vertex to the result vertex that matches the query. The routing strategy affects both the efficiency and accuracy of the search [39]. A recent study [58] splits routing into two stages: the far stage (S1) and the close stage (S2), based on how far they are from the query object.

Our routing. Algorithm 4 illustrates our random two-stage routing strategy. In S1, we randomly select ⌈ R/h ⌉ neighbors from N ( u i ) (where 1 ≤ h ≤ R , and R is the maximum out-degree), and calculate their distances to the query object q . This enables us to quickly approach the neighborhood of q . In S2, we evaluate the distances of all neighbors in N ( u i ) to q , following a similar approach to the greedy search [19]. The transition from S1 to S2 occurs when S1 reaches a local optimum, indicated by no further updates to the result set R . In S2, we continue updating R by checking all neighbors of the visited vertex. The process terminates when R no longer receives any updates.

Theorem 4 In the worst case, our routing has the same Recall @ k as the current greedy search [19].

Algorithm 4: Routing ( G , q , P )

According to Theorem 4 (See Appendix L for the proof), our random strategy in S1 does not reduce the accuracy because most vertices do not require distance calculation from the query in S1 and we can recover a small number of possibly missed vertices in S2.

Complexity. Previous works [36, 19, 58] show that the time complexity of greedy search on a state-of-the-art PG (e.g., HNSW [36], NSG [19]) is O ( R · log( | V | )) , where R ≪| V | is the maximum number of neighbors per vertex and log( | V | ) is roughly the average routing path length. In our routing, we use l 1 and l 2 to denote the average routing path lengths in S1 and S2, respectively; then we have l 1 + l 2 = log( | V | ) . Thus, our routing's time complexity is O (( ⌈ R/h ⌉ ) · l 1 + R · l 2 ) .

4.3 NPG with Our Edge Selection and Routing

We present two NPGs, NPG nsw and NPG kgraph , that use our edge selection and routing strategies on two basal PGs: NSW [35] and KGraph [14]. See Appendix M for implementation details.

5 Experiments

5.1 Experimental Setting

Datasets. We use ten real-world datasets, including one newly released dataset called Paper . They span various modalities, such as video, image, audio, and text. We summarize their main characteristics in Tab. 1. With the exception of the Paper dataset, the remaining datasets solely consist of high-dimensional feature vectors without any original attributes. Therefore, we generate attributes for each object in these datasets using the similar method described in [51, 56]. For instance, in SIFT1M, we augment each image with attributes such as date, location, and size , thereby creating an object set that comprises both feature vectors and attributes.

Compared methods. We compare our HQ methods with seven existing ones that have been used in many high-tech companies. ADBV [54] is a cost-based HQ method proposed by Alibaba. It optimizes IVFPQ [28] for ANNS. Milvus [38, 51] divides the object set through frequently used attributes, and deploys ADBV [54] on each subset. Vearch [29, 32] is developed by Jingdong, which implements the HQ working off Strategy B. NGT [59] is a ANNS library released by Yahoo Japan, which answers a HQ to conduct attribute filtering atop the candidates recalled by NGT (Strategy B). Faiss [17] is a ANNS library developed by Facebook, which answers a HQ based on IVFPQ [28] and Strategy A. SPTAG [37] is a PG-based ANNS library from Microsoft, which answers HQ on Strategy B. Filtered-DiskANN [21] proposes two optimizations based on DiskANN: FilteredVamana and StitchedVamana. FilteredVamana connects vertices with shared attributes. StitchedVamana builds separate graph indexes for each filter and overlays them. NHQNPG nsw and NHQNPG kgraph are our HQ methods based on NHQ framework integrating two NPGs.

Metrics. We measure the search efficiency by queries per second ( QPS ), which is the number of queries ( # q ) divided by the search time ( t ), i.e., # q/t . We use the Recall rate to evaluate the search accuracy, which is defined by Eq. 3. Unlike ANNS, hybrid query also requires attribute constraints in Eq. 3, i.e., the elements in R ∩ T must have the same attributes as the query object.

Implementation setup. All codes are written in C++, and are compiled by g++ 6.5. All experiments are conducted on a Linux server with an Intel(R) Xeon(R) Gold 6248R CPU at 3.00GHz, and a 755G memory. We use 64 threads to build all the indexes in parallel. We use a single thread for search, which is a common setting in related work [19, 18]. We report the average results from three trials.

0.80

0.85

0.90

0.95

1.00

Recall

@10

10 1

10 2

10 3

10 4

QPS

NHQ

-

NPGkgraph

NHQ

-

NPGnsw

Vearch

ADBV

Faiss

SPTAG

NGT

(a) SIFT1M

0.80

0.85

0.90

0.95

1.00

Recall

@10

10 3

10 4

QPS

(b) Audio

0.6

0.7

0.8

0.9

1.0

Recall

@10

10 3.5

10 3.8

10 4.0

10 4.2

QPS

NHQ

DiskANN

FilteredVamana

StitchedVamana

(a) In-memory

0.5

0.6

0.7

0.8

0.9

1.0

Recall

@10

10 1.5

10 2.0

10 2.5

10 3.0

QPS

NHQ

DiskANN

Filtered

DiskANN

(b) On-disk

5.2 HQPerformance

Fig. 4 shows that our methods outperform existing methods in terms of QPS vs Recall trade-off on all datasets. For example, NHQNPG kgraph achieves two to three orders of magnitude higher QPS than others when Recall@10 > 0 . 99 . To execute Filtered-DiskANN, we only test single-attribute queries in Fig. 5. To eliminate other factors, we implement NHQ on DiskANN (NHQ can be easily extended to the current graph index), named NHQ-DiskANN. We keep the same parameters in DiskANN. From the results, NHQ-DiskANN outperforms Filtered-DiskANN, in both memory and disk versions.

0.85

0.90

0.95

1.00

Recall

@10

10 1

10 2

10 3

QPS

NHQ

-

DiskANN

AF

-

DiskANN

(a) BIGANN100M

0.85

0.90

0.95

1.00

Recall

@10

0

1000

2000

3000

Mean I

/

Os

(b) BIGANN100M

0.7

0.8

0.9

1.0

Recall

@10

10 2

10 3

10 4

QPS

NHQ

-

HNSW

(

Strategy C

)

HNSW

(

Strategy B

)

IVFPQ

(

Strategy A

)

(a) HNSW (SIFT1M)

0.7

0.8

0.9

1.0

Recall

@10

10 2

10 3

10 4

QPS

NHQ

-

NSG

(

Strategy C

)

NSG

(

Strategy B

)

IVFPQ

(

Strategy A

)

(b) NSG (SIFT1M)

5.3 Scalability on Larger Dataset

To test the scalability of NHQ, we implement HQ on the state-of-the-art disk-resident PG, DiskANN [27], using 'first ANNS, then AF' strategy (AF-DiskANN) and our NHQ (NHQ-DiskANN). Both implementations have the same hyper-parameters for DiskANN. Fig. 6(a) and (b) show the QPS and Mean I/Os results for different Recall on BIGANN100M dataset. NHQ-DiskANN consistently beats AF-DiskANN by a large margin. For QPS , NHQ-DiskANN is 315 × faster than AF-DiskANN when Recall@10=0.95, because NHQ-DiskANN reduces expensive random disk I/Os. For example, to get a recall of 0.95, AF-DiskANN needs 1,303 disk I/Os, while NHQ-DiskANN only needs 30 I/Os, saving about 97.7% of disk I/Os. This shows the scalability of NHQ to handle larger dataset.

5.4 Ablation Study

Validation of NHQ framework. We test the universality of our NHQ (Strategy C) by applying the HQ model with 'decomposition-assembly' to different PGs (including HNSW [36] and NSG [19]) based on 'first ANNS, then AF' (Strategy B in Appendix A). We also use IVFPQ [28] in 'first AF, then ANNS' (Strategy A in Appendix A) following state-of-the-art implementation [54, 51] because Strategy A does not support PG ( L4 in §1.1 ). We integrate HNSW and NSG into NHQ to form NHQ-HNSW and NHQ-NSG, respectively. As Fig. 7 shows, NHQ outperforms other strategies on different PGs, and keeps stable QPS advantage on different datasets. Due to IVFPQ's limitations, the HQ based on Strategy A have low accuracy, e.g., Recall @10 < 0 . 8 on SIFT1M.

0.80

0.85

0.90

0.95

1.00

Recall

@10

10 1.7

10 2.0

10 2.5

10 3.0

Speedup

Ours

NGT

HNSW

HCNNG

NSG

(a) SIFT1M

0.80

0.85

0.90

0.95

1.00

Recall

@10

10 1.0

10 1.5

10 2.0

10 2.5

Speedup

Ours

NGT

HNSW

HCNNG

NSG

(b) GIST1M

0.85

0.90

0.95

1.00

Recall

@10

10 4

QPS

NHQ

-

NPGkgraph

NHQ

-

HNSW

NHQ

-

NSG

(a) SIFT1M

0.85

0.90

0.95

1.00

Recall

@10

10 3

10 4

QPS

Effect of our edge selection strategy. We compare our edge selection strategy with four existing ones: NGT [59], HNSW [36], HCNNG [39], and NSG [19] on SIFT1M and GIST1M datasets using a recent evaluation framework [52]. All competitors use the same routing inference and distance function. We measure the Speedup -Recall metric, where Speedup is relative to brute force. Fig. 8

shows that our strategy outperforms the others. For example, at Recall@10=0.9 on SIFT1M, our strategy achieves 1.1 × , 1.4 × , and 25.9 × speedup over HCNNG/NSG, HNSW, and NGT, respectively.

Effect of our routing strategy. We compare our routing strategy with three existing ones: TOGG [58], FINGER [8], and HCNNG [39] within the HNSW index framework with consistent parameters. Tab. 2 shows the speedup of each optimized strategy over the original HNSW at Recall@10 = 0.9 on SIFT1M. All optimized strategies are faster than the original HNSW, indicating the benefit of optimizing the routing procedure. Our strategy has the lowest storage cost, as it does not require extra structures. Moreover, it delivers a significant improvement in search performance.

Effect of different PGs under NHQ. We compare the performance of three PGs on NHQ: HNSW [36], NSG [19], and our NPG kgraph . We call them NHQ-HNSW, NHQ-NSG, and NHQNPG kgraph respectively. Fig. 9 shows their results on SIFT1M and GloVe. NHQNPG kgraph outperforms the others, especially on the harder GloVe dataset.

6 Discussion

Memory overhead. NHQhas a higher memory cost than Faiss when considering their best trade-offs. This is a common drawback of graph-based methods compared to PQ-based methods in the ANNS community. This is mainly because graph-based methods build an extra proximity graph index (stored as an adjacency list) to speed up the online search process. While graph-based methods have a higher storage cost, they achieve a significantly better trade-off between accuracy and efficiency and have become the mainstream algorithms in most vector databases (such as Milvus [51]). Our NHQ framework enhances the ability of current graph-based methods to handle ANNS + AF.

10 1

10 2

10 3

QPS

NHQ

-

NPGkgraph

Vearch

ADBV

Faiss

SPTAG

NGT

0.75

0.80

0.85

0.90

0.95

1.00

Recall

@10

10 1

10 2

10 3

10 4

QPS

(a)

z

= 36

0.80

0.85

0.90

0.95

1.00

Recall

@10

10 2

10 3

10 4

QPS

(b)

z

= 972

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Recall

@10

10 2

10 3

10 4

QPS

(c)

z

= 26244

0.80

0.85

0.90

0.95

1.00

Figure 10: Hybrid query performance of different number of attribute combinations ( ). Number of attribute combinations. We conduct QPS -Recall comparisons across different attribute dimensions { 3 , 6 , 9 } in Fig. 10. The corresponding number of attribute combinations (represented by z ) is { 36 , 972 , 26244 } . We find that increasing z can pose challenges for both the NHQ and existing frameworks. However, NHQ demonstrates higher speedup over existing frameworks in scenarios with a large z , indicating its potential advantage in handling diverse attribute sets efficiently.

7 Conclusion

In this paper, we address HQ, which is ANNS with attribute constraint. We present a NHQ framework that adapts to most existing PGs (including disk-resident PGs, e.g., DiskANN [27]) and optimizes them for HQ. We introduce two NPGs with enhanced edge selection and routing strategies that achieve better ANNS performance than original PGs. We integrate several PGs into NHQ to implement HQ methods that are up to 315 × faster than the state-of-the-art alternatives. A possible future and ongoing work is to apply our framework to multimodal search to handle more complex query requirements.

Acknowledgments

This work was supported by the Primary R&D Plan of Zhejiang (2021C03156 and 2023C03198) and the National NSF of China (62072149).

References

A comprehensive survey and experimental comparison of graph-based approximate nearest neighbor search. https://github.com/Lsyhprum/WEAVESS/tree/dev/dataset, 2021.

Akhil Arora, Sakshi Sinha, Piyush Kumar, and Arnab Bhattacharya. Hd-index: Pushing the scalability-accuracy boundary for approximate knn search in high-dimensional spaces. PVLDB , 11(8):906-919, 2018.

Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. Towards open-set identity preserving face synthesis. In CVPR , pages 6713-6722, 2018.

Dmitry Baranchuk, Dmitry Persiyanov, Anton Sinitsin, and Artem Babenko. Learning to route in similarity graphs. In ICML , volume 97, pages 475-484, 2019.

Erik Bernhardsson, Martin Aumüller, and Alexander Faithfull. Benchmarking nearest neighbors. https://github.com/erikbern/ann-benchmarks, 2018.

Prosenjit Bose, Vida Dujmovi´ c, Ferran Hurtado, John Iacono, Stefan Langerman, Henk Meijer, Vera Sacristán, Maria Saumell, and David R Wood. Proximity graphs: E, δ , δ , χ and ω . International Journal of Computational Geometry & Applications , 22(05):439-469, 2012.

Antoine Boutet, Anne-Marie Kermarrec, Nupur Mittal, and François Taïani. Being prepared in a sparse world: The case of KNN graph construction. In ICDE , pages 241-252, 2016.

Patrick Chen, Wei-Cheng Chang, Jyun-Yu Jiang, Hsiang-Fu Yu, Inderjit Dhillon, and Cho-Jui Hsieh. Finger: Fast inference for graph-based approximate nearest neighbor search. In WWW , pages 3225-3235, 2023.

Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. SPANN: highly-efficient billion-scale approximate nearest neighborhood search. In NeurIPS , pages 5199-5212, 2021.

Shivani Choudhary, Tarun Luthra, Ashima Mittal, and Rajat Singh. A survey of knowledge graph embedding and their applications. CoRR , abs/2107.07842, 2021.

Benjamin Coleman, Santiago Segarra, Alexander J Smola, and Anshumali Shrivastava. Graph reordering for cache-efficient near neighbor search. In NeurIPS , pages 38488-38500, 2022.

John P. Collomosse, Tu Bui, and Hailin Jin. Livesketch: Query perturbations for guided sketch-based visual search. In CVPR , pages 2879-2887, 2019.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. CoRR , abs/1810.04805, 2018.

Wei Dong, Moses Charikar, and Kai Li. Efficient k-nearest neighbor graph construction for generic similarity measures. In WWW , pages 577-586, 2011.

Karima Echihabi. High-dimensional vector similarity search: From time series to deep network embeddings. In SIGMOD , pages 2829-2832, 2020.

Karima Echihabi, Kostas Zoumpatianos, and Themis Palpanas. High-dimensional similarity search for scalable data science. In ICDE , pages 2369-2372, 2021.

Facebook. A library for efficient similarity search and clustering of dense vectors. https://github.com/facebookresearch/faiss, 2018.

Cong Fu, Changxu Wang, and Deng Cai. High dimensional similarity search with satellite system graph: Efficiency, scalability, and unindexed query compatibility. TPAMI , 2021.

Cong Fu, Chao Xiang, Changxu Wang, and Deng Cai. Fast approximate nearest neighbor search with the navigating spreading-out graph. PVLDB , 12(5):461-474, 2019.

Aristides Gionis, Piotr Indyk, and Rajeev Motwani. Similarity search in high dimensions via hashing. In VLDB , pages 518-529, 1999.

Siddharth Gollapudi, Neel Karia, Varun Sivashankar, Ravishankar Krishnaswamy, Nikit Begwani, Swapnil Raz, Yiyong Lin, Yin Zhang, Neelam Mahapatro, Premkumar Srinivasan, et al. Filtered-diskann: Graph algorithms for approximate nearest neighbor search with filters. In WWW , pages 3406-3416, 2023.

Long Gong, Huayi Wang, Mitsunori Ogihara, and Jun Xu. idec: Indexable distance estimating codes for approximate nearest neighbor search. PVLDB , 13(9):1483-1497, 2020.

Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. Accelerating large-scale inference with anisotropic vector quantization. In ICML , pages 3887-3896, 2020.

Qiang Huang, Jianlin Feng, Yikai Zhang, Qiong Fang, and Wilfred Ng. Query-aware localitysensitive hashing for approximate nearest neighbor search. PVLDB , 9(1):1-12, 2015.

Qiujun Huang, Jingli Mao, and Yong Liu. An improved grid search algorithm of svr parameters optimization. In ICCT , pages 1022-1026, 2012.

Xiao Huang, Jundong Li, and Xia Hu. Label informed attributed network embedding. In WSDM , pages 731-739, 2017.

Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnawamy, and Rohan Kadekodi. Diskann: Fast accurate billion-point nearest neighbor search on a single node. In NeurIPS , volume 32, 2019.

Hervé Jégou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. TPAMI , 33(1):117-128, 2011.

Jingdong. A distributed system for embedding-based retrieval. https://github.com/vearch/vearch, 2020.

Amar Viswanathan Kannan, Dmitriy Fradkin, Ioannis Akrotirianakis, Tugba Kulahcioglu, Arquimedes Canedo, Aditi Roy, Shih-Yuan Yu, Arnav V. Malawade, and Mohammad Abdullah Al Faruque. Multimodal knowledge graph for deep learning papers and code. In CIKM , pages 3417-3420, 2020.

Conglong Li, Minjia Zhang, David G. Andersen, and Yuxiong He. Improving approximate nearest neighbor search through learned adaptive early termination. In SIGMOD , pages 25392554, 2020.

Jie Li, Haifeng Liu, Chuanghua Gui, Jianyu Chen, Zhenyuan Ni, Ning Wang, and Yuan Chen. The design and implementation of a real time visual search system on JD e-commerce platform. In Proceedings of the 19th International Middleware Conference , pages 9-16, 2018.

Wen Li, Ying Zhang, Yifang Sun, Wei Wang, Mingjie Li, Wenjie Zhang, and Xuemin Lin. Approximate nearest neighbor search on high dimensional data - experiments, analyses, and improvement. TKDE , 32(8):1475-1488, 2020.

Peng Cheng Lin and Wan Lei Zhao. Graph based nearest neighbor search: Promises and failures. CoRR , abs/1904.02077, 2019.

Yury Malkov, Alexander Ponomarenko, Andrey Logvinov, and Vladimir Krylov. Approximate nearest neighbor algorithm based on navigable small world graphs. Information Systems , 45:61-68, 2014.

Yury A. Malkov and D. A. Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. TPAMI , 42(4):824-836, 2020.

Microsoft. Sptag: A library for fast approximate nearest neighbor search. https://github.com/microsoft/SPTAG, 2020.

Milvus. The world's most advanced open-source vector database. https://github.com/milvusio/milvus, 2019.

Javier Alvaro Vargas Muñoz, Marcos André Gonçalves, Zanoni Dias, and Ricardo da Silva Torres. Hierarchical clustering-based graphs for large scale approximate nearest neighbor search. Pattern Recognition , 96:106970, 2019.

Liudmila Prokhorenkova and Aleksandr Shekhovtsov. Graph-based nearest neighbor search: From practice to theory. In ICML , pages 7803-7813, 2020.

Jianbin Qin, Wei Wang, Chuan Xiao, and Ying Zhang. Similarity query processing for highdimensional data. PVLDB , 13(12):3437-3440, 2020.

Jianbin Qin, Wei Wang, Chuan Xiao, Ying Zhang, and Yaoshu Wang. High-dimensional similarity query processing for data science. In SIGKDD , pages 4062-4063, 2021.

Jie Ren, Minjia Zhang, and Dong Li. HM-ANN: efficient billion-point nearest neighbor search on heterogeneous memory. In NeurIPS , 2020.

Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogerio Feris, Raja Giryes, and Alex Bronstein. Delta-encoder: an effective sample synthesis method for few-shot object recognition. NeurIPS , 31, 2018.

Larissa Capobianco Shimomura, Marcos R Vieira, and Daniel S Kaster. Performance analysis of graph-based methods for exact and approximate similarity search in metric spaces. In International Conference on Similarity Search and Applications , pages 18-32, 2018.

Chanop Silpa-Anan and Richard I. Hartley. Optimised kd-trees for fast image descriptor matching. In CVPR , 2008.

Harsha Vardhan Simhadri, George Williams, Martin Aumüller, Matthijs Douze, Artem Babenko, Dmitry Baranchuk, Qi Chen, Lucas Hosseini, Ravishankar Krishnaswamy, Gopal Srinivasa, Suhas Jayaram Subramanya, and Jingdong Wang. Results of the neurips'21 challenge on billion-scale approximate nearest neighbor search. In NeurIPS , pages 177-189, 2021.

Zhenan Sun and Tieniu Tan. Ordinal measures for iris recognition. TPAMI , 31(12):2211-2226, 2008.

Godfried T Toussaint. The relative neighbourhood graph of a finite planar set. Pattern recognition , 12(4):261-268, 1980.

Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, and James Hays. Composing text and image for image retrieval - an empirical odyssey. In CVPR , pages 6439-6448, 2019.

Jianguo Wang, Xiaomeng Yi, Rentong Guo, Hai Jin, Peng Xu, Shengjun Li, Xiangyu Wang, Xiangzhou Guo, Chengming Li, Xiaohai Xu, Kun Yu, Yuxing Yuan, Yinghao Zou, Jiquan Long, Yudong Cai, Zhenxiang Li, Zhifeng Zhang, Yihua Mo, Jun Gu, Ruiyi Jiang, Yi Wei, and Charles Xie. Milvus: A purpose-built vector data management system. In SIGMOD , pages 2614-2627, 2021.

Mengzhao Wang, Xiaoliang Xu, Qiang Yue, and Yuxiang Wang. A comprehensive survey and experimental comparison of graph-based approximate nearest neighbor search. PVLDB , 14(11):1964-1978, 2021.

Zhenhua Wang, Bin Fan, Gang Wang, and Fuchao Wu. Exploring local and overall ordinal information for robust feature description. TPAMI , 38(11):2198-2211, 2015.

Chuangxian Wei, Bin Wu, Sheng Wang, Renjie Lou, Chaoqun Zhan, Feifei Li, and Yuanzhe Cai. Analyticdb-v: A hybrid analytical engine towards query fusion for structured and unstructured data. PVLDB , 13(12):3152-3165, 2020.

Suping Xu, Lin Shang, and Furao Shen. Latent semantics encoding for label distribution learning. In IJCAI , pages 3982-3988, 2019.

Xiaoliang Xu, Chang Li, Yuxiang Wang, and Yixing Xia. Multiattribute approximate nearest neighbor search based on navigable small world graph. Concurrency and Computation: Practice and Experience , 32(24), 2020.

Xiaoliang Xu, Jun Liu, Yuxiang Wang, and Xiangyu Ke. Academic expert finding via ( k, P ) -core based embedding over heterogeneous graphs. In ICDE , pages 338-351, 2022.

Xiaoliang Xu, Mengzhao Wang, Yuxiang Wang, and Dingcheng Ma. Two-stage routing with optimized guided search and greedy algorithm on proximity graph. Knowledge-Based Systems , page 107305, 2021.

Yahoo. Nearest neighbor search with neighborhood graph and tree for high-dimensional data. https://github.com/yahoojapan/NGT, 2016.

Wen Yang, Tao Li, Gai Fang, and Hong Wei. Pase: Postgresql ultra-high-dimensional approximate nearest neighbor search extension. In SIGMOD , page 2241-2253, 2020.

Zhao Zhang, Fuzhen Zhuang, Hengshu Zhu, Zhiping Shi, Hui Xiong, and Qing He. Relational graph neural network with hierarchical attention for knowledge graph completion. In AAAI , pages 9612-9619, 2020.

Wan-Lei Zhao, Hui Wang, and Chong-Wah Ngo. Approximate k-nn graph construction: A generic online approach. IEEE Transactions on Multimedia , 2021.

Weijie Zhao, Shulong Tan, and Ping Li. SONG: approximate nearest neighbor search on GPU. In ICDE , pages 1033-1044, 2020.

Yuke Zhu, Ce Zhang, Christopher Ré, and Li Fei-Fei. Building a large-scale multimodal knowledge base for visual question answering. CoRR , abs/1507.05670, 2015.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel approximate nearest neighbor search framework which baked in attribute constraints via a single composite index compared to many existing two stage solutions. The framework is mainly relying on a newly proposed distance function that fuses both feature vector distances and attribute vector distances. Based on the new distance function, several proximity graph based ANNS methods have been developed. The paper presents an ablation study showing the merits of the methods. The experimental results shown on multiple popular landmark datasets outperform baseline methods.\n\nSTRENGTHS:\nThe proposed fusion distance is an excellent idea that enables the opportunities of the single composite index construction as well as the advantages over the legacy two-stage models. \n\nEach step of the method has been well-explained. In addition, most of the components have provided theoretical proof or support analysis.\n\nANN problems are often complicated in comparisons considering the nature of trade-offs among different factors including memory, accuracy, speed, etc. The paper has conducted sufficient experiments, ablation studies, and explanations of results. These strengthen the hypnosis and make the work overall to be solid.\n\nWEAKNESSES:\nDetails about config or setups for some baselines are missing. For example, the parameters or usages of FAISS are not discussed. It makes it harder to understand why FAISS gets saturated at recall 80% as shown in Section 5.2. Is it possible that the number of probes in IVF has not increased sufficiently?\n\nQUESTIONS:\nThere will be other challenges as the number of attributes increases. It may be worth discussing solutions if there are very high diversities of attributes among the data to be indexed. For example, different categories in product search may not share the same attributes (names). The total number of attributes could be considerably large, and it will also increase the attribute vector distance computation cost. The two stage approaches, e.g. AF + ANNS, are less likely to be constrained.\n\nLIMITATIONS:\nNo limitations of the work are discussed in the paper. \n\nPlease refer to my questions and comments in Weakness and Questions.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, authors tackle the problem of retrieving nearest neighbor items under constraints on attributes of the retrieved items. Each item is described by a feature vector and a set of discrete attributes. \n\nAuthors propose to use a distance function that uses weighted combination of feature vector based distance and discrete attribute based distance. \n\nTo support efficient retrieval for a given query and attribute constraints, authors use graph-based nearest neighbor search indices. The proposed hybrid distance function is used to build the graph in an offline step and to navigate the graph during test-time search.\n\nAuthors also propose two heuristics to improve graph construction and test-time graph navigation, and overall the proposed approach yields improvement over baselines.\n\nUpdate: I am leaning towards accepting the paper and have updated my rating from  `5: borderline accept` to `7: accept` after reading clarifications from the authors.\n\nSTRENGTHS:\n- The proposed idea of using a hybrid distance function (which combines feature vector and attribute based distances) provides significant improvement over two-stage inference pipelines that either retrieve based on feature vectors and then filter based on attributes or first filter based on attribute and then search over filtered items using feature vector.\n- The proposed approach outperforms popular baselines on a variety of datasets.\n\nWEAKNESSES:\n- Some missing baselines/ablations\n    - The three main contribution of the paper are\n        - a) New distance function for hybrid queries.\n        - b) New algorithm for constructing graph over items.\n        - b) New heuristic for efficiently navigating the graph at test-time.\n    - The experiments clearly show the advantage of using the hybrid distance function over two-stage search (which separately performs nearest neighbor retrieval based on feature vector and then filters based on attributes).\n    - But individual contribution of the proposed graph construction strategy and the proposed graph navigation strategy is not clear.\n    - The proposed graph construction algorithm should be compared with NGT, HNSW, Munoz et al. (2019) while keeping every other design variable the same i.e. with the same test-time inference as well as with the same distance function.\n    - Similarly, the proposed graph search method should be compared with existing methods for speeding up search such as TOGG (mentioned in the paper), Chen et al., (2023), Munoz et al., (2019).\n- The presentation of the paper can be further improved.\n    - Most algorithms are described in text but it would help to present them in Algorithm boxes.\n    - While it is okay to use appendix for extra results, theorem proofs etc, I think some important details such as experiment setup, datasets and baseline description has also been moved to appendix. Reading the paper involved too many jumps between the main paper and the appendix. Authors could make the hybrid distance section more concise to make some space or move some extra results to appendix while keeping only main results in the paper.\n    - Proof for theorem 4 is missing\n\n*Patrick Chen, Wei-Cheng Chang, Jyun-Yu Jiang, Hsiang-Fu Yu, Inderjit Dhillon, and Cho-Jui Hsieh. 2023. FINGER: Fast Inference for Graph-based Approximate Nearest Neighbor Search. In Proceedings of the ACM Web Conference 2023 (WWW '23)*\n\nMunoz, Javier Vargas, et al. \"Hierarchical clustering-based graphs for large scale approximate nearest neighbor search.\" *Pattern Recognition*\n 96 (2019): 106970.\n\nQUESTIONS:\n- Performance for NPG_{kgraph} on SIFT1M dataset is reported in Fig 5c, 6c and 7a. But the three graph do not seem to match.\n- In Fig 5c), does NHQ-NPG_{kgraph} use proposed routing method and do other baselines (NHQ-HNSW and NHQ-NSG) use naive greedy search?\n- What hyper-parameters were tried for ANNS + AF filtering approach?\n- The proposed search strategy has two stages S1 and S2. How does transition from S1 to S2 happen?\n- Complexity in Sec 4.1 : If C(u_i) is selected through an additional index then it will not be a constant time operation. Also in the final results, is C(u_i) selected randomly or using some additional index (and what index)?\n- Some suggestion of writing and presentation\n    - Including gridlines in the plot can help reading the graph.\n\nLIMITATIONS:\nNo discussion", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper discusses how hybrid query finds objects that are both similar to a feature vector and match some structured attributes. However, existing methods handle ANNS and attribute filtering separately, leading to inefficiency and inaccuracy. The paper proposes a new efficient and robust framework called native hybrid query (NHQ) and two new navigable PGs (NPGs) with optimized edge selection and routing, which improve the overall ANNS performance.

STRENGTHS:
1. Optimized edge selection and routing are proposed which is efficient for ANNS problems.

2. The authors perform a sufficient complexity analysis of the proposed method. This helps readers understand the superiority and limitations of the proposed method.

3. Many experiments have been conducted. The authors have conducted experiments on multiple datasets to show their superiority in terms of accuracy, efficiency, and memory usage.

WEAKNESSES:
1. Compared to PQ-based methods, the PGs need more storage in runtime. Thus, a discussion on storage cost theoretically and experimentally of NHQ with PQ-based methods is necessary. There is still an improvement space on trade-off strategy on storage and efficiency.

2. More related works (HQ-based methods and edge selection strategies) are needed.

3. Lack of results of NHQ without edge selection strategy in ablution study.

QUESTIONS:
1. Discuss the storage cost theoretically and experimentally of NHQ with PQ-based methods.

2. Include results of NHQ without edge selection strategy. This comparison can help readers understand the strengths of the proposed edge selection strategy.

3. Include more related works (HQ-based methods and edge selection strategies).

LIMITATIONS:
Please refer to Paper Weakness.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAttribute filtering (AF) is an important part of many scenarios using nearest neighbor search. Here, each data points has a feature vector in a geometric space and also a set of attributes (e.g., data, author) and queries must be matched to nearest vectors satisfying some attribute constraints.\n\nWhile many algorithms have been studied for the classic ANNS problem, ANNS + AF is hard and needs new algorithms. Recently, there has been a flurry of attempts at this. Some of the basic approaches include filtering results of classic ANNS (which tends be yield poor results) or building separate indices for each attribute (Which leads to duplication).\n\nThis paper proposed that a better way to address this might be to create a fusion distance that incorporates geometric distance between feature vectors and suitably normalized similary score between attribute vectors. They then argue that a proximity graph data structure can be built using this distance. And that this performs better than other baselines selected in the paper.\n\nSTRENGTHS:\nThe authors identify and articulate an important problem.\nEmpirical comparisons are made with many baselines.\n\nWEAKNESSES:\nThere is no description of the dataset design in the main section. In appendix O, the datasets are described as usual vector datasets with 3 attributes (e.g., date, location, size) It looks like each vector can only have one possible combination of the attributes. So one can model it is one attribute dimension (cross-product) In such a case, why it not be easier and faster to build separate indices for each possible choice of attributes? Isnt complex index design only needed when datasets have multiple labels with an attribute dimension? In any case, it is impossible to evaluate the algorithm without well motivated datasets.\n\nMissing baseline: filter-diskann [www'23], methods therein and equivalent code (in-memory and on-disk) are not compared. Instead weaker baselines are compared.\n\nThere is far too much greek notation. The notation can be greatly simplified for better readability. Simple pseudocode would also make for much better reading.\n\nQUESTIONS:\nIn preliminaries, can you clearly state the attribution filtering problem and the defintion of recall for attribute filtering problem? It is unclear how the attribute vectors or indexed data and queries are matched.\n\n\n Please describe the datasets clearly in the main section. Please describe the encoding l(.) function for your datasets.\n\nwhy are proximity graphs (PG) and not navigation graphs (e.g., NSG, HNSW) not the starting points for your construction. Proximity graphs tend to be disconnected even for non-AF use cases. Would like to see some analysis of connectivity of PG on AF-datasets.\n\nLIMITATIONS:\n-", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}