Hypergraph Propagation and Community Selection for Objects Retrieval
Authors: Guoyuan An
Yuchi Huo
Sung-eui Yoon
Venue: NeurIPS 2021 Poster (2021)
Keywords: Image search
Landmark retrieval
Uncertainty
Diffusion
Query expansion
Matching
Hypergraph

Abstract: Spatial verification is a crucial technique for particular object retrieval. It utilizes spatial information for the accurate detection of true positive images. However, existing query expansion and diffusion methods cannot efficiently propagate the spatial information in an ordinary graph with scalar edge weights, resulting in low recall or precision. To tackle these problems, we propose a novel hypergraph-based framework that efficiently propagates spatial information in query time and retrieves an object in the database accurately. Additionally, we propose using the image graph's structure information through community selection technique, to measure the accuracy of the initial search result and to provide correct starting points for hypergraph propagation without heavy spatial verification computations. Experiment results on ROxford and RParis show that our method  significantly outperforms the existing query expansion and diffusion methods.

Hypergraph Propagation and Community Selection for Objects Retrieval
Guoyuan An', Yuchi Huo and Sung-Eui Yoon
School of Computing, KAIST
2Zeejiang Lab
3 State Key Lab of CAD&CG, Zhejiang University
Abstract
Spatial verification is crucial technique for particular object retrieval. It utilizes spatial information for the accurate detection of true positive images. However, existing query expansion and diffusion methods cannot efficiently propagate the spatial information in an ordinary graph with scalar edge weights, resulting in low recall or precision. To tackle these problems, we propose novel hypergraph- based framework that efficiently propagates spatial information in query time and retrieves object in the database accurately. Additionally, propose using the image graph's 's structure information through community selection technique, to measure the accuracy of the initial search result and provide correct starting points for hypergraph propagation without heavy spatial verification computations. Experiment results ROxford and RParis show that method significantly outperforms the existing query expansion and diffusion methods.
Introduction
Image search is fundamental problem in computer vision with numerous applications such as
content-based image browsing [6 19 1. visual localization 2 30,, and 3D reconstruction [15] (1)
To retrieve the database images that contain the same object of query, a search engine represents images using global or local features [5] 21 33 26 3] that describe their similarities, and then performs a nearest neighbor search for the query image.
An object is depicted in various conditions in the real world 25 Global features of images depicting the same object form manifold in the feature space 39 Faclidean-distance--ass
nearest neighbor search 220 4 does not work well on this manifold. To improve the retrieval performance, query expansion 13 27 (1 issues new query by aggregating useful information of nearest neighbors of the prior query. Unfortunately, this approach only explores the neighborhood of very similar images, resulting in low recall [16]. Diffusion [16 39 explores more images on a neighborhood graph of
the dataset. It, however, adopts false positive items in practice, leading to a low precision (2]..
Spatial verification [23] uses spatial information to detect the true positive images. However, due to its heavy computational overhead, it is impractical to apply spatial verification to all the images during diffusion. This paper aims pre-compute the spatial matching information of database images offline and quickly propagate them for given query online. To achieve this goal, we propose hypergraph propagation model. Our model efficiently builds hypergraph for given query and propagates the spatial relationship with the query through hyperedges in a sequence of images.
When the query image is not in the database, diffusion represents it using its top-ranked images in the initial search [16] 37 However, not all the top-ranked images contain the object the query [16 25 Some works
25
show that applying spatial verification before diffusion improves the retrieval
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

performance, but it requires heavy computational overheads [25] paper is the proposal of an initial search technique, community selection, which reduces the overhead of spatial verification while maintaining high accuracy.
The second contribution of this
Experimental result shows that our hypergraph propagation significantly outperforms the existing query expansion and diffusion methods. On the hard protocol of the ROxford dataset, our hypergraph- propagation approach achieves the impressive mAPs of 73.0 and 60.5 with and without RIM distractors, respectively.
In conclusion, our contributions are: propose novel hypergraph model efficiently propagate the spatial information in a sequence of images to improve the search performance; 2) we propose community selection to predict the accuracy of the initial search and provide correct starting points for hypergraph diffusion; 3) we achieve state-of-the-art performance on the ROxford dataset with and without RIM M distractors.
2 Background
2.1 Query expansion and diffusion: relation based search
Query expansion (QE) [9] aggregates the top-ranked initial candidates into an expanded query, which is then used to search more images in the database. The critical factor here is how to aggregate only helpful information and cull out unrelated information.
Average query expansion (AQE) [9] mean-aggregates the top k retrieved images as the expanded query. Average query expansion with decay (AQEwD) [13 gives the top k images the monotonically decaying weights over their rank, Alpha query expansion (aQE) [27 uses the power-normalized similarity between the query and the top-ranked images the aggregation weights. Discriminative query expansion (DQE) 3 also uses the weighted average, where the weight is the dual-form solution of SVM, which classifies the positive (top-ranked) and negative (low-ranked) images of query. Learnable attention-based query expansion (LAuIE)) [4 uses self-attention (34] to share information between the query and the top-ranked items to compute better aggregation weights.
Unfortunately, these QE methods only explore the neighborhood of very similar images of the query. The recursive methods 9 improve the results by recursively generating new queries from all spatially verified results returned so far. However, this is at the cost of increased query time.
Diffusion 39 16
ameliorates the problem mentioned above by propagating similarities through a pairwise affinity matrix. The works of Chang al. (6) and Liu er al. (17) are two novel variants of this line. The critical issue in diffusion is how to calculate the affinity matrix. Some works [24 [6] use the reciprocal neighborhood relations to refine the search results, and other 2 6
use the inliers number of SP to re-weight the similarity. However, all of these methods compress the relations between two images as a single scala. As a result, these prior methods lost the spatial matching detail. Although the propagation methods are elegant, they have difficulties recovering good relations to guide the diffusion direction.
We propose a novel and straightforward hypergraph model to utilize the relations among database images better. It connects the corresponding local features among images using hyperedges; in sequence of similar images, it detects the image from which the following images are no longer relevant the query.
2.2 Local descriptors and spatial verification
Local descriptors are the representations of an image's important patches. Some popular local descriptors are RootSIFT [3], deep local feature (DELF) [2]] and deep local and global feature
(DELG) [.
Local descriptors' location information is usually used in the spatial verification (SP) 22 stage. standard SP pipeline first using a KD-tree [22] to find the nearest neighbors between local descriptors in two images. Then it uses the RANSAC algorithm [12) to estimate the homography matrix [32] and the number of inlier correspondences between two images. Existing image search engines commonly apply SP on the shortlist, say the top 100 images ranked by global descriptor.

---

Y2
77
76
Yo
Intra-image Hyperedge
X3
X2
X1
X2
e
Y/2
V11
Inter-image Hyperedge
X4
X5
X4
X5
33
True Positive
6
44-5
Y4-6
V9
True Positive
y1
y
X5
False Positive
e
Y12
Y7-8
110-1
True Negative
X4
Ambiguity
Resolved
Ordinary Graph Propagation
(b) Hypergraph Propagation
Figure I: a) shows a part of an ordinary graph with scalar-weighted, i.e., similarity, edges. Orange frames the common visible regions among images N1, X2, X3, and X4. Purple frames the common visible regions between images X2 and X5. X3 and X; are close neighbors to image X2. While X3 related X₃ by sharing the orange frame, X5 is not. Utilizing scalar-weighted edges cannot propagate the query in the ordinary graph without this ambiguity issue. b) shows the corresponding hypergraph of a). Inter-image hyperedges e! are shown in yellow, intra-image hyperedges ef are blue, and local features y, are in green. A hypergraph path connects local features from y1 to Yy in X1 and X3, but no path connects local features in X1 and X5. large version of this figure is in the appendix.
SP is known to be crucial for verifying true positive images (25 Performing SP before diffusion improves the final retrieval performance [25 6 However, it is also the slowest process [ 23 in image search. In this paper, we propose community selection reduce the computing overhead of SP. Community selection frames the diffusion initialization task as selecting a community for the given query rather than verifying the top-ranked images one by one. Experiment shows that community selection reduces much SP overhead without hurting the accuracy.
3 Overview
We first discuss issues of query expansions and diffusion process of ordinary graphs in Sec Sec and novel hypergraph based propagation (Sec. 3.2) and initialization scheme utilizing the concept of community (Sec. 3.3) for achieving accurate diftusion process.
3.1 Propagating in ordinary graph
We can use graph G (X, F) to represent the database images. x (Xi,..., 3 ) is the node set, where item x; could either an image or an image region depending on chosen retrieval method; F is the set of edges whose scalar weight represents the similarity between two nodes connected by each edge. Utilizing information in this ordinary graph is an important strategy to improve retrieval results, and, thus, widely used in various methods, such query expansion 9 27 [4] and diffusion 16 39). Both diffusion and query expansion can be represented by the following function:
r= I(f").
where / represents the post-processing process, fu. E IR. is the ranking score for x,,, 12 is the initial ranking vector with f = if x, query and f = 0 otherwise, and f" is the ranking vector after query expansion or diffusion. Diffusion methods diffuse the label the query node to its neighbors along the graph edges until reaching stationary state. The stationary state is found by repeating the iteration function: +++ off (1 a ff",, where O is the normalized affinity matrix that describes the weights of graph edges, a a jumping probability. Query expansion gathers useful information in the query node's nearest neighbors update the ranking list.
Diffusion and query expansion have different advantages and disadvantages. Diffusion can efficiently propagate information to the whole graph through compact formulation. However, the propagation is

---

conducted through the affinity matrix O 339 16], whose scalar value cannot represent accurate spatial information, resulting in low precision. Take Figure aas an example, where images of x and x; are incorrectly connected through image X2 in the ordinary graph with scalar edge weights; note that they do not share common regions. This leads to false positives in the diffusion process. We denote this as the ambiguity problem of propagation. In contrast, query expansion uses sophisticated approaches, such as transformer [[4] and spatial verification O 31. to guarantee that only shared information between query and its nearest neighbors are used to update the ranking list. This approach, however, only explores the neighborhood of very similar images due to the computational overhead [25] 9 resulting in low recall.
This paper proposes novel hypergraph propagation and community selection, settling the ambiguity problem separately in the propagation and initialization stages. We describe these two techniques next.
3.2 Extension to hypergraph
One meaningful direction to overcome the drawbacks of diffusion and query expansion is to handle the ambiguity problem of graph propagation by explicitly resolving the propagation in a more informative space. The local features contain informative spatial information that can help to distinguish the ambiguity through geometric verification [22]_ However, considering the large scale of the image database, it is impractical to do the geometric verification
all the pairs. In practice, image search engines usually build the kNN graph instead of the complete graph, which means every image only connects with its top k similar images (16). To utilize the spatial information on the kNN graph, we propose a novel hypergraph propagation model to efficiently propagate spatial information through hyperedges, connecting an arbitrary number of nodes. This technique appropriate for the real- world situation where the same object in different images has a different number of local features. In the example of Figure [[] image X1 can connect image X3 by matching the local features with those of image X2, but will not connect image X5.
When query is given, we define hypergraph H (Y, E) on top of the ordinary image graph, where its node set y = [y,,,.., Yn contains all the local features extracted, and the edge set E consists of hyperedges, which are used connect the local features related to the query object in different images, Our model then propagates the label of local features of the query to their neighbors and uses an aggregation function find the final ranking list of images, as follows:
1" = [,))) r = (I),
(2)
where 12t and I are the initial and final ranking lists of local features respectively, 1" is the ranking list of images, Jn is the propagation function in a hypergraph, and Agg is the aggregation function that converts ranking scores of local features to ranking scores of images. We will discuss the construction of hypergraph in Section 4.1 and the propagation process in Section 4.2
3.3 Handling new queries: community selection
Diffusion methods standard approach to handle a new query is representing the query using its top-ranked images in the initial search start the propagation [16]. However, is hard to guarantee that these global descriptor- based nearest neighbors contain the same object as the query. We call it the ambiguity problem of initialization. Recent studies [25] 6 perform online spatial verification on 100 nearest neighbors of the query and delete the wrong neighbors to improve the quality of diffusion. Unfortunately, conducting spatial verification for 100 images in query time is a heavy computational load for a search engine.
38
usually consider the query point to be contained in the dataset. The
We find that the structure of the image graph 223 contains helpful information to measure the accuracy of the nearest neighbors. Following the term in graph theory, we call tightly connected groups communities, which are sets of nodes with many connections inside and few to outside. We observe that the items containing the same objects have high probability of composing a community and having short geodesic distances. Based on this observation, we frame the diffusion initialization task as selecting community for the given query instead of using spatial verification to delete the wrong neighbors. This graph-based approach improves both accuracy and speed. We will give the detail of community selection in Section 4.3

---

4 Methods
4.1 Construction
[x1: Xm where N2 is an image, and its corresponding K-nearest neighbor (Knn) graph (39 10 G = (X,F), where F is the ordinary edge sets. For hypergraph propagation, we define a hypergraph H = (Y.E), E). where Y = (y.:...,y,, .,,, is the set of vertices and y; represents a local feature in an image. E = E U is the union of the inter-image hyperedges set E = e and the intra-image hyperedges set E² = {ef, eg....e].. An inter-image hyperedge connects local features in two images; we also say it connect two images in this paper. An intra-image hyperedge connects local features in the same image. These hyperedges are shown by the orange and blue groups in Figure b). The hypergraph has two incidence matrixes s' = (s and S (s ). where % is 1 if y; E e and 0 otherwise. We build the ordinary Knn offline and build the hypergraph on top of the Knn graph online. We call the local features in the hypergraph activated local features. We now describe how constructe the hypegraph.
Assume that we have an image dataset X
In order to solve the ambiguity problem of graph propagation, hypergraph propagation propagates only among the spatially matched local features of images. This paper uses RANSAC [12] the most common features matching method in the spatial verification stage of image search 225 21 5. to pre-compute the homography matrix [12) between two images offline. We then use these pre- computed homography matrixes to verify if two features in two images are matched when building the hyperedges at query time.
We assume the query image can be represented by local features in and will discuss how to handle a new query in Section 4.3 Because a propagation process of a query is progressive, we expand hyperedges on-demand. To judge if an image is related to the query in a sequence of similar images, we only need to see if it has the activated local features. An inter-image hyperedge is then defined as follows:
Definition 4.1 (Inter-image hyperedge). For two images x. and Xg, let V. C Y be the activated local features set of Xu, and I, C be the local features set of xu. The inter-image hyperedge among features in x, and x, is: e = ( In = Vu. OUT = ( (y, E Ry, TN is matched with y, by the pre-computed homography between x,, and x,
If the matching inliers between two images are less than threshold, they could be directly verified as irrelevant with each other. As the target of hypergraph is to search the related images for the query, we do not build the hyperedge for them. We set the threshold as 20 in this paper, which is the standard threshold in the spatial verification stage of the image search systems [23 7.. Note that even an image has more than 20 matching inliers with another one, its activated local features may not be matched.
Similar to object detection 288 we use bounding box to describe the object region. In image, local features depicting an object locate in the object's bounding box. As a homography does not guarantee to find all the matched features between two images, the inter-image hyperedges do not connect all the features in an object's bounding box. Unfortunately, the unmatched related features also important for propagation. As shown Figure features y1, Y2 and y, are in the same region and describing the same building, but only y1 and Ys are connected within image X2 using an inter-image hyperedge to image X1. Without an intra-image hyperedge connecting Y1 Y2 and y3. we cannot propagate information from images x1 to X3. To recover the unmatched yet related local features, we define the intra-image hypergraph follows:
Definition 4.2 (Intra-image hyperedge). Given a feature set in an image X let V. C Y be the activated local features set of X., and I, C Y be the set of all the local features of x,, Let the smallest rectangle covering all the features in V,, be the bounding box b. The intra-image hyperedge in x,, is defined as e IIN- = Vu, OUT {y, E Jui;, exists in b})).
Due to the high computing cost, it is impractical perform the propagation on large-scale dataset consisting of millions of images. Inspired by the idea of truncation (16) in diffusion, we build inter-image hyperedges only among the hop-N neighbors ] of given query image. Specifically, we build the inter-image hyperedges from image x,, to x,, only if both of them are in the hop-N neighbors of the query.
Node A is in B's hop-N neighbors if A can be connected with B using not more than N different hyperedges.
5

---

4.2 Propagation
After building the hypergraph, we use a propagation model in the hypergraph to achieve the final ranking scores of retrieved images. Specifically, we define propagation matrix P= (Ptz) and set the propagation weight between y, E X,, and y, E X, as Pij = d(x,, x.) maxate (sta &uu )c) where d(xu, X.) is the Euclidean distance between the global features of X and X and $
maxabe (s ta ha bee records whether y; and y, are connected in the hypergraph. The propagation matrix is normalized using the degree matrix = (tfag(P1,) as P = PD. For aggregation function, aju E A is set b. y; E Xu. and 0 otherwise, where 6,, is the number of activated local features in X.. Then the propagation function in Equation 2 can be written as:
rr Ln (1)) 1 1'p' 10p'2
1"p' rr Agg(1") I'A.
(3)
Intuitively, this propagation function gives high scores to the images with shorter hypergraph geodesic distances the query while considering similarities of global features.
4.3 Community selection for new query
The hypergraph propagation described in Sec. 4 assumes the query is in the database, which may not be true in general retrieval scenario. Now consider how to find the good starting nodes for new query.
As discussed in Section 3 observe that the items containing the same object usually belong to the same community in G. By checking whether the top S items of the initial search are in the same community,
evaluate the quality of the initial search without query time spatial verification. As shown in Figure [ the quality of the global-feature- beasd search of Q1 is better than Q2 because its top-ranked items in the same community.
Specifically, after the initial search, we use the top images and their connected edges in to build a subgraph G.. G. consists of one more components, and each component represents community G. We call the top-1 image of initial search the dominant image, its component in G2 the dominant component, and its community in G the dominant community. We call other components in G2 the opposition components. Intuitively all the top S images in G2 constitute a single component, all of them may contain the same object with the query image. Otherwise, the uncertainty will increase if the number and size of the opposition components increase. This observation also agrees with our experimental results. Inspired by Shannon entropy, define the uncertainty index, U, of the initial search as:
no
U -=- -
log P(C.),
(4)
where ng is the number of component in G,, C2 is a component and P (C.) is the proportion of C, in G..
Community selection is the pre-stage of hypergraph propagation. The overall retrieval process with community selection is to perform the initial search firstly and then calculate the uncertainty of the initial search result. If the uncertainty index low, say U lower than 1, hypergraph propagation uses images in the dominant community to represent the query image and starts the propagation process from them. If the uncertainty index high, the search engine does the spatial verification on
Figure 2: Two queries on an image graph with three communities. The numbers on the nodes are their rankings in the initial search. The uncertainty of the initial search result of Q1 is lower than that of Q2 as most retrieved items of QI distribute in the same community.

---

the top T images of the initial search to find a new dominant community for the query. In this way, we significantly reduce the number of spatial verification while keeping very high accuracy.
Experiment
5.1 Implementation
We compare hypergraph propagation against the existing QE/diflusion methods: average query expansion [9] average query expansion with decay [13], query expansion and diffusion [. We evaluate the performance of hypergraph propagation under the ordinary retrieval pipeline: performing the initial search, followed by conducting QE/diffusion. The focus of hypergraph propagation is on improving the QE/diffusion process, so we do not consider more complex approaches such repeating QE and diffusion in different stages (c.g., initial search+ QE spatial verification + Diffusion [6] and ASMK + spatial verification + Diffusion [25 )- Fortunately, our system can be combined with other methods and produce stronger performance thanks to its generality. We do not use community selection and always start the hypergraph propagation from the initial dominant community for all the queries make the comparison fair.
For hypergraph propagation and all the compared QE/diffusion methods, we use the DELG model [5] trained on GLDv2 [35] to extract both global and local features. We find the nearest neighbor number K does not influence the final result a lot, and we report the performance of setting K as 200. After the propagation, we reorder the initial ranking list using f" to get the final results.
For community selection, we S, the number of images for calculating the uncertainty, as 20. When testing the performance of combining community selection and hypergraph propagation, we set the uncertainty threshold as I, which we find is good balance the accuracy and computation overhead. For the query with an uncertainty index higher than the threshold, we do the spatial verification for the top 100 items using SIFT features [18]. Once we find an item with more than 20 inliers, we treat its community hypergraph propagation in the new dominant community. The code of this work is publicly available on httpss://gvr kaist. kr/
the new dominant community. stop the spatial verification, and apply
We evaluate our methods on two well-known landmark retrieval benchmarks: revisited Oxford (ROxf) and revisited Paris (RPar) [25] There are 4993 (6322) database images in the ROxf (RPar) dataset, and different query set for each dataset, both with 70 images. We also report the result with the RIM distractor set, which contains IM images. The accuracy is measured using mean average precision (mAP). We conduct the offline process on Intel(R) Core(TM) 19-9900K CPUs @ 3.60GHz and 896GB of RAM and implement the online hypergraph propagation and community selection one CPU and 50GB of memory.
5.2 Comparison to the state-of-the-art
Table compares the accuracy of hypergraph propagation and other QE/diffusion methods. Our method obtains the best result on ROxford and Raaris, both with and without the RIM distractor set. Compared to RParis, ROxford more challenging as contains more difficult retrieval cases such as the change of viewpoint 25 The performance improvement of our method on ROxford hard cases is impressive; we get the mAP 70.3 and 60 with and without IM distractors, outperforming the traditional diffusion method 18.5% and 55%, respectively. To the best of our knowledge, these are the state-of-the-e MAP even compared with other techniques methods integrating additional steps 25 6
with more computation and memory requirements.
Ordinary diffusion better than QE methods on all the datasets except ROxford with RIM distractors. We think that the texture and shape of medieval buildings in ROxford are common and tend to appear in other buildings around the world. When retrieving these objects with RIM distractons, its ambiguity problem explores more false positive items, leading to a low mAp than QE methods. Our hypergraph propagation keeps the spatial information during propagation, thus avoiding the false positive issue of ordinary diffusion, resulting in high accuracy.
7

---

Table 1: Results (% mAP) the ROxf/RPar datasets and their large-scale versions ROxf+IM/RPar+IM, with both Medium and Hard evaluation protocols.
ROxf
ROxf+RIM
RPar
RPar+ + I
Method DELG (Global)
H
H
M
M
5
76.3
55.6
63.7
37.5
86.6
72.4
70.6
46.9
DELG (Global + Local) ) 5
81.2
64.0
69.1
47.5
87.2
72.8
71.5
48.7
Average QE
77.2
57.1
68.5
43.0
87.6
74.3
75.4
54.8
Average QE with decay 13
78.4
58.0
70.4
44.7
88.2
75.3
76.2
56.0
QE 27
65.2
43.2
57.0
30.2
91.0
81.2
81.0
64.1
Diffusion [66
81.0
59.3
63.9
38.7
91.4
82.7
80.0
64.9
Hypergrah Propagation (Ours)
85.7
70.3
78.0
60.0
92.6
83.3
86.6
72.7
5.3 Effectivness of community selection
Figure 3 shows the mAP of the initial search results in ROxford with different uncertainty thresholds. As the uncertainty increases, the mAP of the initial search result decreases. The uncertainty index predicts the quality of the initial search without any user feedback. Only for the high uncertainty retrieval results, use the computationally heavy approach, i.e., spatial verification, to improve the precision nearest neighbors before the hypergraph propagation.
Table 2 compares the performances between with and without community selection (CS) as the pre-stage when using spatial verification (SP) to initialize the hypergraph propagation (HP). With CS, the search engine improves the performance of hypergraph propagation in ROxford very efficiently by only conducting SP on 4 and queries with and without RIM distractors, respectively, The performance improvement with RIM distractors is less than that without them. We think its reason is that the correct items with the RIM distractors are ranked outside the top 100 images and thus are not detected by SP. Although CS does not improve the retrieval results of these cases, it successfully tags them as uncertain cases. We think this result leads to a promising future direction. Because once a search engine can automatically detect the failed query cases, it is possible to refine the representation ability by relabeling and training only for the failed cases instead of training on a large dataset.
Thanks to the improved representation ability of DELG (5) features, the nearest neighbors of all queries RParis dataset have high precision. The average precisions of the top 20 items of the initial result of all the queries in RParis are 0.97 and 0.90 in medium and hard protocols, respectively. As the initial dominant communities for all the queries in RParis are correct, online SP is not helpful for the propagation process. CS avoids these unnecessary SPs in these cases as the uncertainties of all the queries are low, shown in Table CS significantly improves the retrieval performance for some hard cases. Figure 4 shows an impressive example in ROxford, in which the mAP increases from 0.077 and 0.062 to 0.988 and 0.987 in medium and hard protocols, respectively.
In summary, the main benefits of CS include: 1) CS quickly detects the low-quality retrieval result, which is either because of the difficult query or the insufficient feature representation ability; 2) CS avoids the unnecessary computationally heavy spatial verification process; 3) CS provides significant performance improvement for some hard cases.
Discussion
While we have shown benefits of our method, it has limitations and interesting future work. First, matching using the homography matrix calculated by RANSAC might not the best option for
0.8
0.6
mAp
0.4
0.2
ROxford
ROxford R1M
M
0.5
1
1.5
2
2.5
1.5
2.5
Uncertainty threshold
Uncertainty threshold
Figure 3: mAP of the query results above the uncertainty threshold. Uncertainty predicts the query quality well.

---

Table 2: Comparison between with and without community selection (CS) as the pre-stage when using spatial verification (SP) to initialize the hypergraph propagation (HP). This table shows both the mAP and the total number of queries which need SP when testing the corresponding dataset.
HP
CS + SP + HP
SP + HP
MAP
mAP
total SP
mAP
total SP
ROXf(M/H)
85.7/70.3
88.4/73.0
4
88.4/73.0
70
ROxf+R (M//H))
78.0/60.0
79.1/60.5
7
29.1/60.3
70
MP/H
92.6/83.3
92.6/83.3
0
92.6/83.3
70
RPar+R (M/HH))
86.6/72.7
86.6/72.7
0
86.6/72.7
70
hypergraph propagation. It does not consider local features' contextual cues when applying matching, and has the offline homography calculation overhead. Recent deep-learning-hasse matching techniques such as SuperGlue [29] show performance improvement in many tasks in terms of both speed and accuracy and can also be helpful for our hypergraph propagation. Second, to achieve better result, both hypergraph propagation and community selection can be combined with existing query expansion, diffusion, and spatial verification methods. For example, it is possible to adapt region diffusion [10 to the hypergraph model adapt community selection on ordinary diffusion methods.
Image search techniques might be related to surveillance techniques. But our work does study human-related problems, so we think the negative social impact is minor.
Overall, we believe that our hypergraph propagation method combined with the community selection technique takes a meaningful step towards designing highly accurate image search techniques. Especially the hard protocol of the ROxford dataset, hypergraph propagation achieved the impressive mAPs of 73.0 and 60.5 with and without R RIM distractors, respectively. To the best of our knowledge, these the ttate-of-the results.
Query
The initial search result ( uncertainty: 1.77, Medium MAP: 0.077, Hard mAP: 0.062)
The result after CS and (Medium mAP: 0.988, Hard MAP: 0.987)
Figure 4: In this example, the top 11 images and the initial dominant community is unrelated to the query. Its uncertainty index is 1.77, which means the search engine is very uncertain about the accuracy of initial dominant community. So CS finds a new dominant community using spatial verification. After hypergraph propagation in this new dominant community, the retrieval performance is significantly improved.

---

Acknowledgments and Disclosure of Funding
We would like to thank the anonymous reviewers for their valueable comments and insightful sug- gestions. Sung-Eui Yoon and Yuchi Huo are co-corresponding authors of the paper. This work was supported by Institute of Information & communications Technology Planning & Evaluation (ITTP) grant funded by the Korea government (MSIT) LUTP-2015-0-0019, Proximity computing and its applications to autonomous vehicle, image search, and 3D printing) and the National Research Foun- dation of Korea (NRF) grant funded by the Korea (DV (No. 2019R IAC3300283))
References
[1] Stefano Alletto, Davide Abati, Giuseppe Serra, and Rita Cucchiara. Exploring architectural details through wearable egocentric vision device. Sensors, 16(2):237, 2016.
[2] Relja Arandjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Netvlad: Can architecture for weakly supervised place recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5297-5307, 2016.
[3] Relja Arandjelović and Andrew Zisserman. Three things everyone should know to improve object retrieval. In 2012 IEEE Conference on Computer Vision and Pottern Recognition, pages 2911 1-1988. IEEE, 2012.
[4] Artem Babenko and Victor Lempitsky. Efficient indexing of billion-scalc datascts of deep descriptors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3055-2063, 2016.
[5] Bingyi Cao, André Araujo, and Jack Sim. Unifying deep local and global features for image search. In European Conference on Computer Vision, pages 723-743. Springer, 2020.
[6] Cheng Chang, Guangwei Yu, Chundi Liu, and Maksims Volkovs. Explore-exploit graph traversal for image retrieval. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9423 9431, 2019.
[7] Ondrej Chum et al. Large-scale discovery of spatially related images. IEFE transactions pattern analysis and machine intelligence, 32(2):371-377, 2009.
Ondfej Chum, Andrej Mikulik, Michal Perdoch, and Jifí Matas. Total recall i:: Query expansion revisited. In CVPR 201 pages 889-896. IEEE, 2011.
[9] Ondrej Chum, James Philbin, Josef Sivie, Michael Isard, and Andrew Zisserman. Total recall: Automatic query expansion with a generative feature model for object retrieval. In 2007 IEEE 11th International Conference on Computer Vision, pages 1-8, IEEE, 2007.
[10] Wei Dong, Charikar Moses, and Kai Li. Efficient k-ncarest neighbor graph construction for generic similarity measures. In Proceedings of the 20th international conference on World wide web, pages 577 777-586,20 2011.
[11] Michacl Donoser and Horst Bischof. Diffusion processes for retrieval revisited. In Proceedings of the IEEE conference on computer vision and pultern recognition, pages 1320-1327, 2013.
[12] Martin A Fischler and Robert C Bolles. Random sample consensus: paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-39 38555. 1981.
[13] Albert Gordo, Jon Almazan, Jerome Revaud, and Diane Larlus. End-to-end learning of deep visual representations for image retrieval. International Journal of Computer Vision, 124(2):337-554 2017.
[14] Albert Gordo, Filip Radenovic, and Tamara Berg. Attention-based query expansion learning. In European Conference on Computer Vision, pages 172-188. Springer, 2020.
[15] Jared Heinly, Johannes L Schonberger. Enrique Dunn, and Man-Michael Frahm. Reconstructing the world* in days* captured by the yahoo 100 million image dataset). Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 3287-3295, 2015.
[16] Ahmet Iseen, Giorgos Tolias, Yannis Avrithis, Teddy Furon, and Ondrej Chum. Efficient diffusion on region manifolds: Recovering small objects with compact can representations. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 2077-200 2086, 2017.
10

---

[17] Chundi Liu, Guangwei Yu, Maksims Volkovs, Cheng Chang, Himanshu Rai, Junwei Ma, and Satya Krishna Gorti. Guided similarity separation for image retrieval. 2019.
[18] David Lowe. Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference on computer vision, volume 2, pages 1150-1157 1157. Ieee, 1999.
[19] Andrej Mikulik, Oudiej Chum, and Jif Matas. Image retrieval for online browsing in large image collections. In International Conference on Similarity Search and Applications, pages 3-15. Springer, 2013.
[20] Marius Muja and David G Lowe. Scalable nearest neighbor algorithms for high dimensional data. IEEE transactions on pattern analysis and machine intelligence. 36(1 (6(11))2227 22220220,,222 2014.
[21] Hyconwoo Noh, Andre Araujo, Jack Sim, Tobias Weyand, and Bohyung Han. Large-scale image retrieval with attentive deep local features. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
[22] James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman. Object retrieval with large vocabularies and fast spatial matching. In 2007 IEEE conference on computer vision and pattern recognition, pages 1-8. IEEE, 2007.
[23] James Philbin and Andrew 7isserman. Ohject mining using a matching graph on very large image collections. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 738-745. IEEF, 2008.
[24] Danfeng Qin, Stephan Gammeter, Lukas Bossard, Till Quack, and Lue Van Gool. Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors. In CVPR 2011, pages 777-784. IEEE, 2011.
[25] Filip Radenović, Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondřej Chum. Revisiting oxford and paris: Large-scale image retrieval benchmarking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5706-5715, 2018.
[26] Filip Radenović, Giorgos Tolias, and Ondřej Chum. Cun image retrieval learns from bow: Unsupervised fine-tuning with hard examples. In European conference computer vision, pages 3-20. Springer, 2016.
[27] Filip Radenović, Giorgos Tolias, and Ondřej Chum. Fine-tuning enn image retrieval with no human annotation. IEEE transactions on pattern analysis and machine intelligence,
2018
[28] Shaoqing Ren, Kaiming Hc, Ross Girshick, and Jian Sun. Faster r-cnm: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01 497, 2015.
[29] Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz. and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pages 4938 4947, 2020.
[30] Torsten Sattler, Tobias Weyand, Bastian Leibe, and Leif Kobbelt. Image retrieval for image-based localization revisited. In BMVC, volume 1, page 4, 2012.
[31] Johannes L Schonberger, Filip Radenovic, Ondrej Chum, and Jan-Michael Frahm. From single image query to detailed 3d reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5126-5134, 2015.
[32] Richard Szeliski. Computer vision: algorithms and applications. Springer Science & Business Media, 2010.
[33] Giorgos Tolias, Ronan Sicre, and Hervé Jégou. Particular object retrieval with integral max-pooling of cnn activations. arXiv preprint [t1i::5111.0877. 2015.
[34] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. arXiv preprint arXiv:1706.03700 037622 2017.
[35] Tobias Weyand, Andre Araujo, Bingyi Cao, and Jack Sim. Google landmarks datasct v2-a large-scale benchmark for instance- recognition and retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 2575-2584, 2020.
[36] Tobias Weyand and Bastian Leibe. Discovering favorite views of popular places with iconoid shift. In 2011 International Conference Computer Vision, pages 1132 1139. IEEE, 2011.
11

---

[37] Fan Yang, Ryota Hinami, Yusuke Matsui. Steven Ly, and Shin ichi Satoh. Efficient image retrieval via decoupling difflusion into online and offline processing. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pages 9087-9094, 2019.
[38] Dengyong 7hou, Olivier Bousquet, Thomas N I La, Jason Weston, and Berhhard Schölkopf. Learning with local and global consistency. In Advances neural information processing systems, pages 321-328, 2004.
[39] Dengyong Zhou, Jason Weston, Arthur Gretton, Olivier Bousquet, and Bernhard Schölkopf. Ranking on data manifolds. Advances neural information processing systems, 16: 16:1669,76, 2004.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information how to answer these questions. For each question, change the default [TODO] to [Yes]. [No], , or [N/A] . You are strongly encouraged to include justification your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
Did you include the license the code and datasets? [Yes] See Section ??.
Did you include the license the code and datasets? [No] The code and the data proprictary.
Did you include the license the code and datasets? [N/A]
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 6
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
12

---

(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "v3Ew1FgrSoQ", "reviewer": "Reviewer_2ivT", "summary": "The paper addresses the object retrieval problem from images. Traditional query expansion methods can only utilize very similar images, resulting in low recall. On the other hand, diffusion propagation methods on a image graph often ambiguity problem of propagation. Instead, spatial verification has to be used for the accurate detection of true positive images. However, in an ordinary graph with scalar edge weights, spatial verification cannot be done. Furthermore, spatial verification is very computationally heavy. To address the problem, a hypergraph is proposed to map image's internal structural information, and a community selection technique is employed for hypergraph propagation without heavy spatial verification. Experiments are conducted which show promising results.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strong points:\n- The solution is well motivated, intutive and reasonable.\n- The paper is well presented and easy to follow, even for someone not specifically working on object retrieval\n- Experiments show that the solution is promising, with both quantitative and qualitative results.\n\nWeak points:\n- The solution appears straightforward and quite heuristic. Though technically valid, it is not sure if there is any guarantee on the community based method. \n- The community method: only top 1 image is treated as the dominant community. There could be a robustness problem if top 1 image happens to be a false positive.\n- I did not figure out how the communities are found. Different community detection method may have an impact on the retrieval performance.\n\n-- update after rebuttal --\nBased on the rebuttal, I will maintain my current rating.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "fine with me.\n\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "W8jvhpw-iRZ", "reviewer": "Reviewer_PYXJ", "summary": "This work tackles the problem of image retrieval, specifically in the domain of re-ranking with ambiguous objects. It proposes two core ideas to improve upon existing query expansion and diffusion techniques, namely\n\n1) Introduction of the hypergraph which unifies both global and local relations in the graph propagation process, hence resolving potential ambiguities.\n2) Community selection, which resolves the ambiguities in initializing the graphs.\n\nThe proposed method shows strong performance on the standard ROxf-RPar (and the 1M distractor set) datasets over existing image retrieval methods with the re-ranking process. Qualitative results are also provided to help understand the way it increases the precision of top-ranked results.", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Originality\nTo my best knowledge, the concept of hypergraphs for re-ranking is novel to the image search problem. In fact, this work is also the first attempt to combine the two existing classes of re-ranking methods: local descriptor-based geometric verification and global descriptor-based (query expansion and diffusion) into a single framework with the proposal of the hypergraph. I think this is a very elegant way of solving the existing ambiguities mentioned by the authors and the results speak for themselves.\n\nThe other novelty proposed in the work, Community selection, is less technical. However, this does not prevent it from being a good novelty as the motivation for it is intuitively explained and has the potential to be adopted in all general image retrieval pipelines.\n\n## Quality\nThe submission is technically sound. \n\nAfter assessing the methods and experiments sections, I think the contribution of both novelties is well-justified by the experimental results and ablation studies. It is truly impressive the addition of hypergraph propagation alone already outperforms state-of-the-art (SoTA) in the ROxf-RPar [25] benchmark by a huge margin (12.5% increase in mAP over DELG [5] with local re-ranking on ROxf+1M-Hard, which is notoriously difficult due to the queries having cropped regions that cause extreme zooms *v.s.* database images, a lot of ambiguous regions/textures/objects, let alone the 1-million distractors). The impact of community selection is also clearly justified by the results in Table 2, and the great addition of Figure 4 to qualitatively show how the proposed re-ranking components significantly improve the retrieval results on arguably the most difficult query among the whole dataset.\n\nThere are a few minor questions, which clarifications from the author would be greatly appreciated:\n1) It is hypothesized in L237-238 that “if all the top $S$ images in $G_s$ constitute a single component, we are very certain that all of them contain the same object with the query image”. Although this is intuitively correct, I wonder if the authors had a sanity check of whether this is actually true, *e.g.* by actually detecting and classifying the objects and checking whether they all fall into the same “community”? Although this does not affect the validity of the contribution of community selection, I think these results would be very interesting for the image retrieval community and provide domain knowledge for future works.\n2) Why are the results from [14] not included in Table 1, since they are a very recent work? It would also be interesting to see if the hypergraph propagation can be incorporated with a Transformer (just out of curiosity, not asking for extra experiments).\n3) In L277-278 the method runs on multiple CPUs and requires a huge amount of memory (896GB). \n\n    a) Would the performance drop a lot if you limit the RAM usage to a more normal amount (*e.g.* <50GB)?\n\n    b) Could the authors provide a runtime breakdown analysis, especially how long it takes to run on the 1M datasets? \n\n    c) Is GPU acceleration an option? If so, how much does it improve the runtime?\n\n## Clarity\nThe paper is very well-written, easy to follow. I believe the technical details presented allow for reproduction of the reported results.\n\n## Significance\nWith reference to the **Quality** section, I believe the contributions in the work would be impactful for the field of image/object search & retrieval, not only by advancing the state-of-the-art by a big step, but also potentially inspire a lot of future work along this line. Clarifications from the authors on 3c) above in **Quality** would be a bonus, as it helps understand whether the compute requirement would be too much for other researchers to replicate and build upon the impressive results reported in this work.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have addressed the limitations and potential negative societal impact of this work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "Ri_YGYsZ9fN", "reviewer": "Reviewer_YeE2", "summary": "This paper proposes a new query expansion method for image object retrieval. Firstly, they propose a hypergraph among local features of base images. To expand new queries, it is time-consuming to check whether the query is related to initial search results. They propose a community selection method to speed up the procedure. In experiments, the proposed method outperforms traditional methods. \n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positive points:\nThe proposed community selection method seems reasonable and is efficient in new query expansion.\n\nMy concerns and questions:\n1. The motivation for using hypergraphs is weird. At least the example in Figure 1 is misleading. In Figure 1 (a), the ordinary graph is constructed at the image level, so it leads to ambiguity. Can the ambiguity be solved by constructing ordinary graphs by local features as in Figure 1 (b)? Say, x_2 and x_5 are connected based on local features in purple frames while x_2 and x_3 are connected based on orange frames. In this way, the ambiguity would be solved. \n2. Inter hyperedges are only constructed between two images. Is it possible to construct a hyperedge involves more than two images?\n3. Section 4.1 is difficult to follow. What is the difference between \"activated local features\" and \"local features\"? What are \"IN\" and \"OUT\"? If they are concepts from other papers. They should be introduced here to let this paper be self-contained. These concepts are important for understanding.\n\nMinor problems:\n1. On my side, Figure 1 (b) is difficult to read after printing.\n2. Typo: Line 36, 'research' -> 'search'.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "5z_K78ohZm", "reviewer": "Reviewer_fAs6", "summary": "This paper proposes a hypergraph model to settle the ambiguity problem of propagation.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "originality, from technical perspectives, the proposed model is quite simple and common. No many brilliant designs are shown.\nquality, it outperforms the listed state-of-the-art methods.\nclarity, in the introduction, there is no sufficient instruction to make me understand the necessity of using the hypergraphs.\nsignificance, there are novel designs but this work also has some problems.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1, In the introduction section of the story's beginning, there is no sufficient instruction on why we have to use a hypergraph. What are the benefits of the hypergraph?\n\n2, According Fig. 1 a), the ordinary graph is not the key factor to link false-positive images, since we could also link the similar features in local ranges to break the connection between x1 and x5.\n\n3，The propagation method is still quite simple based on Euclidean distance and constrained by hyperedges. There is not an explicit difference between the proposed hypergraph and an ordinary graph.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1 day"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This seems like a good approach which reviewers considered well motivated, and with promising results. Some weak points were identified such as lack of theoretical guarantees, and minor weaknesses/lack of clarity in the presentation. Reviewers largely appreciated the author response, and it seems if the changes are made the manuscript can be of good value to the NeurIPS audience.
Decision: Accept (Poster)