Robust Graph Structure Learning via Multiple Statistical Tests
Authors: Yaohua Wang
Fangyi Zhang
Ming Lin
Senzhang Wang
Xiuyu Sun
Rong Jin
Venue: NeurIPS 2022 Accept (2022)
Keywords: Graph Structure Learning
Graph Convolutional Networks (GCNs)
Computer Vision

Abstract: Graph structure learning aims to learn connectivity in a graph from data. It is particularly important for many computer vision related tasks since no explicit graph structure is available for images for most cases. A natural way to construct a graph among images is to treat each image as a node and assign pairwise image similarities as weights to corresponding edges. It is well known that pairwise similarities between images are sensitive to the noise in feature representations, leading to unreliable graph structures. We address this problem from the viewpoint of statistical tests. By viewing the feature vector of each node as an independent sample, the decision of whether creating an edge between two nodes based on their similarity in feature representation can be thought as a ${\it single}$ statistical test. To improve the robustness in the decision of creating an edge, multiple samples are drawn and integrated by ${\it multiple}$ statistical tests to generate a more reliable similarity measure, consequentially more reliable graph structure. The corresponding elegant matrix form named $\mathcal{B}$$\textbf{-Attention}$ is designed for efficiency. The effectiveness of multiple tests for graph structure learning is verified both theoretically and empirically on multiple clustering and ReID benchmark datasets. Source codes are available at https://github.com/Thomas-wyh/B-Attention.

Robust Graph Structure Learning via Multiple Statistical Tests
Yaohua Wang
Fangyi Zhang
Alibaba Group
Queensland University of Technology
xiachen. wwweeeiaaaaaaalliianaaaaaaaaa com
Centre for Robotics (QCR) fangyi zhang@qut edu. au
Ming Lin Amazon
Senzhang Wang Central South University
Xiuyu Sun Alibaba Group
mingl www.aamzzzzz.cooom
szwangecsu. eww.eeccsuuuuuuc.omm en
xiuyu.sxy@al hachaallbbbbaa ibaba-inc com
Rong Jin Twitter rongj mengjinemi11@@mi..comm
Abstract
Graph structure learning aims to learn connectivity in graph from data. It is particularly important for many computer vision related tasks since no explicit graph structure is available for images for most cases. A natural way to construct graph among images is to treat each image as a node and assign pairwise image similarities as weights corresponding edges. It is well known that pairwise similarities between images are sensitive to the noise in feature representations, leading unreliable graph structures. We address this problem from the viewpoint of statistical tests. By viewing the feature vector of each node as an independent sample, the decision of whether creating an edge between two nodes based on their similarity in feature representation can be thought as a single statistical test. To improve the robustness the decision of creating an edge, multiple samples are drawn and integrated by multiple statistical tests to generate a more reliable simi- larity measure, consequentially more reliable graph structure. The corresponding elegant matrix form named B-Attention is designed for efficiency. The effective- ness of multiple tests for graph structure learning verified both theoretically and empirically multiple clustering and ReID benchmark datasets. Source codes are available at htttt:://// //gi /ig thub. com/ Thomas- wwhhB Attention
Introduction
Graph structure learning plays an important role in Graph Convolutional Networks (GCNs). It seeks discover underlying graph structures for better graph representation learning when graph structures are unavailable, noisy or corrupted. These issues are typical in the field of vision tasks [3X]]]] 63 62
20
As an example, there no explicit graph structure between photos in your phone albums, so the graph-based approaches (20 22 57 in photo management softwares
50 57
seek learn graph structure. A popular research direction on it is consider graph structure
"Equal contribution.
i Work done while at Alibaba. i "Correppnniigg author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

learning as similarity measure learning upon the node embedding space 160| They learn weighted as graphs and feed them to GCNs to learn
adjacency matrices 220
25 5
representations.
However, the main problem of the above weighted-based methods is that weights not always accurate and reliable: two nodes with smaller associations may have a higher weight score than those with larger associations. The root of this problem is closely related to the noise in the high dimensional feature vectors extracted by the deep learning model, as similarities between nodes are usually decided based on their feature vectors. The direct consequence of using an inaccurate graph structure is so called feature pollution [57], leading to performance degradation in downstream tasks. In our empirical studies, we found that close to 1/3 of total weights are assigned noisy edges when unnormalized cosine similarities between feature vectors are used to calculate weights.
Many methods have been developed to improve similarity measure between nodes in graph, including adding learnable parameters y
28
introducing nonlinearity ICC 1 [4], Gaussian kernel func-
42] Despite the progress, they are limited to modifying similarity function, without explicitly addressing the noise in feature vectors, a more fundamental issue for graph structure learning. One way to capture the noise in feature vectors is to view the feature vector of each node as an independent sample from some unknown distribution associated with each node. As result, the similarity score between two nodes can be thought as the expectation of a test to decide if two nodes share the same distribution. Since, only one vector is sampled for each node in the above methods, the statistical test is essentially based on a single sample from both nodes, making the calculated weights inaccurate and unreliable.
tions 25 35 and self-attention
153
An possible solution towards addressing the noise in feature vectors is to run multiple tests, a common approach to reduce variance in sampling and decision making E DI 26. Multiple samples are drawn for each node, and each sample can be used to run an independent test to decide if two nodes share the same distribution. Based on the results from multiple tests, we develop novel and robust similarity measure that significantly improves the robustness and reliability of calculated weights. This paper will show how to accomplish multiple tests in the absence of graph structures and theoretically prove why the proposed approach can produce a better similarity measure under specified conditions. This similarity measure is also crafted into an efficient and elegant form of matrix, named B-Attention. B-A Ateentio utilizes the fourth order statistics of the features, which is a significant difference with popular attentions. Benefiting from a better similarity measure produced by multiple tests, graph structures can be significantly improved, resulting better graph representations and therefore boosting the performance of downstream tasks. We empirically verify the effectivness of the proposed method over the task of clustering and ReID.
In summary, this paper makes the following three major contributions: (1) To the best of our knowledge, this is the first paper to address inaccurate edge weights problem by statistical tests on the task of graph structure learning over images. (2) A novel and robust similarity measure is proposed based on multiple tests, proven theoretically to be superior and designed in an elegant matrix form to improve graph structure quality. (3) Extensive experiments further verify the robustness of the graph structure learning method against noise. State-of-thee of-the-art (SOTA) performances are achieved on multiple clustering and RelD benchmarks with the learned graph structures.
2 Methodology
To handle the noise in feature vectors, we model the feature vector of each node as a sample drawn from the underlying distribution associated with that node, and the similarity score between two nodes as the expectation of statistical test. The test is to compare their similarity to given threshold to decide if two nodes share the same distribution and that decision can be captured by binary Bernoulli random variable. The comparison in statistical test may lead error edges, i.c., noisy edges (an edge connects two nodes of different categories) or missing edges (two nodes of the same category should be connected but are not). The superiority of the similarity measure can be revealed by comparing the probability of creating an error edge. The proposed similarity measure will be introduced in Section 2.2 and its further design in matrix form will be presented in Section
2.1 Preliminary
Chernoff Bounds. Sums of random variables have been studied for long time 4 Bounds [ provide sharply decreasing bounds that are in the exponential form on tail distributions
26 Chernoff

---

of sums of independent random variables. It is widely used fundamental tool in mathematics and computer science 322 An often used and looser form of Chernoff Bounds is follows.
Lemma 1 (Chernoff Bounds M.). Suppose X; is binary random variable taking value in set {0, 13 with P(X, 1) Pi- Define X = ΣT X; where {X1, X2, n are independently
sampled. Then for any 0 < é 1
P(X 2 (1 ()))) S Se-fu
P(X $ (1 c)p) S 5
shows that with a high probability 1- we have X $ (1+))u, or (1-e)ı4 where == =
GCNs, A GCN network normally consists of multiple GCN layers. Given an input graph 9 (X,, with L vertices, the output of normal GCN layer 34
x = =D- AD XWI), where Ā A D
A i
(1)
j=1
X RLXM is input vertex features with M dimensions, A E RIXL is their adjacency matrix; I LRL is an identity matrix, D E IR Lx is the diagonal degree matrix of A; Wi RMM is
trainable matrix, and (()) is an activation function such as ReLU 19: X &RLXM" is the output vertex features with M' dimensions, M = M in this work.
2.2 Multiple tests on similarity measure
Let V {u U2 .... UN} } be the collection of N nodes. Each node is assumed to be in one of two categories: +1 or -1 for simplicity. For each node vi the subset of candidate nodes to connect denoted by V,. This subset can be decided by a simple criterion such as by k nearest neighbours. When calculating the proposed similarity between V; and U, V., the common candidate nodes between i; and By, i... V., V; n Vj are first identified. The nodes in the Vi, are treated as the multiple samples drawn for the comparison of and vy. Then, each Uy E Ves performs single test twice, one for vk and iú and one for ei and ey. The single test here is a comparison using the pairwise similarity based on the features of and U; be cosine similarity, Gaussian kernel similarity or any other form, and denoted by Sim-S. The proposed similarity score between and vj increases by one if the two tests yield C(o:)) C(vs) and C(vx) = (vv)), and zero otherwise, where C(v) is the category of node v. contains multiple single tests, hence the name multiple tests, corresponding similarity measure denoted by Sim-M.
and 1/. The pairwise similarity can
For the convenience of analysis, few assumptions are further introduced. First, the size of Vi.j is m, and among the nodes in Vej. , am nodes share the same category as and (1 - at) m nodes belong to the opposite category, with a > $. For any node E V,, its probability of being connected with v. according to the statistical test, g E [0, 1] if C(T:) * C(u,). and the probability increases to C(tz). This section will show that, if m is sufficiently large (i.e., enough number of tests can be run), there will exist an appropriate threshold for the Sim-M such that the number of error edges can be reduced significantly compared to Sim-S.
1 12 > q if C(x))
Proposition For the single test method, in expectation, the number of noisy edges ES [Nnonyy] = (1 ax) qm and the number of missing edges E(Nmiss) = a (1 p) m.
The theorem below shows that both E(Nnoisy) } and E [(miis)) can be reduced by a significant portion when m is large enough.
Theorem 1. Suppose y > 1 and
3((p 2)²² + (2a 1) 1))) log( min((1 o)g, a(1 -p))
m pq((p q) (2a 1)(12 2)2
We have E-(Nnossy) } = (1 - a) qm/? and [E/ = a (1 (1-p)m) (1-p)m/7--
Consider node v; and one of its candidate node ny E V, St, is denoted as the similarity score between iú and vy based on the statistical test with respect uk. As shown in Figure I
Proof.
Up may
3

---

Q.1 XW
acdiltion
Audf
Quif
Katf
X.
E(St,) pq.1f ((m)
Kadf
An aofuru
AlAttantion
9
A W
Aboi
Quet
K A. W
#(S)) of 1. -0)+ (()))
Figure 1: Illustration of [ in Sim-M nodes from different categories and the same category. Nodes of different colors are from different categories.
Figure Illustration the B-Attention mechanism. The self-attention part is the same that in Transformers. The Q-Attention part generates A x and then pay attention to generate the Aqart. The two output attention maps fused as the final output Aband-
be of the same category as v; or not (denoted by v'x for a better show). Evidently, E[St,] = pq C(u;) * C(uy), and E[sk] (.j) ap² + (1 a) if ((v)) = C(vz).
The Sim-M between e; and V; is defined as:
S
Σ S
(3)
m KEV.
Using the Chernoff Bounds
we have with probability 1- 5,
Sig 1. m EkEV,, Sk +
31og(1/5)
}Pb
C(vx) # C(ty),
p477
(4)
St. 1 ExEV,, Sx 2 (1 (app F(1 =l1" ym (ap² + +(1 a))a),
3log(1/5)
C(os) = C(t)),
By assuming that m is large enough, i.e.,
3 log (1/8) pqm
(p 9)- (2a 1)p
(5)
(p + q)a + (2a 1) q92)
by choosing the similarity threshold
3log(1/8)
pq
3log(1/6) (ap² +11 2)
ap² + (1 =)
S = (1+ V
pqm
+
V
2
(6)
only with probability 6, we will miss correct edge, and with probability 5 we will introduce noisy edge. We complete the proof by substituting 5 in Equation 5 with
8 min((1 a)g a11 -)))
We can guarantee that the Sim-M between two nodes from the same category is strictly larger than that for two nodes from different categories, leading the distillation of all noisy edges.
2.3 Multiple tests on GCNs
With a better similarity measure prototype above, we further craft into an elegant and efficient matrix form for adaptation of graph structure learning in the real-world. The analysis in Theorem can be

---

easily extended to real similarities, instead of binary simiarities, by using the generalized version of Chernoff inequality detailed in SectionEoot Appendix. Given the L2 normalized original node features X, the idea of Sim-S can be implemented by inner product of features, i.e. Ax XX*. The twice- run tests on each node can be achieved by Ax multiplying itself. Inspired by the design of popular Transformers [X some learnable parameters are also introduced. It contains the fourth-order statistics of features, which is a significant difference, since self-attention only has second-order, self-attention is regarded as Duet of Attention, our method is analogous to Quertet of Attention (Q- Attention for short). The vectorized representation of Equation @-Attention A then implemented as follows:
part E RLXL is
A Aqart Qgart K gart: where Quart = Ax W
gart: K Kaart = Ax W
(8)
garts
W gart E BLXXL and W is also applied to Ax.
gart E RRLLL' are learnable weights, L' L in this work. L2 normalization
Motivated by the further enhancements from combinations of intrinsic graph structures and implicit graph structures in literature 335 & R6 the self-attention can play an intrinsic role to fuse with the Q-Attention in vision tasks, although intrinsic graph structure exists between images all. The self-attention part is the same as that in Transformers [33
The attention map Asef E IR LxL is:
A [ Kells"
where Quels = XW self' K = XW
selly self)
VMd
W self E RM and W.K. sets E MMX" are the learnable weights, M is the dimension of query and key features. The combined form blends the duet and quartet of attention, hence the name Band of Attention (B-Attention for short). The overview of B-Attention mechanism shown in Figure 2
The fusion of A and A qart is defined as (where softmax (-) is a softmax function):
Aband softmax ( Aart sett Agel))
(10)
When combined with the neural network layer in Equation the final form is:
X' o bbba
(11)
In real implementation. considering the graph size can be very large for large-scale datasets, sub- sub-graph sampling strategy (57 55 is adopted to make it more computationally efficient and scalable. In particular, kiseed nodes are first selected as seeds from a dataset in each sampling step. Then the seed nodes together with their A: nearest neighbours (kNN) [4] form the nodes of sub-graph. As result, Equations X and D are applied to the sub-graph during training and inference. For a dataset of size Na. it takes about Na/k /iedd iterations to process the whole dataset on average. The pseudo code for self-attention and Q-Attention is detailed in Section of Appendix.
Experiments
Experiments are designed with three levels. Firstly, experiments are conducted to analyse the advan- tage of Sim-M over that of Sim-S and its robustness noise. Secondly, armed with multiple tests on GCNs, i.e., B-Attennion is further investigated in comparison with the self-attention and commonly used form of GCNs on robustness and superiority. Ablation experiments are then conducted to study how each part of B-Attention contributes to and influences the performance. Finally, with the robust graph structure learned by B-Attention, comparison experiments downstream graph-based clustering and RelD tasks are conducted to evaluate the benefits of the proposed approach.
3.1 Experimental setups
Datasets Experiments conducted on commonly used visual datasets with three different types of objects, i.e., MS-Celeb (21) (human faces), MSMT17 (59] (human bodies) and VeRi- 776 37 (vehicles), to verify the generalization of the proposed method. In these datasets, each sample has specific category, and most categories have rich samples. The tasks on these datasets are popular and representative in computer vision. It is worth noting that MS-Celeb divided into 10 parts by identities (Part0-9): PartO for training and Part1-9 for testing, to maintain comparability and consistency with previous works [6
4 50
Details of the dataset are in Section A.
'DECLARATION: Considering the ethical and privacy issues, all the datasets used in this paper are feature vectors which can not be restored to images.
5

---

Metrics The Area Under the Curve (AUC) is adopted to directly evaluate the discriminatory power of similarity measures between node pairs, Edge Noise Rate (ENR) illustrates the amount of noise in graphs quantitatively. For node Us. ENR N where Na is the number of noisy edges and Ni
Nall is the amount of all edges for ot. The average ENR is defined as the average of the ENRs of all nodes. The feature quality of GCNs output is investigated to prove the effectiveness of graph structure learning: The mean Average Precision (mAP) [68] is used from the perspective of identification and ROC curves are shown from the perspective of verification. The higher the quality of the features the better the graph structure learning. In downstream ReID tasks, mApR and Rankl accuracy (R1) are used. Compared to mAP, mAp additionally takes into account the camera id information, i.c., samples from the same camera the probe will not be counted in evaluation. BCubed (FB) B and Pairwise (Fp) 5 F-scores are used for evaluating clustering tasks.
3.2 Effectiveness of the Sim-M
To verify the robustness of Sim-M, the study is first conducted on MS-Celeb (Partl) by evaluating the dis- criminatory ability on different average ENRs compar- ing with Sim-S. The Sim-S uses the cosine similarity here for simplicity and without loss of generality. For the sake of sparsity and efficiency it brings, the graphs are built based on kNN via the approximate nearest neighbour algorithm. The average ENR of graphs is controlled by using different i values when searching neighbours. AUC is adopted score between node pairs which have kNN relations.
Table 1: Average ENRs of differ- ent k values and the AUC of Sim-S (AUCs), Sim-M (AUCM) and their dif- ference (AUC-=AUCM-AUCG) on MS- Celeb (partl). The values of AUC are scaled by a factor of 100 for better display.
k
ENR
AUCs
AUCM
AUCs
evaluate the similarity
5
0.05
96.80
97.59
0.79
10
0.09
96.21
97.72
1.51
Table presents that the k increases, more and more noise is added (ENR gets larger). For Sim-M, however, more samples can be drawn and more tests can be run, so the advantage of Sim-M over Sim-S becomes in- creasingly obvious. This demonstrates the robustness of Sim-M to noise. This phenomenon also conforms to the condition and conclusion in Theorem [ However, as increases and ENR gets larger, too much noise can not be handled by multiple tests, so the performance of Sim-M begins to drop. This downward trend can be slowed down by the careful design of B-Atttenionn. The same experiments are conducted on the MSMT17 and VeRi-776, with similar findings in Section A.2
20
0.14
95.33
97.21
1.88
40
0.23
94.00
95.66
1.66
80
0.39
92.05
91.32
-0.71
120
0.54
90.76
86.58
-4.17
160
0.64
90.65
83.90
-6.75
3.3 Effectiveness of the B-Attention
To demonstrate the robustness of B-Attention mechanism, the study is first conducted on MS- Celeb (Partl) by evaluating the performance of B-Attention with different average ENRs, comparing it to other baselines. Then, comparison experiments are also conducted on MSMT17, VeRi-776 and also larger scales (Part3-9) of MS-Celeb with the best ENR settings for each dataset. Comparison baselines include the original features of each dataset, commonly used form of GCN 34 57 Transformer, GCN with the self-attention (GCN with A Transformer with the B-Attention mechanism (Trans with Abuna), and GCN with B-Attention mechanism (GCN with Abone). Trans- former here only used as an encoder for structure learning, no positional encoding is required.
Network architectures and settings The model is composed of multiple GCN layers with B- Attention, and an extra fully connected layer with PReLU 23 activation. The output features have a dimension of 2048 for all datasets. The number of GCN layers is also tuned respectively for different datasets: MS-Celeb uses three GCN layers; MSMT17 and VeRi-776 use one GCN layer. Each node acts as the probe to search its &NN construct the graph. The l value is tuned respectively for different datasets: 120 for MS-Celeb, 30 for MSMT17, and 80 for VeRi-776. Fach node in a dataset
can be enhanced multiple times, either probe or a neighbour. The Hinge Loss 45 577 is used to classify node pairs. The initial learning rate is 0,008 with the cosine annealing strategy.
Performance on different average ENRs Models are trained on PartO and tested on Partl of MS-Celeb. All basclines use three GCN/Transformet layers for MS-Celebb except the GCN with

---

Aceff which uses two GCN layers (observed to have better performance than three GCN layers). For a certain ENR, the same value used in training and testing. Figure shows how the output feature qualities (mAP) change with different average ENRs for all baselines. It shows that the GCN and Transformer with band work the best in most average ENRs in terms of both mAP values and their robustness to different ENRs. The GCN has the worst results, and the performances of the ones with Aeets are in between. This validates that a commonly used form GCN is sensitive to noisy edges in input graphs; the proposed B-Attention can well alleviate the influence of noisy edges and show the strongest robustness when learning graph structure; the A also has some power of dealing with noisy edges. but it is worse than band- When & value is too small g.. 5, 10), the performances of Atand and Afelf are similar. This indicates that the Q-Attention part of B-Attention can hardly bring a positive effect when there are too few context neighbours, i.e. the number of single tests gathered is insufficient. In addition, GCN and Transformer have very similar performance, either with Aband or s This indicates that base networks are not critical, and both attention mechanisms work for the two base networks. The performance degradation of B-Attention with very large ENR (e.g., 0.64) indicates that the current form B-Attention still has improvement room, although works much better than the self-attention. Some empirical case studies on self attention, Q-Attention and B-Attention are shown in Section F of Appendix to better understand the effect of multiple tests.
0.95
Table 2: Feature quality (mAP) of different methods on visual datasets of different types of objects.
0,90
mAp
0.85
Dataset
MS-Celeb
MSMT17
VeRi-776
0.80
Aeert
GCH wt Aus
Original Feature
80.07
74.46
81.91
Trans
GCN
90.15
83,51
85,27
0.75
OCN
Transformer
93.86
85.44
85.92
Average Edge Noise Rate (ENR)
0.1
0.2
0.3
0.4
0.5 0.6
GCN w/ Ase!f
94.09
85.40
86.88
Trans w/ Abond
95.70
85.92
87.07
Figure 3: Feature quality (mAP) of various methods with different average ENRs MS-Celeb Partl.
GCN w/ Aband
95.97
86.04
87.59
Performance different types of objects To further validate the B-Attention mechanism, comparisons are also made on datasets with other types of objects: MSMT17 and VeRi-776. In these two datasets, k values and network depths are also tuned for all baselines, aiming for their best performances. For both datasets, one GCN layer is used for GCN, GCN with Aband, and Transformer with Aband; three GCN layers are used for GCN or Transformer with A In MSMT17, k=10 for GCN and "GCN w/ " k=20 for Transformer; k=30 for the two B-Attention methods. In VeRi-776, k=80 for "GCN w/ Aband" and "Trans w/ Aband", ", and k=20 for the rest of baselines. The best mAP results of all baselines are listed in Table [ where the results for MS-Celeb are the peak values in Figure 3 From Table 2 the B-Attention works the best in all three datasets: Ase!/ works better than GCN; and all output features are much better than the original ones, although the original features of MSMT17 and VeRi-776 are current SOTA ones. The best performances are achieved by the GCN with Aband in all datasets. Their ROC curves are shown in Figure where the same observations can be seen from verification perspective. As shown in Figure and Table 2 the performance differences among those baselines with attention mechanisms are less significant in MSMT17 where & values are relatively small and close across baselines. This observation similar to that in Figure 3 the performance margin between Aband and A s is less significant when average ENRs (i.c., k) are relatively small. This further shows that the benefit of B-Attention is limited when with a small number of neighbours, as not many single tests to run and leverage. Better performance can not be achieved by enlarging values for MSMT17, as its average sample amount for each individual is only 31. Our analysis assumes that for any node v, there are more nodes in the I nearest neighbors sharing the same category as e than those sharing different categories (i.e. )). By significantly enlarging k. we will end up with the assumption of our algorithm. Too large & values would result in too big ENR which can not be handled by the current form of B-Attention as discussed on Figure
2. in k nearest neighbors, which fails
Experiments are also conducted to compare these baselines on larger scales MS-Celeb (i.e., Part3-9) in Section A.3 of Appendix. Results in Table 6 and Figure 5 show that all baselines

---

1.0
1.0
1.0
True positive rate
True positive rate
Thue positive rate
0.8
0.8
0.8
0.6-
0.4
OC
0.4
Oc n/ Auoe
0.4
6C
of Aue
haxe n/ Aue
Trne
6C4
oCV
0.0
0.0
0.0
False positive rate
False positive rate
False positive rate
(a) MS- Partl
(b) MSMT17
(c) VcRi-776
Figure 4: ROC curves of different methods on MS-Celeb Partl (a), MSMT17 (b) and VeRi-776 (c).
Table 3: Clustering performance on MS-Celeb.
Dataset
Partl
Part3
Part5
Part7
Part9
Metries
Fp
FB
Fp
FB
Fp
FB
Fp
F8
Fp
F
K-Mcans
79.21
81.23
73.04
75.20
69.83
72.34
67.90
70.57
66.47
69.42
HAC
70.63
70.46
54.40
69.53
11.08
68.62
1.40
67.69
0.37
66.96
DBSCAN
67.93
67.17
63.41
66.53
52.50
66.26
45.24
44.87
44.94
44.74
Clusformer
88.20
87.17
84.60
84.05
82,79
82.30
81.03
80.51
79.91
79.95
STAR-FC
91.97
90.21
88.28
86.26
86.17
84.13
84.70
82.63
83.46
81.47
Ada-NETS
92.79
91.40
89.33
87.98
87.50
86.03
85.40
84.48
83.99
83.28
Original+G-cut
69.63
73.62
63.31
66.56
60.61
62.91
57.13
58.72
54,42
58.54
Ours+G-cut 93.90
92.47
90.54
89.23
88.70
87.13
86.18
85.59
84.36
84.10
Original+Infomap
93.91
92.42
90.10
89.08
88.49
87.30
85.31
86.01
83.08
85.01
Ours+Infomap 94.94
93.67
91.74
90.81
89.50
89.15
87.04
87.81
85.40
86.76
have performance degradation when the scale enlarges while the degradation of that "w/ A Bandd is much smaller than baselines. These results for the cases with different ENRs, different types of objects and larger scales testsets, together validate the robustness and effectiveness of proposed B-Attention in graph structure learning.
Ablation experiments on the design of the topological architecture and attention fusion method also conducted in Section A.4 of Appendix.
3.4 Effectiveness of the downstream tasks
The improvement of graph structure learning also be proven through the downstream tasks by exploiting the features it produces. The better the graph structure is learned, the better the performance on downstream tasks. Graphs have important applications in large-scale clustering and re-reranking in RelD. However, they suffer from inaccurate weight problems when learning and working with the graph structure. Armed the GCN with Atand, many SOTA results are achieved.
Performance on clustering tasks The clustering tasks are evaluated on MS-Celeb and MSMT17. Both of them have large number of classes and are suitable as clustering benchmarks. The same features output by "GCN w/ Apand" are used as that in Section A.3 for MS-Celeb and Section 3.3 for MSMT17. The same clustering method (denoted "G-cut") is adopted that in 20 57 where pairs of nodes are linked when their similarity scores are larger than a tuned threshold, and clustering finally completed by transitively merging all links via a union-find algorithm [2]. The threshold is tuned for each dataset, aiming for the best balance between BCubed and Pairwise F-Scores. In addition, map equation based unsupervised clustering algorithm Infomap
46
o1
is also adopted
here. Their BCubed and Pairwise F-scores are listed in Table and Table 10 where the results of current SOTA GCN/Transformer methods (Clusformer
42 STAR-FC [50] and Ada-NETS (57])
and three conventional clustering approaches (K-Means listed. The results show that the features enhanced via our approach have the best clustering results in all datasets, either with G-cut or Infomap, both of which outperform the current SOTA methods Ada-NETS and STAR-FC with large margins. comparison, results of the ones with Infomap better than those with G-cut.
38 HAC [52) and DBSCAN 17/] are also
8

---

Performance on RelD tasks The ReID tasks is evaluated on two ReID datasets: MSMT17 and VeRi-776. The enhanced features used are the same as those in Table 2 mApR and R1 are evaluated following the standard probe and gallery protocols of MSMT17 and VeRi-776. The results are shown in Ta- ble g As the output feature of learned graph can also be understood as rerank process in ReID, comparisons
Table ReID performance on MSMT17 and VeRi-776.
Datasets
MSMT17
VeRi-776
Metrics
mAp
R1
mAp
R1
Original Feature
62.76
82.36
79.00
96.60
GCN
74.66
85.36
82.31
96.60
Transformer
77.60
86.37
82.62
96.54
GCN w/ Aself
77.34
85.49
84.29
96.30
Trans w/ Aband
77.49
86.54
83.98
96.72
GCN w/ Aband
78.28
86.64
84.74
96.78
also made on commonly used rerank meth- ods: QE 13, LBR 39 KR (67] and
Original+QE
71.10
84.37
83,09
95.47
Original+LBR
70.36
85,28
83.63
97.13
ECN 47 Table f shows that the "GCN w/ A band achieves the best compa- rable performance, and all enhanced fea- tures are much better than the original SOTA ones. In MSMT17, similar the results in Section 3.3
Original+KR
76.46
85.48
82.32
96.84
Original+ECN 78.79
86.16
84,25
97.13
the baselines with attention mechanism have very close performance, but work much better than the GCN. In VeRi-776, from the mApR perspective. B--Attention works better than Aself, and both of them are better than the GCN, but their RI results are comparable. These results still support the superiority of the proposed approach, although RI results of VeRi-776 are comparable, as mApR is more comprehensive indication of feature quality for RelD). The experiments on the combination of "GCN w/ A bend" with these commonly used rerank methods are also conducted in Section A.6 achieving more SOTA performances and proving the complementarity between them.
4 Related Work
extends the operation of convolution from the grid-data (c.g. images, videos) to non-grid data which is more common in the real world. The classic GCN 3E is proposed based the spectral theory and achieves promising results citation network datasets: Citeseer, Cora and Pubmed 49). GraphSAGE [2] further extends GCNs from transductive framework to inductive framework and obtains better results. GAT
Graph Convolutional Networks. GCNs 34
22
[44 introduces the
attention mechanism in GCN, making it more expressive. Fast-GCN 6 interprets graph convolutions integral transforms of embedding functions under probability measures, which not only is efficient
for training but also generalizes well for inference. Some research works 48 66
27
56 / also
advance GCNs for relational data modeling. However, these GCNs are all investigated relational datasets that have explicit graph structures. This work aims to extend the capacity of GCNs to work on datasets (e.g., image objects) which have explicitly defined graph structures.
Graph Structure Learning. The literature 60 suggests that research on graph structure learning for GCNs can be roughly divided into discrete-based and srighted-basee ones. Some discrete- based methods [6 B1 40 [8] treat the graph structures as random distributions and sample binary adjacency matrices from them. Another way [57] constructs discrete graphs based heuristic proxy objectiveness aiming for clean yet rich neighbours for each node. The binary matrices methods are difficult to optimize because of the gradient breakage,
such methods resort to variational inference (e.g., approaches in [6 40)). reinforcement learning (e.g., approaches in (31) or optimization in stages (e.g., approaches in [57) which may lead to sub-optimal solutions. In addition, the parameter size of distributions 18 can be very large (e.g., O(Nk) where N is the number of nodes and ls the number of neighbours for each node) and therefore not suitable for large datasets. The weighted- based methods D 10
42), which are the focus of this paper, learn weighted adjacency matrices that it can optimized by SGD techniques 44 33 [. The main difference between these works 9 JC
43
64
28
8, 25
35 20,
4R
64
28
R3 is in
and kernel- based [25] 355 roughly. To enhance the discriminatory power of the similarity measure, they have adopted various forms: adding learnable parameters 9
the similarity measure, which can be classified into attention-base
28
X introducing nonlinearity [TC [k] (4]
Gaussian kernel functions 25 35 or self-attention 20
etc. However, these methods are sensitive to the noise from feature representations, particular when each node is represented by a high dimensional vector, leading to the problem of inaccurate weights and vulnerable graph structure.
42
53
12

---

This work aims to propose multiple samples based similarity measure to improve the robustness of graph structure learning. Another approach uses the low-rank, sparsity and feature smoothness properties of the real-world graph to optimize the graph structure [29, but these priors are suitable for community networks rather than graph of image objects. Some work [30] learns augment the given graph with new edges by reinforcement learning to improve the performance of downstream tasks, but existing noisy edges problem in graphs cannot be handled.
Conclusion
The problem of inaccurate edge weights is identified in graph structure learning, analysed from the perspective of noise feature vectors, and handled based on several tests. A novel and robust similarity measure Sim-M is proposed, proven to be superior and verified in experiments. Then it designed in an elegant matrix form, i.e., B-Attention mechanism, to improve the graph structure quality in GCNs. The superiority and robustness of B-AAttenton validated by comprehensive experiments and achieve many SOTA results on clustering and ReID benchmarks.
Acknowledgments and Disclosure of Funding
We thank many colleagues DAMO Academy of Alibaba, in particular, Yuqi Zhang, Weihua Chen and Hao Luo for useful discussions.
This research was partially funded by the National Science Foundation of China (No. 62172443) and Hunan Provincial Natural Science Foundation of China (No. 2022J130053).
References
[1] Enrique Amigó, Julio Gonzalo, Javier Artiles, and Felisa Verdejo. comparison of extrinsic clustering evaluation metrics based formal constraints. Information retrieval, 12(4):461 486, 2009.
[2] Richard J Anderson and Heather Woll. Wait-free parallel algorithms for the union-find problem. In Proceedings of the twenty-third annual ACM symposium Theory of computing, pages 370-380, 1991.
[3] Amit Bagga and Breck Baldwin. Algorithms for scoring coreference chains. In The first international conference on language resources and evaluation workshop linguistics coreference, volume 1, pages 563-566. Citeseer, 1998.
[4] Trénéc-Jules Bienaymé. Considérations à l' t'appui de la découverte de Laplace sur la loi de probabilité dans la méthode des moindres carrés. Imprimerie de
1853.
[5] Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. Siam Review, 60(2):23333 3113,,1122200 2018.
[6] Chen, Tengfei Ma, and Cao Xiao. Fastgen: fast learning with graph convolutional networks via importance sampling. In International conference on learning representations (ICLR), 2018.
Yiqi Chen, Tieyun Qian, Wanli Li, and Yile Liang. Exploiting composite relation graph convolution for attributed network embedding. Journal of Computer Research and Development, pages 1674-1682, 2020.
[8] Yu Chen, Lingfei Wu, and Mohammed Zaki. Iterative deep graph learning for graph neural networks: Better robust node embeddings. Advances in Neural Information Processing Systems, 11119941-99339 2020.
[9] Yu Chen, Lingfei Wu, and Mohammed Zaki. Graphflow: Exploiting conversation flow with graph neural networks for conversational machine comprehension. arXiv preprint Arx: 00059, 2019.
[10] Yu Chen, Lingfei Wu, and Mohammed Zaki. Reinforcement learning based graph-to-sequence model for natural question generation. In International conference on learning representations (ICLR), 2020.
[11] Herman Chernoff. A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. The Annals of Mathematical Statistics, pages 493-507, 1952.
[12] E. Choi, Zhen Xu, Yujia Li, Michael W. Dusenberry, Gerardo Flores, Yuan Xue, and Andrew M. Dai. Learning the graphical structure of electronic health records with graph convolutional transformer. In AAAI, 2020.
10

---

[13] Ondrej Chum, James Philbin, Josef Sivic, Michael Isard, and Andrew Zisserman. Total recall: Automatic query expansion with generative feature model for object retrieval. In 2007 IEEE 11th International Conference on Computer Vision, pages 1-8, Rio de Janeiro, Brazil, 2007. IEEE
[14] Thomas Cover and Peter Hart. Nearest neighbor pattern classification. IEEE transactions information theory, 13(1):21-27. 1967.
[15] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4690 4690-4699, 2019.
Additive angular margin loss
[16] Pantelis Hlinas, Edwin Bonilla, and Louis Tiao. Variational inference for graph convolutional networks in the absence of graph data and adversarial settings. Advances in Neural Information Processing Systems, 33: 10111868-111 18660, 2020.
[17] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial datahases with noise. In Kdd, volume 96, pages 226-231, 1996.
[18] Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. Learning discrete structures for graph neural networks. In International conference on machine learning, pages 1972-1982. PMLR, 2019.
[19] Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 315-323, JMLR Workshop and Conference Proceedings, 2011.
[20] Senhui Guo, Jing Xu, Dapeng Chen, Chao Zhang, Xiaogang Wang, and Rui Zhao. Density-aware feature embedding for face clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6698-6706, 2020.
[21] Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb-Im: A dataset and benchmark for large-scale face recognition. In European conference computer vision, pages 87-102 Springer, 2016.
[22] William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. arXiv preprint /706::/006.0221 2017.
[23] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance imagenet classification. Proceedings the IEEE international conference on computer vision, pages 1026-1034, 2015.
[24] Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformer based object re-identification. Proceedings the IEEF/CVF International Conference on Computer Vision (ICCV), pages 15513-15022, Octoher 2021.
the
[25] Mikael Henaff, Joan Bruna, and Yann LeCun. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163 2015.
[26] Wassily Hoeffding. On the distribution of the number of successes in independent trials. The Annals of Mathematical Statistics, pages 713-721, 1956.
[27] Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. Heterogeneous graph transformer. Proceedings of The Web Conference 2020, pages 2704-2710, 2020.
[28] Deng Huang, Peihao Chen, Runhao Zeng, Qing Du, Mingkui Tan, and Chuang Gan. Location aware graph convolutional networks for video question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1021-11028, 2020.
[29] Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang. Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
[30] Daniel D. Johnson, H. Larochelle, and Daniel Tarlow. Learning graph structure with finite-state automaton layer. In Advances in Neural Information Processing Systems, 2020.
[31] Anees Kazi, Luca Di Cosmo, Nassir Navab, and Michael M. Bronstein. Differentiable graph module (dgm) graph convolutional networks. ArXiv, abs/2002.04999, 2020.
[32] Michael J Kearns and Umesh Vazirani. An introduction to computational learning theory. MIT press. 1994.

---

[33] Jack Kiefer, Jacob Wolfowitz. et al. Stochastic estimation of the maximum of regression function. The Annals of Mathematical Statistics, 2363):622-66,, 1952.
[34] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In
International Conference on Learning Representations (ICLR), 2017.
[35] Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. Adaptive graph convolutional neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[36] Shangqing Liu, Yu Chen, Xiaofei Xie, Jingkai Siow, and Yang Liu. Retrieval-augmenne generation for code summarization via hybrid gnn. arXiv preprint arXiv:2006.054. 05405, 2020.
[37] Xinchen Liu, Wu Liu, Huadong Ma, and Huiyuan Fu. Large-scale vehicle re-identification in urban surveillance videos. In 2016 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6, 2016.
[38] Stuart Lloyd. Least squares quantization in pem. IEEE transactions on information theory, 28(2):129-133 1982.
[39] Chuanchen Luo, Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang. Spectral feature transformation for person re-identification. Proceedings of the IEEE International Conference on Computer Vision, pages 4976-4985, 2019.
[40] Jiaqi Ma, Weijing Tang, Ji Zhu, and Qiaozhu Mei. A flexible generative framework for graph-based semi-supervised learning. Advances in Neural Information Processing Systems, 32, 2019.
[41] Rajecv Motwani and Prabhakar Raghavan. Randomized algorithms. Cambridge university press, 1995.
[42] Xuan-Bat Nguyen, Due Toan Bui, Chi Nhan Duong, Tien Bui, Khoa Luu. Clusformer: transformer based clustering approach to unsupervised large-scale face and visual landmark recognition. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 10847-10856, 2021.
[43] Kyoung-Wont Woon On, Eun-Sol Kim, Yu-Jung Hco, and Byoung- Tak Zhang. Cut-based graph learning networks to discover compositional structure of sequential vidco data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5315-5322, 2020.
[44] Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pages 400-407, 1951.
[45] Lorenzo Rosasco, Ernesto De Vito, Andrea Caponnetto, Michele Piana, and Alessandro Verri. Are loss functions all the same? Neural computation, 16(5):1063-1076, 2004.
[46] M. Rosvall, D. Axelsson, and C. T. Bergstrom. The map equation. The European Physical Journal Special Topics, 178(1):13 23, 2009.
[47] M Saquib Sarfraz, Ame Schumann, Andreas Eberle, and Rainer Stiefelhagen. A pose-sensitive embedding for person re-identification with expanded cross neighborhood re-ranking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 420-429, 2018
[48] Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European semantic web conference, pages 593-607. Springer, 2018
[49] Prithviraj Sen, Galilco Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Radd. Collective classification in network data. Al magazine, 29(3):93-93, 2008
[50] Shuai Shen. Wanhua Li, Zheng Zhu, Guan Huang, Dalong Du, Jiwen Lu, and Zhou. Structure- aware face clustering on a large-scale graph with 107 nodes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[51] Yichun Shi, Charles Otto, and Anil K Jain. Face clustering: representation and pairwise constraints. JEEE Transactions on Information Forensics and Security, 13(7):1626-1666 2018.
[52] Robin Sibson. Slink: an optimally efficient algorithm for the single-link cluster method. The computer journal, 3611),30-3,, 1973.
[53] Ashish Vaswani. Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
12

---

[54] Petar Velickovie, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3. 2018, Conference Truck Proceedings. OpenReview.nel, 2018.
[55] Minjie Wang, Da Zheng. Zihao Ye. Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Gai. Tianjun Xiao, Tong He. George Karypis. Jinyang Li, and Zheng Zhang. Deep graph library: graph-centric, perfolrmaprooma package for graph neural networks. arXiv preprint arXiv:1909.01315. 2019.
[56] Xiao Wang, Houye Ji, Chuan Shi, Bai Wang. Yanfang Ye, Peng Cui, and Philip S Yu. Heterogencous graph attention network. In The world wide web conference, pages 2022 2022-003,222 2019.
[57] Yaohua Wang, Yaobin Zhang. Fangyi Zhang, Senzhang Wang. Ming Lin, YuQi Zhang, and Xiuyu Sun. Ada-nets: Face clustering via adaptive neighbour discovery in the structure space. In International conference on learning representations (ICLR), 2022.
[58] Zhongdao Wang. Liang Zheng. Yali Li, and Shengjin Wang. Linkage based face clustering via graph convolution network. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 1117-1125, 2019.
[59] Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. Person transfer gan to bridge domain gap for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 79-88, 2018.
[60] Lingfei Wu, Peng Cui, Jian Pei, and Liang Zhao. Graph Neural Networks: Foundations, Frontiers, and Applications. Springer Singapore, Singapore, 2022.
[61] Yongfu Xiong. face- cluster- by-infomap 2020.
flce-cluster-by-ine bn-iooa.........
httt:://ggthubb..mm/zaaaaoooooo
[62] Lei Yang, Dapeng Chen, Xiaohang Zhan, Rui Zhao, Chen Change Loy, and Dahua Lin. Learning to cluster faces via confidence and connectivity estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13369-13378, 2020.
[63] Lei Yang, Xiaohang Zhan, Dapeng Chen, Junjie Yan, Chen Change Loy, and Dahua Lin. Learning to cluster faces on an affinity graph. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2298-23066 2019.
[64] Zhilin Yang, Jake 7han, Bhuwan Dhingra, Kaiming He, William W Cohen, Russ R Salakhutdinov, and Yann LeCun. Glomo: Unsupervised learning of transferable relational graphs. Advances in Neural Information Processing Systems, 31, 2018
[65] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang. Ling Shao, and Steven C. H. Hoi. Deep learning for person re-identification: A survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
[66] Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. Heterogeneous graph neural network. Proceedings of 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 793-803, 2019.
[67] Zhun Zhong, I Liang Zheng, Donglin Cao, and Shaozi Li. Re-ranking person re-identifiiation with reciprocal encoding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1318 1327,,11722200 2017.
[68] Mu Zhu. Recall, precision and average precision. Department of Statistics and Actuarial Science, 2, 2004.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] See abstract and Section !1
(b) Did you describe the limitations of your work? [Yes] See Section in Appendix.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section in Appendix.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] We have read the ethics guidelines on the official website.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 2.2
(b) Did you include complete proofs of all theoretical results? [Yes] See Section 2.2
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [No] The code and the data are proprictary. The instructions is in Section 5
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See in Section B.I paragraph of Network architectures and settings and the paragraphs describing the corresponding experiment in Section 3.3
(c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)? [No] Error bars are not reported because would too computationally expensive.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section A.7
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... [Yes] See the Datasets
(a) If your work uses existing assets, did you cite the creators?
paragraph in Section 3.1
(b) Did you mention the license of the assets? [No] The used assets are very common in papers of computer vision.
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you't 'e using/curating? [No] The used assets are very common papers of computer vision. (e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [Yes] See the footnote in Datasets paragraph in Section T.T Considering the ethical and privacy issues, all the datasets used this paper are feature vectors which can not be restored to images.
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "d3KFjhCzuze", "reviewer": "Reviewer_Xw91", "summary": "The paper attempts to build a robust graph structure learning approach over images from the viewpoint of multiple statistical tests. The method is novel and proven theoretically according to the Chernoff Bounds. The corresponding elegant matrix form of that method, named Bttention, is designed for efficiency. The experimental results are solid, and the analysis and conclusions are very sufficient. The story sounds interesting and meaningful. It is a fundamental research to the NeurIPS community.", "questions": "(1) How to choose the optimal k value for new datasets?\n(2) The reviewer is concerned about whether the code will be made public.\n", "limitations": "The authors have discussed the limitations of this paper in detail in the appendix, which is already comprehensive.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n(1) The effectiveness of multiple tests for graph structure learning is verified both theoretically and empirically on multiple clustering and ReID benchmark datasets. The originality is sufficient.\n(2) The Bttention is well-designed in the elegant matrix form which is easy to implement. A high quality contribution on technology.\n(3) The logical flow is clear. It first indicates the unreliability of pairwise similarity, then proposes their Sim-M and Bttention in Section 2. Finally, it validates their robustness to noise and shows SOTA performances on downstream tasks in Section 3.\n(4) The experiments are very sufficient and well organized. It significantly outperforms previous SOAT methods in clustering and ReID tasks. These results can support the conclusions.\n\nWeaknesses:\n(1) This method cannot be adapted to various datasets. As proven in Section 2.2 and empirically shown in Section 3.3, it is only advantageous when the number of samples in each category is large enough in the dataset.\n(2) The performance of Bttention will degrade when with too large ENR as discussed in Section 3.3.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "VUZCQNfGKqC", "reviewer": "Reviewer_dFZi", "summary": "This paper proposes a graph structure learning for images that have no graph structure naturally. Instead of simply computing the pairwise similarity of images to form the edge, This paper proposes to leverage a single statistical test to form the edges between images, namely Sim-S.  In addition, this paper proposed to leverage the multiple statistical tests for robust graph structure learning, namely Sim-M.  Bttention is also proposed to craft the proposed method into an elegant and efficient matrix form for efficacy in real-world tasks. The experiments demonstrate the effectiveness of the proposed method, Sim-S, Sim-M, and Bttention. ", "questions": "1. I would like to see the potential solution to reduce the time of the fourth-order statistics in the **Weaknesses 1** \n2. As footnote 1 said, “all the datasets used in this paper are feature vectors” (Page 5).  Thus I wonder how the proposed method performs while it uses the original features. Since the learned features would be meaningful and less noisy than the original features. \n3.  I would like to see some case studies to show the results of the proposed method, for example, a small graph learned in the real-world graph.\n4. I would suggest that there should be more introduction to Statistical Tests in Section 2.1.\n", "limitations": "None", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. This paper is well-motivated. Since the simple similarity between images can not often capture the relation between the image pairs, thus a more accurate way to compute the similarity between image pairs is needed. \n2. The proposed method is technically sound.  The effectiveness of the single test and multiple tests is well supported by the theoretical analysis in  Proposition 1 and Theorem 1.  The proposed method is also practical since it provides an elegant and efficient matrix form.\n3. The experimental results are convincing to me. Sections 3.3 and 3.4 show the effectiveness of the proposed methods, also the robustness of Sim-M to noise.  Table 3 shows the superiority of the proposed methods for clustering tasks. This paper also demonstrates the superiority of the proposed method on downstream tasks.\n4. The paper is well organized and easy to follow. The proposed method is easy to implement and reproduce.\n\n**Weaknesses**\n\n1. Although the proposal is simple and practical, fourth-order statistics of features are used in Bttention. Such an operation would be time-consuming.\n2. As “Better performance can not be achieved by enlarging k values for MSMT17, as its average sample amount 251 for each individual is only 31” (Line 251) said,  the performance of Bttention will go bad in some cases, and this paper didn’t provide the insightful analysis for this phenomenon.\n3. The Preliminary part lacks the introduction of  Statistical Tests. And the Chernoff Bounds part in the section is not clear. The usage and advantages of this term should be discussed.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "3NFAM8fVcu1", "reviewer": "Reviewer_1q9s", "summary": "The core target of this paper aims to use GCNs with the common neighbor sets to alleviate the impact of noises in expectation, through modeling the vision tasks as the *node classification*. \n\nAt first, the authors theoretically show that using the common node set $\\mathcal V_{ij}$ to compute the similarity of $v_i$ and $v_j$,  would help to reduce the variance of the estimation of pairwise nodes. Then, similar to the popular self-attention (which is widely used in Transformer), two graph matrices, $A_{band}$ and $A_{self}$, are designed. Some experiments of ReID and clustering are conducted. ", "questions": "1. What's the connection between Eq. 10 and $k$NN? How does the model obtain the sparse graph (as the authors claim in Line 179)? \n\n2. Is the model expensive, from both time and space perspectives? It is indeed an intrinsic problem of graph models, but the authors should also consider the negative impact. \n\n3. Could the authors please clarify the connection between Section 2.2 and Section 2.3 more formally? ", "limitations": "It has been discussed in the appendix. ", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Pros: \n\n1. The paper is well-written and easy to follow.  \n2. The motivation is technically sound. Using more nodes/points to reduce the variance is a reliable scheme. \n\n### Cons: \n\n1. As the vision task is viewed as the *node classification*, the size of graph would increase with the size of the dataset. It indicates that the inefficiency problem is an inescapable (and severe) problem. However, there is no discussion in Eq. 8-11. It seems that the model computes a dense similarity matrix, while the authors also claim that the graph is constructed by $k$NN. What's the connection between Eq. 10 and $k$NN? Does it compute $A_{band}$ firstly and then find a $k$NN-approximation matrix that only retains the $k$  largest entries?  Besides, $A_X=XX^T$ is already time-consuming.  It would be better if the authors provide the pseudo-code of the whole procedure in the appendix. \n\n2. Although the motivation to reduce the impact of noises is convincing, the design in Section 2.3 seems to imitate the self-attention mechanism. In other words, it seems a little irrelevant to Section 2.2 since Eq. 8-11 seems not to be strongly driven by the theoretical results. It looks like Theorem 1 is the explanation of Eq. 8-11, rather than the motivation. \n\n3. The conclusion of Theorem 1 should be more formal. In other words, it should be described mathematically. What's more, Line 79-81 should appear as a lemma.   \n\n4. The GCN formulation in Eq. 1 is not standard. The authors should clarify it.  \n\n5. I suggest using $\\mathcal{Q}$-Attention and $\\mathcal B$-Attention. \n\nOverall, my main concerns come from two aspects, the inefficiency problem and the weak connection between Sections 2.2 and 2.3. I would like to update my score after reading other reviews and the response.    ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept