Original: Robust Graph Structure Learning over Images via Multiple Statistical Tests
Revised: Robust Graph Structure Learning over Images via Multiple Statistical Tests

Original: Abstract
Revised: Abstract

Original: Graph structure learning aims to learn connectivity in a graph from data.
Revised: Graph structure learning aims to learn connectivity in a graph from data.

Original: It isparticularly important for many computer vision related tasks since no explicitgraph structure is available for images for most cases.
Revised: It is particularly important for many computer vision related tasks since no explicit graph structure is available for images for most cases.

Original: A natural way to construct agraph among images is to treat each image as a node and assign pairwise imagesimilarities as weights to corresponding edges.
Revised: A natural way to construct a graph among images is to treat each image as a node and assign pairwise image similarities as weights to corresponding edges.

Original: It is well known that pairwisesimilarities between images are sensitive to the noise in feature representations,leading to unreliable graph structures.
Revised: It is well known that pairwise similarities between images are sensitive to the noise in feature representations, leading to unreliable graph structures.

Original: We address this problem from the viewpointof statistical tests.
Revised: We address this problem from the viewpoint of statistical tests.

Original: By viewing the feature vector of each node as an independentsample, the decision of whether creating an edge between two nodes based ontheir similarity in feature representation can be thought as a single statisticaltest.
Revised: By viewing the feature vector of each node as an independent sample, the decision of whether creating an edge between two nodes based on their similarity in feature representation can be thought as a single statistical test.

Original: To improve the robustness in the decision of creating an edge, multiplesamples are drawn and integrated by multiple statistical tests to generate a morereliable similarity measure, consequentially more reliable graph structure.
Revised: To improve the robustness in the decision of creating an edge, multiple samples are drawn and integrated by multiple statistical tests to generate a more reliable similarity measure, consequentially more reliable graph structure.

Original: Thecorresponding elegant matrix form named B ttention is designed for efficiency.
Revised: The corresponding elegant matrix form named B -Attention is designed for efficiency.

Original: The effectiveness of multiple tests for graph structure learning is verified boththeoretically and empirically on multiple clustering and ReID benchmark datasets.
Revised: The effectiveness of multiple tests for graph structure learning is verified both theoretically and empirically on multiple clustering and ReID benchmark datasets.

Revised: Source codes are available at https://github.com/Thomas-wyh/B-Attention .

Original:  Introduction
Revised:  Introduction

Original: Graph structure learning plays an important role in Graph Convolutional Networks (GCNs).
Revised: Graph structure learning plays an important role in Graph Convolutional Networks (GCNs).

Original: It seeksto discover underlying graph structures for better graph representation learning when graph structuresare unavailable, noisy or corrupted.
Revised: It seeks to discover underlying graph structures for better graph representation learning when graph structures are unavailable, noisy or corrupted.

Original: These issues are typical in the field of vision tasks [54, 59, 58,19, 40, 47, 53, 12, 37, 62, 45].
Revised: These issues are typical in the field of vision tasks [58, 63, 62, 20, 42, 50, 57, 13, 39, 67, 47].

Original: As an example, there is no explicit graph structure between photos inyour phone albums so the graph-based approaches [19, 40, 53] in photo management softwares seekto learn graph structure.
Revised: As an example, there is no explicit graph structure between photos in your phone albums, so the graph-based approaches [20, 42, 57] in photo management softwares seek to learn graph structure.

Original: A popular research direction on it is to consider graph structure learning assimilarity measure learning upon the node embedding space [56].
Revised: A popular research direction on it is to consider graph structure learning as similarity measure learning upon the node embedding space [60].

Original: They learn weighted adjacencymatrices [19, 40, 8, 9, 41, 60, 26, 7, 24, 33] as graphs and feed them to GCNs to learn representations.
Revised: They learn weighted adjacency matrices [20, 42, 9, 10, 43, 64, 28, 8, 25, 35] as graphs and feed them to GCNs to learn representations.

Original: However, the main problem of the above weighted-based methods is that weights are not alwaysaccurate and reliable: two nodes with smaller associations may have a higher weight score thanthose with larger associations.
Revised: However, the main problem of the above weighted-based methods is that weights are not always accurate and reliable: two nodes with smaller associations may have a higher weight score than those with larger associations.

Original: The root of this problem is closely related to the noise in the highdimensional feature vectors extracted by the deep learning model, as similarities between nodes areusually decided based on their feature vectors.
Revised: The root of this problem is closely related to the noise in the high dimensional feature vectors extracted by the deep learning model, as similarities between nodes are usually decided based on their feature vectors.

Original: The direct consequence of using an inaccurate graphstructure is so called feature pollution [53], leading to a performance degradation in downstreamtasks.
Revised: The direct consequence of using an inaccurate graph structure is so called feature pollution [57], leading to a performance degradation in downstream tasks.

Original: In our empirical studies, we found that close to 1 / 3 of total weights are assigned to noisy edgeswhen unnormalized cosine similarities between feature vectors are used to calculate weights.
Revised: In our empirical studies, we found that close to 1 / 3 of total weights are assigned to noisy edges when unnormalized cosine similarities between feature vectors are used to calculate weights.

Original: Many methods have been developed to improve similarity measure between nodes in graph, includingadding learnable parameters [8, 26, 7], introducing nonlinearity [9, 41, 60], Gaussian kernel functions [24, 33] and self-attention [50, 11, 19, 40].
Revised: Many methods have been developed to improve similarity measure between nodes in graph, including adding learnable parameters [9, 28, 8], introducing nonlinearity [10, 43, 64], Gaussian kernel functions [25, 35] and self-attention [53, 12, 20, 42].

Original: Despite the progress, they are limited to modifyingsimilarity function, without explicitly addressing the noise in feature vectors, a more fundamentalissue for graph structure learning.
Revised: Despite the progress, they are limited to modifying similarity function, without explicitly addressing the noise in feature vectors, a more fundamental issue for graph structure learning.

Original: One way to capture the noise in feature vectors is to view thefeature vector of each node as an independent sample from some unknown distribution associatedwith each node.
Revised: One way to capture the noise in feature vectors is to view the feature vector of each node as an independent sample from some unknown distribution associated with each node.

Original: As a result, the similarity score between two nodes can be thought as the expectationof a test to decide if two nodes share the same distribution.
Revised: As a result, the similarity score between two nodes can be thought as the expectation of a test to decide if two nodes share the same distribution.

Original: Since, only one vector is sampled foreach node in the above methods, the statistical test is essentially based on a single sample from bothnodes, making the calculated weights inaccurate and unreliable.
Revised: Since, only one vector is sampled for each node in the above methods, the statistical test is essentially based on a single sample from both nodes, making the calculated weights inaccurate and unreliable.

Original: An possible solution towards addressing the noise in feature vectors is to run multiple tests, a commonapproach to reduce variance in sampling and decision making [4, 10, 25].
Revised: An possible solution towards addressing the noise in feature vectors is to run multiple tests, a common approach to reduce variance in sampling and decision making [4, 11, 26].

Original: Multiple samples aredrawn for each node, and each sample can be used to run an independent test to decide if twonodes share the same distribution.
Revised: Multiple samples are drawn for each node, and each sample can be used to run an independent test to decide if two nodes share the same distribution.

Original: Based on the results from multiple tests , we develop a noveland robust similarity measure that significantly improves the robustness and reliability of calculatedweights.
Revised: Based on the results from multiple tests , we develop a novel and robust similarity measure that significantly improves the robustness and reliability of calculated weights.

Original: This paper will show how to accomplish multiple tests in the absence of graph structuresand theoretically prove why the proposed approach can produce a better similarity measure underspecified conditions.
Revised: This paper will show how to accomplish multiple tests in the absence of graph structures and theoretically prove why the proposed approach can produce a better similarity measure under specified conditions.

Original: This similarity measure is also crafted into an efficient and elegant form ofmatrix, named B ttention .
Revised: This similarity measure is also crafted into an efficient and elegant form of matrix, named B -Attention .

Original: B ttention utilizes the fourth order statistics of the features, which is asignificant difference with popular attentions.
Revised: B -Attention utilizes the fourth order statistics of the features, which is a significant difference with popular attentions.

Original: Benefiting from a better similarity measure produced bymultiple tests, graph structures can be significantly improved, resulting in better graph representationsand therefore boosting the performance of downstream tasks.
Revised: Benefiting from a better similarity measure produced by multiple tests, graph structures can be significantly improved, resulting in better graph representations and therefore boosting the performance of downstream tasks.

Original: We empirically verify the effectivnessof the proposed method over the task of clustering and ReID.
Revised: We empirically verify the effectivness of the proposed method over the task of clustering and ReID.

Original: In summary, this paper makes the following three major contributions: (1) To the best of ourknowledge, this is the first paper to address inaccurate edge weights problem by statistical tests on thetask of graph structure learning over images.
Revised: In summary, this paper makes the following three major contributions: (1) To the best of our knowledge, this is the first paper to address inaccurate edge weights problem by statistical tests on the task of graph structure learning over images.

Original: (2) A novel and robust similarity measure is proposedbased on multiple tests, proven theoretically to be superior and designed in an elegant matrix formto improve graph structure quality.
Revised: (2) A novel and robust similarity measure is proposed based on multiple tests, proven theoretically to be superior and designed in an elegant matrix form to improve graph structure quality.

Original: (3) Extensive experiments further verify the robustness of thegraph structure learning method against noise.
Revised: (3) Extensive experiments further verify the robustness of the graph structure learning method against noise.

Original: State-of-the-art (SOTA) performances are achieved onmultiple clustering and ReID benchmarks with the learned graph structures.
Revised: State-of-the-art (SOTA) performances are achieved on multiple clustering and ReID benchmarks with the learned graph structures.

Original:  Methodology
Revised:  Methodology

Original: To handle the noise in feature vectors, we model the feature vector of each node as a sample drawnfrom the underlying distribution associated with that node, and the similarity score between twonodes as the expectation of a statistical test.
Revised: To handle the noise in feature vectors, we model the feature vector of each node as a sample drawn from the underlying distribution associated with that node, and the similarity score between two nodes as the expectation of a statistical test.

Original: The test is to compare their similarity to a given thresholdto decide if two nodes share the same distribution and that decision can be captured by a binary
Revised: The test is to compare their similarity to a given threshold to decide if two nodes share the same distribution and that decision can be captured by a binary

Original: Bernoulli random variable.
Revised: Bernoulli random variable.

Original: The comparison in statistical test may lead to error edges, i.e., noisyedges (an edge connects two nodes of different categories) or missing edges (two nodes of the samecategory should be connected but are not).
Revised: The comparison in statistical test may lead to error edges, i.e., noisy edges (an edge connects two nodes of different categories) or missing edges (two nodes of the same category should be connected but are not).

Original: The superiority of the similarity measure can be revealedby comparing the probability of creating an error edge.
Revised: The superiority of the similarity measure can be revealed by comparing the probability of creating an error edge.

Original: The proposed similarity measure will beintroduced in Section 2.2, and its further design in matrix form will be presented in Section 2.3.
Revised: The proposed similarity measure will be introduced in Section 2.2, and its further design in matrix form will be presented in Section 2.3.

Original: 2.1 Preliminary
Revised: 2.1 Preliminary

Original: Chernoff Bounds.
Revised: Chernoff Bounds.

Original: Sums of random variables have been studied for a long time [4, 10, 25].
Revised: Sums of random variables have been studied for a long time [4, 11, 26].

Original: Chernoff
Revised: Chernoff

Original: Bounds [10] provide sharply decreasing bounds that are in the exponential form on tail distributionsof sums of independent random variables.
Revised: Bounds [11] provide sharply decreasing bounds that are in the exponential form on tail distributions of sums of independent random variables.

Original: It is a widely used fundamental tool in mathematics andcomputer science [30, 39].
Revised: It is a widely used fundamental tool in mathematics and computer science [32, 41].

Original: An often used and looser form of Chernoff Bounds is as follows.
Revised: An often used and looser form of Chernoff Bounds is as follows.

Revised: Lemma 1 (Chernoff Bounds [11]) .

Original: Let X = (cid:80) n 1 X i  ,where X i = 1  with the probability p i and X i = 0 with the probability 1 − p i . 
Revised: Suppose X i is a binary random variable taking value in set { 0 , 1 } with P ( X i = 1) = p i . Define X

Original: All
Revised: =

Original: X i are independent random variables. Let µ = E ( X ) = (cid:80) n 1 p i .
Revised: (cid:80) n 1 X i where { X 1 , X 2 , · · · , X n } are independently sampled.

Original: Then for all 0 < ϵ < 1 ,
Revised: Then for any 0 < ϵ < 1 ,

Original: [Equation]
Revised: [Equation]

Original: 3 µ . It shows that with a high probability 1 − δ , we have X ≤ (1+ ϵ ) µ , or X ≥ (1 − ϵ ) µ , where δ = e − ϵ 2 3 .
Revised: It shows that with a high probability 1 − δ , we have X ≤ (1+ ϵ ) µ , or X ≥ (1 − ϵ ) µ , where δ = e − ϵ 2 3 .

Original: GCNs.
Revised: GCNs.

Original: A GCN network normally consists of multiple GCN layers.
Revised: A GCN network normally consists of multiple GCN layers.

Original: Given an input graph G ( X , A )with L vertices, the output of a normal GCN layer  is:
Revised: Given an input graph G ( X , A ) with L vertices, the output of a normal GCN layer [34] is:

Original: [Equation]
Revised: [Equation]

Original: where ˜A =
Revised: where ˜A =

Original: A + I , ˜D ii =
Revised: A + I , ˜D ii =

Original: L (cid:88)
Revised: L (cid:88)

Original: j
Revised: j

Original: = ˜A i,j , (1)
Revised: = ˜A i,j , (1)

Original: X ∈ R L × M is input vertex features with M dimensions, A ∈ R L × L is their adjacency matrix;
Revised: X ∈ R L × M is input vertex features with M dimensions, A ∈ R L × L is their adjacency matrix;

Original: I ∈
Revised: I ∈

Original: R L × L is an identity matrix, ˜D ∈ R L × L is the diagonal degree matrix of ˜A ; W l ∈ R M × M ′ is atrainable matrix, and σ ( · ) is an activation function such as ReLU [18]; X ′ ∈ R L × M ′ is the outputvertex features with M ′ dimensions, M ′ =
Revised: R L × L is an identity matrix, ˜D ∈ R L × L is the diagonal degree matrix of ˜A ; W l ∈ R M × M ′ is a trainable matrix, and σ ( · ) is an activation function such as ReLU [19]; X ′ ∈ R L × M ′ is the output vertex features with M ′ dimensions, M ′ =

Original: M in this work.
Revised: M in this work.

Original: 2.2 Multiple tests on similarity measure
Revised: 2.2 Multiple tests on similarity measure

Original: Let V = { v 1 , v 2 , ..., v N } be the collection of N nodes.
Revised: Let V = { v 1 , v 2 , ..., v N } be the collection of N nodes.

Original: Each node is assumed to be in one of twocategories: +1 or − 1 for simplicity.
Revised: Each node is assumed to be in one of two categories: +1 or − 1 for simplicity.

Original: For each node v i , the subset of candidate nodes to connect itis denoted by V i .
Revised: For each node v i , the subset of candidate nodes to connect it is denoted by V i .

Original: This subset can be decided by a simple criterion such as by k nearest neighbours.
Revised: This subset can be decided by a simple criterion such as by k nearest neighbours.

Original: When calculating the proposed similarity between v i and v j ∈ V i , the common candidate nodesbetween v i and v j , i.e., V i,j = V i ∩ V j are first identified.
Revised: When calculating the proposed similarity between v i and v j ∈ V i , the common candidate nodes between v i and v j , i.e., V i,j = V i ∩ V j are first identified.

Original: The nodes in the V i,j are treated as themultiple samples drawn for the comparison of v i and v j .
Revised: The nodes in the V i,j are treated as the multiple samples drawn for the comparison of v i and v j .

Original: Then, each v k ∈ V i,j performs singletest twice, one for v k and v i and one for v k and v j .
Revised: Then, each v k ∈ V i,j performs single test twice, one for v k and v i and one for v k and v j .

Original: The single test here is a comparison using thepairwise similarity based on the features of v k and v i , or v k and v j .
Revised: The single test here is a comparison using the pairwise similarity based on the features of v k and v i , or v k and v j .

Original: The pairwise similarity canbe cosine similarity, Gaussian kernel similarity or any other form, and is denoted by Sim-S .
Revised: The pairwise similarity can be cosine similarity, Gaussian kernel similarity or any other form, and is denoted by Sim-S .

Original: Theproposed similarity score between v i and v j increases by one if the two tests yield C ( v i ) 
Revised: The proposed similarity score between v i and v j increases by one if the two tests yield C ( v i ) =

Original: = C ( v k )and C ( v k ) =
Revised: C ( v k ) and C ( v k ) =

Original: C ( v j ) , and zero otherwise, where C ( v ) is the category of node v .
Revised: C ( v j ) , and zero otherwise, where C ( v ) is the category of node v .

Original: It contains multiplesingle tests, hence the name multiple tests , corresponding similarity measure denoted by Sim-M .
Revised: It contains multiple single tests, hence the name multiple tests , corresponding similarity measure denoted by Sim-M .

Original: For the convenience of analysis, a few assumptions are further introduced.
Revised: For the convenience of analysis, a few assumptions are further introduced.

Original: First, the size of V i,j is m ,and among the nodes in V i,j , αm nodes share the same category as v i and (1 − α ) m nodes belongto the opposite category, with α > 12 .
Revised: First, the size of V i,j is m , and among the nodes in V i,j , αm nodes share the same category as v i and (1 − α ) m nodes belong to the opposite category, with α > 12 .

Original: For any node v j ∈ V i , its probability of being connected withv i , according to the statistical test, is q ∈ [0 , 1] if C ( v i )
Revised: For any node v j ∈ V i , its probability of being connected with v i , according to the statistical test, is q ∈ [0 , 1] if C ( v i )

Original: ̸ = C ( v j ) , and the probability increases to1 ≥ p > q if C ( v i ) =
Revised: ̸ = C ( v j ) , and the probability increases to 1 ≥ p > q if C ( v i ) =

Original: C ( v j ) .
Revised: C ( v j ) .

Original: This section will show that, if m is sufficiently large (i.e., enoughnumber of tests can be run), there will exist an appropriate threshold for the Sim-M such that thenumber of error edges can be reduced significantly compared to Sim-S.
Revised: This section will show that, if m is sufficiently large (i.e., enough number of tests can be run), there will exist an appropriate threshold for the Sim-M such that the number of error edges can be reduced significantly compared to Sim-S.

Original: Proposition 1.
Revised: Proposition 1.

Original: For the single test method, in expectation, the number of noisy edges is (1 − α ) qmand the number of missing edges is α (1 − p )m .
Revised: For the single test method, in expectation, the number of noisy edges E { N noisy } = (1 − α ) qm and the number of missing edges E { N miss } = α (1 − p )

Original:  The theorem below shows that both the noisy  and missing edges can be reduced by a significantportion when m is not too small.
Revised: m . The theorem below shows that both E { N noisy } and E { N miss } can be reduced by a significant portion when m is large enough.

Original: Theorem 1.
Revised: Theorem 1.

Original: Assume for γ > 1 , we have
Revised: Suppose γ > 1 and

Original: [Equation]
Revised: [Equation]

Original: Then, for the multiple test method, in expectation, it can reduce the number of noisy edges and missingedges of the single test method by 1 /γ .
Revised: We have E { N noisy } = (1 − α ) qm/γ and E { N miss } = α (1 − p ) m/γ .

Original: Proof.
Revised: Proof.

Original: Consider node v i and one of its candidate node v j ∈ V i .
Revised: Consider node v i and one of its candidate node v j ∈ V i .

Original: S ki,j is denoted as the similarity scorebetween v i and v j based on the statistical test with respect to v k .
Revised: S ki,j is denoted as the similarity score between v i and v j based on the statistical test with respect to v k .

Original: As shown in Figure 1, v k maybe of the same category as v i or not (denoted by v ′ k for a better show).
Revised: As shown in Figure 1, v k may 𝒗 ! 𝒗 𝒊 … 𝒗 𝒌 𝑣 $% … 𝑝 𝑞 𝑞 𝑝 𝐸 𝑆 &,! $ = 𝛼∗𝑝𝑞+ 1−𝛼 ∗𝑞𝑝 =𝑝𝑞,𝑖𝑓 𝐶 𝑣 & ≠𝐶(𝑣 ! )

Original: Evidently, E [ S ki,j ] =
Revised: 𝒗 ! 𝒗 𝒊 … 𝒗 𝒌 … 𝑝 𝑞 𝑞 𝑝 𝐸 𝑆 &,! $ =𝛼∗𝑝 ( + 1−𝛼 ∗𝑞 ( ,𝑖𝑓 𝐶 𝑣 & =𝐶(𝑣 ! ) 𝑣 $% Self Attention

Revised: Features of nodes dot product addition -Attention be of the same category as v i or not (denoted by v ′ k for a better show).

Revised: Evidently, E [ S ki,j ] =

Original: pq if
Revised: pq if

Original: C ( v i )
Revised: C ( v i )

Original: ̸ = C ( v j ) , and E [ S ki,j ] =
Revised: ̸ = C ( v j ) , and E [ S ki,j ] =

Original: αp 2 + (1 − α )
Revised: αp 2 + (1 − α )

Original: q 2 if C ( v i ) =
Revised: q 2 if C ( v i ) =

Original: C ( v j ) .
Revised: C ( v j ) .

Original: The Sim-M between v i and v j is defined as:
Revised: The Sim-M between v i and v j is defined as:

Original: [Equation]
Revised: [Equation]

Original: Using the Chernoff Bounds [10], we have with a probability 1 − δ ,
Revised: Using the Chernoff Bounds [11], we have with a probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: 𝒗 ! 𝒗 𝒊 … 𝒗 𝒌 𝑣 $% … 𝑝 𝑞 𝑞 𝑝 𝐸 𝑆 &,!

Original: $ = 𝛼∗𝑝𝑞+ 1−𝛼 ∗𝑞𝑝 =𝑝𝑞,𝑖𝑓 𝐶 𝑣 & ≠𝐶(𝑣 ! )

Original: 𝒗 ! 𝒗 𝒊 … 𝒗 𝒌 … 𝑝 𝑞 𝑞 𝑝 𝐸 𝑆 &,!

Original: $ =𝛼∗𝑝 ( + 1−𝛼 ∗𝑞 ( ,𝑖𝑓 𝐶 𝑣 & =𝐶(𝑣 ! )

Original: 𝑣 $%

Original: Self Attention

Original: Features of nodes dot product addition

Original: By assuming that m is large enough, i.e.,
Revised: By assuming that m is large enough, i.e.,

Original: [Equation]
Revised: [Equation]

Original: 5) by choosing the similarity threshold
Revised: by choosing the similarity threshold

Original: [Equation]
Revised: [Equation]

Original: only with a probability δ , we will miss a correct edge, and with a probability δ we will introduce anoisy edge.
Revised: only with a probability δ , we will miss a correct edge, and with a probability δ we will introduce a noisy edge.

Original: We complete the proof by substituting δ  with
Revised: We complete the proof by substituting δ in Equation 5 with

Original: [Equation]
Revised: [Equation]

Original: We can guarantee that the Sim-M between two nodes from the same category is strictly larger thanthat for two nodes from different categories, leading to the distillation of all noisy edges.
Revised: We can guarantee that the Sim-M between two nodes from the same category is strictly larger than that for two nodes from different categories, leading to the distillation of all noisy edges.

Original: 2.3 Multiple tests on GCNs
Revised: 2.3 Multiple tests on GCNs

Original: With a better similarity measure prototype above, we further craft it into an elegant and efficientmatrix form for adaptation of graph structure learning in the real-world.
Revised: With a better similarity measure prototype above, we further craft it into an elegant and efficient matrix form for adaptation of graph structure learning in the real-world.

Revised: The analysis in Theorem 1 can be easily extended to real similarities, instead of binary simiarities, by using the generalized version of

Revised: Chernoff inequality detailed in Section E of Appendix.

Original: Given the L2 normalizedoriginal node features X , the idea of Sim-S can be implemented by inner product of features, i.e. 
Revised: Given the L2 normalized original node features X , the idea of Sim-S can be implemented by inner product of features, i.e. A X =

Original: A X = XX T .
Revised: XX T .

Original: The twice-run tests on each node can be achieved by A X multiplying itself.
Revised: The twicerun tests on each node can be achieved by A X multiplying itself.

Original: Inspiredby the design of popular Transformers [50], some learnable parameters are also introduced.
Revised: Inspired by the design of popular Transformers [53], some learnable parameters are also introduced.

Original: It containsthe fourth-order statistics of features , which is a significant difference, since self-attention onlyhas second-order.
Revised: It contains the fourth-order statistics of features , which is a significant difference, since self-attention only has second-order.

Original: If self-attention is regarded as Duet of Attention , our method is analogous to Quertet of Attention ( Q ttention for short).
Revised: If self-attention is regarded as Duet of Attention , our method is analogous to Quertet of Attention ( QAttention for short).

Original: The vectorized representation of Equation 3, Q ttention
Revised: The vectorized representation of Equation 3, Q -Attention

Original: A qart ∈ R L × L , is then implemented as follows:
Revised: A qart ∈ R L × L , is then implemented as follows:

Original: [Equation]
Revised: [Equation]

Original: Kqart ∈ R L × L
Revised: where Q qart = A X [Equation] [Equation]

Revised: qart

Original: [Equation]
Revised: [Equation]

Revised: X

Original: are learnable weights, L′ =
Revised: [Equation] W [Equation] ∈ R L

Revised: [Equation]

Revised: L

Revised: [Equation]

Revised: and W Kqart ∈ R L × L ′ are learnable weights, L ′ =

Original: L in this work.
Revised: L in this work.

Revised: L2 normalization is also applied to A X .

Original: Motivated by the further enhancements from combinations of intrinsic graph structures and implicitgraph structures in literature [33, 7, 34], the self-attention can play an intrinsic role to fuse with the
Revised: Motivated by the further enhancements from combinations of intrinsic graph structures and implicit graph structures in literature [35, 8, 36], the self-attention can play an intrinsic role to fuse with the

Original: Q ttention in vision tasks, although no intrinsic graph structure exists between images at all.
Revised: Q -Attention in vision tasks, although no intrinsic graph structure exists between images at all.

Original: Theself-attention part is the same as that in Transformers [50].
Revised: The self-attention part is the same as that in Transformers [53].

Original: The attention map A self ∈ R L × L is:
Revised: The attention map A self ∈ R L × L is:

Original: [Equation]
Revised: [Equation]

Original: , where Q self =
Revised: , where Q self =

Original: XW Qself , Kself = [Equation] Kself
Revised: XW Qself , K

Original: [Equation]
Revised: [Equation]

Original: W Qself ∈ R M × M d and W Kself ∈ R M × M d are the learnable weights, M d is the dimension of queryand key features.
Revised: W Qself ∈ R M × M d and W Kself ∈ R M × M d are the learnable weights, M d is the dimension of query and key features.

Original: The combined form blends the duet and quartet of attention, hence the name Bandof Attention ( B ttention for short).
Revised: The combined form blends the duet and quartet of attention, hence the name Band of Attention ( B -Attention for short).

Original: The overview of B ttention mechanism is shown in Figure 2.
Revised: The overview of B -Attention mechanism is shown in Figure 2.

Original: The fusion of A self and A qart is defined as (where softmax ( · ) is a softmax function):
Revised: The fusion of A self and A qart is defined as (where softmax ( · ) is a softmax function):

Original: [Equation]
Revised: [Equation]

Original: When combined with the neural network layer in Equation 1, the final form is:
Revised: When combined with the neural network layer in Equation 1, the final form is:

Original: [Equation]
Revised: [Equation]

Original: In real implementation, considering the graph size can be very large for large-scale datasets, sub-graphsampling strategy [53, 52] is adopted to make it more computationally efficient and scalable.
Revised: In real implementation, considering the graph size can be very large for large-scale datasets, sub-graph sampling strategy [57, 55] is adopted to make it more computationally efficient and scalable.

Original: Inparticular, k seed nodes are first selected as seeds from a dataset in each sampling step.
Revised: In particular, k seed nodes are first selected as seeds from a dataset in each sampling step.

Original: Then the seednodes together with their k nearest neighbours ( k NN) [13] form the nodes of a sub-graph.
Revised: Then the seed nodes together with their k nearest neighbours ( k NN) [14] form the nodes of a sub-graph.

Original: As a result,
Revised: As a result,

Original: Equations 8 and 9 are applied to the sub-graph during training and inference.
Revised: Equations 8 and 9 are applied to the sub-graph during training and inference.

Original: For a dataset of size
Revised: For a dataset of size

Original: N d , it takes about N d /k seed iterations to process the whole dataset on average.
Revised: N d , it takes about N d /k seed iterations to process the whole dataset on average.

Revised: The pseudo code for self-attention and Q -Attention is detailed in Section D of Appendix.

Original:  Experiments
Revised:  Experiments

Original: 3.1 Experimental setups
Revised: Experiments are designed with three levels. Firstly, experiments are conducted to analyse the advantage of Sim-M over that of Sim-S and its robustness to noise. Secondly, armed with multiple tests on GCNs, i.e., B -Attention is further investigated in comparison with the self-attention and a commonly used form of GCNs on robustness and superiority.

Revised: Ablation experiments are then conducted to study how each part of B -Attention contributes to and influences the performance.

Revised: Finally, with the robust graph structure learned by B -Attention , comparison experiments in downstream graph-based clustering and ReID tasks are conducted to evaluate the benefits of the proposed approach.

Original: Datasets Experiments are conducted on commonly used visual datasets 1 with three differenttypes of objects, i.e., MS-Celeb [20] (human faces), MSMT17 [55] (human bodies) and VeRi776 [35] (vehicles), to verify the generalization of the proposed method.
Revised: Datasets Experiments are conducted on commonly used visual datasets 4 with three different types of objects, i.e., MS-Celeb [21] (human faces), MSMT17 [59] (human bodies) and VeRi776 [37] (vehicles), to verify the generalization of the proposed method.

Original: In these datasets, eachsample has a specific category, and most categories have rich samples.
Revised: In these datasets, each sample has a specific category, and most categories have rich samples.

Original: The tasks on these datasetsare popular and representative in computer vision.
Revised: The tasks on these datasets are popular and representative in computer vision.

Original: It is worth noting that MS-Celeb is divided into 10parts by identities (Part0-9): Part0 for training and Part1-9 for testing, to maintain comparability andconsistency with previous works [59, 59, 19, 53, 40, 47].
Revised: It is worth noting that MS-Celeb is divided intoparts by identities (Part0-9): Part0 for training and Part1-9 for testing, to maintain comparability and consistency with previous works [63, 63, 20, 57, 42, 50].

Original: Details of the dataset are in Section A.1.
Revised: Details of the dataset are in Section A.1.

Original: Metrics The Area Under the Curve ( AUC ) is adopted to directly evaluate the discriminatory powerof similarity measures between node pairs.
Revised: Metrics The Area Under the Curve ( AUC ) is adopted to directly evaluate the discriminatory power of similarity measures between node pairs.

Original: Edge Noise Rate ( ENR ) illustrates the amount of noise ingraphs quantitatively.
Revised: Edge Noise Rate ( ENR ) illustrates the amount of noise in graphs quantitatively.

Original: For node v i , ENR = N noisei N alli , where N noisei is the number of noisy edges and
Revised: For node v i , ENR = N noisei N alli , where N noisei is the number of noisy edges and

Original: N alli is the amount of all edges for v i .
Revised: N alli is the amount of all edges for v i .

Original: The average ENR is defined as the average of the ENRs of allnodes.
Revised: The average ENR is defined as the average of the ENRs of all nodes.

Original: The feature quality of GCNs output is investigated to prove the effectiveness of graph structurelearning: The mean Average Precision ( mAP ) [63] is used from the perspective of identification and
Revised: The feature quality of GCNs output is investigated to prove the effectiveness of graph structure learning: The mean Average Precision ( mAP ) [68] is used from the perspective of identification and

Original: ROC curves are shown from the perspective of verification.
Revised: ROC curves are shown from the perspective of verification.

Original: The higher the quality of the featuresthe better the graph structure learning.
Revised: The higher the quality of the features the better the graph structure learning.

Original: In downstream ReID tasks, mAP R and Rank1 accuracy ( R1 )are used.
Revised: In downstream ReID tasks, mAP R and Rank1 accuracy ( R1 ) are used.

Original: Compared to mAP, mAP R additionally takes into account the camera
Revised: Compared to mAP, mAP R additionally takes into account the camera

Original: id information, i.e.,samples from the same camera as the probe will not be counted in evaluation.
Revised: id information, i.e., samples from the same camera as the probe will not be counted in evaluation.

Original: BCubed ( F B ) [3, 1]and Pairwise ( F P ) [48] F-scores are used for evaluating clustering tasks.
Revised: BCubed ( F B ) [3, 1] and Pairwise ( F P ) [51] F-scores are used for evaluating clustering tasks.

Original: 3.2 Effectiveness of the Sim-M
Revised: 3.2 Effectiveness of the Sim-M

Revised: [Table]

Original: To verify the robustness of Sim-M, the study is first conducted on MS-Celeb (Part1) by evaluating thediscriminatory ability on different average ENRs comparing with Sim-S. The Sim-S uses the cosinesimilarity here for simplicity and without loss of generality.
Revised: To verify the robustness of Sim-M, the study is first conducted on MS-Celeb (Part1) by evaluating the discriminatory ability on different average ENRs comparing with Sim-S. The Sim-S uses the cosine similarity here for simplicity and without loss of generality.

Original: For the sake of sparsity and efficiencyit brings, the graphs are built based on k NN via the approximate nearest neighbour algorithm.
Revised: For the sake of sparsity and efficiency it brings, the graphs are built based on k NN via the approximate nearest neighbour algorithm.

Original: Theaverage ENR of graphs is controlled by using different k values when searching neighbours.
Revised: The average ENR of graphs is controlled by using different k values when searching neighbours.

Original: AUC is adopted to evaluate the similarity score between node pairs which have k NN relations.
Revised: AUC is adopted to evaluate the similarity score between node pairs which have k NN relations.

Original: Table 1presents that as the k increases, more and more noise is added (ENR gets larger).
Revised: Table 1 presents that as the k increases, more and more noise is added (ENR gets larger).

Original: For Sim-M,however, more samples can be drawn and more tests can be run, so the advantage of Sim-M over Sim-S becomes increasingly obvious.
Revised: For Sim-M, however, more samples can be drawn and more tests can be run, so the advantage of Sim-M over Sim-S becomes increasingly obvious.

Original: This demonstrates the robustness of Sim-M to noise.
Revised: This demonstrates the robustness of Sim-M to noise.

Original: Thisphenomenon also conforms to the condition and conclusion in Theorem 1 .
Revised: This phenomenon also conforms to the condition and conclusion in Theorem 1.

Original: However, as k increasesand ENR gets larger, too much noise can not be handled by multiple tests, so the performance of SimM begins to drop.
Revised: However, as k increases and ENR gets larger, too much noise can not be handled by multiple tests, so the performance of Sim-M begins to drop.

Original: This downward trend can be slowed down by the careful design of B ttention .
Revised: This downward trend can be slowed down by the careful design of B -Attention .

Original: Thesame experiments are conducted on the MSMT17 and VeRi-776, with similar findings in Section A.2.
Revised: The same experiments are conducted on the MSMT17 and VeRi-776, with similar findings in Section A.2.

Original: 3.3 Effectiveness of the B ttention
Revised: 3.3 Effectiveness of the B -Attention

Original: [Table] To demonstrate the robustness of B ttention mechanism, the study is first conducted on MS-Celeb (Part1)by evaluating the performance of B ttention with different average ENRs, comparing it to other baselines.
Revised: To demonstrate the robustness of B -Attention mechanism, the study is first conducted on MSCeleb (Part1) by evaluating the performance of B -Attention with different average ENRs, comparing it to other baselines.

Original: Then, comparison experiments are also conducted on MSMT17, VeRi-776 and also larger scales(Part3-9) of MS-Celeb with the best ENR settings foreach dataset.
Revised: Then, comparison experiments are also conducted on MSMT17, VeRi-776 and also larger scales (Part3-9) of MS-Celeb with the best ENR settings for each dataset.

Original: Comparison baselines include the original features of each dataset, a commonly used formof GCN [32, 53], Transformer, GCN with the selfattention (GCN with A self ), Transformer with the
Revised: Comparison baselines include the original features of each dataset, a commonly used form of GCN [34, 57],

Original:  B ttention mechanism (Trans with A band ), and GCNwith B ttention mechanism (GCN with A band ).
Revised: Transformer, GCN with the self-attention (GCN with A self ), Transformer with the B -Attention mechanism (Trans with A band ), and GCN with B -Attention mechanism (GCN with A band ).

Original: Transformer here is only used as an encoder for structurelearning, no positional encoding is required.
Revised: Transformer here is only used as an encoder for structure learning, no positional encoding is required.

Original: Network architectures and settings The model is composed of multiple GCN layers with B ttention , and an extra fully connected layer with PReLU [22] activation.
Revised: Network architectures and settings The model is composed of multiple GCN layers with BAttention , and an extra fully connected layer with PReLU [23] activation.

Original: The output featureshave a dimension of 2048 for all datasets.
Revised: The output features have a dimension of 2048 for all datasets.

Original: The number of GCN layers is also tuned respectively fordifferent datasets: MS-Celeb uses three GCN layers; MSMT17 and VeRi-776 use one GCN layer.
Revised: The number of GCN layers is also tuned respectively for different datasets: MS-Celeb uses three GCN layers; MSMT17 and VeRi-776 use one GCN layer.

Original: Each node acts as the probe to search its k NN to construct the graph.
Revised: Each node acts as the probe to search its k NN to construct the graph.

Original: The k value is tuned respectivelyfor different datasets: 120 for MS-Celeb, 30 for MSMT17, and 80 for VeRi-776.
Revised: The k value is tuned respectively for different datasets: 120 for MS-Celeb, 30 for MSMT17, and 80 for VeRi-776.

Original: Each node in adataset can be enhanced multiple times, as either a probe or a neighbour.
Revised: Each node in a dataset can be enhanced multiple times, as either a probe or a neighbour.

Original: The Hinge Loss [43, 53] isused to classify node pairs.
Revised: The Hinge Loss [45, 57] is used to classify node pairs.

Original: The initial learning rate is 0.008 with the cosine annealing strategy.
Revised: The initial learning rate is 0.008 with the cosine annealing strategy.

Original: Performance on different average ENRs Models are trained on Part0 and tested on Part1 of
Revised: Performance on different average ENRs Models are trained on Part0 and tested on Part1 of

Original: MS-Celeb.
Revised: MS-Celeb.

Original: All baselines use three GCN/Transformer layers for MS-Celeb, except the GCN with
Revised: All baselines use three GCN/Transformer layers for MS-Celeb, except the GCN with

Original: A self which uses two GCN layers (observed to have better performance than three GCN layers).
Revised: A self which uses two GCN layers (observed to have better performance than three GCN layers).

Original: Fora certain ENR, the same k value is used in training and testing.
Revised: For a certain ENR, the same k value is used in training and testing.

Original: Figure 3 shows how the output featurequalities (mAP) change with different average ENRs for all baselines.
Revised: Figure 3 shows how the output feature qualities (mAP) change with different average ENRs for all baselines.

Original: It shows that the GCN and
Revised: It shows that the GCN and

Original: Transformer with A band work the best in most average ENRs in terms of both mAP values and theirrobustness to different ENRs.
Revised: Transformer with A band work the best in most average ENRs in terms of both mAP values and their robustness to different ENRs.

Original: The GCN has the worst results, and the performances of the ones with
Revised: The GCN has the worst results, and the performances of the ones with

Original: A self are in between.
Revised: A self are in between.

Original: This validates that a commonly used form of GCN is sensitive to noisy edgesin input graphs; the proposed B ttention can well alleviate the influence of noisy edges and show thestrongest robustness when learning graph structure; the A self also has some power of dealing withnoisy edges, but it is worse than A band .
Revised: This validates that a commonly used form of GCN is sensitive to noisy edges in input graphs; the proposed B -Attention can well alleviate the influence of noisy edges and show the strongest robustness when learning graph structure; the A self also has some power of dealing with noisy edges, but it is worse than A band .

Original: When k value is too small (e.g., 5, 10), the performances of
Revised: When k value is too small (e.g., 5, 10), the performances of

Original: A band and A self are similar.
Revised: A band and A self are similar.

Original: This indicates that the Q ttention part of B ttention can hardly bring apositive effect when there are too few context neighbours, i.e. the number of single tests gathered isinsufficient.
Revised: This indicates that the Q -Attention part of B -Attention can hardly bring a positive effect when there are too few context neighbours, i.e. the number of single tests gathered is insufficient.

Original: In addition, GCN and Transformer have very similar performance, either withA band or
Revised: In addition, GCN and Transformer have very similar performance, either with

Original: A  self .
Revised: A band or A self .

Original: This indicates that base networks are not critical, and both attention mechanisms work forthe two base networks.
Revised: This indicates that base networks are not critical, and both attention mechanisms work for the two base networks.

Original: The performance degradation of B ttention with very large ENR (e.g., 0.64)indicates that the current form B ttention still has improvement room, although works much betterthan the self-attention.
Revised: The performance degradation of B -Attention with very large ENR (e.g., 0.64) indicates that the current form B -Attention still has improvement room, although works much better than the self-attention.

Original: [Figure]
Revised: Some empirical case studies on self attention, Q -Attention and B -Attention are shown in Section F of Appendix to better understand the effect of multiple tests.

Original: [Figure]
Revised: [Figure]

Original: Dataset

Original: MS-Celeb

Original: MSMT

Original: VeRiOriginal Feature 80.

Original: [Table]
Revised: [Table]

Original: Performance on different types of objects To further validate the B ttention mechanism, comparisons are also made on datasets with other types of objects: MSMT17 and VeRi-776.
Revised: Performance on different types of objects To further validate the B -Attention mechanism, comparisons are also made on datasets with other types of objects: MSMT17 and VeRi-776.

Original: In thesetwo datasets, k values and network depths are also tuned for all baselines, aiming for their bestperformances.
Revised: In these two datasets, k values and network depths are also tuned for all baselines, aiming for their best performances.

Original: For both datasets, one GCN layer is used for GCN, GCN with A band , and Transformerwith A band ; three GCN layers are used for GCN or Transformer with A self .
Revised: For both datasets, one GCN layer is used for GCN, GCN with A band , and Transformer with A band ; three GCN layers are used for GCN or Transformer with A self .

Original: In MSMT17, k =10for GCN and “GCN w/ A self ”; k =20 for Transformer; k =30 for the two B ttention methods.
Revised: In MSMT17, k = for GCN and “GCN w/ A self ”; k =20 for Transformer; k =30 for the two B -Attention methods.

Original: In
Revised: In

Original: VeRi-776, k =80 for “GCN w/ A band ” and “Trans w/ A band ”, and k =20 for the rest of baselines.
Revised: VeRi-776, k =80 for “GCN w/ A band ” and “Trans w/ A band ”, and k =20 for the rest of baselines.

Original: The best mAP results of all baselines are listed in Table 2, where the results for MS-Celeb are thepeak values in Figure 3.
Revised: The best mAP results of all baselines are listed in Table 2, where the results for MS-Celeb are the peak values in Figure 3.

Original: From Table 2, the B ttention works the best in all three datasets; A selfworks better than GCN; and all output features are much better than the original ones, although theoriginal features of MSMT17 and VeRi-776 are current SOTA ones.
Revised: From Table 2, the B -Attention works the best in all three datasets; A self works better than GCN; and all output features are much better than the original ones, although the original features of MSMT17 and VeRi-776 are current SOTA ones.

Original: The best performances areachieved by the GCN with A band in all datasets.
Revised: The best performances are achieved by the GCN with A band in all datasets.

Original: Their ROC curves are shown in Figure 4, wherethe same observations can be seen from verification perspective.
Revised: Their ROC curves are shown in Figure 4, where the same observations can be seen from verification perspective.

Original: As shown in Figure 4 and Table 2,the performance differences among those baselines with attention mechanisms are less significantin MSMT17 where k values are relatively small and close across baselines.
Revised: As shown in Figure 4 and Table 2, the performance differences among those baselines with attention mechanisms are less significant in MSMT17 where k values are relatively small and close across baselines.

Original: This observation issimilar to that in Figure 3: the performance margin between A band and A self is less significantwhen average ENRs (i.e., k ) are relatively small.
Revised: This observation is similar to that in Figure 3: the performance margin between A band and A self is less significant when average ENRs (i.e., k ) are relatively small.

Original: This further shows that the benefit of B ttention islimited when with a small number of neighbours, as not many single tests to run and leverage.
Revised: This further shows that the benefit of B -Attention is limited when with a small number of neighbours, as not many single tests to run and leverage.

Original: Betterperformance can not be achieved by enlarging k values for MSMT17, as its average sample amountfor each individual is only 31.
Revised: Better performance can not be achieved by enlarging k values for MSMT17, as its average sample amount for each individual is only 31.

Revised: Our analysis assumes that for any node v , there are more nodes in the k nearest neighbors sharing the same category as v than those sharing different categories (i.e. α > 1 2 ).

Revised: By significantly enlarging k , we will end up with α < 1 2 in k nearest neighbors, which fails the assumption of our algorithm.

Original: Too large k values would result in too big ENR which can not behandled by the current form of B ttention as discussed on Figure 3.
Revised: Too large k values would result in too big ENR which can not be handled by the current form of B -Attention as discussed on Figure 3.

Original: Experiments are also conducted to compare these baselines on larger scales of MS-Celeb (i.e.,
Revised: Experiments are also conducted to compare these baselines on larger scales of MS-Celeb (i.e.,

Original: Part3-9) in Section A.3 of Appendix.
Revised: Part3-9) in Section A.3 of Appendix.

Original: Results in Table 6 and Figure 5 show that all baselines haveperformance degradation when the scale enlarges while the degradation of that “w/ A band ” is muchsmaller than baselines.
Revised: Results in Table 6 and Figure 5 show that all baselines [Figure] [Figure] [Figure]

Revised: [Figure]

Revised: rate

Revised: [Figure]

Revised: [Figure]

Revised: T r u e p

Revised: o

Revised: s i

Revised: t i v e r a t

Revised: e

Revised: [Figure]

Revised: [Figure]

Revised: / A self

Revised: TransGCNOriginal (

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: p

Revised: [Figure]

Revised: TransGCNOriginal (

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: T r u

Revised: [Figure]

Revised: p o s i t i v e r a t e

Revised: GCN w/ A

Revised: [Figure]

Revised: Trans w/

Revised: A band GCN w/

Revised: [Figure]

Revised: self

Revised: TransGCNOriginal (c) VeRi-

Revised: Dataset

Revised: [Table]

Revised: Part

Revised: Part

Revised: Part

Revised: Part

Revised: Metrics

Revised: F P

Revised: F B

Revised: F P

Revised: F B

Revised: F P

Revised: F B

Original: These results for the cases with different ENRs, on different types of objectsand larger scales of testsets, together validate the robustness and effectiveness of proposed Bttentionin graph structure learning.
Revised: F P F B F P F B

Original: Ablation experiments on the design of the topological architecture and attention fusion method arealso conducted in Section A.4 of Appendix.
Revised: K-Means 79. 69.42 HAC 70. 66.96 DBSCAN 67.

Original: [Table]
Revised: [Table]

Revised: have performance degradation when the scale enlarges while the degradation of that “w/ A band ” is much smaller than baselines.

Original:  of the downstream tasks
Revised: These results for the cases with different ENRs, on different types of objects and larger scales of testsets, together validate the robustness and effectiveness of proposed B -Attention in graph structure learning. Ablation experiments on the design of the topological architecture and attention fusion method are also conducted in Section A.4 of Appendix. 3.4 Effectiveness of the downstream tasks

Original: The improvement of graph structure learning can also be proven through the downstream tasks byexploiting the features it produces.
Revised: The improvement of graph structure learning can also be proven through the downstream tasks by exploiting the features it produces.

Original: The better the graph structure is learned, the better the performanceon downstream tasks.
Revised: The better the graph structure is learned, the better the performance on downstream tasks.

Original: Graphs have important applications in large-scale clustering and re-rerankingin ReID.
Revised: Graphs have important applications in large-scale clustering and re-reranking in ReID.

Original: However, they suffer from inaccurate weight problems when learning and working with thegraph structure.
Revised: However, they suffer from inaccurate weight problems when learning and working with the graph structure.

Original: Armed the GCN with A band , many SOTA results are achieved.
Revised: Armed the GCN with A band , many SOTA results are achieved.

Original: [Table]

Original: Performance on clustering tasks
Revised: Performance on clustering tasks

Original: Theclustering tasks are evaluated on MSCeleb and MSMT17.
Revised: The clustering tasks are evaluated on MS-Celeb and MSMT17.

Original: Both of them havea large number of classes and are suitable as clustering benchmarks.
Revised: Both of them have a large number of classes and are suitable as clustering benchmarks.

Original: The samefeatures output by “GCN w/ A band ” areused as that in Section A.3 for MS-Celeband Section 3.3 for MSMT17.
Revised: The same features output by “GCN w/ A band ” are used as that in Section A.3 for MS-Celeb and Section 3. for MSMT17.

Original: The sameclustering method (denoted “G-cut”) isadopted as that in [19, 53], where pairsof nodes are linked when their similarity scores are larger than a tuned threshold, and clustering is finally completedby transitively merging all links via aunion-find algorithm [2].
Revised: The same clustering method (denoted “G-cut”) is adopted as that in [20, 57], where pairs of nodes are linked when their similarity scores are larger than a tuned threshold, and clustering is finally completed by transitively merging all links via a union-find algorithm [2].

Original: The thresholdis tuned for each dataset, aiming for the best balance between BCubed and Pairwise F-Scores.
Revised: The threshold is tuned for each dataset, aiming for the best balance between BCubed and Pairwise F-Scores.

Original: Inaddition, map equation based unsupervised clustering algorithm Infomap [44, 57] is also adoptedhere.
Revised: In addition, map equation based unsupervised clustering algorithm Infomap [46, 61] is also adopted here.

Original: Their BCubed and Pairwise F-scores are listed in Table 3 and Table 10, where the results ofcurrent SOTA GCN/Transformer methods (Clusformer [40], STAR-FC [47] and Ada-NETS [53])and three conventional clustering approaches (K-Means [36], HAC [49] and DBSCAN [16]) are alsolisted.
Revised: Their BCubed and Pairwise F-scores are listed in Table 3 and Table 10, where the results of current SOTA GCN/Transformer methods (Clusformer [42], STAR-FC [50] and Ada-NETS [57]) and three conventional clustering approaches (K-Means [38], HAC [52] and DBSCAN [17]) are also listed.

Original: The results show that the features enhanced via our approach have the best clustering resultsin all datasets, either with G-cut or Infomap, both of which outperform the current SOTA methods
Revised: The results show that the features enhanced via our approach have the best clustering results in all datasets, either with G-cut or Infomap, both of which outperform the current SOTA methods

Original: Ada-NETS and STAR-FC
Revised: Ada-NETS and STAR-FC

Original: with large margins.
Revised: with large margins.

Original: In comparison, the results of the ones with Infomapare better than those with G-cut.
Revised: In comparison, the results of the ones with Infomap are better than those with G-cut.

Revised: [Table]

Original: Performance on ReID tasks The ReID tasks is evaluated on two ReID datasets: MSMT17 and VeRi-776.
Revised: The ReID tasks is evaluated on two ReID datasets: MSMT17 and VeRi-776.

Original: The enhanced features used are the same as those in Table 2.
Revised: The enhanced features used are the same as those in Table 2.

Original: mAP R and R1 are evaluatedfollowing the standard probe and gallery protocols of MSMT17 and VeRi-776.
Revised: mAP R and R1 are evaluated following the standard probe and gallery protocols of MSMT17 and VeRi-776.

Original: The results are shownin Table 4.
Revised: The results are shown in Table 4.

Original: As the output feature of learned graph can also be understood as a rerank process in ReID,comparisons are also made on commonly used rerank methods: QE [12], LBR [37], KR [62] and
Revised: As the output feature of learned graph can also be understood as a rerank process in ReID, comparisons are also made on commonly used rerank methods: QE [13], LBR [39], KR [67] and

Original: ECN [45].
Revised: ECN [47].

Original: Table 4 shows that the “GCN w/ A band ” achieves the best or comparable performance, andall enhanced features are much better than the original SOTA ones.
Revised: Table 4 shows that the “GCN w/ A band ” achieves the best or comparable performance [Table] and all enhanced features are much better than the original SOTA ones.

Original: In MSMT17, similar to the resultsin Section 3.3, the baselines with attention mechanism have very close performance, but work much better than the GCN.
Revised: In MSMT17, similar to the results in Section 3.3, the baselines with attention mechanism have very close performance, but work much better than the GCN.

Original: In VeRi-776, from the mAP R perspective, B ttention works better than A self ,and both of them are better than the GCN, but their R1 results are comparable.
Revised: In VeRi-776, from the mAP R perspective, B -Attention works better than A self , and both of them are better than the GCN, but their R1 results are comparable.

Original: These results stillsupport the superiority of the proposed approach, although R1 results of VeRi-776 are comparable,as mAP R is a more comprehensive indication of feature quality for ReID.
Revised: These results still support the superiority of the proposed approach, although R1 results of VeRi-776 are comparable, as mAP R is a more comprehensive indication of feature quality for ReID.

Original: The experiments on thecombination of “GCN w/ A band ” with these commonly used rerank methods are also conducted in Section A.6, achieving more SOTA performances and proving the complementarity between them.
Revised: The experiments on the combination of “GCN w/ A band ” with these commonly used rerank methods are also conducted in Section A.6, achieving more SOTA performances and proving the complementarity between them.

Original:  Related Work
Revised:  Related Work

Original: Graph Convolutional Networks.
Revised: Graph Convolutional Networks.

Original: GCNs [32, 21, 51] extends the operation of convolution fromthe grid-data (e.g. images, videos) to non-grid data which is more common in the real world.
Revised: GCNs [34, 22, 54] extends the operation of convolution from the grid-data (e.g. images, videos) to non-grid data which is more common in the real world.

Original: Theclassic GCN [32] is proposed based on the spectral theory and achieves promising results on citationnetwork datasets: Citeseer, Cora and Pubmed [46].
Revised: The classic GCN [34] is proposed based on the spectral theory and achieves promising results on citation network datasets: Citeseer, Cora and Pubmed [49].

Original: GraphSAGE [21] further extends GCNs fromtransductive framework to inductive framework and obtains better results.
Revised: GraphSAGE [22] further extends GCNs from transductive framework to inductive framework and obtains better results.

Original: GAT [51] introduces theattention mechanism in GCN, making it more expressive.
Revised: GAT [54] introduces the attention mechanism in GCN, making it more expressive.

Original: Fast-GCN [6] interprets graph convolutionsas integral transforms of embedding functions under probability measures, which not only is efficientfor training but also generalizes well for inference.
Revised: Fast-GCN [6] interprets graph convolutions as integral transforms of embedding functions under probability measures, which not only is efficient for training but also generalizes well for inference.

Revised: Some research works [48, 66, 27, 56, 7] also advance GCNs for relational data modeling.

Original: However, these GCNs are all investigated onrelational datasets that have explicit graph structures.
Revised: However, these GCNs are all investigated on relational datasets that have explicit graph structures.

Original: This work aims to extend the capacity of GCNsto work on datasets (e.g., image objects) which have no explicitly defined graph structures.
Revised: This work aims to extend the capacity of GCNs to work on datasets (e.g., image objects) which have no explicitly defined graph structures.

Original: Graph Structure Learning.
Revised: Graph Structure Learning.

Original: The literature [56] suggests that research on graph structure learningfor GCNs can be roughly divided into discrete-based and weighted-based ones.
Revised: The literature [60] suggests that research on graph structure learning for GCNs can be roughly divided into discrete-based and weighted-based ones.

Original: Some discretebased methods [15, 29, 38, 17] treat the graph structures as random distributions and sample binaryadjacency matrices from them.
Revised: Some discretebased methods [16, 31, 40, 18] treat the graph structures as random distributions and sample binary adjacency matrices from them.

Original: Another way [53] constructs discrete graphs based on heuristicproxy objectiveness aiming for clean yet rich neighbours for each node.
Revised: Another way [57] constructs discrete graphs based on heuristic proxy objectiveness aiming for clean yet rich neighbours for each node.

Original: The binary matricesmethods are difficult to optimize because of the gradient breakage, so such methods resort tovariational inference (e.g., approaches in [15, 38]), reinforcement learning (e.g., approaches in [29])or optimization in stages (e.g., approaches in [53]), which may lead to sub-optimal solutions.
Revised: The binary matrices methods are difficult to optimize because of the gradient breakage, so such methods resort to variational inference (e.g., approaches in [16, 40]), reinforcement learning (e.g., approaches in [31]) or optimization in stages (e.g., approaches in [57]), which may lead to sub-optimal solutions.

Original: Inaddition, the parameter size of distributions [17] can be very large (e.g., O ( Nk ) where N is thenumber of nodes and k is the number of neighbours for each node) and therefore not suitable for largedatasets.
Revised: In addition, the parameter size of distributions [18] can be very large (e.g., O ( Nk ) where N is the number of nodes and k is the number of neighbours for each node) and therefore not suitable for large datasets.

Original: The weighted-based methods [8, 9, 41, 60, 26, 7, 24, 33, 19, 40], which are the focus of thispaper, learn weighted adjacency matrices so that it can be optimized by SGD techniques [42, 31, 5].
Revised: The weighted-based methods [9, 10, 43, 64, 28, 8, 25, 35, 20, 42], which are the focus of this paper, learn weighted adjacency matrices so that it can be optimized by SGD techniques [44, 33, 5].

Original: The main difference between these works [8, 9, 41, 60, 26, 7, 24, 33] is in the similarity measure,which can be classified into attention-based [8, 9, 41, 60, 26, 7] and kernel-based [24, 33] roughly.
Revised: The main difference between these works [9, 10, 43, 64, 28, 8, 25, 35] is in the similarity measure, which can be classified into attention-based [9, 10, 43, 64, 28, 8] and kernelbased [25, 35] roughly.

Original: Toenhance the discriminatory power of the similarity measure, they have adopted various forms: addinglearnable parameters [8, 26, 7], introducing nonlinearity [9, 41, 60], Gaussian kernel functions [24, 33]or self-attention [19, 40, 50, 11] etc.
Revised: To enhance the discriminatory power of the similarity measure, they have adopted various forms: adding learnable parameters [9, 28, 8], introducing nonlinearity [10, 43, 64], Gaussian kernel functions [25, 35] or self-attention [20, 42, 53, 12] etc.

Original: However, these methods are sensitive to the noise from featurerepresentations, particular when each node is represented by a high dimensional vector, leading to theproblem of inaccurate weights and vulnerable graph structure.
Revised: However, these methods are sensitive to the noise from feature representations, particular when each node is represented by a high dimensional vector, leading to the problem of inaccurate weights and vulnerable graph structure.

Original: This work aims to propose a multiplesamples based similarity measure to improve the robustness of graph structure learning over images.
Revised: This work aims to propose a multiple samples based similarity measure to improve the robustness of graph structure learning over images.

Original: Another approach uses the low-rank, sparsity and feature smoothness properties of the real-worldgraph to optimize the graph structure [27], but these priors are suitable for community networksrather than graph of image objects.
Revised: Another approach uses the low-rank, sparsity and feature smoothness properties of the real-world graph to optimize the graph structure [29] but these priors are suitable for community networks rather than graph of image objects.

Original: Some work [28] learns to augment the given graph with newedges by reinforcement learning to improve the performance of downstream tasks, but existing noisyedges problem in graphs cannot be handled.
Revised: Some work [30] learns to augment the given graph with new edges by reinforcement learning to improve the performance of downstream tasks, but existing noisy edges problem in graphs cannot be handled.

Original:  Conclusion
Revised:  Conclusion

Original: The problem of inaccurate edge weights is identified in graph structure learning, analysed fromthe perspective of noise in feature vectors, and handled based on several tests.
Revised: The problem of inaccurate edge weights is identified in graph structure learning, analysed from the perspective of noise in feature vectors, and handled based on several tests.

Original: A novel and robustsimilarity measure Sim-M is proposed, proven to be superior and verified in experiments.
Revised: A novel and robust similarity measure Sim-M is proposed, proven to be superior and verified in experiments.

Original: Then it isdesigned in an elegant matrix form, i.e., B ttention mechanism, to improve the graph structure qualityin GCNs.
Revised: Then it is designed in an elegant matrix form, i e., B -Attention mechanism, to improve the graph structure quality in GCNs.

Original: The superiority and robustness of B ttention are validated by comprehensive experimentsand achieve many SOTA results on clustering and ReID benchmarks. Checklist
Revised: The superiority and robustness of B -Attention are validated by comprehensive experiments and achieve many SOTA results on clustering and ReID benchmarks

Original: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope? [Yes] See abstract and Section 1.(b) Did you describe the limitations of your work? [Yes] See Section B in Appendix.(c)
Revised: Acknowledgments and Disclosure of Funding

Original: Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See

Original: Section C in Appendix.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes] We have read the ethics guidelines on the official website.2. If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 2.2.
Revised: We thank many colleagues at DAMO Academy of Alibaba, in particular, Yuqi Zhang, Weihua Chen and Hao Luo for useful discussions

Original: b

Original: ) Did you include complete proofs of all theoretical results

Original: [Yes] See Section

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] The code andthe data are proprietary The instructions is in Section 3.(b)
Revised: This research was partially funded by the National Science Foundation of China (No. 62172443) and

Original: Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] See in Section 3.1, paragraph of Network architectures andsettings and the paragraphs describing the corresponding experiment in Section 3.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running exper-

Original: iments multiple times)?

Original: [No] Error bars are not reported because it would be too

Original: computationally expensive.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section A.7.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See the Datasets

Original: paragraph in Section 3.1.

Original: (b) Did you mention the license of the assets?

Original: [No] The used assets are very common in

Original: papers of computer vision.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?

Original: [No] The used assets are very common in papers of computer vision.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?

Original: [Yes] See the footnote in Datasets paragraph in

Original: Section 3.1: Considering the ethical and privacy issues, all the datasets used in this

Original: paper are feature vectors which can not be restored to images.

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable? [N/A] [Table] [Table]
Revised: Hunan Provincial Natural Science Foundation of China (No. 2022JJ30053)

Original: A ExperimentsA.1 Details of the dataset
Revised: A Experiments A.1 Details of the dataset

Original: MS-Celeb [20] is a dataset with 10M face images of nearly 100K individuals.
Revised: MS-Celeb [21] is a dataset with 10M face images of nearly 100K individuals.

Original: A cleaned version [14]with about 5.8M face images of 86K identities is used in this paper.
Revised: A cleaned version [15] with about 5.8M face images of 86K identities is used in this paper.

Original: The protocol and features arethe same as that in [59], where the dataset is divided into 10 parts by identities (Part0-9): Part0 fortraining and Part1-9 for testing.
Revised: The protocol and features are the same as that in [63], where the dataset is divided into 10 parts by identities (Part0-9): Part0 for training and Part1-9 for testing.

Original: The feature dimension is 256.
Revised: The feature dimension is 256.

Original: The data amounts of Part 0-9 (0, 1, 3,5, 7 and 9) are respectively: 584K, 584K, 1.74M, 2.89M, 4.05M and 5.21M.
Revised: The data amounts of Part 0-9 (0, 1, 3, 5, 7 and 9) are respectively: 584K, 584K, 1.74M, 2.89M, 4.05M and 5.21M.

Original: MSMT17 [55] is the current largest ReID dataset.
Revised: MSMT17 [59] is the current largest ReID dataset.

Original: It contains 32,621 images of 1,041 individuals fortraining and 93,820 images of 3,060 individuals for testing.
Revised: It contains 32,621 images of 1,041 individuals for training and 93,820 images of 3,060 individuals for testing.

Original: These images are captured by 15 camerasunder different time periods, weather and light conditions.
Revised: These images are captured by 15 cameras under different time periods, weather and light conditions.

Original: This work uses the SOTA features from
Revised: This work uses the SOTA features from

Original: AGW [61] with a dimension of 2048.
Revised: AGW [65] with a dimension of 2048.

Original: VeRi-776 [35] is a commonly used vehicle ReID benchmark, which contains over 40K boundingboxes of 619 vehicles captured by 20 cameras in unconstrained traffic scenes.
Revised: VeRi-776 [37] is a commonly used vehicle ReID benchmark, which contains over 40K bounding boxes of 619 vehicles captured by 20 cameras in unconstrained traffic scenes.

Original: The dataset is dividedinto two parts: 37,715 images of 576 vehicles for training, and 13,257 images of 200 vehicles fortesting.
Revised: The dataset is divided into two parts: 37,715 images of 576 vehicles for training, and 13,257 images of 200 vehicles for testing.

Original: This work uses the features extracted by the SOTA ViT-B/16 Baseline in TransReID
Revised: This work uses the features extracted by the SOTA ViT-B/16 Baseline in TransReID

Original: [23],with 768 dimensions.
Revised: [24], with 768 dimensions.

Original: A.2 Effectiveness of the Sim-M
Revised: A.2 Effectiveness of the Sim-M

Original: The experiments on MSMT17 and VeRi-776 also show that the advantage of Sim-M over Sim-S isincreasingly obvious as k increases and then drops for introduction of too much noise as in Table 5.
Revised: The experiments on MSMT17 and VeRi-776 also show that the advantage of Sim-M over Sim-S is increasingly obvious as k increases and then drops for introduction of too much noise as in Table 5.

Original: a factor of 100 for a better show.

Original: [Table]
Revised: [Table]

Original: A.3 Performance on larger scales of MS-Celeb
Revised: A.3 Performance on larger scales of MS-Celeb

Original: Experiments are also conducted to compare these baselines on larger scales of MS-Celeb (i.e.,Part3-
Revised: Experiments are also conducted to compare these baselines on larger scales of MS-Celeb (i.e.,

Original:  The same model for MS-Celeb Part1 is used as in Section 3.3, but tested on Part3-9.
Revised: Part3-9). The same model for MS-Celeb Part1 is used as in Section 3.3, but tested on Part3-9.

Original: The results in Table 6 show that all baselines have performance degradation when the scale enlarges.
Revised: The results in Table 6 show that all baselines have performance degradation when the scale enlarges.

Original: The B ttentionobtains the best results in all parts; A self works better than GCN; all output features are much betterthan the original ones.
Revised: The B -Attention obtains the best results in all parts; A self works better than GCN; all output features are much better than the original ones.

Original: Among the baselines with attention mechanisms, the ones using GCN-basednetworks have better performances.
Revised: Among the baselines with attention mechanisms, the ones using GCN-based networks have better performances.

Original: The GCN with A band has the highest mAP scores.
Revised: The GCN with A band has the highest mAP scores.

Original: Figure 5shows the ROC curves of GCN, “GCN w/ A self ” and “GCN w/ A band ” across all parts of MS-Celeb,from which we can also see the performance degradation with larger testsets.
Revised: Figure 5 shows the ROC curves of GCN, “GCN w/ A self ” and “GCN w/ A band ” across all parts of MS-Celeb, from which we can also see the performance degradation with larger testsets.

Original: The performancedegradation of “GCN w/ A band ” is much smaller than the other two baselines, which is shown moreclearly from the differences of AUC in Table 7.
Revised: The performance degradation of “GCN w/ A band ” is much smaller than the other two baselines, which is shown more clearly from the differences of AUC in Table 7.

Original: In Table 7, differences on AUC of two adjacent parts(e.g., part1 - part3, part3 - part5, part5 - part7, part7 - part9) and their summation for each method areshown.
Revised: In Table 7, differences on AUC of two adjacent parts (e.g., part1 - part3, part3 - part5, part5 - part7, part7 - part9) and their summation for each method are shown.

Original: It can be observed that the summation of the differences of “GCN w/ A band ” is 1.49, whichis only 58.30% and 67.26% of the ones of GCN (2.56) and “GCN with A self " (2.22).
Revised: It can be observed that the summation of the differences of “GCN w/ A band ” is 1.49, which is only 58.30% and 67.26% of the ones of GCN (2.56) and “GCN with A self " (2.22).

Original: This validatesthat the features from “GCN w/ A band ” have a much smaller performance degradation than the othertwo baselines with larger testsets from the perspective of verification.
Revised: This validates that the features from “GCN w/ A band ” have a much smaller performance degradation than the other two baselines with larger testsets from the perspective of verification.

Original: Similar observations can alsobe seen in Table 6, where the performance degradation of the baselines with attention mechanisms ismuch smaller than GCN and original features.
Revised: Similar observations can also be seen in Table 6, where the performance degradation of the baselines with attention mechanisms is much smaller than GCN and original features.

Original: 1.0 False positive rate 0.
Revised: 1.0 False positive rate 0.

Original: T r u e p
Revised: T r u e p

Original: o
Revised: o

Original: s i
Revised: s i

Original: t i v e r a t
Revised: t i v e r a t

Original: e
Revised: e

Original: Part1Part3Part5Part7Part (a) GCN 0.
Revised: Part1Part3Part5Part7Part (a) GCN 0.

Original: 1.0 False positive rate 0.
Revised: 1.0 False positive rate 0.

Original: T r u e p
Revised: T r u e p

Original: o
Revised: o

Original: s i
Revised: s i

Original: t i v e r a t
Revised: t i v e r a t

Original: e
Revised: e

Original: Part1Part3Part5Part7Part (b) GCN with A self 0.
Revised: Part1Part3Part5Part7Part (b) GCN with A self 0.

Original: 1.0 False positive rate 0.
Revised: 1.0 False positive rate 0.

Original: T r u e p
Revised: T r u e p

Original: o
Revised: o

Original: s i
Revised: s i

Original: t i v e r a t
Revised: t i v e r a t

Original: e
Revised: e

Original: Part1Part3Part5Part7Part (c) GCN with A band
Revised: Part1Part3Part5Part7Part (c) GCN with A band

Original: Partition
Revised: Partition

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: GCN 90.
Revised: GCN 90.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Transformer93.
Revised: Transformer

Revised: [Table]

Original: GCN w/
Revised: GCN w

Original: [Table]
Revised: [Table]

Original: self 94.

Original: Trans w/
Revised: Trans w/

Original: A band 95.
Revised: A band 95.

Original: GCN w/ A band 95.
Revised: GCN w/ A band 95.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: A.4 Ablation study on B ttention
Revised: A.4 Ablation study on B -Attention

Original: Ablation experiments are conducted mainly on MS-Celeb to study the design of the topologicalarchitecture and attention fusion method of B ttention .
Revised: Ablation experiments are conducted mainly on MS-Celeb to study the design of the topological architecture and attention fusion method of B -Attention .

Original: Similar to Section 3.3, models are trained on Part0 and tested on Part1.
Revised: Similar to Section 3.3, models are trained on Part0 and tested on Part1.

Original: All settings other than the factor being investigated are the same as that in Section 3.3 for “GCN with A band ” with k = 120 .
Revised: All settings other than the factor being investigated are the same as that in Section 3.3 for “GCN with A band ” with k = 120 .

Original: A.4.1 topological architecture design
Revised: A.4.1 topological architecture design

Original: This study compares the topological architecture of the B ttention with three other designs: (1) onewith only the A qart of B ttention ; (2) one using only the Q ttention part but with the A X replaced by A self in Equation 8, denoted as (cid:101)
Revised: This study compares the topological architecture of the B -Attention with three other designs: (1) one with only the A qart of B -Attention ; (2) one using only the Q -Attention part but with the A X replaced by A self in Equation 8, denoted as (cid:101)

Original: A qart ; (3) and one with B ttention but with the A qart replaced by(cid:101)
Revised: A qart ; (3) and one with B -Attention but with the A qart replaced by (cid:101)

Original: A qart , denoted as (cid:101)
Revised: A qart , denoted as (cid:101)

Original: A band .
Revised: A band .

Original: Table 8 shows their mAP results, where the results of “GCN w/ A band ”and one with only A self are also listed for comparison.
Revised: Table 8 shows their mAP results, where the results of “GCN w/ A band ” and one with only A self are also listed for comparison.

Original: For the purpose of ablation study, the onewith only A self here has the same depth (i.e., three layers) with “GCN w/ A band ”, which is differentfrom that in Table 2 (two layers).
Revised: For the purpose of ablation study, the one with only A self here has the same depth (i.e., three layers) with “GCN w/ A band ”, which is different from that in Table 2 (two layers).

Original: The comparisons are made on two conditions: with or without W qart (i.e., W Q qart and W K qart ) in Equation 8.
Revised: The comparisons are made on two conditions: with or without W qart (i.e., W Q qart and W K qart ) in Equation 8.

Original: For the cases without W qart , W Q qart and W K qartare both identity matrices, i.e., Q qart =
Revised: For the cases without W qart , W Q qart and W Kqart are both identity matrices, i.e., Q qart =

Original: K qart =
Revised: K qart =

Original: A X .
Revised: A X .

Original: Their ROC curves are shown in Figure 6.
Revised: Their ROC curves are shown in Figure 6.

Original: It can be seen that the B ttention design achieves the best results in both mAP and ROC; A qarthas better performance than A self ; (cid:101)
Revised: It can be seen that the B -Attention design achieves the best results in both mAP and ROC; A qart has better performance than A self ; (cid:101)

Original: A qart and (cid:101)
Revised: A qart and (cid:101)

Original: A band are worse than A qart and A band respectively.
Revised: A band are worse than A qart and A band respectively.

Original: This shows that the combination of A self and A qart and the use of A X (instead of A self ) in A qart both contribute to the performance improvement.
Revised: This shows that the combination of A self and A qart and the use of A X (instead of A self ) in A qart both contribute to the performance improvement.

Original: In addition, the baselines with W qart have betterperformances (higher mAP scores and better ROC) than those without, indicating that W Qqart and W Kqart are also critical for better graph structure learning.
Revised: In addition, the baselines with W qart have better performances (higher mAP scores and better ROC) than those without, indicating that W Qqart and W Kqart are also critical for better graph structure learning.

Original: [Table]
Revised: [Table]

Original: The ablation experiments are also conducted on attention map fusion method in Section A.4.2in Appendix .
Revised: The ablation experiments are also conducted on attention map fusion method in Section A.4. in Appendix .

Original: The results in Table 9 and Figure 7 together will prove that the proposed form in
Revised: The results in Table 9 and Figure 7 together will prove that the proposed form in

Original: Equation 11 has the best performance across all datasets.
Revised: Equation 11 has the best performance across all datasets.

Original: A.4.2 Attention map fusion method
Revised: A.4.2 Attention map fusion method

Original: The attention map fusion method is studied by comparing it to two other designs: direct element-wiseaddition without weighting ( A qart + A self ) and element-wise multiplication ( A qart ⊙ A self ) in
Revised: The attention map fusion method is studied by comparing it to two other designs: direct element-wise addition without weighting ( A qart + A self ) and element-wise multiplication ( A qart ⊙ A self ) in

Original: Equation 11.
Revised: Equation 11.

Original: The results are shown in Table 9 and Figure 7: The θ qart A qart + θ self A self designworks the best, and the performance of A qart ⊙ A self is very close to θ qart A qart + θ self A self .
Revised: The results are shown in Table 9 and Figure 7: The θ qart A qart + θ self A self design works the best, and the performance of A qart ⊙ A self is very close to θ qart A qart + θ self A self .

Original: However, results of MSMT17 and VeRi-776 in Table 9 shows our proposed method has the bestperformance across all datasets.
Revised: However, results of MSMT17 and VeRi-776 in Table 9 shows our proposed method has the best performance across all datasets.

Original: [Table]
Revised: [Table]

Revised: T r u e p

Original:  [Figure]
Revised: o s i t i v e r [Figure]

Original: [Table]
Revised: [Table]

Original: A.5 Clustering performance on MSMT17 
Revised: A.5 Clustering performance on MSMT

Original: The clustering tasks are also evaluated on MSMT17 and the results are in Table 10.
Revised: The clustering tasks are also evaluated on MSMT17 and the results are in Table 10.

Original: A.6 Ensemble Performance on ReID tasks
Revised: A.6 Ensemble Performance on ReID tasks

Original: In addition to directly comparing with the commonly used rerank methods (e.g., QE [12], LBR [37],
Revised: In addition to directly comparing with the commonly used rerank methods (e.g., QE [13], LBR [39],

Original: KR [62] and ECN [45]) in Section 4.4, the “GCN with A band " can also be combined with them.
Revised: KR [67] and ECN [47]) in Section 4.4, the “GCN with A band " can also be combined with them.

Original: The commonly used rerank methods can utilize the enhanced features from “GCN with A band " forbetter ranking.
Revised: The commonly used rerank methods can utilize the enhanced features from “GCN with A band " for better ranking.

Original: The results are shown in Table 11.
Revised: The results are shown in Table 11.

Original: Since mAP R can provide a more comprehensiveevaluation of the feature quality than R1 for ReID, the mAP R is discussed more often.
Revised: Since mAP R can provide a more comprehensive evaluation of the feature quality than R1 for ReID, the mAP R is discussed more often.

Original: It can beobserved that new SOTA performances are obtained on both datasets.
Revised: It can be observed that new SOTA performances are obtained on both datasets.

Original: The mAP R on MSMT17 is ashigh as 79.67 by “ A band +KR" and on VeRi-776 is as high as 85.92 by “ A band +LBR".
Revised: The mAP R on MSMT17 is as high as 79.67 by “ A band +KR" and on VeRi-776 is as high as 85.92 by “ A band +LBR".

Original: For most ofthe commonly used rerank methods, the mAP R of reranking with A band is higher than with originalfeatures and also higher than “GCN w/ A band ”.
Revised: For most of the commonly used rerank methods, the mAP R of reranking with A band is higher than with original features and also higher than “GCN w/ A band ”.

Original: The only exception is “ A band +ECN" which is higherthan “GCN w/ A band ” but only comparable with “original+ECN".
Revised: The only exception is “ A band +ECN" which is higher than “GCN w/ A band ” but only comparable with “original+ECN".

Original: The results show that there is acomplementary relationship between “GCN with A band " and commonly used rerank methods.
Revised: The results show that there is a complementary relationship between “GCN with A band " and commonly used rerank methods.

Original: The“GCN with A band " can provide better features so that the commonly used rerank methods can befurther improved, although the R 1 results are only comparable.
Revised: The “GCN with A band " can provide better features so that the commonly used rerank methods can be further improved, although the R 1 results are only comparable.

Original: [Table]
Revised: [Table]

Original: GCN w/ A band 78.

Original: Original+QE 71.

Original: Original+LBR 70.

Original: Original+KR 76.

Original: Original+ECN 78.

Original: A band +QE 79.

Original: A band +LBR 79.

Original: A band +KR 79.

Original: A band +ECN 78.

Original: A.7 Declaration of the type of computing resources
Revised: A.7 Declaration of the type of computing resources

Original: All experiments are conducted on a single machine, with 54 E5-2682 v4 CPUs, 8 NVIDIA P100cards and 400G memory.
Revised: All experiments are conducted on a single machine, with 54 E5-2682 v4 CPUs, 8 NVIDIA P cards and 400G memory.

Original: B Limitations
Revised: B Limitations

Original: Our approach achieves promising results in graph structure learning, but it still has the followinglimitations:
Revised: Our approach achieves promising results in graph structure learning, but it still has the following limitations:

Original: • The B ttention requires more computation and number of parameters as shown in Section 2.3.
Revised: • The B -Attention requires more computation and number of parameters as shown in Section 2.3.

Original: However, the baseline methods can not achieve the same performance even adding their network depth.
Revised: However, the baseline methods can not achieve the same performance even adding their network depth.

Original: The layer number of network in each method is tuned in all experiments.
Revised: The layer number of network in each method is tuned in all experiments.

Original: • The superiority of B ttention is not obvious enough on datasets with few samples in each category compared with self attention as revealed in Section 3.3.
Revised: • The superiority of B -Attention is not obvious enough on datasets with few samples in each category compared with self attention as revealed in Section 3.3.

Original: • The performance of B ttention will also degrade when with large ENR as discussed inSection 3.3, indicating the current form still has improvement room on the robustnessagainst noise to further investigate.
Revised: • The performance of B -Attention will also degrade when with large ENR as discussed in Section 3.3, indicating the current form still has improvement room on the robustness against noise to further investigate.

Original: • The outputs B ttention will vary depending on the choice of nearest neighbours.
Revised: • The outputs B -Attention will vary depending on the choice of nearest neighbours.

Original: However,the superiority of this method can be guaranteed as long as the conditions in Section 2.2 aremet.
Revised: However, the superiority of this method can be guaranteed as long as the conditions in Section 2.2 are met.

Original: C Potential Negative Societal Impacts
Revised: C Potential Negative Societal Impacts

Original: The proposed graph structure learning method inherently is harmless just like many other AI technologies.
Revised: The proposed graph structure learning method inherently is harmless just like many other AI technologies.

Original: This technology may have negative societal impacts if someone uses it for maliciousapplications such as surveillance, personal information collections.
Revised: This technology may have negative societal impacts if someone uses it for malicious applications such as surveillance, personal information collections.

Original: As authors, we advocate forproper technology usage.
Revised: As authors, we advocate for proper technology usage.

Original: D Pseudocode for self-attention and Q ttention
Revised: D Pseudocode for self-attention and Q -Attention

Original: # X: the input feature matrix, with shape (N, D) def self_attention(X):
Revised: # X: the input feature matrix, with shape (N, D) def self_attention(X):

Original: Q_self=
Revised: Q_self=

Original: torch.matmul(X, W_self_Q)
Revised: torch.matmul(X, W_self_Q)

Original: K_self =
Revised: K_self =

Original: torch.matmul(X, W_self_K)
Revised: torch.matmul(X, W_self_K)

Original: self_attn = torch.matmul(Q_self, K_self.transpose(-2, -1)) return self_attn def Q_attention(X):
Revised: self_attn = torch.matmul(Q_self, K_self.transpose(-2, -1)) return self_attn def Q_attention(X):

Original: A_X=torch.matmul(X, X.transpose(-2, -1))
Revised: A_X=torch.matmul(X, X.transpose(-2, -1))

Original: Q_qart =
Revised: Q_qart =

Original: torch.matmul(A_X, W_qart_Q)
Revised: torch.matmul(A_X, W_qart_Q)

Original: K_qart = torch.matmul(A_X, W_qart_K)
Revised: K_qart = torch.matmul(A_X, W_qart_K)

Original: Q_attn = torch.matmul(Q_qart, K_qart.transpose(-2, -1)) return Q_attn
Revised: Q_attn = torch.matmul(Q_qart, K_qart.transpose(-2, -1)) return Q_attn

Original: E Detailed Proof of with generalized version of Chernoff Bounds
Revised: E Detailed Proof of with generalized version of Chernoff Bounds

Original: Below is a full analysis for the practical algorithm in Section 2.3 that does not assume a binaryrelationship between two nodes in A X .
Revised: Below is a full analysis for the practical algorithm in Section 2.3 that does not assume a binary relationship between two nodes in A X .

Original: Let X i denote a random variable with no distribution assumptions such that a ≤ X i ≤ b for all i .
Revised: Let X i denote a random variable with no distribution assumptions such that a ≤ X i ≤ b for all i .

Original: Define X = (cid:80) m 1 X
Revised: Define X = (cid:80) m 1 X

Original: i , µ = E ( X ) .
Revised: i , µ = E ( X ) .

Original: Then for all 0 < ϵ < 1 , we have the general version of Chernoffinequality as follows.
Revised: Then for all 0 < ϵ < 1 , we have the general version of Chernoff inequality as follows.

Original: ( i ) Upper Tail: P ( X ≥ (1 + ϵ ) µ )
Revised: ( i ) Upper Tail: P ( X ≥ (1 + ϵ ) µ )

Original: ≤ e − ϵ 2 µ 2 m ( b − a )2 ;( ii ) Lower Tail: P ( X ≤ (1 − ϵ ) µ )
Revised: ≤ e − ϵ 2 µ 2 m ( b − a )2 ; ( ii ) Lower Tail: P ( X ≤ (1 − ϵ ) µ )

Original: ≤ e − ϵ 2 µ 2 m ( b − a )2 .
Revised: ≤ e − ϵ 2 µ 2 m ( b − a )2 .

Original: The above show that with probability at least 1 − δ , (1 − ϵ ) µ ≤ X ≤ (1 + ϵ ) µ where δ :=
Revised: The above show that with probability at least 1 − δ , (1 − ϵ ) µ ≤ X ≤ (1 + ϵ ) µ where δ = e − ϵ 2 µ [Equation] b − a [Equation]

Original: e − ϵ 2 µ 2 m ( b − a )2or equivalently ϵ = (cid:113) m ( b − a ) 2 log (1 /δ )
Revised: or equivalently ϵ = (cid:113) m ( b − a [Equation] 2 log (1 /

Revised: [Equation]

Original: µ.
Revised: µ.

Original: Instead of assuming p and q for making connections in the case of C ( v i )
Revised: Instead of assuming p and q for making connections in the case of C ( v i )

Original: =
Revised: =

Original: C ( v j ) and C ( v i )
Revised: C ( v j ) and C ( v i )

Original: ̸ = C ( v j ) ,we assume that the similarity score s i,j is sampled from a bounded distribution (bounded between-1 and +1) with mean of s + when C ( v i ) =
Revised: ̸ = C ( v j ) , we assume that the similarity score s i,j is sampled from a bounded distribution (bounded between -1 and +1) with mean of s + when C ( v i ) =

Original: C ( v j ) , and s i,j is sampled from a bounded distribution(bound between -1 and +1) with mean of s − when C ( v i )
Revised: C ( v j ) , and s i,j is sampled from a bounded distribution (bound between -1 and +1) with mean of s − when C ( v i )

Original: =
Revised: =

Original: C ( v j ) .
Revised: C ( v j ) .

Original: We of course assume s + > s − .
Revised: We of course assume s + > s − .

Original: In expectation, the number of edges connected under S ki,j on V
Revised: In expectation, the number of edges connected under S ki,j on V

Original: i,j is
Revised: i,j is

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where C ( v ) is the category of node v .
Revised: where C ( v ) is the category of node v .

Original: It should be noted that ( b − a ) 2 = 4 under our assumption.
Revised: It should be noted that ( b − a ) 2 = 4 under our assumption.

Original: Therefore, when v i , v j are sampled fromdifferent categories, we have, with probability at least 1 − δ ,[Equation] i
Revised: Therefore, when v i , v j are sampled from different categories, we have, with probability at least 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: When v i
Revised: When v i

Original: [Equation]
Revised: [Equation]

Original: v j are sampled from the same category
Revised: v j are sampled from the same category

Original: [Equation]
Revised: [Equation]

Original: we have, with probability at least 1 − δ ,
Revised: we have [Equation] with probability at least 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: The two distributions can be well separated if the above two bounds do not overlap .
Revised: The two distributions can be well separated if the above two bounds do not overlap .

Original: That is, werequire that
Revised: That is, we require that

Original: [Equation]
Revised: [Equation]

Original: On the other hand, it is obvious s + s − < αs 2+ + (1 − α )
Revised: On the other hand, it is obvious s + s − < αs 2+ + (1 − α )

Original: s 2 − , so we know
Revised: s 2 − , so we know

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Set ϵ
Revised: Set ϵ

Original: =
Revised: =

Original: ϵ − to have a larger step and a stricter condition, we have
Revised: ϵ − to have a larger step and a stricter condition

Original: [Equation]
Revised: [Equation]

Revised: we have

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Because α > 12 and s + > s − ,
Revised: Because α > 12 and s + > s − ,

Original: [Equation]
Revised: [Equation]

Original: So we have
Revised: So we have

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We complete the proof by substituting δ with
Revised: We complete the proof by substituting δ with

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: 21) where γ > 1 is a constant.
Revised: where γ > 1 is a constant.

Original: Finally, we prove that
Revised: Finally, we prove that

Original: [Equation]
Revised: [Equation]

Revised: (cid:18) γ min((1 − α

Revised: [Equation]

Original:  We can guarantee that when the above condition is met, the proposed similarity measure between twonodes from the same category is strictly larger than that for two nodes from different categories.
Revised: − , α (1 − s [Equation] ) (cid:19) . We can guarantee that when the above condition is met, the proposed similarity measure between two nodes from the same category is strictly larger than that for two nodes from different categories.

Original: Itis able to reduce the number of noisy edges and missing edges of the single test method by 1 /γ inexpectation.
Revised: It is able to reduce the number of noisy edges and missing edges of the single test method by 1 /γ in expectation.

Original: F Case Studies on self attention, Q ttention and B ttention
Revised: F Case Studies on self attention, Q -Attention and B -Attention

Original: To better understand the effect of multiple tests (i.e. the fourth order of statistics), we illustrate theeffect of self-attention, Q ttention and B ttention with the help of the tool BertViz 2 .
Revised: To better understand the effect of multiple tests (i.e. the fourth order of statistics), we illustrate the effect of self-attention, Q -Attention and B -Attention with the help of the tool BertViz 5 .

Original: [Figure]
Revised: [Figure]

Original: As shown in the left panel of Figure 8, self-attention introduces a number of noisy connectionsbetween nodes belonging to different categories.
Revised: As shown in the left panel of Figure 8, self-attention introduces a number of noisy connections between nodes belonging to different categories.

Original: In contrast, according to the middle panel of
Revised: In contrast, according to the middle panel of

Original: Figure 8, Q ttention is able to introduce many more connections between nodes from the same category, and at the same time significantly reduces the weights for those noisy connections.
Revised: Figure 8, Q -Attention is able to introduce many more connections between nodes from the same category, and at the same time significantly reduces the weights for those noisy connections.

Original: Inthe last panel, the combination self-attention with the fourth order statistics, The B ttention furtherremoves those noisy connections while keeps most of the clean connections between nodes.
Revised: In the last panel, the combination self-attention with the fourth order statistics, The B -Attention further removes those noisy connections while keeps most of the clean connections between nodes.