Original: Robust Graph Structure Learning over Images via Multiple Statistical Tests
Revised: Robust Graph Structure Learning over Images via Multiple Statistical Tests

Original: Abstract
Revised: Abstract

Original: Graph structure learning aims to learn connectivity in a graph from data.
Revised: Graph structure learning aims to learn connectivity in a graph from data.

Original: It isparticularly important for many computer vision related tasks since no explicitgraph structure is available for images for most cases.
Revised: It isparticularly important for many computer vision related tasks since no explicitgraph structure is available for images for most cases.

Original: A natural way to construct agraph among images is to treat each image as a node and assign pairwise imagesimilarities as weights to corresponding edges.
Revised: A natural way to construct agraph among images is to treat each image as a node and assign pairwise imagesimilarities as weights to corresponding edges.

Original: It is well known that pairwisesimilarities between images are sensitive to the noise in feature representations,leading to unreliable graph structures.
Revised: It is well known that pairwisesimilarities between images are sensitive to the noise in feature representations,leading to unreliable graph structures.

Original: We address this problem from the viewpointof statistical tests.
Revised: We address this problem from the viewpointof statistical tests.

Original: By viewing the feature vector of each node as an independentsample, the decision of whether creating an edge between two nodes based ontheir similarity in feature representation can be thought as a single statisticaltest.
Revised: By viewing the feature vector of each node as an independentsample, the decision of whether creating an edge between two nodes based ontheir similarity in feature representation can be thought as a single statisticaltest.

Original: To improve the robustness in the decision of creating an edge, multiplesamples are drawn and integrated by multiple statistical tests to generate a morereliable similarity measure, consequentially more reliable graph structure.
Revised: To improve the robustness in the decision of creating an edge, multiplesamples are drawn and integrated by multiple statistical tests to generate a morereliable similarity measure, consequentially more reliable graph structure.

Original: Thecorresponding elegant matrix form named B ttention is designed for efficiency.
Revised: Thecorresponding elegant matrix form named B ttention is designed for efficiency.

Original: The effectiveness of multiple tests for graph structure learning is verified boththeoretically and empirically on multiple clustering and ReID benchmark datasets.
Revised: The effectiveness of multiple tests for graph structure learning is verified boththeoretically and empirically on multiple clustering and ReID benchmark datasets.

Original:  Introduction
Revised:  Introduction

Original: Graph structure learning plays an important role in Graph Convolutional Networks (GCNs).
Revised: Graph structure learning plays an important role in Graph Convolutional Networks (GCNs).

Original: It seeksto discover underlying graph structures for better graph representation learning when graph structuresare unavailable, noisy or corrupted.
Revised: It seeksto discover underlying graph structures for better graph representation learning when graph structuresare unavailable, noisy or corrupted.

Original: These issues are typical in the field of vision tasks [54, 59, 58,19, 40, 47, 53, 12, 37, 62, 45].
Revised: These issues are typical in the field of vision tasks [52, 57, 56,18, 39, 46, 51, 12, 36, 60, 44].

Original: As an example, there is no explicit graph structure between photos inyour phone albums so the graph-based approaches [19, 40, 53] in photo management softwares seekto learn graph structure.
Revised: As an example, there is no explicit graph structure between photos inyour phone albums so the graph-based approaches [18, 39, 51] in photo management softwares seekto learn graph structure.

Original: A popular research direction on it is to consider graph structure learning assimilarity measure learning upon the node embedding space [56].
Revised: A popular research direction on it is to consider graph structure learning assimilarity measure learning upon the node embedding space [54].

Original: They learn weighted adjacencymatrices [19, 40, 8, 9, 41, 60, 26, 7, 24, 33] as graphs and feed them to GCNs to learn representations.
Revised: They learn weighted adjacencymatrices [18, 39, 8, 9, 40, 58, 25, 7, 23, 32] as graphs and feed them to GCNs to learn representations.

Original: However, the main problem of the above weighted-based methods is that weights are not alwaysaccurate and reliable: two nodes with smaller associations may have a higher weight score thanthose with larger associations.
Revised: However, the main problem of the above weighted-based methods is that weights are not alwaysaccurate and reliable: two nodes with smaller associations may have a higher weight score thanthose with larger associations.

Original: The root of this problem is closely related to the noise in the highdimensional feature vectors extracted by the deep learning model, as similarities between nodes areusually decided based on their feature vectors.
Revised: The root of this problem is closely related to the noise in the highdimensional feature vectors extracted by the deep learning model, as similarities between nodes areusually decided based on their feature vectors.

Original: The direct consequence of using an inaccurate graphstructure is so called feature pollution [53], leading to a performance degradation in downstreamtasks.
Revised: The direct consequence of using an inaccurate graphstructure is so called feature pollution [51], leading to a performance degradation in downstreamtasks.

Original: In our empirical studies, we found that close to 1 / 3 of total weights are assigned to noisy edgeswhen unnormalized cosine similarities between feature vectors are used to calculate weights.
Revised: In our empirical studies, we found that close to 1 / 3 of total weights are assigned to noisy edgeswhen unnormalized cosine similarities between feature vectors are used to calculate weights.

Original: Many methods have been developed to improve similarity measure between nodes in graph, includingadding learnable parameters [8, 26, 7], introducing nonlinearity [9, 41, 60], Gaussian kernel functions [24, 33] and self-attention [50, 11, 19, 40].
Revised: Many methods have been developed to improve similarity measure between nodes in graph, includingadding learnable parameters [8, 25, 7], introducing nonlinearity [9, 40, 58], Gaussian kernel functions [23, 32] and self-attention [49, 11, 18, 39].

Original: Despite the progress, they are limited to modifyingsimilarity function, without explicitly addressing the noise in feature vectors, a more fundamentalissue for graph structure learning.
Revised: Despite the progress, they are limited to modifyingsimilarity function, without explicitly addressing the noise in feature vectors, a more fundamentalissue for graph structure learning.

Original: One way to capture the noise in feature vectors is to view thefeature vector of each node as an independent sample from some unknown distribution associatedwith each node.
Revised: One way to capture the noise in feature vectors is to view thefeature vector of each node as an independent sample from some unknown distribution associatedwith each node.

Original: As a result, the similarity score between two nodes can be thought as the expectationof a test to decide if two nodes share the same distribution.
Revised: As a result, the similarity score between two nodes can be thought as the expectationof a test to decide if two nodes share the same distribution.

Original: Since, only one vector is sampled foreach node in the above methods, the statistical test is essentially based on a single sample from bothnodes, making the calculated weights inaccurate and unreliable.
Revised: Since, only one vector is sampled foreach node in the above methods, the statistical test is essentially based on a single sample from bothnodes, making the calculated weights inaccurate and unreliable.

Original: An possible solution towards addressing the noise in feature vectors is to run multiple tests, a commonapproach to reduce variance in sampling and decision making [4, 10, 25].
Revised: An possible solution towards addressing the noise in feature vectors is to run multiple tests, a commonapproach to reduce variance in sampling and decision making [4, 10, 24].

Original: Multiple samples aredrawn for each node, and each sample can be used to run an independent test to decide if twonodes share the same distribution.
Revised: Multiple samples aredrawn for each node, and each sample can be used to run an independent test to decide if twonodes share the same distribution.

Original: Based on the results from multiple tests , we develop a noveland robust similarity measure that significantly improves the robustness and reliability of calculatedweights.
Revised: Based on the results from multiple tests , we develop a noveland robust similarity measure that significantly improves the robustness and reliability of calculatedweights.

Original: This paper will show how to accomplish multiple tests in the absence of graph structuresand theoretically prove why the proposed approach can produce a better similarity measure underspecified conditions.
Revised: This paper will show how to accomplish multiple tests in the absence of graph structuresand theoretically prove why the proposed approach can produce a better similarity measure underspecified conditions.

Original: This similarity measure is also crafted into an efficient and elegant form ofmatrix, named B ttention .
Revised: This similarity measure is also crafted into an efficient and elegant form ofmatrix, named B ttention .

Original: B ttention utilizes the fourth order statistics of the features, which is asignificant difference with popular attentions.
Revised: B ttention utilizes the fourth order statistics of the features, which is asignificant difference with popular attentions.

Original: Benefiting from a better similarity measure produced bymultiple tests, graph structures can be significantly improved, resulting in better graph representationsand therefore boosting the performance of downstream tasks.
Revised: Benefiting from a better similarity measure produced bymultiple tests, graph structures can be significantly improved, resulting in better graph representationsand therefore boosting the performance of downstream tasks.

Original: We empirically verify the effectivnessof the proposed method over the task of clustering and ReID.
Revised: We empirically verify the effectivnessof the proposed method over the task of clustering and ReID.

Original: In summary, this paper makes the following three major contributions: (1) To the best of ourknowledge, this is the first paper to address inaccurate edge weights problem by statistical tests on thetask of graph structure learning over images.
Revised: In summary, this paper makes the following three major contributions: (1) To the best of ourknowledge, this is the first paper to address inaccurate edge weights problem by statistical tests on thetask of graph structure learning over images.

Original: (2) A novel and robust similarity measure is proposedbased on multiple tests, proven theoretically to be superior and designed in an elegant matrix formto improve graph structure quality.
Revised: (2) A novel and robust similarity measure is proposedbased on multiple tests, proven theoretically to be superior and designed in an elegant matrix formto improve graph structure quality.

Original: (3) Extensive experiments further verify the robustness of thegraph structure learning method against noise.
Revised: (3) Extensive experiments further verify the robustness of thegraph structure learning method against noise.

Original: State-of-the-art (SOTA) performances are achieved onmultiple clustering and ReID benchmarks with the learned graph structures.
Revised: State-of-the-art (SOTA) performances are achieved onmultiple clustering and ReID benchmarks with the learned graph structures.

Original:  Methodology
Revised:  Methodology

Original: To handle the noise in feature vectors, we model the feature vector of each node as a sample drawnfrom the underlying distribution associated with that node, and the similarity score between twonodes as the expectation of a statistical test.
Revised: To handle the noise in feature vectors, we model the feature vector of each node as a sample drawnfrom the underlying distribution associated with that node, and the similarity score between twonodes as the expectation of a statistical test.

Original: The test is to compare their similarity to a given thresholdto decide if two nodes share the same distribution and that decision can be captured by a binary
Revised: The test is to compare their similarity to a given thresholdto decide if two nodes share the same distribution and that decision can be captured by a binary

Original: Bernoulli random variable.
Revised: Bernoulli random variable.

Original: The comparison in statistical test may lead to error edges, i.e., noisyedges (an edge connects two nodes of different categories) or missing edges (two nodes of the samecategory should be connected but are not).
Revised: The comparison in statistical test may lead to error edges, i.e., noisyedges (an edge connects two nodes of different categories) or missing edges (two nodes of the samecategory should be connected but are not).

Original: The superiority of the similarity measure can be revealedby comparing the probability of creating an error edge.
Revised: The superiority of the similarity measure can be revealedby comparing the probability of creating an error edge.

Original: The proposed similarity measure will beintroduced in Section 2.2, and its further design in matrix form will be presented in Section 2.3.
Revised: The proposed similarity measure will beintroduced in Section 2.2, and its further design in matrix form will be presented in Section 2.3.

Original: 2.1 Preliminary
Revised: 2.1 Preliminary

Original: Chernoff Bounds.
Revised: Chernoff Bounds.

Original: Sums of random variables have been studied for a long time [4, 10, 25].
Revised: Sums of random variables have been studied for a long time [4, 10, 24].

Original: Chernoff
Revised: Chernoff

Original: Bounds [10] provide sharply decreasing bounds that are in the exponential form on tail distributionsof sums of independent random variables.
Revised: Bounds [10] provide sharply decreasing bounds that are in the exponential form on tail distributionsof sums of independent random variables.

Original: It is a widely used fundamental tool in mathematics andcomputer science [30, 39].
Revised: It is a widely used fundamental tool in mathematics andcomputer science [29, 38].

Original: An often used and looser form of Chernoff Bounds is as follows.
Revised: An often used and looser form of Chernoff Bounds is as follows.

Original: Let X = (cid:80) n 1 X i , where X i = 1 with the probability p i and X i = 0 with the probability 1 − p i .
Revised: Let X = (cid:80) n 1 X i , where X i = 1 with the probability p i and X i = 0 with the probability 1 − p i .

Original: All
Revised: All

Original: X i are independent random variables.
Revised: X i are independent random variables.

Original: Let µ = E ( X ) =
Revised: Let µ = E ( X ) =

Original: (cid:80) n 1 p
Revised: (cid:80) n 1 p

Original: i .
Revised: i .

Original: Then for all 0 < ϵ < 1 ,
Revised: Then for all 0 < ϵ < 1 ,

Original: [Equation]
Revised: [Equation]

Original: 3 µ .
Revised: 3 µ .

Original: It shows that with a high probability 1 − δ , we have X ≤ (1+ ϵ ) µ , or X ≥ (1 − ϵ ) µ , where δ = e − ϵ 2 3 .
Revised: It shows that with a high probability 1 − δ , we have X ≤ (1+ ϵ ) µ , or X ≥ (1 − ϵ ) µ , where δ = e − ϵ 2 3 .

Original: GCNs.
Revised: GCNs.

Original: A GCN network normally consists of multiple GCN layers.
Revised: A GCN network normally consists of multiple GCN layers.

Original: Given an input graph G ( X , A )with L vertices, the output of a normal GCN layer is:
Revised: Given an input graph G ( X , A )with L vertices, the output of a normal GCN layer is:

Original: [Equation]
Revised: [Equation]

Original: where ˜A =
Revised: where ˜A =

Original: A + I , ˜D ii =
Revised: A + I , ˜D ii =

Original: L (cid:88)
Revised: L (cid:88)

Original: j
Revised: j

Original: = ˜A i,j , (1)
Revised: = ˜A i,j , (1)

Original: X ∈ R L × M is input vertex features with M dimensions, A ∈ R L × L is their adjacency matrix;
Revised: X ∈ R L × M is input vertex features with M dimensions, A ∈ R L × L is their adjacency matrix;

Original: I ∈
Revised: I ∈

Original: R L × L is an identity matrix, ˜D ∈ R L × L is the diagonal degree matrix of ˜A ; W l ∈ R M × M ′ is atrainable matrix, and σ ( · ) is an activation function such as ReLU [18]; X ′ ∈ R L × M ′ is the outputvertex features with M ′ dimensions, M ′ =
Revised: R L × L is an identity matrix, ˜D ∈ R L × L is the diagonal degree matrix of ˜A ; W l ∈ R M × M ′ is atrainable matrix, and σ ( · ) is an activation function such as ReLU [17]; X ′ ∈ R L × M ′ is the outputvertex features with M ′ dimensions, M ′ =

Original: M in this work.
Revised: M in this work.

Original: 2.2 Multiple tests on similarity measure
Revised: 2.2 Multiple tests on similarity measure

Original: Let V = { v 1 , v 2 , ..., v N } be the collection of N nodes.
Revised: Let V = { v 1 , v 2 , ..., v N } be the collection of N nodes.

Original: Each node is assumed to be in one of twocategories:
Revised: Each node is assumed to be in one of twocategories:

Original: +1 or − 1 for simplicity.
Revised: +1 or − 1 for simplicity.

Original: For each node v i , the subset of candidate nodes to connect itis denoted by V i .
Revised: For each node v i , the subset of candidate nodes to connect itis denoted by V i .

Original: This subset can be decided by a simple criterion such as by k nearest neighbours.
Revised: This subset can be decided by a simple criterion such as by k nearest neighbours.

Original: When calculating the proposed similarity between v i and v j ∈ V i , the common candidate nodesbetween v i and v j , i.e., V i,j = V i ∩ V j are first identified.
Revised: When calculating the proposed similarity between v i and v j ∈ V i , the common candidate nodesbetween v i and v j , i.e., V i,j = V i ∩ V j are first identified.

Original: The nodes in the V i,j are treated as themultiple samples drawn for the comparison of v i and v j .
Revised: The nodes in the V i,j are treated as themultiple samples drawn for the comparison of v i and v j .

Original: Then, each v k ∈ V i,j performs singletest twice, one for v k and v i and one for v k and v j .
Revised: Then, each v k ∈ V i,j performs singletest twice, one for v k and v i and one for v k and v j .

Original: The single test here is a comparison using thepairwise similarity based on the features of v k and v i , or v k and v j .
Revised: The single test here is a comparison using thepairwise similarity based on the features of v k and v i , or v k and v j .

Original: The pairwise similarity canbe cosine similarity, Gaussian kernel similarity or any other form, and is denoted by Sim-S .
Revised: The pairwise similarity canbe cosine similarity, Gaussian kernel similarity or any other form, and is denoted by Sim-S .

Original: Theproposed similarity score between v i and v j increases by one if the two tests yield C ( v i )
Revised: Theproposed similarity score between v i and v j increases by one if the two tests yield C ( v i )

Original: =
Revised: =

Original: C ( v k )and C ( v k ) =
Revised: C ( v k )and C ( v k ) =

Original: C ( v j ) , and zero otherwise, where C ( v ) is the category of node v .
Revised: C ( v j ) , and zero otherwise, where C ( v ) is the category of node v .

Original: It contains multiplesingle tests, hence the name multiple tests , corresponding similarity measure denoted by Sim-M .
Revised: It contains multiplesingle tests, hence the name multiple tests , corresponding similarity measure denoted by Sim-M .

Original: For the convenience of analysis, a few assumptions are further introduced.
Revised: For the convenience of analysis, a few assumptions are further introduced.

Original: First, the size of V i,j is m ,and among the nodes in V i,j , αm nodes share the same category as v i and (1 − α ) m nodes belongto the opposite category, with α > 12 .
Revised: First, the size of V i,j is m ,and among the nodes in V i,j , αm nodes share the same category as v i and (1 − α ) m nodes belongto the opposite category, with α > 12 .

Original: For any node v j ∈ V i , its probability of being connected withv i , according to the statistical test, is q ∈ [0 , 1] if C ( v i )
Revised: For any node v j ∈ V i , its probability of being connected withv i , according to the statistical test, is q ∈ [0 , 1] if C ( v i )

Original: ̸ = C ( v j ) , and the probability increases to1 ≥ p > q if C ( v i ) =
Revised: ̸ = C ( v j ) , and the probability increases to1 ≥ p > q if C ( v i ) =

Original: C ( v j ) .
Revised: C ( v j ) .

Original: This section will show that, if m is sufficiently large (i.e., enoughnumber of tests can be run), there will exist an appropriate threshold for the Sim-M such that thenumber of error edges can be reduced significantly compared to Sim-S.
Revised: This section will show that, if m is sufficiently large (i.e., enoughnumber of tests can be run), there will exist an appropriate threshold for the Sim-M such that thenumber of error edges can be reduced significantly compared to Sim-S.

Original: Proposition 1.
Revised: Proposition 1.

Original: For the single test method, in expectation, the number of noisy edges is (1 − α ) qmand the number of missing edges is α (1 − p ) m .
Revised: For the single test method, in expectation, the number of noisy edges is (1 − α ) qmand the number of missing edges is α (1 − p ) m .

Original: The theorem below shows that both the noisy and missing edges can be reduced by a significantportion when m is not too small.
Revised: The theorem below shows that both the noisy and missing edges can be reduced by a significantportion when m is not too small.

Original: Theorem 1.
Revised: Theorem 1.

Original: Assume for γ > 1 , we have
Revised: Assume for γ > 1 , we have

Original: [Equation]
Revised: [Equation]

Original: Then, for the multiple test method, in expectation, it can reduce the number of noisy edges and missingedges of the single test method by 1 /γ .
Revised: Then, for the multiple test method, in expectation, it can reduce the number of noisy edges and missingedges of the single test method by 1 /γ .

Original: Proof.
Revised: Proof.

Original: Consider node v i and one of its candidate node v j ∈ V i .
Revised: Consider node v i and one of its candidate node v j ∈ V i .

Original: S ki,j is denoted as the similarity scorebetween v i and v j based on the statistical test with respect to v k .
Revised: S ki,j is denoted as the similarity scorebetween v i and v j based on the statistical test with respect to v k .

Original: As shown in Figure 1, v k maybe of the same category as v i or not (denoted by v ′ k for a better show).
Revised: As shown in Figure 1, v k maybe of the same category as v i or not (denoted by v ′ k for a better show).

Original: Evidently, E [ S ki,j ] =
Revised: Evidently, E [ S ki,j ] =

Original: pq if
Revised: pq if

Original: C ( v i )
Revised: C ( v i )

Original: ̸ = C ( v j ) , and E [ S ki,j ] =
Revised: ̸ = C ( v j ) , and E [ S ki,j ] =

Original: αp 2 + (1 − α )
Revised: αp 2 + (1 − α )

Original: q 2 if C ( v i ) =
Revised: q 2 if C ( v i ) =

Original: C ( v j ) .
Revised: C ( v j ) .

Original: The Sim-M between v i and v j is defined as:
Revised: The Sim-M between v i and v j is defined as:

Original: [Equation]
Revised: [Equation]

Original: Using the Chernoff Bounds [10], we have with a probability 1 − δ ,
Revised: Using the Chernoff Bounds [10], we have with a probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: 𝒗 ! 𝒗 𝒊 … 𝒗 𝒌 𝑣 $% … 𝑝 𝑞 𝑞 𝑝 𝐸 𝑆 &,!
Revised: 𝒗 ! 𝒗 𝒊 … 𝒗 𝒌 𝑣 $% … 𝑝 𝑞 𝑞 𝑝 𝐸 𝑆 &,!

Original: $ = 𝛼∗𝑝𝑞+ 1−𝛼 ∗𝑞𝑝 =𝑝𝑞,𝑖𝑓 𝐶 𝑣 & ≠𝐶(𝑣 ! )
Revised: $ = 𝛼∗𝑝𝑞+ 1−𝛼 ∗𝑞𝑝 =𝑝𝑞,𝑖𝑓 𝐶 𝑣 & ≠𝐶(𝑣 ! )

Original: 𝒗 ! 𝒗 𝒊 … 𝒗 𝒌 … 𝑝 𝑞 𝑞 𝑝 𝐸 𝑆 &,!
Revised: 𝒗 ! 𝒗 𝒊 … 𝒗 𝒌 … 𝑝 𝑞 𝑞 𝑝 𝐸 𝑆 &,!

Original: $ =𝛼∗𝑝 ( + 1−𝛼 ∗𝑞 ( ,𝑖𝑓 𝐶 𝑣 & =𝐶(𝑣 ! )
Revised: $ =𝛼∗𝑝 ( + 1−𝛼 ∗𝑞 ( ,𝑖𝑓 𝐶 𝑣 & =𝐶(𝑣 ! )

Original: 𝑣 $%
Revised: 𝑣 $%

Original: Self Attention
Revised: Self Attention

Original: Features of nodes dot product addition
Revised: Features of nodes dot product addition

Original: By assuming that m is large enough, i.e.,
Revised: By assuming that m is large enough, i.e.,

Original: [Equation]
Revised: [Equation]

Original: 5) by choosing the similarity threshold
Revised: 5) by choosing the similarity threshold

Original: [Equation]
Revised: [Equation]

Original: only with a probability δ , we will miss a correct edge, and with a probability δ we will introduce anoisy edge.
Revised: only with a probability δ , we will miss a correct edge, and with a probability δ we will introduce anoisy edge.

Original: We complete the proof by substituting δ with
Revised: We complete the proof by substituting δ with

Original: [Equation]
Revised: [Equation]

Original: We can guarantee that the Sim-M between two nodes from the same category is strictly larger thanthat for two nodes from different categories, leading to the distillation of all noisy edges.
Revised: We can guarantee that the Sim-M between two nodes from the same category is strictly larger thanthat for two nodes from different categories, leading to the distillation of all noisy edges.

Original: 2.3 Multiple tests on GCNs
Revised: 2.3 Multiple tests on GCNs

Original: With a better similarity measure prototype above, we further craft it into an elegant and efficientmatrix form for adaptation of graph structure learning in the real-world.
Revised: With a better similarity measure prototype above, we further craft it into an elegant and efficientmatrix form for adaptation of graph structure learning in the real-world.

Original: Given the L2 normalizedoriginal node features X , the idea of Sim-S can be implemented by inner product of features, i.e.
Revised: Given the L2 normalizedoriginal node features X , the idea of Sim-S can be implemented by inner product of features, i.e.

Original: A X = XX T .
Revised: A X = XX T .

Original: The twice-run tests on each node can be achieved by A X multiplying itself.
Revised: The twice-run tests on each node can be achieved by A X multiplying itself.

Original: Inspiredby the design of popular Transformers [50], some learnable parameters are also introduced.
Revised: Inspiredby the design of popular Transformers [49], some learnable parameters are also introduced.

Original: It containsthe fourth-order statistics of features , which is a significant difference, since self-attention onlyhas second-order.
Revised: It containsthe fourth-order statistics of features , which is a significant difference, since self-attention onlyhas second-order.

Original: If self-attention is regarded as Duet of Attention , our method is analogous to
Revised: If self-attention is regarded as Duet of Attention , our method is analogous to

Original: Quertet of Attention ( Q ttention for short).
Revised: Quertet of Attention ( Q ttention for short).

Original: The vectorized representation of Equation 3, Q ttention
Revised: The vectorized representation of Equation 3, Q ttention

Original: A qart ∈ R L × L , is then implemented as follows:
Revised: A qart ∈ R L × L , is then implemented as follows:

Original: [Equation]
Revised: [Equation]

Original: Kqart ∈ R L × L
Revised: Kqart ∈ R L × L

Original: [Equation]
Revised: [Equation]

Original: are learnable weights, L ′ =
Revised: are learnable weights, L ′ =

Original: L in this work.
Revised: L in this work.

Original: Motivated by the further enhancements from combinations of intrinsic graph structures and implicitgraph structures in literature [33, 7, 34], the self-attention can play an intrinsic role to fuse with the
Revised: Motivated by the further enhancements from combinations of intrinsic graph structures and implicitgraph structures in literature [32, 7, 33], the self-attention can play an intrinsic role to fuse with the

Original: Q ttention in vision tasks, although no intrinsic graph structure exists between images at all.
Revised: Q ttention in vision tasks, although no intrinsic graph structure exists between images at all.

Original: Theself-attention part is the same as that in Transformers [50].
Revised: Theself-attention part is the same as that in Transformers [49].

Original: The attention map A self ∈ R L × L is:
Revised: The attention map A self ∈ R L × L is:

Original: [Equation]
Revised: [Equation]

Original: , where Q self =
Revised: , where Q self =

Original: XW Qself , K self =
Revised: XW Qself , K self =

Original: [Equation]

Original:  Kself 
Revised: XW Kself ,

Original: [Equation]
Revised: [Equation]

Original: W Qself ∈ R M × M d and W Kself ∈ R M × M d are the learnable weights, M d is the dimension of queryand key features.
Revised: W Qself ∈ R M × M d and W Kself ∈ R M × M d are the learnable weights, M d is the dimension of queryand key features.

Original: The combined form blends the duet and quartet of attention, hence the name Bandof Attention ( B ttention for short).
Revised: The combined form blends the duet and quartet of attention, hence the name Bandof Attention ( B ttention for short).

Original: The overview of B ttention mechanism is shown in Figure 2.
Revised: The overview of B ttention mechanism is shown in Figure 2.

Original: The fusion of A self and A qart is defined as (where softmax ( · ) is a softmax function):
Revised: The fusion of A self and A qart is defined as (where softmax ( · ) is a softmax function):

Original: [Equation]
Revised: [Equation]

Original: When combined with the neural network layer in Equation 1, the final form is:
Revised: When combined with the neural network layer in Equation 1, the final form is:

Original: [Equation]
Revised: [Equation]

Original: In real implementation, considering the graph size can be very large for large-scale datasets, sub-graphsampling strategy [53, 52] is adopted to make it more computationally efficient and scalable.
Revised: Experiments are designed with three levels. Firstly, experiments are conducted to analyse theadvantage of Sim-M over that of Sim-S and its robustness to noise.

Original: Inparticular, k seed nodes are first selected as seeds from a dataset in each sampling step. Then the seednodes together withtheir k nearest neighbours ( k NN) [13] form the nodes of a sub-graph.
Revised: Secondly, armed with multipletests on GCNs, i.e., B ttention is further investigated in comparison with the self-attention anda commonly used form of GCNs on robustness and superiority.

Original: As a result,
Revised: Ablation experiments are thenconducted to study how each part of B ttention contributes to and influences the performance.

Original: Equations 8 and 9 are applied to  the sub-graph during training and inference. For a dataset ofsize N d , it takes about N d /k seed iterations to process the whole dataset on average.  Experiments
Revised: Finally,with the robust graph structure learned by B ttention , comparison experiments in downstream graphbased clustering and ReID tasks are conducted to evaluate the benefits of the proposed approach.

Original: 3.1 Experimental setups
Revised: 3.1 Experimental setups

Original: Datasets Experiments are conducted on commonly used visual datasets 1 with three differenttypes of objects, i.e., MS-Celeb [20] (human faces), MSMT17 [55] (human bodies) and VeRi776 [35] (vehicles), to verify the generalization of the proposed method.
Revised: Datasets Experiments are conducted on commonly used visual datasets 1 with three differenttypes of objects, i.e., MS-Celeb [19] (human faces), MSMT17 [53] (human bodies) and VeRi776 [34] (vehicles), to verify the generalization of the proposed method.

Original: In these datasets, eachsample has a specific category, and most categories have rich samples.
Revised: In these datasets, eachsample has a specific category, and most categories have rich samples.

Original: The tasks on these datasetsare popular and representative in computer vision.
Revised: The tasks on these datasetsare popular and representative in computer vision.

Original: It is worth noting that MS-Celeb is divided into 10parts by identities (Part0-9): Part0 for training and Part1-9 for testing, to maintain comparability andconsistency with previous works [59, 59, 19, 53, 40, 47].
Revised: It is worth noting that MS-Celeb is divided into 10parts by identities (Part0-9): Part0 for training and Part1-9 for testing, to maintain comparability andconsistency with previous works [57, 57, 18, 51, 39, 46].

Original: Details of the dataset are in Section A.1.
Revised: Details of the dataset are in Section A.1.

Original: Metrics The Area Under the Curve ( AUC ) is adopted to directly evaluate the discriminatory powerof similarity measures between node pairs.
Revised: Metrics The Area Under the Curve ( AUC ) is adopted to directly evaluate the discriminatory powerof similarity measures between node pairs.

Original: Edge Noise Rate ( ENR ) illustrates the amount of noise ingraphs quantitatively.
Revised: Edge Noise Rate ( ENR ) illustrates the amount of noise ingraphs quantitatively.

Original: For node v i , ENR = N noisei N alli , where N noisei is the number of noisy edges and
Revised: For node v i , ENR = N noisei N alli , where N noisei is the number of noisy edges and

Original: N alli is the amount of all edges for v i .
Revised: N all i is the amount of all edges for v i .

Original: The average ENR is defined as the average of the ENRs of allnodes.
Revised: The average ENR is defined as the average of the ENRs of allnodes.

Original: The feature quality of GCNs output is investigated to prove the effectiveness of graph structurelearning: The mean Average Precision ( mAP ) [63] is used from the perspective of identification and
Revised: The feature quality of GCNs output is investigated to prove the effectiveness of graph structurelearning: The mean Average Precision ( mAP ) [61] is used from the perspective of identification and

Original: ROC curves are shown from the perspective of verification.
Revised: ROC curves are shown from the perspective of verification.

Original: The higher the quality of the featuresthe better the graph structure learning.
Revised: The higher the quality of the featuresthe better the graph structure learning.

Original: In downstream ReID tasks, mAP R and Rank1 accuracy ( R1 )are used.
Revised: In downstream ReID tasks, mAP R and Rank1 accuracy ( R1 )are used.

Original: Compared to mAP, mAP R additionally takes into account the camera
Revised: Compared to mAP, mAP R additionally takes into account the camera

Original: id information, i.e.,samples from the same camera as the probe will not be counted in evaluation.
Revised: id information, i.e.,samples from the same camera as the probe will not be counted in evaluation.

Original: BCubed ( F B ) [3, 1]and Pairwise ( F P ) [48] F-scores are used for evaluating clustering tasks.
Revised: BCubed ( F B ) [3, 1]and Pairwise ( F P ) [47] F-scores are used for evaluating clustering tasks.

Original: 3.2 Effectiveness of the Sim-M
Revised: 3.2 Effectiveness of the Sim-M

Original: To verify the robustness of Sim-M, the study is first conducted on MS-Celeb (Part1) by evaluating thediscriminatory ability on different average ENRs comparing with Sim-S. The Sim-S uses the cosinesimilarity here for simplicity and without loss of generality.
Revised: To verify the robustness of Sim-M, the study is first conducted on MS-Celeb (Part1) by evaluating thediscriminatory ability on different average ENRs comparing with Sim-S. The Sim-S uses the cosinesimilarity here for simplicity and without loss of generality.

Original: For the sake of sparsity and efficiencyit brings, the graphs are built based on k NN via the approximate nearest neighbour algorithm.
Revised: For the sake of sparsity and efficiencyit brings, the graphs are built based on k nearest neighbours ( k NN) via the approximate nearest neighbour algorithm.

Original: Theaverage ENR of graphs is controlled by using different k values when searching neighbours.
Revised: The average ENR of graphs is controlled by using different k values whensearching neighbours.

Original: AUC is adopted to evaluate the similarity score between node pairs which have k NN relations.
Revised: AUC is adopted to evaluate the similarity score between node pairs whichhave k NN relations.

Original: Table 1presents that as the k increases, more and more noise is added (ENR gets larger).
Revised: Table 1 presents that as the k increases, more and more noise is added (ENRgets larger).

Original: For Sim-M,however, more samples can be drawn and more tests can be run, so the advantage of Sim-M over Sim-S becomes increasingly obvious.
Revised: For Sim-M, however, more samples can be drawn and more tests can be run, so theadvantage of Sim-M over Sim-S becomes increasingly obvious.

Original: This demonstrates the robustness of Sim-M to noise.
Revised: This demonstrates the robustness of Sim-M to noise.

Original: Thisphenomenon also conforms to the condition and conclusion in Theorem 1 .
Revised: This phenomenon also conforms to the condition and conclusion in Theorem 1 .

Original: However, as k increasesand ENR gets larger, too much noise can not be handled by multiple tests, so the performance of SimM begins to drop.
Revised: However, as k increases and ENR gets larger, too much noise can not be handled by multiple tests, sothe performance of Sim-M begins to drop.

Original: This downward trend can be slowed down by the careful design of B ttention .
Revised: This downward trend can be slowed down by the carefuldesign of B ttention .

Original: Thesame experiments are conducted on the MSMT17 and VeRi-776, with similar findings in Section A.2.
Revised: The same experiments are conducted on the MSMT17 and VeRi-776, withsimilar findings in Section A.2.

Original: 3.3 Effectiveness of the B ttention
Revised: 3.3 Effectiveness of the B ttention

Original: [Table]
Revised: [Table]

Original: To demonstrate the robustness of B ttention mechanism, the study is first conducted on MS-Celeb (Part1)by evaluating the performance of B ttention with different average ENRs, comparing it to other baselines.
Revised: To demonstrate the robustness of B ttention mechanism, the study is first conducted on MS-Celeb (Part1)by evaluating the performance of B ttention with different average ENRs, comparing it to other baselines.

Original: Then, comparison experiments are also conducted on MSMT17, VeRi-776 and also larger scales(Part3-9) of MS-Celeb with the best ENR settings foreach dataset.
Revised: Then, comparison experiments are also conducted on MSMT17, VeRi-776 and also larger scales(Part3-9) of MS-Celeb with the best ENR settings foreach dataset.

Original: Comparison baselines include the original features of each dataset, a commonly used formof GCN [32, 53], Transformer, GCN with the selfattention (GCN with A self ), Transformer with the
Revised: Comparison baselines include the original features of each dataset, a commonly used formof GCN [31, 51], Transformer, GCN with the selfattention (GCN with A self ), Transformer with the

Original: B ttention mechanism (Trans with A band ), and GCNwith B ttention mechanism (GCN with A band ).
Revised: B ttention mechanism (Trans with A band ), and GCNwith B ttention mechanism (GCN with A band ).

Original: Transformer here is only used as an encoder for structurelearning, no positional encoding is required.
Revised: Transformer here is only used as an encoder for structurelearning, no positional encoding is required.

Original: Network architectures and settings The model is composed of multiple GCN layers with
Revised: Network architectures and settings The model is composed of multiple GCN layers with

Original: B ttention , and an extra fully connected layer with PReLU [22] activation.
Revised: B ttention , and an extra fully connected layer with PReLU [21] activation.

Original: The output featureshave a dimension of 2048 for all datasets.
Revised: The output featureshave a dimension of 2048 for all datasets.

Original: The number of GCN layers is also tuned respectively fordifferent datasets: MS-Celeb uses three GCN layers; MSMT17 and VeRi-776 use one GCN layer.
Revised: The number of GCN layers is also tuned respectively fordifferent datasets: MS-Celeb uses three GCN layers; MSMT17 and VeRi-776 use one GCN layer.

Original: Each node acts as the probe to search its k NN to construct the graph.
Revised: Each node acts as the probe to search its k NN to construct the graph.

Original: The k value is tuned respectivelyfor different datasets: 120 for MS-Celeb, 30 for MSMT17, and 80 for VeRi-776.
Revised: The k value is tuned respectivelyfor different datasets: 120 for MS-Celeb, 30 for MSMT17, and 80 for VeRi-776.

Original: Each node in adataset can be enhanced multiple times, as either a probe or a neighbour.
Revised: Each node in adataset can be enhanced multiple times, as either a probe or a neighbour.

Original: The Hinge Loss [43, 53] isused to classify node pairs.
Revised: The Hinge Loss [42, 51] isused to classify node pairs.

Original: The initial learning rate is 0.008 with the cosine annealing strategy.
Revised: The initial learning rate is 0.008 with the cosine annealing strategy.

Original: Performance on different average ENRs Models are trained on Part0 and tested on Part1 of
Revised: Performance on different average ENRs Models are trained on Part0 and tested on Part1 of

Original: MS-Celeb.
Revised: MS-Celeb.

Original: All baselines use three GCN/Transformer layers for MS-Celeb, except the GCN with
Revised: All baselines use three GCN/Transformer layers for MS-Celeb, except the GCN with

Original: A self which uses two GCN layers (observed to have better performance than three GCN layers).
Revised: A self which uses two GCN layers (observed to have better performance than three GCN layers).

Original: Fora certain ENR, the same k value is used in training and testing.
Revised: Fora certain ENR, the same k value is used in training and testing.

Original: Figure 3 shows how the output featurequalities (mAP) change with different average ENRs for all baselines.
Revised: Figure 3 shows how the output featurequalities (mAP) change with different average ENRs for all baselines.

Original: It shows that the GCN and
Revised: It shows that the GCN and

Original: Transformer with A band work the best in most average ENRs in terms of both mAP values and theirrobustness to different ENRs.
Revised: Transformer with A band work the best in most average ENRs in terms of both mAP values and theirrobustness to different ENRs.

Original: The GCN has the worst results, and the performances of the ones with
Revised: The GCN has the worst results, and the performances of the ones with

Original: A self are in between.
Revised: A self are in between.

Original: This validates that a commonly used form of GCN is sensitive to noisy edgesin input graphs; the proposed B ttention can well alleviate the influence of noisy edges and show thestrongest robustness when learning graph structure; the A self also has some power of dealing withnoisy edges, but it is worse than A band .
Revised: This validates that a commonly used form of GCN is sensitive to noisy edgesin input graphs; the proposed B ttention can well alleviate the influence of noisy edges and show thestrongest robustness when learning graph structure; the A self also has some power of dealing withnoisy edges, but it is worse than A band .

Original: When k value is too small (e.g., 5, 10), the performances of
Revised: When k value is too small (e.g., 5, 10), the performances of

Original: A band and A self are similar.
Revised: A band and A self are similar.

Original: This indicates that the Q ttention part of B ttention can hardly bring apositive effect when there are too few context neighbours, i.e. the number of single tests gathered isinsufficient.
Revised: This indicates that the Q ttention part of B ttention can hardly bring apositive effect when there are too few context neighbours, i.e. the number of single tests gathered isinsufficient.

Original: In addition, GCN and Transformer have very similar performance, either with A band or
Revised: In addition, GCN and Transformer have very similar performance, either with A band or

Original: A self .
Revised: A self .

Original: This indicates that base networks are not critical, and both attention mechanisms work forthe two base networks.
Revised: This indicates that base networks are not critical, and both attention mechanisms work forthe two base networks.

Original: The performance degradation of B ttention with very large ENR (e.g., 0.64)indicates that the current form B ttention still has improvement room, although works much betterthan the self-attention.
Revised: The performance degradation of B ttention with very large ENR (e.g., 0.64)indicates that the current form B ttention still has improvement room, although works much betterthan the self-attention.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Dataset
Revised: Dataset

Original: MS-Celeb
Revised: MS-Celeb

Original: MSMT
Revised: MSMT

Original: VeRiOriginal Feature 80.
Revised: VeRiOriginal Feature 80.

Original: [Table]
Revised: [Table]

Original: Performance on different types of objects To further validate the B ttention mechanism, comparisons are also made on datasets with other types of objects: MSMT17 and VeRi-776.
Revised: Performance on different types of objects To further validate the B ttention mechanism, comparisons are also made on datasets with other types of objects: MSMT17 and VeRi-776.

Original: In thesetwo datasets, k values and network depths are also tuned for all baselines, aiming for their bestperformances.
Revised: In thesetwo datasets, k values and network depths are also tuned for all baselines, aiming for their bestperformances.

Original: For both datasets, one GCN layer is used for GCN, GCN with A band , and Transformerwith A band ; three GCN layers are used for GCN or Transformer with A self .
Revised: For both datasets, one GCN layer is used for GCN, GCN with A band , and Transformerwith A band ; three GCN layers are used for GCN or Transformer with A self .

Original: In MSMT17, k =10for GCN and “GCN w/ A self ”; k =20 for Transformer; k =30 for the two B ttention methods.
Revised: In MSMT17, k =10for GCN and “GCN w/ A self ”; k =20 for Transformer; k =30 for the two B ttention methods.

Original: In
Revised: In

Original: VeRi-776, k =80 for “GCN w/ A band ” and “Trans w/ A band ”, and k =20 for the rest of baselines.
Revised: VeRi-776, k =80 for “GCN w/ A band ” and “Trans w/ A band ”, and k =20 for the rest of baselines.

Original: The best mAP results of all baselines are listed in Table 2, where the results for MS-Celeb are thepeak values in Figure 3.
Revised: The best mAP results of all baselines are listed in Table 2, where the results for MS-Celeb are thepeak values in Figure 3.

Original: From Table 2, the B ttention works the best in all three datasets; A selfworks better than GCN; and all output features are much better than the original ones, although theoriginal features of MSMT17 and VeRi-776 are current SOTA ones.
Revised: From Table 2, the B ttention works the best in all three datasets; A selfworks better than GCN; and all output features are much better than the original ones, although theoriginal features of MSMT17 and VeRi-776 are current SOTA ones.

Original: The best performances areachieved by the GCN with A band in all datasets.
Revised: The best performances areachieved by the GCN with A band in all datasets.

Original: Their ROC curves are shown in Figure 4, wherethe same observations can be seen from verification perspective.
Revised: Their ROC curves are shown in Figure 4, wherethe same observations can be seen from verification perspective.

Original: As shown in Figure 4 and Table 2,the performance differences among those baselines with attention mechanisms are less significantin MSMT17 where k values are relatively small and close across baselines.
Revised: As shown in Figure 4 and Table 2,the performance differences among those baselines with attention mechanisms are less significantin MSMT17 where k values are relatively small and close across baselines.

Original: This observation issimilar to that in Figure 3: the performance margin between A band and A self is less significantwhen average ENRs (i.e., k ) are relatively small.
Revised: This observation issimilar to that in Figure 3: the performance margin between A band and A self is less significantwhen average ENRs (i.e., k ) are relatively small.

Original: This further shows that the benefit of B ttention islimited when with a small number of neighbours, as not many single tests to run and leverage.
Revised: This further shows that the benefit of B ttention islimited when with a small number of neighbours, as not many single tests to run and leverage.

Original: Betterperformance can not be achieved by enlarging k values for MSMT17, as its average sample amountfor each individual is only 31.
Revised: Betterperformance can not be achieved by enlarging k values for MSMT17, as its average sample amountfor each individual is only 31.

Original: Too large k values would result in too big ENR which can not behandled by the current form of B ttention as discussed on Figure 3.
Revised: Too large k values would result in too big ENR which can not behandled by the current form of B ttention as discussed on Figure 3.

Original: Experiments are also conducted to compare these baselines on larger scales of MS-Celeb (i.e.,
Revised: Experiments are also conducted to compare these baselines on larger scales of MS-Celeb (i.e.,

Original: Part3-9) in Section A.3 of Appendix.
Revised: Part3-9) in Section A.3 of Appendix.

Original: Results in Table 6 and Figure 5 show that all baselines haveperformance degradation when the scale enlarges while the degradation of that “w/ A band ” is muchsmaller than baselines.
Revised: Results in Table 6 and Figure 5 show that all baselines haveperformance degradation when the scale enlarges while the degradation of that “w/ A band ” is muchsmaller than baselines.

Original: These results for the cases with different ENRs, on different types of objectsand larger scales of testsets, together validate the robustness and effectiveness of proposed B ttentionin graph structure learning.
Revised: These results for the cases with different ENRs, on different types of objectsand larger scales of testsets, together validate the robustness and effectiveness of proposed B ttentionin graph structure learning.

Original: Ablation experiments on the design of the topological architecture and attention fusion method arealso conducted in Section A.4 of Appendix.
Revised: Ablation experiments on the design of the topological architecture and attention fusion method arealso conducted in Section A.4 of Appendix.

Original: [Table]
Revised: [Table]

Original: of the downstream tasks
Revised: of the downstream tasks

Original: The improvement of graph structure learning can also be proven through the downstream tasks byexploiting the features it produces.
Revised: The improvement of graph structure learning can also be proven through the downstream tasks byexploiting the features it produces.

Original: The better the graph structure is learned, the better the performanceon downstream tasks.
Revised: The better the graph structure is learned, the better the performanceon downstream tasks.

Original: Graphs have important applications in large-scale clustering and re-rerankingin ReID.
Revised: Graphs have important applications in large-scale clustering and re-rerankingin ReID.

Original: However, they suffer from inaccurate weight problems when learning and working with thegraph structure.
Revised: However, they suffer from inaccurate weight problems when learning and working with thegraph structure.

Original: Armed the GCN with A band , many SOTA results are achieved.
Revised: Armed the GCN with A band , many SOTA results are achieved.

Original: [Table]
Revised: [Table]

Original: Performance on clustering tasks
Revised: Performance on clustering tasks

Original: Theclustering tasks are evaluated on MSCeleb and MSMT17.
Revised: Theclustering tasks are evaluated on MSCeleb and MSMT17.

Original: Both of them havea large number of classes and are suitable as clustering benchmarks.
Revised: Both of them havea large number of classes and are suitable as clustering benchmarks.

Original: The samefeatures output by “GCN w/ A band ” areused as that in Section A.3 for MS-Celeband Section 3.3 for MSMT17.
Revised: The samefeatures output by “GCN w/ A band ” areused as that in Section A.3 for MS-Celeband Section 3.3 for MSMT17.

Original: The sameclustering method (denoted “G-cut”) isadopted as that in [19, 53], where pairsof nodes are linked when their similarity scores are larger than a tuned threshold, and clustering is finally completedby transitively merging all links via aunion-find algorithm [2].
Revised: The sameclustering method (denoted “G-cut”) isadopted as that in [18, 51], where pairsof nodes are linked when their similarity scores are larger than a tuned threshold, and clustering is finally completedby transitively merging all links via aunion-find algorithm [2].

Original: The thresholdis tuned for each dataset, aiming for the best balance between BCubed and Pairwise F-Scores.
Revised: The thresholdis tuned for each dataset, aiming for the best balance between BCubed and Pairwise F-Scores.

Original: Inaddition, map equation based unsupervised clustering algorithm Infomap [44, 57] is also adoptedhere.
Revised: Inaddition, map equation based unsupervised clustering algorithm Infomap [43, 55] is also adoptedhere.

Original: Their BCubed and Pairwise F-scores are listed in Table 3 and Table 10, where the results ofcurrent SOTA GCN/Transformer methods (Clusformer [40], STAR-FC [47] and Ada-NETS [53])and three conventional clustering approaches (K-Means [36], HAC [49] and DBSCAN [16]) are alsolisted.
Revised: Their BCubed and Pairwise F-scores are listed in Table 3 and Table 10, where the results ofcurrent SOTA GCN/Transformer methods (Clusformer [39], STAR-FC [46] and Ada-NETS [51])and three conventional clustering approaches (K-Means [35], HAC [48] and DBSCAN [15]) are alsolisted.

Original: The results show that the features enhanced via our approach have the best clustering resultsin all datasets, either with G-cut or Infomap, both of which outperform the current SOTA methods
Revised: The results show that the features enhanced via our approach have the best clustering resultsin all datasets, either with G-cut or Infomap, both of which outperform the current SOTA methods

Original: Ada-NETS and STAR-FC
Revised: Ada-NETS and STAR-FC

Original: with large margins.
Revised: with large margins.

Original: In comparison, the results of the ones with Infomapare better than those with G-cut.
Revised: In comparison, the results of the ones with Infomapare better than those with G-cut.

Original: Performance on ReID tasks The ReID tasks is evaluated on two ReID datasets: MSMT17 and
Revised: Performance on ReID tasks The ReID tasks is evaluated on two ReID datasets: MSMT17 and

Original: VeRi-776.
Revised: VeRi-776.

Original: The enhanced features used are the same as those in Table 2.
Revised: The enhanced features used are the same as those in Table 2.

Original: mAP R and R1 are evaluatedfollowing the standard probe and gallery protocols of MSMT17 and VeRi-776.
Revised: mAP R and R1 are evaluatedfollowing the standard probe and gallery protocols of MSMT17 and VeRi-776.

Original: The results are shownin Table 4.
Revised: The results are shownin Table 4.

Original: As the output feature of learned graph can also be understood as a rerank process in ReID,comparisons are also made on commonly used rerank methods: QE [12], LBR [37], KR [62] and
Revised: As the output feature of learned graph can also be understood as a rerank process in ReID,comparisons are also made on commonly used rerank methods: QE [12], LBR [36], KR [60] and

Original: ECN [45].
Revised: ECN [44].

Original: Table 4 shows that the “GCN w/ A band ” achieves the best or comparable performance, andall enhanced features are much better than the original SOTA ones.
Revised: Table 4 shows that the “GCN w/ A band ” achieves the best or comparable performance, andall enhanced features are much better than the original SOTA ones.

Original: In MSMT17, similar to the resultsin Section 3.3, the baselines with attention mechanism have very close performance, but work much better than the GCN.
Revised: In MSMT17, similar to the resultsin Section 3.3, the baselines with attention mechanism have very close performance, but work much better than the GCN.

Original: In VeRi-776, from the mAP R perspective, B ttention works better than A self ,and both of them are better than the GCN, but their R1 results are comparable.
Revised: In VeRi-776, from the mAP R perspective, B ttention works better than A self ,and both of them are better than the GCN, but their R1 results are comparable.

Original: These results stillsupport the superiority of the proposed approach, although R1 results of VeRi-776 are comparable,as mAP R is a more comprehensive indication of feature quality for ReID.
Revised: These results stillsupport the superiority of the proposed approach, although R1 results of VeRi-776 are comparable,as mAP R is a more comprehensive indication of feature quality for ReID.

Original: The experiments on thecombination of “GCN w/ A band ” with these commonly used rerank methods are also conducted in
Revised: The experiments on thecombination of “GCN w/ A band ” with these commonly used rerank methods are also conducted in

Original: Section A.6, achieving more SOTA performances and proving the complementarity between them.
Revised: Section A.6, achieving more SOTA performances and proving the complementarity between them.

Original:  Related Work
Revised:  Related Work

Original: Graph Convolutional Networks.
Revised: Graph Convolutional Networks.

Original: GCNs [32, 21, 51] extends the operation of convolution fromthe grid-data (e.g. images, videos) to non-grid data which is more common in the real world.
Revised: GCNs [31, 20, 50] extends the operation of convolution fromthe grid-data (e.g. images, videos) to non-grid data which is more common in the real world.

Original: Theclassic GCN [32] is proposed based on the spectral theory and achieves promising results on citationnetwork datasets: Citeseer, Cora and Pubmed [46].
Revised: Theclassic GCN [31] is proposed based on the spectral theory and achieves promising results on citationnetwork datasets: Citeseer, Cora and Pubmed [45].

Original: GraphSAGE [21] further extends GCNs fromtransductive framework to inductive framework and obtains better results.
Revised: GraphSAGE [20] further extends GCNs fromtransductive framework to inductive framework and obtains better results.

Original: GAT [51] introduces theattention mechanism in GCN, making it more expressive.
Revised: GAT [50] introduces theattention mechanism in GCN, making it more expressive.

Original: Fast-GCN [6] interprets graph convolutionsas integral transforms of embedding functions under probability measures, which not only is efficientfor training but also generalizes well for inference.
Revised: Fast-GCN [6] interprets graph convolutionsas integral transforms of embedding functions under probability measures, which not only is efficientfor training but also generalizes well for inference.

Original: However, these GCNs are all investigated onrelational datasets that have explicit graph structures.
Revised: However, these GCNs are all investigated onrelational datasets that have explicit graph structures.

Original: This work aims to extend the capacity of GCNsto work on datasets (e.g., image objects) which have no explicitly defined graph structures.
Revised: This work aims to extend the capacity of GCNsto work on datasets (e.g., image objects) which have no explicitly defined graph structures.

Original: Graph Structure Learning.
Revised: Graph Structure Learning.

Original: The literature [56] suggests that research on graph structure learningfor GCNs can be roughly divided into discrete-based and weighted-based ones.
Revised: The literature [54] suggests that research on graph structure learningfor GCNs can be roughly divided into discrete-based and weighted-based ones.

Original: Some discretebased methods [15, 29, 38, 17] treat the graph structures as random distributions and sample binaryadjacency matrices from them.
Revised: Some discretebased methods [14, 28, 37, 16] treat the graph structures as random distributions and sample binaryadjacency matrices from them.

Original: Another way [53] constructs discrete graphs based on heuristicproxy objectiveness aiming for clean yet rich neighbours for each node.
Revised: Another way [51] constructs discrete graphs based on heuristicproxy objectiveness aiming for clean yet rich neighbours for each node.

Original: The binary matricesmethods are difficult to optimize because of the gradient breakage, so such methods resort tovariational inference (e.g., approaches in [15, 38]), reinforcement learning (e.g., approaches in [29])or optimization in stages (e.g., approaches in [53]), which may lead to sub-optimal solutions.
Revised: The binary matricesmethods are difficult to optimize because of the gradient breakage, so such methods resort tovariational inference (e.g., approaches in [14, 37]), reinforcement learning (e.g., approaches in [28])or optimization in stages (e.g., approaches in [51]), which may lead to sub-optimal solutions.

Original: Inaddition, the parameter size of distributions [17] can be very large (e.g., O ( Nk ) where N is thenumber of nodes and k is the number of neighbours for each node) and therefore not suitable for largedatasets.
Revised: Inaddition, the parameter size of distributions [16] can be very large (e.g., O ( Nk ) where N is thenumber of nodes and k is the number of neighbours for each node) and therefore not suitable for largedatasets.

Original: The weighted-based methods [8, 9, 41, 60, 26, 7, 24, 33, 19, 40], which are the focus of thispaper, learn weighted adjacency matrices so that it can be optimized by SGD techniques [42, 31, 5].
Revised: The weighted-based methods [8, 9, 40, 58, 25, 7, 23, 32, 18, 39], which are the focus of thispaper, learn weighted adjacency matrices so that it can be optimized by SGD techniques [41, 30, 5].

Original: The main difference between these works [8, 9, 41, 60, 26, 7, 24, 33] is in the similarity measure,which can be classified into attention-based [8, 9, 41, 60, 26, 7] and kernel-based [24, 33] roughly.
Revised: The main difference between these works [8, 9, 40, 58, 25, 7, 23, 32] is in the similarity measure,which can be classified into attention-based [8, 9, 40, 58, 25, 7] and kernel-based [23, 32] roughly.

Original: Toenhance the discriminatory power of the similarity measure, they have adopted various forms: addinglearnable parameters [8, 26, 7], introducing nonlinearity [9, 41, 60], Gaussian kernel functions [24, 33]or self-attention [19, 40, 50, 11] etc.
Revised: Toenhance the discriminatory power of the similarity measure, they have adopted various forms: addinglearnable parameters [8, 25, 7], introducing nonlinearity [9, 40, 58], Gaussian kernel functions [23, 32]or self-attention [18, 39, 49, 11] etc.

Original: However, these methods are sensitive to the noise from featurerepresentations, particular when each node is represented by a high dimensional vector, leading to theproblem of inaccurate weights and vulnerable graph structure.
Revised: However, these methods are sensitive to the noise from featurerepresentations, particular when each node is represented by a high dimensional vector, leading to theproblem of inaccurate weights and vulnerable graph structure.

Original: This work aims to propose a multiplesamples based similarity measure to improve the robustness of graph structure learning over images.
Revised: This work aims to propose a multiplesamples based similarity measure to improve the robustness of graph structure learning over images.

Original: Another approach uses the low-rank, sparsity and feature smoothness properties of the real-worldgraph to optimize the graph structure [27], but these priors are suitable for community networksrather than graph of image objects.
Revised: Another approach uses the low-rank, sparsity and feature smoothness properties of the real-worldgraph to optimize the graph structure [26], but these priors are suitable for community networksrather than graph of image objects.

Original: Some work [28] learns to augment the given graph with newedges by reinforcement learning to improve the performance of downstream tasks, but existing noisyedges problem in graphs cannot be handled.
Revised: Some work [27] learns to augment the given graph with newedges by reinforcement learning to improve the performance of downstream tasks, but existing noisyedges problem in graphs cannot be handled.

Original:  Conclusion
Revised:  Conclusion

Original: The problem of inaccurate edge weights is identified in graph structure learning, analysed fromthe perspective of noise in feature vectors, and handled based on several tests.
Revised: The problem of inaccurate edge weights is identified in graph structure learning, analysed fromthe perspective of noise in feature vectors, and handled based on several tests.

Original: A novel and robustsimilarity measure Sim-M is proposed, proven to be superior and verified in experiments.
Revised: A novel and robustsimilarity measure Sim-M is proposed, proven to be superior and verified in experiments.

Original: Then it isdesigned in an elegant matrix form, i.e., B ttention mechanism, to improve the graph structure qualityin GCNs.
Revised: Then it isdesigned in an elegant matrix form, i.e., B ttention mechanism, to improve the graph structure qualityin GCNs.

Original: The superiority and robustness of B ttention are validated by comprehensive experimentsand achieve many SOTA results on clustering and ReID benchmarks.
Revised: The superiority and robustness of B ttention are validated by comprehensive experimentsand achieve many SOTA results on clustering and ReID benchmarks.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] See abstract and Section 1.(b)
Revised: [Yes] See abstract and Section 1.

Revised: [Table]

Original:  Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section B in Appendix.(c)
Revised: [Yes] See Section B in Appendix.

Original:  Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See
Revised: [Yes] SeeSection C in Appendix.

Original: Section C in Appendix.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to [Table] them?

Original: [Yes] We have read the ethics guidelines on the official website.2.
Revised: [Yes] We have read the ethics guidelines on the official website. [Table]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Revised: [Table]

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Section 2.2.
Revised: [Yes] See Section 2.2.

Revised: [Table]

Original: b ) Did you include complete proofs of all theoretical results
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Section 
Revised: [Yes] See Section 2.

Original: If you ran experiments...
Revised: If you ran experiments...

Original:  (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: [Table] (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?

Original: [No] The code andthe data are proprietary
Revised: [No] The code andthe data are proprietary.

Original: The instructions is in Section 3.(b)
Revised: The instructions is in Section 3.

Revised: [Table]

Original:  Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they [Table] were chosen)?

Original: [Yes] See in Section 3.1, paragraph of Network architectures andsettings and the paragraphs describing the corresponding experiment in Section 3.
Revised: [Yes] See in Section 3.1, paragraph of Network architectures and settings and the paragraphs describing the corresponding experiment in Section 3.3. [Table]

Original: (c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] Error bars are not reported because it would be too 
Revised: [No] Error bars are not reported because it would be too [Table]

Original: computationally expensive.
Revised: computationally expensive.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section A.7.
Revised: [Yes] See Section A.7.4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See the Datasets
Revised: [Yes] See the Datasets

Original: paragraph in Section 3.1.
Revised: paragraph in Section 3.1.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [No] The used assets are very common in
Revised: [No] The used assets are very common in

Original: papers of computer vision.
Revised: papers of computer vision.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [No] The used assets are very common in papers of computer vision.
Revised: [No] The used assets are very common in papers of computer vision.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [Yes] See the footnote in Datasets paragraph in
Revised: [Yes] See the footnote in Datasets paragraph in

Original: Section 3.1: Considering the ethical and privacy issues, all the datasets used in this
Revised: Section 3.1: Considering the ethical and privacy issues, all the datasets used in this

Original: paper are feature vectors which can not be restored to images.
Revised: paper are feature vectors which can not be restored to images.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table]
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: spent on participant compensation?

Original: [Table]
Revised: [N/A]

Original: A ExperimentsA.1 Details of the dataset
Revised: A ExperimentsA.1 Details of the dataset

Original: MS-Celeb [20] is a dataset with 10M face images of nearly 100K individuals.
Revised: MS-Celeb [19] is a dataset with 10M face images of nearly 100K individuals.

Original: A cleaned version [14]with about 5.8M face images of 86K identities is used in this paper.
Revised: A cleaned version [13]with about 5.8M face images of 86K identities is used in this paper.

Original: The protocol and features arethe same as that in [59], where the dataset is divided into 10 parts by identities (Part0-9): Part0 fortraining and Part1-9 for testing.
Revised: The protocol and features arethe same as that in [57], where the dataset is divided into 10 parts by identities (Part0-9): Part0 fortraining and Part1-9 for testing.

Original: The feature dimension is 256.
Revised: The feature dimension is 256.

Original: The data amounts of Part 0-9 (0, 1, 3,5, 7 and 9) are respectively: 584K, 584K, 1.74M, 2.89M, 4.05M and 5.21M.
Revised: The data amounts of Part 0-9 (0, 1, 3,5, 7 and 9) are respectively: 584K, 584K, 1.74M, 2.89M, 4.05M and 5.21M.

Original: MSMT17 [55] is the current largest ReID dataset.
Revised: MSMT17 [53] is the current largest ReID dataset.

Original: It contains 32,621 images of 1,041 individuals fortraining and 93,820 images of 3,060 individuals for testing.
Revised: It contains 32,621 images of 1,041 individuals fortraining and 93,820 images of 3,060 individuals for testing.

Original: These images are captured by 15 camerasunder different time periods, weather and light conditions.
Revised: These images are captured by 15 camerasunder different time periods, weather and light conditions.

Original: This work uses the SOTA features from
Revised: This work uses the SOTA features from

Original: AGW [61] with a dimension of 2048.
Revised: AGW [59] with a dimension of 2048.

Original: VeRi-776 [35] is a commonly used vehicle ReID benchmark, which contains over 40K boundingboxes of 619 vehicles captured by 20 cameras in unconstrained traffic scenes.
Revised: VeRi-776 [34] is a commonly used vehicle ReID benchmark, which contains over 40K boundingboxes of 619 vehicles captured by 20 cameras in unconstrained traffic scenes.

Original: The dataset is dividedinto two parts: 37,715 images of 576 vehicles for training, and 13,257 images of 200 vehicles fortesting.
Revised: The dataset is dividedinto two parts: 37,715 images of 576 vehicles for training, and 13,257 images of 200 vehicles fortesting.

Original: This work uses the features extracted by the SOTA ViT-B/16 Baseline in TransReID
Revised: This work uses the features extracted by the SOTA ViT-B/16 Baseline in TransReID

Original: [23],with 768 dimensions.
Revised: [22],with 768 dimensions.

Original: A.2 Effectiveness of the Sim-M
Revised: A.2 Effectiveness of the Sim-M

Original: The experiments on MSMT17 and VeRi-776 also show that the advantage of Sim-M over Sim-S isincreasingly obvious as k increases and then drops for introduction of too much noise as in Table 5.
Revised: The experiments on MSMT17 and VeRi-776 also show that the advantage of Sim-M over Sim-S isincreasingly obvious as k increases and then drops for introduction of too much noise as in Table 5.

Original: a factor of 100 for a better show.
Revised: a factor of 100 for a better show.

Original: [Table]
Revised: [Table]

Original: A.3 Performance on larger scales of MS-Celeb
Revised: A.3 Performance on larger scales of MS-Celeb

Original: Experiments are also conducted to compare these baselines on larger scales of MS-Celeb (i.e., Part3-
Revised: Experiments are also conducted to compare these baselines on larger scales of MS-Celeb (i.e., Part39).

Original: The same model for MS-Celeb Part1 is used as in Section 3.3, but tested on Part3-9.
Revised: The same model for MS-Celeb Part1 is used as in Section 3.3, but tested on Part3-9.

Original: The results in
Revised: The results in

Original: Table 6 show that all baselines have performance degradation when the scale enlarges.
Revised: Table 6 show that all baselines have performance degradation when the scale enlarges.

Original: The B ttentionobtains the best results in all parts; A self works better than GCN; all output features are much betterthan the original ones.
Revised: The B ttentionobtains the best results in all parts; A self works better than GCN; all output features are much betterthan the original ones.

Original: Among the baselines with attention mechanisms, the ones using GCN-basednetworks have better performances.
Revised: Among the baselines with attention mechanisms, the ones using GCN-basednetworks have better performances.

Original: The GCN with A band has the highest mAP scores.
Revised: The GCN with A band has the highest mAP scores.

Original: Figure 5shows the ROC curves of GCN, “GCN w/ A self ” and “GCN w/ A band ” across all parts of MS-Celeb,from which we can also see the performance degradation with larger testsets.
Revised: Figure 5shows the ROC curves of GCN, “GCN w/ A self ” and “GCN w/ A band ” across all parts of MS-Celeb,from which we can also see the performance degradation with larger testsets.

Original: The performancedegradation of “GCN w/ A band ” is much smaller than the other two baselines, which is shown moreclearly from the differences of AUC in Table 7.
Revised: The performancedegradation of “GCN w/ A band ” is much smaller than the other two baselines, which is shown moreclearly from the differences of AUC in Table 7.

Original: In Table 7, differences on AUC of two adjacent parts(e.g., part1 - part3, part3 - part5, part5 - part7, part7 - part9) and their summation for each method areshown.
Revised: In Table 7, differences on AUC of two adjacent parts(e.g., part1 - part3, part3 - part5, part5 - part7, part7 - part9) and their summation for each method areshown.

Original: It can be observed that the summation of the differences of “GCN w/ A band ” is 1.49, whichis only 58.30% and 67.26% of the ones of GCN (2.56) and “GCN with A self " (2.22).
Revised: It can be observed that the summation of the differences of “GCN w/ A band ” is 1.49, whichis only 58.30% and 67.26% of the ones of GCN (2.56) and “GCN with A self " (2.22).

Original: This validatesthat the features from “GCN w/ A band ” have a much smaller performance degradation than the othertwo baselines with larger testsets from the perspective of verification.
Revised: This validatesthat the features from “GCN w/ A band ” have a much smaller performance degradation than the othertwo baselines with larger testsets from the perspective of verification.

Original: Similar observations can alsobe seen in Table 6, where the performance degradation of the baselines with attention mechanisms ismuch smaller than GCN and original features.
Revised: Similar observations can alsobe seen in Table 6, where the performance degradation of the baselines with attention mechanisms ismuch smaller than GCN and original features.

Original: 1.0 False positive rate 0.
Revised: 1.0 False positive rate 0.

Original: T r u e p
Revised: T r u e p

Original: o
Revised: o

Original: s i
Revised: s i

Original: t i v e r a t
Revised: t i v e r a t

Original: e
Revised: e

Original: Part1Part3Part5Part7Part (a) GCN 0.
Revised: Part1Part3Part5Part7Part (a) GCN 0.

Original: 1.0 False positive rate 0.
Revised: 1.0 False positive rate 0.

Original: T r u e p
Revised: T r u e p

Original: o
Revised: o

Original: s i
Revised: s i

Original: t i v e r a t
Revised: t i v e r a t

Original: e
Revised: e

Original: Part1Part3Part5Part7Part (b) GCN with A self 0.
Revised: Part1Part3Part5Part7Part (b) GCN with A self 0.

Original: 1.0 False positive rate 0.
Revised: 1.0 False positive rate 0.

Original: T r u e p
Revised: T r u e p

Original: o
Revised: o

Original: s i
Revised: s i

Original: t i v e r a t
Revised: t i v e r a t

Original: e
Revised: e

Original: Part1Part3Part5Part7Part (c) GCN with A band
Revised: Part1Part3Part5Part7Part (c) GCN with A band

Original: Partition
Revised: Partition

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: GCN 90.
Revised: GCN 90.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Transformer 93.
Revised: Transformer 93.

Original: GCN w/
Revised: GCN w/

Original: [Table]
Revised: [Table]

Original: self 94.
Revised: self 94.

Original: Trans w/
Revised: Trans w/

Original: A band 95.
Revised: A band 95.

Original: GCN w/ A band 95.
Revised: GCN w/ A band 95.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: A.4 Ablation study on B ttention
Revised: A.4 Ablation study on B ttention

Original: Ablation experiments are conducted mainly on MS-Celeb to study the design of the topologicalarchitecture and attention fusion method of B ttention .
Revised: Ablation experiments are conducted mainly on MS-Celeb to study the design of the topologicalarchitecture and attention fusion method of B ttention .

Original: Similar to Section 3.3, models are trained on
Revised: Similar to Section 3.3, models are trained on

Original: Part0 and tested on Part1.
Revised: Part0 and tested on Part1.

Original: All settings other than the factor being investigated are the same as that in
Revised: All settings other than the factor being investigated are the same as that in

Original: Section 3.3 for “GCN with A band ” with k = 120 .
Revised: Section 3.3 for “GCN with A band ” with k = 120 .

Original: A.4.1 topological architecture design
Revised: A.4.1 topological architecture design

Original: This study compares the topological architecture of the B ttention with three other designs: (1) onewith only the A qart of B ttention ; (2) one using only the Q ttention part but with the A X replaced by
Revised: This study compares the topological architecture of the B ttention with three other designs: (1) onewith only the A qart of B ttention ; (2) one using only the Q ttention part but with the A X replaced by

Original: A self in Equation 8, denoted as (cid:101)
Revised: A self in Equation 8, denoted as (cid:101)

Original: A qart ; (3) and one with B ttention but with the A qart replaced by(cid:101)
Revised: A qart ; (3) and one with B ttention but with the A qart replaced by(cid:101)

Original: A qart , denoted as (cid:101)
Revised: A qart , denoted as (cid:101)

Original: A band .
Revised: A band .

Original: Table 8 shows their mAP results, where the results of “GCN w/ A band ”and one with only A self are also listed for comparison.
Revised: Table 8 shows their mAP results, where the results of “GCN w/ A band ”and one with only A self are also listed for comparison.

Original: For the purpose of ablation study, the onewith only A self here has the same depth (i.e., three layers) with “GCN w/ A band ”, which is differentfrom that in Table 2 (two layers).
Revised: For the purpose of ablation study, the onewith only A self here has the same depth (i.e., three layers) with “GCN w/ A band ”, which is differentfrom that in Table 2 (two layers).

Original: The comparisons are made on two conditions: with or without
Revised: The comparisons are made on two conditions: with or without

Original: W qart (i.e., W Q qart and W K qart ) in Equation 8.
Revised: W qart (i.e., W Q qart and W K qart ) in Equation 8.

Original: For the cases without W qart , W Q qart and W K qartare both identity matrices, i.e., Q qart =
Revised: For the cases without W qart , W Q qart and W K qartare both identity matrices, i.e., Q qart =

Original: K qart =
Revised: K qart =

Original: A X .
Revised: A X .

Original: Their ROC curves are shown in Figure 6.
Revised: Their ROC curves are shown in Figure 6.

Original: It can be seen that the B ttention design achieves the best results in both mAP and ROC; A qarthas better performance than A self ; (cid:101)
Revised: It can be seen that the B ttention design achieves the best results in both mAP and ROC; A qarthas better performance than A self ; (cid:101)

Original: A qart and (cid:101)
Revised: A qart and (cid:101)

Original: A band are worse than A qart and A band respectively.
Revised: A band are worse than A qart and A band respectively.

Original: This shows that the combination of A self and A qart and the use of A X (instead of A self ) in A qart both contribute to the performance improvement.
Revised: This shows that the combination of A self and A qart and the use of A X (instead of A self ) in A qart both contribute to the performance improvement.

Original: In addition, the baselines with W qart have betterperformances (higher mAP scores and better ROC) than those without, indicating that W Qqart and
Revised: In addition, the baselines with W qart have betterperformances (higher mAP scores and better ROC) than those without, indicating that W Qqart and

Original: W Kqart are also critical for better graph structure learning.
Revised: W Kqart are also critical for better graph structure learning.

Original: [Table]
Revised: [Table]

Original: The ablation experiments are also conducted on attention map fusion method in Section A.4.2in Appendix .
Revised: The ablation experiments are also conducted on attention map fusion method in Section A.4.2in Appendix .

Original: The results in Table 9 and Figure 7 together will prove that the proposed form in
Revised: The results in Table 9 and Figure 7 together will prove that the proposed form in

Original: Equation 11 has the best performance across all datasets.
Revised: Equation 11 has the best performance across all datasets.

Original: A.4.2 Attention map fusion method
Revised: A.4.2 Attention map fusion method

Original: The attention map fusion method is studied by comparing it to two other designs: direct element-wiseaddition without weighting ( A qart + A self ) and element-wise multiplication ( A qart ⊙ A self ) in
Revised: The attention map fusion method is studied by comparing it to two other designs: direct element-wiseaddition without weighting ( A qart + A self ) and element-wise multiplication ( A qart ⊙ A self ) in

Original: Equation 11.
Revised: Equation 11.

Original: The results are shown in Table 9 and Figure 7: The θ qart A qart + θ self A self designworks the best, and the performance of A qart ⊙ A self is very close to θ qart A qart + θ self A self .
Revised: The results are shown in Table 9 and Figure 7: The θ qart A qart + θ self A self designworks the best, and the performance of A qart ⊙ A self is very close to θ qart A qart + θ self A self .

Original: However, results of MSMT17 and VeRi-776 in Table 9 shows our proposed method has the bestperformance across all datasets.
Revised: However, results of MSMT17 and VeRi-776 in Table 9 shows our proposed method has the bestperformance across all datasets.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: A.5 Clustering performance on MSMT17
Revised: A.5 Clustering performance on MSMT17

Original: The clustering tasks are also evaluated on MSMT17 and the results are in Table 10.
Revised: The clustering tasks are also evaluated on MSMT17 and the results are in Table 10.

Original: A.6 Ensemble Performance on ReID tasks
Revised: A.6 Ensemble Performance on ReID tasks

Original: In addition to directly comparing with the commonly used rerank methods (e.g., QE [12], LBR [37],
Revised: In addition to directly comparing with the commonly used rerank methods (e.g., QE [12], LBR [36],

Original: KR [62] and ECN [45]) in Section 4.4, the “GCN with A band " can also be combined with them.
Revised: KR [60] and ECN [44]) in Section 4.4, the “GCN with A band " can also be combined with them.

Original: The commonly used rerank methods can utilize the enhanced features from “GCN with A band " forbetter ranking.
Revised: The commonly used rerank methods can utilize the enhanced features from “GCN with A band " forbetter ranking.

Original: The results are shown in Table 11.
Revised: The results are shown in Table 11.

Original: Since mAP R can provide a more comprehensiveevaluation of the feature quality than R1 for ReID, the mAP R is discussed more often.
Revised: Since mAP R can provide a more comprehensiveevaluation of the feature quality than R1 for ReID, the mAP R is discussed more often.

Original: It can beobserved that new SOTA performances are obtained on both datasets.
Revised: It can beobserved that new SOTA performances are obtained on both datasets.

Original: The mAP R on MSMT17 is ashigh as 79.67 by “ A band +KR" and on VeRi-776 is as high as 85.92 by “ A band +LBR".
Revised: The mAP R on MSMT17 is ashigh as 79.67 by “ A band +KR" and on VeRi-776 is as high as 85.92 by “ A band +LBR".

Original: For most ofthe commonly used rerank methods, the mAP R of reranking with A band is higher than with originalfeatures and also higher than “GCN w/ A band ”.
Revised: For most ofthe commonly used rerank methods, the mAP R of reranking with A band is higher than with originalfeatures and also higher than “GCN w/ A band ”.

Original: The only exception is “ A band +ECN" which is higherthan “GCN w/ A band ” but only comparable with “original+ECN".
Revised: The only exception is “ A band +ECN" which is higherthan “GCN w/ A band ” but only comparable with “original+ECN".

Original: The results show that there is acomplementary relationship between “GCN with A band " and commonly used rerank methods.
Revised: The results show that there is acomplementary relationship between “GCN with A band " and commonly used rerank methods.

Original: The“GCN with A band " can provide better features so that the commonly used rerank methods can befurther improved, although the R 1 results are only comparable.
Revised: The“GCN with A band " can provide better features so that the commonly used rerank methods can befurther improved, although the R 1 results are only comparable.

Original: [Table]
Revised: [Table]

Original: GCN w/ A band 78.
Revised: GCN w/ A band 78.

Original: Original+QE 71.
Revised: Original+QE 71.

Original: Original+LBR 70.
Revised: Original+LBR 70.

Original: Original+KR 76.
Revised: Original+KR 76.

Original: Original+ECN 78.
Revised: Original+ECN 78.

Original: A band +QE 79.
Revised: A band +QE 79.

Original: A band +LBR 79.
Revised: A band +LBR 79.

Original: A band +KR 79.
Revised: A band +KR 79.

Original: A band +ECN 78.
Revised: A band +ECN 78.

Original: A.7 Declaration of the type of computing resources
Revised: A.7 Declaration of the type of computing resources

Original: All experiments are conducted on a single machine, with 54 E5-2682 v4 CPUs, 8 NVIDIA P100cards and 400G memory.
Revised: All experiments are conducted on a single machine, with 54 E5-2682 v4 CPUs, 8 NVIDIA P100cards and 400G memory.

Original: B Limitations
Revised: B Limitations

Original: Our approach achieves promising results in graph structure learning, but it still has the followinglimitations:
Revised: Our approach achieves promising results in graph structure learning, but it still has the followinglimitations:

Original: • The B ttention requires more computation and number of parameters as shown in Section 2.3.
Revised: • The B ttention requires more computation and number of parameters as shown in Section 2.3.

Original: However, the baseline methods can not achieve the same performance even adding their
Revised: However, the baseline methods can not achieve the same performance even adding their

Original: network depth.
Revised: network depth.

Original: The layer number of network in each method is tuned in all experiments.
Revised: The layer number of network in each method is tuned in all experiments.

Original: • The superiority of B ttention is not obvious enough on datasets with few samples in each
Revised: • The superiority of B ttention is not obvious enough on datasets with few samples in each

Original: category compared with self attention as revealed in Section 3.3.
Revised: category compared with self attention as revealed in Section 3.3.

Original: • The performance of B ttention will also degrade when with large ENR as discussed inSection 3.3, indicating the current form still has improvement room on the robustnessagainst noise to further investigate.
Revised: • The performance of B ttention will also degrade when with large ENR as discussed inSection 3.3, indicating the current form still has improvement room on the robustness against noise to further investigate.

Original: • The outputs B ttention will vary depending on the choice of nearest neighbours.
Revised: • The outputs B ttention will vary depending on the choice of nearest neighbours.

Original: However,the superiority of this method can be guaranteed as long as the conditions in Section 2.2 aremet.
Revised: However, the superiority of this method can be guaranteed as long as the conditions in Section 2.2 aremet.

Original: C Potential Negative Societal Impacts
Revised: C Potential Negative Societal Impacts

Original: The proposed graph structure learning method inherently is harmless just like many other AI technologies.
Revised: The proposed graph structure learning method inherently is harmless just like many other AI technologies.

Original: This technology may have negative societal impacts if someone uses it for maliciousapplications such as surveillance, personal information collections.
Revised: This technology may have negative societal impacts if someone uses it for maliciousapplications such as surveillance, personal information collections.

Original: As authors, we advocate forproper technology usage.
Revised: As authors, we advocate forproper technology usage.

Original: D Pseudocode for self-attention and Q ttention

Original: # X: the input feature matrix, with shape (N, D) def self_attention(X):

Original: Q_self=

Original: torch.matmul(X, W_self_Q)

Original: K_self =

Original: torch.matmul(X, W_self_K)

Original: self_attn = torch.matmul(Q_self, K_self.transpose(-2, -1)) return self_attn def Q_attention(X):

Original: A_X=torch.matmul(X, X.transpose(-2, -1))

Original: Q_qart =

Original: torch.matmul(A_X, W_qart_Q)

Original: K_qart = torch.matmul(A_X, W_qart_K)

Original: Q_attn = torch.matmul(Q_qart, K_qart.transpose(-2, -1)) return Q_attn

Original: E Detailed Proof of with generalized version of Chernoff Bounds

Original: Below is a full analysis for the practical algorithm in Section 2.3 that does not assume a binaryrelationship between two nodes in A X .

Original: Let X i denote a random variable with no distribution assumptions such that a ≤ X i ≤ b for all i .

Original: Define X = (cid:80) m 1 X

Original: i , µ = E ( X ) .

Original: Then for all 0 < ϵ < 1 , we have the general version of Chernoffinequality as follows.

Original: ( i ) Upper Tail: P ( X ≥ (1 + ϵ ) µ )

Original: ≤ e − ϵ 2 µ 2 m ( b − a )2 ;( ii ) Lower Tail: P ( X ≤ (1 − ϵ ) µ )

Original: ≤ e − ϵ 2 µ 2 m ( b − a )2 .

Original: The above show that with probability at least 1 − δ , (1 − ϵ ) µ ≤ X ≤ (1 + ϵ ) µ where δ :=

Original: e − ϵ 2 µ 2 m ( b − a )2or equivalently ϵ = (cid:113) m ( b − a ) 2 log (1 /δ )

Original: µ.

Original: Instead of assuming p and q for making connections in the case of C ( v i )

Original: =

Original: C ( v j ) and C ( v i )

Original: ̸ = C ( v j ) ,we assume that the similarity score s i,j is sampled from a bounded distribution (bounded between-1 and +1) with mean of s + when C ( v i ) =

Original: C ( v j ) , and s i,j is sampled from a bounded distribution(bound between -1 and +1) with mean of s − when C ( v i )

Original: =

Original: C ( v j ) .

Original: We of course assume s + > s − .

Original: In expectation, the number of edges connected under S ki,j on V

Original: i,j is

Original: [Equation]

Original: where C ( v ) is the category of node v .

Original: It should be noted that ( b − a ) 2 = 4 under our assumption.

Original: Therefore, when v i , v j are sampled fromdifferent categories, we have, with probability at least 1 − δ ,

Original: [Equation]

Original: i

Original: [Equation]

Original: When v i

Original: [Equation]

Original: v j are sampled from the same category

Original: [Equation]

Original: we have, with probability at least 1 − δ ,

Original: [Equation]

Original: The two distributions can be well separated if the above two bounds do not overlap .

Original: That is, werequire that

Original: [Equation]

Original: On the other hand, it is obvious s + s − < αs 2+ + (1 − α )

Original: s 2 − , so we know

Original: [Equation]

Original: Set ϵ

Original: =

Original: ϵ − to have a larger step and a stricter condition, we have

Original: [Equation]

Original: [Equation]

Original: Because α > 12 and s + > s − ,

Original: [Equation]

Original: So we have

Original: [Equation]

Original: [Equation]

Original: We complete the proof by substituting δ with

Original: [Equation]

Original: 21) where γ > 1 is a constant.

Original: Finally, we prove that

Original: [Equation]

Original: We can guarantee that when the above condition is met, the proposed similarity measure between twonodes from the same category is strictly larger than that for two nodes from different categories.

Original: Itis able to reduce the number of noisy edges and missing edges of the single test method by 1 /γ inexpectation.

Original: F Case Studies on self attention, Q ttention and B ttention

Original: To better understand the effect of multiple tests (i.e. the fourth order of statistics), we illustrate theeffect of self-attention, Q ttention and B ttention with the help of the tool BertViz 2 .

Original: [Figure]

Original: As shown in the left panel of Figure 8, self-attention introduces a number of noisy connectionsbetween nodes belonging to different categories.

Original: In contrast, according to the middle panel of

Original: Figure 8, Q ttention is able to introduce many more connections between nodes from the same category, and at the same time significantly reduces the weights for those noisy connections.

Original: Inthe last panel, the combination self-attention with the fourth order statistics, The B ttention furtherremoves those noisy connections while keeps most of the clean connections between nodes.