Original: Randomized Message-Interception Smoothing: Gray-box Certiﬁcates for Graph Neural Networks
Revised: Randomized Message-Interception Smoothing: Gray-box Certiﬁcates for Graph Neural Networks

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Randomized smoothing is one of the most promising frameworks for certifyingthe adversarial robustness of machine learning models, including Graph Neural
Revised: Randomized smoothing is one of the most promising frameworks for certifyingthe adversarial robustness of machine learning models, including Graph Neural

Original: Networks (GNNs).
Revised: Networks (GNNs).

Original: Yet, existing randomized smoothing certiﬁcates for GNNs areoverly pessimistic since they treat the model as a black box, ignoring the underlyingarchitecture.
Revised: Yet, existing randomized smoothing certiﬁcates for GNNs areoverly pessimistic since they treat the model as a black box, ignoring the underlyingarchitecture.

Original: To remedy this, we propose novel gray-box certiﬁcates that exploitthe message-passing principle of GNNs: We randomly intercept messages andcarefully analyze the probability that messages from adversarially controlled nodesreach their target nodes.
Revised: To remedy this, we propose novel gray-box certiﬁcates that exploitthe message-passing principle of GNNs: We randomly intercept messages andcarefully analyze the probability that messages from adversarially controlled nodesreach their target nodes.

Original: Compared to existing certiﬁcates, we certify robustness tomuch stronger adversaries that control entire nodes in the graph and can arbitrarilymanipulate node features.
Revised: Compared to existing certiﬁcates, we certify robustness tomuch stronger adversaries that control entire nodes in the graph and can arbitrarilymanipulate node features.

Original: Our certiﬁcates provide stronger guarantees for attacksat larger distances, as messages from farther-away nodes are more likely to getintercepted.
Revised: Our certiﬁcates provide stronger guarantees for attacksat larger distances, as messages from farther-away nodes are more likely to getintercepted.

Original: We demonstrate the effectiveness of our method on various models anddatasets.
Revised: We demonstrate the effectiveness of our method on various models anddatasets.

Original: Since our gray-box certiﬁcates consider the underlying graph structure,we can signiﬁcantly improve certiﬁable robustness by applying graph sparsiﬁcation.
Revised: Since our gray-box certiﬁcates consider the underlying graph structure,we can signiﬁcantly improve certiﬁable robustness by applying graph sparsiﬁcation.

Original:  Introduction
Revised:  Introduction

Original: The core principle behind the majority of Graph Neural Networks (GNNs) is message passing –the representation of a node is (recursively) computed based on the representations of its neighbors(Gilmer et al., 2017).
Revised: The core principle behind the majority of Graph Neural Networks (GNNs) is message passing –the representation of a node is (recursively) computed based on the representations of its neighbors(Gilmer et al., 2017).

Original: This allows for information to propagate across the graph, e g. in a k-layer
Revised: This allows for information to propagate across the graph, e g. in a k-layer

Original: GNN the prediction for a node depends on the messages received from its k-hop neighborhood.
Revised: GNN the prediction for a node depends on the messages received from its k-hop neighborhood.

Original: Withsuch models, if an adversary controls a few nodes in the graph, they can manipulate node features tocraft adversarial messages that in turn change the prediction of a target node.
Revised: Withsuch models, if an adversary controls a few nodes in the graph, they can manipulate node features tocraft adversarial messages that in turn change the prediction of a target node.

Original: For example in social networks, adversaries may arbitrarily manipulate features of nodes in theircontrol as modelled by recent adversarial attacks (Ma et al., 2020; Zou et al., 2021).
Revised: For example in social networks, adversaries may arbitrarily manipulate features of nodes in theircontrol as modelled by recent adversarial attacks (Ma et al., 2020; Zou et al., 2021).

Original: Such perturbationsmay be unnoticeable and detecting them is a difﬁcult  unsolved task (Carlini and Wagner, 2017a).
Revised: Such perturbationsmay be unnoticeable and detecting them is a difﬁcult yet unsolved task (Carlini and Wagner, 2017a).

Original: How can we limit the inﬂuence of such adversarial attacks?
Revised: How can we limit the inﬂuence of such adversarial attacks?

Original: We introduce a simple but powerful idea:intercept adversarial messages.
Revised: We introduce a simple but powerful idea:intercept adversarial messages.

Original: Speciﬁcally, we propose message-interception smoothing wherewe randomly delete edges and/or randomly ablate (mask) nodes, and analyze the probability thatmessages from adversarially controlled nodes reach the target nodes.
Revised: Speciﬁcally, we propose message-interception smoothing wherewe randomly delete edges and/or randomly ablate (mask) nodes, and analyze the probability thatmessages from adversarially controlled nodes reach the target nodes.

Original: By transforming any messagepassing GNN into a smoothed GNN where the prediction is the majority vote under this randomizedmessage interception we can provide robustness certiﬁcates (compare Figure 1).
Revised: By transforming any messagepassing GNN into a smoothed GNN where the prediction is the majority vote under this randomizedmessage interception we can provide robustness certiﬁcates (compare Figure 1).

Original: Experimentally we obtain signiﬁcantly better robustness guarantees compared to previous (smoothing)certiﬁcates for GNNs (compare Section 6).
Revised: Experimentally we obtain signiﬁcantly better robustness guarantees compared to previous (smoothing)certiﬁcates for GNNs (compare Section 6).

Original: This improvement stems from the fact that our certiﬁcatestake the underlying architecture of the classiﬁer into account.
Revised: This improvement stems from the fact that our certiﬁcatestake the underlying architecture of the classiﬁer into account.

Original: Unlike previous randomized smoothingcertiﬁcates which treat the GNN as a black-box, our certiﬁcates are gray-box .
Revised: Unlike previous randomized smoothingcertiﬁcates which treat the GNN as a black-box, our certiﬁcates are gray-box .

Original: By making thecertiﬁcate message-passing aware we partially open the black-box and obtain stronger guarantees.
Revised: By making thecertiﬁcate message-passing aware we partially open the black-box and obtain stronger guarantees.

Original: Adversarial node
Revised: Adversarial node

Original: Ablated node
Revised: Ablated node

Original: Adversarial message
Revised: Adversarial message

Original: Intercepted message
Revised: Intercepted message

Original: Deleted edge v
Revised: Deleted edge v

Original: Receptive ﬁeld of v v v v ...
Revised: Receptive ﬁeld of v v v v ...

Original: Adversarial messages intercepted in 2 out of 3 random samples
Revised: Adversarial messages intercepted in 2 out of 3 random samples

Original: Our approach is also in contrast to white-box certiﬁcates that apply only to very speciﬁc models, e.g.
Revised: Our approach is also in contrast to white-box certiﬁcates that apply only to very speciﬁc models, e.g.

Original: Zügner and Günnemann (2019) can only certify the GCN model (Kipf and Welling, 2017).
Revised: Zügner and Günnemann (2019) can only certify the GCN model (Kipf and Welling, 2017).

Original: Whilenewly introduced GNNs would require such certiﬁcates to be derived from scratch, our approach ismodel-agnostic and ﬂexible enough to accommodate a large family of message-passing GNNs.
Revised: Whilenewly introduced GNNs would require such certiﬁcates to be derived from scratch, our approach ismodel-agnostic and ﬂexible enough to accommodate a large family of message-passing GNNs.

Original: We demonstrate our certiﬁcates on node classiﬁcation datasets and analyze the robustness of existing
Revised: We demonstrate our certiﬁcates on node classiﬁcation datasets and analyze the robustness of existing

Original: GNN architectures.
Revised: GNN architectures.

Original: By applying simple graph sparsiﬁcation we further increase the certiﬁablerobustness while retaining high accuracy, as sparsiﬁcation reduces the number of messages tointercept.
Revised: By applying simple graph sparsiﬁcation we further increase the certiﬁablerobustness while retaining high accuracy, as sparsiﬁcation reduces the number of messages tointercept.

Original: In stark contrast to previous probabilistic smoothing-based certiﬁcates for GNNs, ourcertiﬁcates require only a few Monte-Carlo samples, are more efﬁcient and practically more relevant:
Revised: In stark contrast to previous probabilistic smoothing-based certiﬁcates for GNNs, ourcertiﬁcates require only a few Monte-Carlo samples, are more efﬁcient and practically more relevant:

Original: For example, our certiﬁcates on Cora-ML saturate in just 17 seconds and assume much strongeradversaries than previous smoothing certiﬁcates (Bojchevski et al., 2020) that take up to 25 minutes.
Revised: For example, our certiﬁcates on Cora-ML saturate in just 17 seconds and assume much strongeradversaries than previous smoothing certiﬁcates (Bojchevski et al., 2020) that take up to 25 minutes.

Original: In short, our main contributions are:
Revised: In short, our main contributions are:

Original: • The ﬁrst gray-box smoothing-based certiﬁcates for GNNs that exploit the underlyingmessage-passing principle for stronger guarantees.
Revised: • The ﬁrst gray-box smoothing-based certiﬁcates for GNNs that exploit the underlyingmessage-passing principle for stronger guarantees.

Original: • Novel randomized smoothing certiﬁcates for strong threat models where adversaries canarbitrarily manipulate
Revised: • Novel randomized smoothing certiﬁcates for strong threat models where adversaries canarbitrarily manipulate

Original: features of a few nodes in their control.
Revised: features of a few nodes in their control.

Original:  Preliminaries and Background
Revised:  Preliminaries and Background

Original: Threat model.
Revised: Threat model.

Original: We develop certiﬁcates against feature perturbations for evasion threat models.
Revised: We develop certiﬁcates against feature perturbations for evasion threat models.

Original: Given attributed graphs G =
Revised: Given attributed graphs G =

Original: ( A , X ) ∈ G encoded via adjacency matrix A ∈ { 0 , 1 } n × n andfeature matrix X ∈ R n × d with n nodes and d features.
Revised: ( A , X ) ∈ G encoded via adjacency matrix A ∈ { 0 , 1 } n × n andfeature matrix X ∈ R n × d with n nodes and d features.

Original: We formally deﬁne the threat model offeature perturbations as a ball B ρ ( G ) (cid:44) { G (cid:48) :
Revised: We formally deﬁne the threat model offeature perturbations as a ball B ρ ( G ) (cid:44) { G (cid:48) :

Original: δ ( G, G (cid:48) ) ≤ ρ } around a given graph G =
Revised: δ ( G, G (cid:48) ) ≤ ρ } around a given graph G =

Original: ( A , X ) ,where δ ( G, G (cid:48) )
Revised: ( A , X ) ,where δ ( G, G (cid:48) )

Original: (cid:44) (cid:80) nv =1 1 x v (cid:54) =
Revised: (cid:44) (cid:80) nv =1 1 x v (cid:54) =

Original: x (cid:48) v denotes the number of nodes whose features differ in at least onedimension when comparing the clean graph G and the perturbed graph G (cid:48)
Revised: x (cid:48) v denotes the number of nodes whose features differ in at least onedimension when comparing the clean graph G and the perturbed graph G (cid:48)

Original: =
Revised: =

Original: ( A (cid:48) , X (cid:48) ) .
Revised: ( A (cid:48) , X (cid:48) ) .

Original: Intuitively,this means adversaries control up to ρ nodes in the graph and can arbitrarily manipulate node features.
Revised: Intuitively,this means adversaries control up to ρ nodes in the graph and can arbitrarily manipulate node features.

Original: Graph neural networks.
Revised: Graph neural networks.

Original: We design robustness certiﬁcates for message-passing GNNs composed of
Revised: We design robustness certiﬁcates for message-passing GNNs composed of

Original: GNN-layers Ψ
Revised: GNN-layers Ψ

Original: ( (cid:96) )
Revised: ( (cid:96) )

Original: that aggregate the node representations of previous layers based on the underlyinggraph structure (Gilmer et al., 2017).
Revised: that aggregate the node representations of previous layers based on the underlyinggraph structure (Gilmer et al., 2017).

Original: Each message-passing layer ﬁrst gathers messages m ( (cid:96) − 1)
Revised: Each message-passing layer ﬁrst gathers messages m ( (cid:96) − 1)

Original: u fromall nodes u in the local neighborhood N ( v ) (cid:44) { u |
Revised: u fromall nodes u in the local neighborhood N ( v ) (cid:44) { u |

Original: A uv = 1 } of node v .
Revised: A uv = 1 } of node v .

Original: The gathered messagesare subsequently transformed and aggregated into the hidden representation m ( (cid:96) ) v of node v .
Revised: The gathered messagesare subsequently transformed and aggregated into the hidden representation m ( (cid:96) ) v of node v .

Original: Moreformally, a message-passing layer can be described as follows:
Revised: Moreformally, a message-passing layer can be described as follows:

Original: [Equation]
Revised: [Equation]

Original: For node classiﬁcation, message-passing GNNs with k GNN-layers can be described as parametrizedfunctions f : G → { 1 , . . .
Revised: For node classiﬁcation, message-passing GNNs with k GNN-layers can be described as parametrizedfunctions f : G → { 1 , . . .

Original: , C } n that assign each node v in graph G class f v ( G ) :
Revised: , C } n that assign each node v in graph G class f v ( G ) :

Original: [Equation]
Revised: [Equation]

Original: where m (0) v (cid:44) x v ∈ R d denotes the input and m ( k ) v ∈ R C the ﬁnal representation of node v .
Revised: where m (0) v (cid:44) x v ∈ R d denotes the input and m ( k ) v ∈ R C the ﬁnal representation of node v .

Original: Randomized smoothing.
Revised: Randomized smoothing.

Original: Our robustness certiﬁcates for GNNs build upon the randomized smoothingframework (Cohen et al., 2019; Lecuyer et al., 2019):
Revised: Our robustness certiﬁcates for GNNs build upon the randomized smoothingframework (Cohen et al., 2019; Lecuyer et al., 2019):

Original: Given any base classiﬁer f , for example amessage-passing GNN, we can build a “smoothed” classiﬁer g that classiﬁes randomly perturbedinput samples, and then takes the “majority vote” among all predictions.
Revised: Given any base classiﬁer f , for example amessage-passing GNN, we can build a “smoothed” classiﬁer g that classiﬁes randomly perturbedinput samples, and then takes the “majority vote” among all predictions.

Original: The goal is to construct asmoothed classiﬁer g that behaves similar to f (for example in terms of accuracy), and for which wecan prove (probabilistic) robustness certiﬁcates.
Revised: The goal is to construct asmoothed classiﬁer g that behaves similar to f (for example in terms of accuracy), and for which wecan prove (probabilistic) robustness certiﬁcates.

Original: Randomized ablation (Levine and Feizi, 2020) is a smoothing-based certiﬁcate that “ablates” the input:
Revised: Randomized ablation (Levine and Feizi, 2020) is a smoothing-based certiﬁcate that “ablates” the input:

Original: Unlike randomized smoothing where the input in randomly perturbed (e.g. by adding Gaussian noiseto images), in randomized ablation the input is randomly masked, for example by replacing parts ofthe input with a special ablation token that “hides” the original information.
Revised: Unlike randomized smoothing where the input in randomly perturbed (e.g. by adding Gaussian noiseto images), in randomized ablation the input is randomly masked, for example by replacing parts ofthe input with a special ablation token that “hides” the original information.

Original: If the perturbed input ismasked for the majority of predictions, we can issue certiﬁcates for the smoothed classiﬁer g .
Revised: If the perturbed input ismasked for the majority of predictions, we can issue certiﬁcates for the smoothed classiﬁer g .

Original:  Randomized Message-Interception Smoothing for Graph Neural Networks
Revised:  Randomized Message-Interception Smoothing for Graph Neural Networks

Original: The main idea of our gray-box smoothing certiﬁcates is to intercept messages from perturbed nodes by(1) deleting edges to disconnect nodes, and/or (2) ablating nodes to mask their features (cf.
Revised: The main idea of our gray-box smoothing certiﬁcates is to intercept messages from perturbed nodes by(1) deleting edges to disconnect nodes, and/or (2) ablating nodes to mask their features (cf.

Original: Figure 1).
Revised: Figure 1).

Original: To implement this we introduce two independent smoothing distributions φ 1 ( A ) and φ 2 ( X ) thatrandomly apply those changes to the input graph: The ﬁrst smoothing distribution φ 1 ( A ) randomlydeletes edges in the adjacency matrix ( 1 → 0 ) with probability p d .
Revised: To implement this we introduce two independent smoothing distributions φ 1 ( A ) and φ 2 ( X ) thatrandomly apply those changes to the input graph: The ﬁrst smoothing distribution φ 1 ( A ) randomlydeletes edges in the adjacency matrix ( 1 → 0 ) with probability p d .

Original: The second smoothing distributionφ 2 ( X ) randomly ablates all features of nodes with probability p a by replacing their feature representations with a ﬁxed representation token t ∈ R d for ablated nodes.
Revised: The second smoothing distributionφ 2 ( X ) randomly ablates all features of nodes with probability p a by replacing their feature representations with a ﬁxed representation token t ∈ R d for ablated nodes.

Original: Introducing two independentsmoothing distributions is important since our base classiﬁers f are GNNs and in practice, GNNsbehave differently to structural changes in the graph than to feature ablation of nodes.
Revised: Introducing two independentsmoothing distributions is important since our base classiﬁers f are GNNs and in practice, GNNsbehave differently to structural changes in the graph than to feature ablation of nodes.

Original: We apply this message-interception smoothing distribution φ ( G ) (cid:44) ( φ 1 ( A ) , φ 2 ( X )) by randomlysampling and then classifying different graphs with a message-passing GNN f .
Revised: We apply this message-interception smoothing distribution φ ( G ) (cid:44) ( φ 1 ( A ) , φ 2 ( X )) by randomlysampling and then classifying different graphs with a message-passing GNN f .

Original: Our smoothedclassiﬁer g ﬁnally takes the majority vote among the predictions of f for the sampled graphs φ ( G ) .
Revised: Our smoothedclassiﬁer g ﬁnally takes the majority vote among the predictions of f for the sampled graphs φ ( G ) .

Original: We formally describe our smoothed g classiﬁer as follows:
Revised: We formally describe our smoothed g classiﬁer as follows:

Original: [Equation]
Revised: [Equation]

Original: where p v,y ( G ) denotes the probability that the base GNN f classiﬁes node v in graph G as class yunder the smoothing distribution φ ( G ) =
Revised: where p v,y ( G ) denotes the probability that the base GNN f classiﬁes node v in graph G as class yunder the smoothing distribution φ ( G ) =

Original: ( φ 1 ( A ) , φ 2 ( X )) .
Revised: ( φ 1 ( A ) , φ 2 ( X )) .

Original: Note that the ablation representation t isa trainable parameter of our smoothed classiﬁer and can be optimized during training.
Revised: Note that the ablation representation t isa trainable parameter of our smoothed classiﬁer and can be optimized during training.

Original:  Provable Gray-box Robustness Certiﬁcates for Graph Neural Networks
Revised:  Provable Gray-box Robustness Certiﬁcates for Graph Neural Networks

Original: We derive provable robustness certiﬁcates for the smoothed classiﬁer g by exploiting that perturbednodes are disconnected or ablated and cannot send messages for the majority of predictions.
Revised: We derive provable robustness certiﬁcates for the smoothed classiﬁer g by exploiting that perturbednodes are disconnected or ablated and cannot send messages for the majority of predictions.

Original: Wetherefore develop a condition that guarantees g v ( G ) =
Revised: Wetherefore develop a condition that guarantees g v ( G ) =

Original: g v ( G (cid:48) ) for any graph G (cid:48)
Revised: g v ( G (cid:48) ) for any graph G (cid:48)

Original: ∈ B ρ ( G ) .
Revised: ∈ B ρ ( G ) .

Original: We assume the worst-case, that is adversaries alter the prediction whenever target nodes receivemessages from perturbed nodes.
Revised: We assume the worst-case, that is adversaries alter the prediction whenever target nodes receivemessages from perturbed nodes.

Original: Formally, let E denote the event that at least one perturbed messagereaches a target node v .
Revised: Formally, let E denote the event that at least one perturbed messagereaches a target node v .

Original: Then we can prove that ∆ (cid:44) p ( E ) quantiﬁes how much probability mass ofthe distribution p v,y ( G ) over classes y is controlled by the worst-case adversary:
Revised: Then we can prove that ∆ (cid:44) p ( E ) quantiﬁes how much probability mass ofthe distribution p v,y ( G ) over classes y is controlled by the worst-case adversary:

Original: Proposition 1.
Revised: Proposition 1.

Original: Given target node v in graph G , and adversarial budget ρ .
Revised: Given target node v in graph G , and adversarial budget ρ .

Original: Let E denote the eventthat the prediction f v ( φ ( G )) receives at least one messages from perturbed nodes.
Revised: Let E denote the eventthat the prediction f v ( φ ( G )) receives at least one messages from perturbed nodes.

Original: Then the changein label probability | p v,y ( G ) − p v,y ( G (cid:48) )
Revised: Then the changein label probability | p v,y ( G ) − p v,y ( G (cid:48) )

Original: | is bounded by the probability ∆ = p ( E ) for all classesy ∈ { 1 , . . .
Revised: | is bounded by the probability ∆ = p ( E ) for all classesy ∈ { 1 , . . .

Original: , C } and graphs G (cid:48) with G (cid:48)
Revised: , C } and graphs G (cid:48) with G (cid:48)

Original: ∈ B ρ ( G ) : | p v,y ( G ) − p v,y ( G (cid:48) )
Revised: ∈ B ρ ( G ) : | p v,y ( G ) − p v,y ( G (cid:48) )

Original: | ≤ ∆ .
Revised: | ≤ ∆ .

Original: Proof sketch .
Revised: Proof sketch .

Original: Whenever we intercept all adversarial messages, adversaries cannot alter the prediction.
Revised: Whenever we intercept all adversarial messages, adversaries cannot alter the prediction.

Original: Thus | p v,y ( G ) − p v,y ( G (cid:48) )
Revised: Thus | p v,y ( G ) − p v,y ( G (cid:48) )

Original: | is bounded by ∆ .
Revised: | is bounded by ∆ .

Original: (cid:3)
Revised: (cid:3)

Original: Full proof in Appendix A. Since the probability ∆ bounds the worst-case change | p v,y ( G ) − p v,y ( G (cid:48) ) |for all classes
Revised: Full proof in Appendix A. Since the probability ∆ bounds the worst-case change | p v,y ( G ) − p v,y ( G (cid:48) ) |for all classes

Original: y , we can utilize ∆ to construct robustness certiﬁcates:
Revised: y , we can utilize ∆ to construct robustness certiﬁcates:

Original: If an adversary cannot shiftenough probability mass to change the majority class, our smoothed classiﬁer is robust.
Revised: If an adversary cannot shiftenough probability mass to change the majority class, our smoothed classiﬁer is robust.

Original: Before we construct our ﬁnal certiﬁcates, we ﬁrst consider a special case of node ablation smoothing,discussing its relation to randomized ablation for image classiﬁers (Levine and Feizi, 2020), and thenwe derive our provably stronger guarantees for the general case of message-interception smoothing.
Revised: Before we construct our ﬁnal certiﬁcates, we ﬁrst consider a special case of node ablation smoothing,discussing its relation to randomized ablation for image classiﬁers (Levine and Feizi, 2020), and thenwe derive our provably stronger guarantees for the general case of message-interception smoothing.

Original: Proposition 2.
Revised: Proposition 2.

Original: For node feature ablation smoothing only ( p d = 0 ), we have ∆ = 1 − p ρa .
Revised: For node feature ablation smoothing only ( p d = 0 ), we have ∆ = 1 − p ρa .

Original: Proof in Appendix B.
Revised: Proof in Appendix B.

Original: In this special case of node feature ablation smoothing, our certiﬁcates for
Revised: In this special case of node feature ablation smoothing, our certiﬁcates for

Original: GNNs are theoretically related to the randomized ablation certiﬁcates for image classiﬁers (Levineand Feizi, 2020).
Revised: GNNs are theoretically related to the randomized ablation certiﬁcates for image classiﬁers (Levineand Feizi, 2020).

Original: We could apply their certiﬁcate to GNNs by randomly ablating features of entirenodes, instead of pixels in an image.
Revised: We could apply their certiﬁcate to GNNs by randomly ablating features of entirenodes, instead of pixels in an image.

Original: However, their approach is speciﬁcally designed for imageclassiﬁers and comes with serious shortcomings when applied to GNNs.
Revised: However, their approach is speciﬁcally designed for imageclassiﬁers and comes with serious shortcomings when applied to GNNs.

Original: In contrast, our certiﬁcatesare designed for graph-structured data, provably tighter and experimentally stronger.
Revised: In contrast, our certiﬁcatesare designed for graph-structured data, provably tighter and experimentally stronger.

Original: We thoroughlydiscuss such differences with more technical details in Appendix C. Most importantly, their certiﬁcateapplied to GNNs ignores the underlying graph structure.
Revised: We thoroughlydiscuss such differences with more technical details in Appendix C. Most importantly, their certiﬁcateapplied to GNNs ignores the underlying graph structure.

Original: Instead, our certiﬁcates are message-passingaware and consider the interception of messages via edge deletion as follows:
Revised: Instead, our certiﬁcates are message-passingaware and consider the interception of messages via edge deletion as follows:

Original: Message-interception.
Revised: Message-interception.

Original: To model the interception of messages from perturbed nodes B , we take thegraph structure A into account:
Revised: To model the interception of messages from perturbed nodes B , we take thegraph structure A into account:

Original: We consider all simple paths P kwv =
Revised: We consider all simple paths P kwv =

Original: { ( e 1 , . . .
Revised: { ( e 1 , . . .

Original: , e i ) | i ≤ k } fromperturbed nodes w ∈ B to target node v of length at most k .
Revised: , e i ) | i ≤ k } fromperturbed nodes w ∈ B to target node v of length at most k .

Original: Intuitively, if any edge e on path p ∈ P kwvis deleted, or the features of w are ablated, messages via path p get intercepted.
Revised: Intuitively, if any edge e on path p ∈ P kwvis deleted, or the features of w are ablated, messages via path p get intercepted.

Original: If all messages fromperturbed nodes get intercepted, adversaries cannot alter the prediction (Proof in Appendix A):
Revised: If all messages fromperturbed nodes get intercepted, adversaries cannot alter the prediction (Proof in Appendix A):

Original: Lemma 1.
Revised: Lemma 1.

Original: Given perturbed nodes B .
Revised: Given perturbed nodes B .

Original: Then f v ( φ ( G ))
Revised: Then f v ( φ ( G ))

Original: =
Revised: =

Original: f v ( φ ( G (cid:48) )) for any graph G (cid:48)
Revised: f v ( φ ( G (cid:48) )) for any graph G (cid:48)

Original: ∈
Revised: ∈

Original: B ρ ( G ) if
Revised: B ρ ( G ) if

Original: [Equation]
Revised: [Equation]

Original: Since k-layer message-passing GNNs aggregate information over local neighborhoods, only featuresof nodes in the receptive ﬁeld affect the prediction of a target node (that is only nodes with a distanceof at most k to v ).
Revised: Since k-layer message-passing GNNs aggregate information over local neighborhoods, only featuresof nodes in the receptive ﬁeld affect the prediction of a target node (that is only nodes with a distanceof at most k to v ).

Original: For any perturbed node w ∈ B outside of the receptive ﬁeld we have P kwv = ∅ andthe message-interception condition of Lemma 1 is always fulﬁlled.
Revised: For any perturbed node w ∈ B outside of the receptive ﬁeld we have P kwv = ∅ andthe message-interception condition of Lemma 1 is always fulﬁlled.

Original: In practice, however, we do not know the nodes B controlled by the adversary.
Revised: In practice, however, we do not know the nodes B controlled by the adversary.

Original: To account for this,we assume adversaries control nodes indicated by ρ v ∈ { 0 , 1 } n that maximize the probability of theevent E ( ρ v ) that target node v receives perturbed messages:
Revised: To account for this,we assume adversaries control nodes indicated by ρ v ∈ { 0 , 1 } n that maximize the probability of theevent E ( ρ v ) that target node v receives perturbed messages:

Original: Theorem 1.
Revised: Theorem 1.

Original: The worst case change in label probability | p v,y ( G ) − p v,y ( G (cid:48) )
Revised: The worst case change in label probability | p v,y ( G ) − p v,y ( G (cid:48) )

Original: | is bounded by
Revised: | is bounded by

Original: [Equation]
Revised: [Equation]

Original: for all classes y ∈ { 1 , . . .
Revised: for all classes y ∈ { 1 , . . .

Original: , C } and any graph G (cid:48)
Revised: , C } and any graph G (cid:48)

Original: ∈ B ρ ( G ) .
Revised: ∈ B ρ ( G ) .

Original: Proof in Appendix A. Note that we show how to compute ∆ in practice in the next section (Section 5).
Revised: Proof in Appendix A. Note that we show how to compute ∆ in practice in the next section (Section 5).

Original: Notably, ∆ does not depend on the base GNN f : The probability to receive at least one message froma perturbed node is only characterized by the number of perturbed nodes ρ , and the probabilities p d foredge deletion and p a for node feature ablation.
Revised: Notably, ∆ does not depend on the base GNN f : The probability to receive at least one message froma perturbed node is only characterized by the number of perturbed nodes ρ , and the probabilities p d foredge deletion and p a for node feature ablation.

Original: Moreover, ∆ is monotonously increasing in ρ , sincethe probability to receive messages from perturb nodes increases the more nodes adversaries control.
Revised: Moreover, ∆ is monotonously increasing in ρ , sincethe probability to receive messages from perturb nodes increases the more nodes adversaries control.

Original: Finally, we derive robustness certiﬁcates for node classiﬁcation:
Revised: Finally, we derive robustness certiﬁcates for node classiﬁcation:

Original: Corollary 1 (Multi-class certiﬁcate) .
Revised: Corollary 1 (Multi-class certiﬁcate) .

Original: Given ∆ as deﬁned in Proposition 1.
Revised: Given ∆ as deﬁned in Proposition 1.

Original: Then we can certify therobustness g v ( G ) =
Revised: Then we can certify therobustness g v ( G ) =

Original: g v ( G (cid:48) ) for any graph G (cid:48)
Revised: g v ( G (cid:48) ) for any graph G (cid:48)

Original: ∈ B ρ ( G ) if
Revised: ∈ B ρ ( G ) if

Original: [Equation]
Revised: [Equation]

Original: where y ∗ (cid:44) g v ( G ) denotes the majority class, and
Revised: where y ∗ (cid:44) g v ( G ) denotes the majority class, and

Original: ˜ y
Revised: ˜ y

Original: the follow-up (second best) class.
Revised: the follow-up (second best) class.

Original: Proof in Appendix A. We also provide a certiﬁcate for binary node classiﬁcation in Appendix A.
Revised: Proof in Appendix A. We also provide a certiﬁcate for binary node classiﬁcation in Appendix A.

Original: Interestingly, since ∆ is monotonously increasing in ρ , there exists a maximally certiﬁable radius thatdepends on the graph structure and changes for each target node.
Revised: Interestingly, since ∆ is monotonously increasing in ρ , there exists a maximally certiﬁable radius thatdepends on the graph structure and changes for each target node.

Original: In the special case of node featureablation smoothing, we can directly determine the maximally certiﬁable radius (Proof in Appendix B):
Revised: In the special case of node featureablation smoothing, we can directly determine the maximally certiﬁable radius (Proof in Appendix B):

Original: Proposition 3.
Revised: Proposition 3.

Original: Given ﬁxed p a > 0 and p d = 0 , it is impossible to certify a radius ρ if p a ≤ ρ √ 0 .
Revised: Given ﬁxed p a > 0 and p d = 0 , it is impossible to certify a radius ρ if p a ≤ ρ √ 0 .

Original: 1 p a .
Revised: 1 p a .

Original: p d (a) .
Revised: p d (a) .

Original: ∆ i 0 .2 .4 .6 .

Revised: ∆ i 0 .2 .4 .6 .

Original: 1 p a .
Revised: 1 p a .

Original: p d (b) .
Revised: p d (b) .

Original: ∆ i 0 .2 .4 .6 .

Revised: ∆ i 0 .2 .4 .6 .

Original: 1 p a .
Revised: 1 p a .

Original: p d (c) .
Revised: p d (c) .

Original: ∆ i
Revised: ∆ i

Original:  Practical Certiﬁcates for Randomized Message-Interception Smoothing
Revised:  Practical Certiﬁcates for Randomized Message-Interception Smoothing

Original: Message-interception certiﬁcates constitute two challenges in practice: (1) computing the boundingconstant ∆ for arbitrary graphs, and (2) computing the label probabilities p v,y ∗ ( G ) and p v, ˜ y
Revised: Message-interception certiﬁcates constitute two challenges in practice: (1) computing the boundingconstant ∆ for arbitrary graphs, and (2) computing the label probabilities p v,y ∗ ( G ) and p v, ˜ y

Original: ( G ) .
Revised: ( G ) .

Original: We address the ﬁrst problem by providing upper bounds on ∆ (i.e. lower bounds on the certiﬁablerobustness).
Revised: We address the ﬁrst problem by providing upper bounds on ∆ (i.e. lower bounds on the certiﬁablerobustness).

Original: For the second problem, we follow existing literature for probabilistic smoothing-basedcertiﬁcates and estimate the smoothed classiﬁer.
Revised: For the second problem, we follow existing literature for probabilistic smoothing-basedcertiﬁcates and estimate the smoothed classiﬁer.

Original: Lower bound on certiﬁable robustness.
Revised: Lower bound on certiﬁable robustness.

Original: Computing ∆ of Theorem 1 poses a problem: First, ﬁndingthe worst-case selection of nodes for arbitrary receptive ﬁelds involves a challenging optimizationproblem over graph structure.
Revised: Computing ∆ of Theorem 1 poses a problem: First, ﬁndingthe worst-case selection of nodes for arbitrary receptive ﬁelds involves a challenging optimizationproblem over graph structure.

Original: Second, computing the probability p ( E ( ρ v )) to receive perturbedmessages is challenging even for a ﬁxed selection of nodes ρ v , since in general, it involves evaluatingthe expensive inclusion-exclusion principle (see Appendix D).
Revised: Second, computing the probability p ( E ( ρ v )) to receive perturbedmessages is challenging even for a ﬁxed selection of nodes ρ v , since in general, it involves evaluatingthe expensive inclusion-exclusion principle (see Appendix D).

Original: We can compute ∆ exactly only forspecial cases such as small or tree-structured receptive ﬁelds (Appendix E).
Revised: We can compute ∆ exactly only forspecial cases such as small or tree-structured receptive ﬁelds (Appendix E).

Original: For arbitrary graphs, weprovide practical upper bounds on ∆ by assuming independence between paths.
Revised: For arbitrary graphs, weprovide practical upper bounds on ∆ by assuming independence between paths.

Original: Due to Corollary 1,upper bounds on ∆ result in lower bounds on the certiﬁable robustness.
Revised: Due to Corollary 1,upper bounds on ∆ result in lower bounds on the certiﬁable robustness.

Original: We ﬁrst derive an upper bound on ∆ for a single perturbed node, and then generalize to multipleperturbed nodes.
Revised: We ﬁrst derive an upper bound on ∆ for a single perturbed node, and then generalize to multipleperturbed nodes.

Original: In the following, let E w denote the event that the target node v receives messagesfrom node w , and ∆ w (cid:44) p ( E w ) .
Revised: In the following, let E w denote the event that the target node v receives messagesfrom node w , and ∆ w (cid:44) p ( E w ) .

Original: Then we can derive the following upper bound for single sources:
Revised: Then we can derive the following upper bound for single sources:

Original: Theorem 2 (Single Source Multiplicative Bound) .
Revised: Theorem 2 (Single Source Multiplicative Bound) .

Original: Given target node v and source node w in thereceptive ﬁeld of a k-layer message-passing GNN f with respect to v .
Revised: Given target node v and source node w in thereceptive ﬁeld of a k-layer message-passing GNN f with respect to v .

Original: Let P kwv denote all simplepaths from w to v of length at most k in graph G .
Revised: Let P kwv denote all simplepaths from w to v of length at most k in graph G .

Original: Then ∆ w ≤ ∆ w for upper bound ∆ w deﬁned as:
Revised: Then ∆ w ≤ ∆ w for upper bound ∆ w deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where | q | denotes the number of edges on the simple path q ∈ P kwv form w to v .
Revised: where | q | denotes the number of edges on the simple path q ∈ P kwv form w to v .

Original: Proof in Appendix F.
Revised: Proof in Appendix F.

Original: We visualize ∆ w for different edge deletion and node ablation probabilitiesin Figure 2.
Revised: We visualize ∆ w for different edge deletion and node ablation probabilitiesin Figure 2.

Original: Notably, the upper bound for single sources is tight for one- and two-layer GNNs( ∆ = ∆ w ), since then all paths from a single source to the target node are independent (Appendix F).
Revised: Notably, the upper bound for single sources is tight for one- and two-layer GNNs( ∆ = ∆ w ), since then all paths from a single source to the target node are independent (Appendix F).

Original: The single source multiplicative bound on ∆ w can only be used to certify a radius of ρ = 1 .
Revised: The single source multiplicative bound on ∆ w can only be used to certify a radius of ρ = 1 .

Original: In thefollowing, we use Theorem 2 for single sources to bound ∆ of Theorem 1 for multiple nodes ρ > 1 :
Revised: In thefollowing, we use Theorem 2 for single sources to bound ∆ of Theorem 1 for multiple nodes ρ > 1 :

Original: Theorem 3 (Generalized multiplicative bound) .
Revised: Theorem 3 (Generalized multiplicative bound) .

Original: Given monotonously decreasing ∆ i for all nodes iin the receptive ﬁeld so that ∆ i ≥ ∆ i +1 .
Revised: Given monotonously decreasing ∆ i for all nodes iin the receptive ﬁeld so that ∆ i ≥ ∆ i +1 .

Original: Given an adversarial budget of ρ nodes.
Revised: Given an adversarial budget of ρ nodes.

Original: Then ∆ ≤ ∆ M for
Revised: Then ∆ ≤ ∆ M for

Original: [Equation]
Revised: [Equation]

Original: Proof in Appendix F. Notably, the multiplicative bound is tighter than a union bound.
Revised: Proof in Appendix F. Notably, the multiplicative bound is tighter than a union bound.

Original: We speciﬁcallyaddress the approximation error in detail in Appendix G. Moreover, the multiplicative bound is tightin the special case of node feature ablation smoothing (Proposition 4 in Appendix B).
Revised: We speciﬁcallyaddress the approximation error in detail in Appendix G. Moreover, the multiplicative bound is tightin the special case of node feature ablation smoothing (Proposition 4 in Appendix B).

Original: Estimating the smoothed classiﬁer in practice.
Revised: Estimating the smoothed classiﬁer in practice.

Original: Computing the probabilities p v,y ∗ ( G ) and p v, ˜ y
Revised: Computing the probabilities p v,y ∗ ( G ) and p v, ˜ y

Original: ( G )exactly is challenging.
Revised: ( G )exactly is challenging.

Original: We thus estimate them similar to previous work by drawing Monte-Carlosamples from φ (Cohen et al., 2019; Levine and Feizi, 2020; Bojchevski et al., 2020).
Revised: We thus estimate them similar to previous work by drawing Monte-Carlosamples from φ (Cohen et al., 2019; Levine and Feizi, 2020; Bojchevski et al., 2020).

Original: We ﬁrstidentify the majority class y ∗ and follow-up class ˜
Revised: We ﬁrstidentify the majority class y ∗ and follow-up class ˜

Original: y using a few samples.
Revised: y using a few samples.

Original: We then use more samplesto estimate a lower bound p v,y ∗ ( G ) on p v,y ∗ ( G ) and an upper bound p v, ˜
Revised: We then use more samplesto estimate a lower bound p v,y ∗ ( G ) on p v,y ∗ ( G ) and an upper bound p v, ˜

Original: y
Revised: y

Original: ( G ) on p v, ˜ y
Revised: ( G ) on p v, ˜ y

Original: ( G ) .
Revised: ( G ) .

Original: We usethe Clopper-Pearson Bernoulli conﬁdence interval and apply Bonferroni correction to ensure that thebounds hold simultaneously with signiﬁcance level α .
Revised: We usethe Clopper-Pearson Bernoulli conﬁdence interval and apply Bonferroni correction to ensure that thebounds hold simultaneously with signiﬁcance level α .

Original: The resulting bounds hold with probabilityof at least 1 − α .
Revised: The resulting bounds hold with probabilityof at least 1 − α .

Original: Moreover, our smoothed classiﬁer abstains if p v,y ∗ ( G ) ≤ p v, ˜ y
Revised: Moreover, our smoothed classiﬁer abstains if p v,y ∗ ( G ) ≤ p v, ˜ y

Original: ( G ) .
Revised: ( G ) .

Original: Finally, ourcertiﬁcates also hold when bounding ∆ and the label probabilities (Proof in Appendix H):
Revised: Finally, ourcertiﬁcates also hold when bounding ∆ and the label probabilities (Proof in Appendix H):

Original: Corollary 2.
Revised: Corollary 2.

Original: We guarantee the robustness g v ( G ) =
Revised: We guarantee the robustness g v ( G ) =

Original: g v ( G (cid:48) ) with probability of at least 1 − α for anygraph G (cid:48) with G (cid:48)
Revised: g v ( G (cid:48) ) with probability of at least 1 − α for anygraph G (cid:48) with G (cid:48)

Original: ∈ B ρ ( G ) if
Revised: ∈ B ρ ( G ) if

Original: [Equation]
Revised: [Equation]

Original: where y ∗ denotes the (estimated) majority class, and ˜
Revised: where y ∗ denotes the (estimated) majority class, and ˜

Original: y
Revised: y

Original: the follow-up class.
Revised: the follow-up class.

Original: Limitations.
Revised: Limitations.

Original: Our certiﬁcates require knowledge about the graph structure A and can only accountfor structural changes with knowledge about the new structure A (cid:48) .
Revised: Our certiﬁcates require knowledge about the graph structure A and can only accountfor structural changes with knowledge about the new structure A (cid:48) .

Original: We cannot provide certiﬁcateswithout A (cid:48) : While edge deletion potentially increases robustness (due to less messages to intercept),adversaries could arbitrarily increase the number of messages via edge insertion.
Revised: We cannot provide certiﬁcateswithout A (cid:48) : While edge deletion potentially increases robustness (due to less messages to intercept),adversaries could arbitrarily increase the number of messages via edge insertion.

Original: Other limitationsinclude that the number of simple paths can be huge ( O ( r !) for receptive ﬁeld size r ).
Revised: Other limitationsinclude that the number of simple paths can be huge ( O ( r !) for receptive ﬁeld size r ).

Original: We argue,however, that (1) graphs are usually sparse, (2) the number of paths can be reduced via sparsiﬁcation,and (3) we have to compute the paths only once for each graph.
Revised: We argue,however, that (1) graphs are usually sparse, (2) the number of paths can be reduced via sparsiﬁcation,and (3) we have to compute the paths only once for each graph.

Original: Discussion.
Revised: Discussion.

Original: Notably, our edge-deletion certiﬁcates ( p d > 0 ) also provide guarantees for nodeinsertion and deletion, as disconnected nodes cannot alter the prediction.
Revised: Notably, our edge-deletion certiﬁcates ( p d > 0 ) also provide guarantees for nodeinsertion and deletion, as disconnected nodes cannot alter the prediction.

Original: 2 As discussed above, wecan only evaluate such certiﬁcates with structural information, that is how inserted/deleted nodes areconnected to target nodes: Given clean graphs (as in our evaluation), we know which nodes adversariescould delete .
Revised: 2 As discussed above, wecan only evaluate such certiﬁcates with structural information, that is how inserted/deleted nodes areconnected to target nodes: Given clean graphs (as in our evaluation), we know which nodes adversariescould delete .

Original: Given perturbed graphs, we know which nodes adversaries could have inserted .
Revised: Given perturbed graphs, we know which nodes adversaries could have inserted .

Original:  Experimental Evaluation
Revised:  Experimental Evaluation

Original: We evaluate our certiﬁcates for different GNN architectures trained on node classiﬁcation datasets.
Revised: We evaluate our certiﬁcates for different GNN architectures trained on node classiﬁcation datasets.

Original: Our certiﬁcates work in standard transductive learning settings used throughout the literature and wereport such results in Appendix J.
Revised: Our certiﬁcates work in standard transductive learning settings used throughout the literature and wereport such results in Appendix J.

Original: However, combining transductive learning with an evasion threatmodel comes with serious shortcomings for the evaluation of certiﬁcates, since no separate test data isavailable.
Revised: However, combining transductive learning with an evasion threatmodel comes with serious shortcomings for the evaluation of certiﬁcates, since no separate test data isavailable.

Original: For example, we can usually achieve high accuracy by overﬁtting a Multi-Layer Perceptron(MLP) to labels predicted by GNNs during training.
Revised: For example, we can usually achieve high accuracy by overﬁtting a Multi-Layer Perceptron(MLP) to labels predicted by GNNs during training.

Original: MLPs do not propagate information throughthe graph at test time and are robust to adversarial messages.
Revised: MLPs do not propagate information throughthe graph at test time and are robust to adversarial messages.

Original: Instead, we evaluate our certiﬁcates insemi-supervised inductive learning settings with hold-out test nodes:
Revised: Instead, we evaluate our certiﬁcates insemi-supervised inductive learning settings with hold-out test nodes:

Original: Experimental setup.
Revised: Experimental setup.

Original: We randomly draw 20 nodes per class for training and validation, and 10% ofnodes for testing.
Revised: We randomly draw 20 nodes per class for training and validation, and 10% ofnodes for testing.

Original: We create separate graphs for training, validation and testing (using the remainingunlabeled nodes for training).
Revised: We create separate graphs for training, validation and testing (using the remainingunlabeled nodes for training).

Original: We train classiﬁers on the training graph, assume adversaries controlnodes in the graph at test time, and compute certiﬁcates for all test nodes.
Revised: We train classiﬁers on the training graph, assume adversaries controlnodes in the graph at test time, and compute certiﬁcates for all test nodes.

Original: We also randomly deleteedges and ablate node features during training (Appendix N).

Original: We use n 0 = 1 , 000 samples forestimating the majority class, n 1 = 2 , 000 samples for certiﬁcation, and α = 0 .
Revised: We use n 0 = 1 , 000 samplesfor estimating the majority class, n 1 = 2 , 000 samples for certiﬁcation, and α = 0 .

Original: We repeat experiments for ﬁve random splits and model initializations, and report averaged results including standarddeviation (shaded areas in the plots).
Revised: We repeatexperiments for ﬁve random splits and model initializations, and report averaged results includingstandard deviation (shaded areas in the plots).

Revised: We compute simple paths using a modiﬁed depth ﬁrstsearch.

Original: We uploaded our implementation as supplemental material.
Revised: We uploaded our implementation as supplemental material.

Original: Datasets and models.
Revised: Datasets and models.

Original: We train our models on citation datasets: Cora-ML (Bojchevski and Günnemann, 2018; McCallum et al., 2000) with 2,810 nodes, 7,981 edges and 7 classes; Citeseer (Sen et al.
Revised: We train our models on citation datasets: Cora-ML (Bojchevski and Günnemann, 2018; McCallum et al., 2000) with 2,810 nodes, 7,981 edges and 7 classes; Citeseer (Sen et al.

Original: with 2,110 nodes, 3,668 edges and 6 classes; and PubMed (Namata et al., 2012) with 19,717nodes, 44,324 edges and 3 classes.
Revised: with 2,110 nodes, 3,668 edges and 6 classes; and PubMed (Namata et al., 2012) with 19,717nodes, 44,324 edges and 3 classes.

Original: We implement smoothed classiﬁers for four architectures withtwo message-passing layers: Graph convolutional networks (GCN) (Kipf and Welling, 2017), graphattention networks (GAT and GATv2)
Revised: We implement smoothed classiﬁers for four architectures withtwo message-passing layers: Graph convolutional networks (GCN) (Kipf and Welling, 2017), graphattention networks (GAT and GATv2)

Original: (Velickovic et al., 2018; Brody et al., 2021), and soft medoidaggregation networks (SMA) (Geisler et al., 2020).
Revised: (Velickovic et al., 2018; Brody et al., 2021), and soft medoidaggregation networks (SMA) (Geisler et al., 2020).

Original: More details in Appendix I. We also computecertiﬁcates for the larger graph
Revised: More details in Appendix I. We also computecertiﬁcates for the larger graph

Original: ogbn-arxiv
Revised: ogbn-arxiv

Original: (Hu et al., 2020) in Appendix K.
Revised: (Hu et al., 2020) in Appendix K.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: We report the classiﬁcation accuracy of the smoothed classiﬁer on the testset ( clean accuracy ), and the certiﬁed ratio , that is the number of test nodes whose predictions arecertiﬁable robust for a given radius.
Revised: We report the classiﬁcation accuracy of the smoothed classiﬁer on the testset ( clean accuracy ), and the certiﬁed ratio , that is the number of test nodes whose predictions arecertiﬁable robust for a given radius.

Original: Since all nodes have different receptive ﬁelds sizes, we alsodivide the certiﬁable radius by the receptive ﬁeld size.
Revised: Since all nodes have different receptive ﬁelds sizes, we alsodivide the certiﬁable radius by the receptive ﬁeld size.

Original: The resulting normalized robustness betterreﬂects how much percentage of the “attack surface” (that is the number of nodes the adversary couldattack) can be certiﬁed.
Revised: The resulting normalized robustness betterreﬂects how much percentage of the “attack surface” (that is the number of nodes the adversary couldattack) can be certiﬁed.

Original: Moreover, we report the area under this (normalized) certiﬁed ratio curve( AUCRC ).
Revised: Moreover, we report the area under this (normalized) certiﬁed ratio curve( AUCRC ).

Original: For completeness, we also report the certiﬁed accuracy in Appendix J, that is the numberof test nodes that are correctly classiﬁed (without abstaining) and certiﬁable robust for a given radius.
Revised: For completeness, we also report the certiﬁed accuracy in Appendix J, that is the numberof test nodes that are correctly classiﬁed (without abstaining) and certiﬁable robust for a given radius.

Original: [Figure]
Revised: [Figure]

Original: Message-interception smoothing.
Revised: Message-interception smoothing.

Original: In Figure 3 we demonstrate our robustness certiﬁcates for speciﬁcedge deletion probabilities p d and node feature ablation probabilities p a .
Revised: In Figure 3 we demonstrate our robustness certiﬁcates for speciﬁcedge deletion probabilities p d and node feature ablation probabilities p a .

Original: By making our certiﬁcatesmessage-passing aware, we can (1) focus on robustness against feature perturbations of entire nodes,(2) analyze robustness locally in the receptive ﬁelds by incorporating the “attack surface”, and(3) provide stronger guarantees at larger distances.
Revised: By making our certiﬁcatesmessage-passing aware, we can (1) focus on robustness against feature perturbations of entire nodes,(2) analyze robustness locally in the receptive ﬁelds by incorporating the “attack surface”, and(3) provide stronger guarantees at larger distances.

Original: As we discuss in the following, this yieldsconsiderably stronger guarantees compared to previous (smoothing) certiﬁcates for GNNs.
Revised: As we discuss in the following, this yieldsconsiderably stronger guarantees compared to previous (smoothing) certiﬁcates for GNNs.

Original: To thebest of our knowledge, we are the ﬁrst to (1) propose gray-box certiﬁcates for GNNs, and (2) certifyrobustness against such strong adversaries that can arbitrarily manipulate features of a few nodes.
Revised: To thebest of our knowledge, we are the ﬁrst to (1) propose gray-box certiﬁcates for GNNs, and (2) certifyrobustness against such strong adversaries that can arbitrarily manipulate features of a few nodes.

Original: [Figure]
Revised: [Figure]

Original: First certiﬁcate for stronger adversaries.
Revised: First certiﬁcate for stronger adversaries.

Original: Existing certiﬁcates for GNNs only certify perturbationsto a few attributes ˜ ρ in the entire graph.
Revised: Existing certiﬁcates for GNNs only certify perturbationsto a few attributes ˜ ρ in the entire graph.

Original: Our certiﬁcates are novel as they provide guarantees forstronger adversaries that can arbitrarily manipulate features of a few nodes in the graph.
Revised: Our certiﬁcates are novel as they provide guarantees forstronger adversaries that can arbitrarily manipulate features of a few nodes in the graph.

Original: To comparethese two approaches, consider a naïve baseline that certiﬁes ρ =
Revised: To comparethese two approaches, consider a naïve baseline that certiﬁes ρ =

Original: ˜
Revised: ˜

Original: ρ/d nodes, where d is the numberof attributes per node.
Revised: ρ/d nodes, where d is the numberof attributes per node.

Original: If each node in the graph had just a single feature, the number of certiﬁablenodes ρ is high.
Revised: If each node in the graph had just a single feature, the number of certiﬁablenodes ρ is high.

Original: As the number of features d per node increases, however, the baseline dramaticallydeteriorates.
Revised: As the number of features d per node increases, however, the baseline dramaticallydeteriorates.

Original: In contrast, our certiﬁcates are entirely independent of the dimension d and holdregardless of how high-dimensional the underlying node data might be.
Revised: In contrast, our certiﬁcates are entirely independent of the dimension d and holdregardless of how high-dimensional the underlying node data might be.

Original: We show this in Figure 4a forthe ﬁrst smoothing-based certiﬁcate for GNNs (Bojchevski et al., 2020), assuming attribute deletionsagainst second-hop nodes ( p + = 0 , p − = 0 . 6 ).
Revised: We show this in Figure 4a forthe ﬁrst smoothing-based certiﬁcate for GNNs (Bojchevski et al., 2020), assuming attribute deletionsagainst second-hop nodes ( p + = 0 , p − = 0 . 6 ).

Original: However, the superiority of our certiﬁcate regardingrobustness against all features of entire nodes holds for any other GNN certiﬁcate proposed so far.
Revised: However, the superiority of our certiﬁcate regardingrobustness against all features of entire nodes holds for any other GNN certiﬁcate proposed so far.

Original: Efﬁcient message-interception smoothing.
Revised: Efﬁcient message-interception smoothing.

Original: We observe that our probabilistic certiﬁcates saturatealready for small amounts of Monte-Carlo samples from φ (see Figure 4 b,c).
Revised: We observe that our probabilistic certiﬁcates saturatealready for small amounts of Monte-Carlo samples from φ (see Figure 4 b,c).

Original: This is in stark contrastto existing smoothing certiﬁcates for GNNs (Bojchevski et al., 2020).
Revised: This is in stark contrastto existing smoothing certiﬁcates for GNNs (Bojchevski et al., 2020).

Original: We hypothesize that the reasonsare twofold:
Revised: We hypothesize that the reasonsare twofold:

Original: First, the probability ∆ to receive messages from perturbed nodes is independent ofthe base classiﬁer f .
Revised: First, the probability ∆ to receive messages from perturbed nodes is independent ofthe base classiﬁer f .

Original: Second, the number of nodes is typically lower than the number of attributesin the graph, and we evaluate redundant receptive ﬁelds during smoothing as entire subgraphs getdisconnected.
Revised: Second, the number of nodes is typically lower than the number of attributesin the graph, and we evaluate redundant receptive ﬁelds during smoothing as entire subgraphs getdisconnected.

Original: For a fair comparison, we adopt their transductive setting, signiﬁcance level α = 0 .
Revised: For a fair comparison, we adopt their transductive setting, signiﬁcance level α = 0 .

Original: 01 ,and compute certiﬁcates for p d = 0 .
Revised: 01 ,and compute certiﬁcates for p d = 0 .

Original: 3 and p a = 0 . 85 .
Revised: 3 and p a = 0 . 85 .

Original: Bojchevski et al.
Revised: Bojchevski et al.

Original: (2020) use 10 6 samples forcertifying test nodes on Cora-ML, which takes up to 25 minutes.
Revised: (2020) use 10 6 samples forcertifying test nodes on Cora-ML, which takes up to 25 minutes.

Original: In contrast, our certiﬁcates saturatealready for 2 , 000 samples in this setting, which takes only 17 seconds (preprocessing Cora-ML takes8 additional seconds).
Revised: In contrast, our certiﬁcates saturatealready for 2 , 000 samples in this setting, which takes only 17 seconds (preprocessing Cora-ML takes8 additional seconds).

Original: Our gray-box certiﬁcates are signiﬁcantly more efﬁcient.
Revised: Our gray-box certiﬁcates are signiﬁcantly more efﬁcient.

Original: [Figure]
Revised: [Figure]

Original: Sparsiﬁcation improves certiﬁable robustness.
Revised: Sparsiﬁcation improves certiﬁable robustness.

Original: Notably, our gray-box certiﬁcates incorporate graphstructure and become stronger for sparser graphs.
Revised: Notably, our gray-box certiﬁcates incorporate graphstructure and become stronger for sparser graphs.

Original: This is in contrast to all black-box certiﬁcates thatignore the underlying message-passing principles.
Revised: This is in contrast to all black-box certiﬁcates thatignore the underlying message-passing principles.

Original: Speciﬁcally, graph sparsiﬁcation signiﬁcantlyimproves robustness while retaining high clean accuracy: First, sparsiﬁcation reduces the numberof paths in the graph and thus reduces the number of messages to intercept.
Revised: Speciﬁcally, graph sparsiﬁcation signiﬁcantlyimproves robustness while retaining high clean accuracy: First, sparsiﬁcation reduces the numberof paths in the graph and thus reduces the number of messages to intercept.

Original: Second, sparsiﬁcationreduces the number of nodes in the receptive ﬁelds and thus the “attack surface”, i.e. the number ofnodes that send messages.
Revised: Second, sparsiﬁcationreduces the number of nodes in the receptive ﬁelds and thus the “attack surface”, i.e. the number ofnodes that send messages.

Original: In Figure 5 (a,b) we apply GDC preprocessing (Gasteiger et al., 2019)
Revised: In Figure 5 (a,b) we apply GDC preprocessing (Gasteiger et al., 2019)

Original: tothe Cora-ML graph at test time.
Revised: tothe Cora-ML graph at test time.

Original: Interestingly, evaluating the model on the sparsiﬁed graph yieldssigniﬁcantly higher certiﬁable robustness, although both approaches show high clean accuracy of 79% .
Revised: Interestingly, evaluating the model on the sparsiﬁed graph yieldssigniﬁcantly higher certiﬁable robustness, although both approaches show high clean accuracy of 79% .

Original: GDC preprocessing yields a directed graph and reduces the number of edges in the graph from 15 , 962to 14 , 606 (setting (cid:15)
Revised: GDC preprocessing yields a directed graph and reduces the number of edges in the graph from 15 , 962to 14 , 606 ( (cid:15)

Original: = 0 . 022 of GDC).
Revised: = 0 . 022 ).

Original: We ignore edge attributes resulting from GDC preprocessing.
Revised: We ignore edge attributes resulting from GDC preprocessing.

Original: Different classiﬁers.
Revised: Different classiﬁers.

Original: In Figure 5c we compare robustness-accuracy tradeoffs for different GNNsagainst second-hop attacks.
Revised: In Figure 5c we compare robustness-accuracy tradeoffs for different GNNsagainst second-hop attacks.

Original: Attention-based message-passing GNNs (Velickovic et al., 2018; Brodyet al., 2021) are dominating.
Revised: Attention-based message-passing GNNs (Velickovic et al., 2018; Brodyet al., 2021) are dominating.

Original: We hypothesize that the degree-normalization of GCN (Kipf and Welling,2017) may be problematic for the performance under randomized edge deletion.
Revised: We hypothesize that the degree-normalization of GCN (Kipf and Welling,2017) may be problematic for the performance under randomized edge deletion.

Original: Our approach maypromote novel message-passing architectures, speciﬁcally designed for smoothed classiﬁers.
Revised: Our approach maypromote novel message-passing architectures, speciﬁcally designed for smoothed classiﬁers.

Original: Skip-connection.
Revised: Skip-connection.

Original: Assuming adversaries do not attack features of target nodes, we can better studyrobustness in the local neighborhood (Figure 3).
Revised: Assuming adversaries do not attack features of target nodes, we can better studyrobustness in the local neighborhood (Figure 3).

Original: We introduce skip-connections from the features ofeach node to their ﬁnal representations.
Revised: We introduce skip-connections from the features ofeach node to their ﬁnal representations.

Original: We use the transformation weights of the same base classiﬁer,but more complex variations are straightforward.
Revised: We use the transformation weights of the same base classiﬁer,but more complex variations are straightforward.

Original: Notably, such skip-connections lead to signiﬁcantlybetter robustness-accuracy tradeoffs (Appendix J).
Revised: Notably, such skip-connections lead to signiﬁcantlybetter robustness-accuracy tradeoffs (Appendix J).

Original: Future work could deploy existing data-dependentsmoothing methods, which are orthogonal to our work, e.g. if nodes represent images, we couldadditionally deploy Gaussian smoothing (Cohen et al., 2019) for target nodes separately.
Revised: Future work could deploy existing data-dependentsmoothing methods, which are orthogonal to our work, e.g. if nodes represent images, we couldadditionally deploy Gaussian smoothing (Cohen et al., 2019) for target nodes separately.

Original: Robustness-accuracy trade-offs.
Revised: Robustness-accuracy trade-offs.

Original: We show that edge deletion and feature ablation smoothing achievesigniﬁcantly better robustness-accuracy tradeoffs against attribute attacks to the second-hop neighborhood and dominate across all datasets (Figure 6).
Revised: We show that edge deletion and feature ablation smoothing achievesigniﬁcantly better robustness-accuracy tradeoffs against attribute attacks to the second-hop neighborhood and dominate across all datasets (Figure 6).

Original: We compare three settings: (1) edge deletion andfeature ablation ( p d > 0 , p a > 0 ), (2) edge deletion only ( p d > 0 , p a = 0 ), and (3) feature ablationonly ( p d = 0 , p a > 0 ).
Revised: We compare three settings: (1) edge deletion andfeature ablation ( p d > 0 , p a > 0 ), (2) edge deletion only ( p d > 0 , p a = 0 ), and (3) feature ablationonly ( p d = 0 , p a > 0 ).

Original: We run 1,000 experiments for each setting, drawing random deletion andablation probabilities from [0 , 1] for each experiment (allowing different probabilities for trainingand inference).
Revised: We run 1,000 experiments for each setting, drawing random deletion andablation probabilities from [0 , 1] for each experiment (allowing different probabilities for trainingand inference).

Original: We compute dominating points on the Pareto front for each setting.
Revised: We compute dominating points on the Pareto front for each setting.

Original: For brevity, weonly show points with clean accuracy of at most 5% below of the maximally achieved performance.
Revised: For brevity, weonly show points with clean accuracy of at most 5% below of the maximally achieved performance.

Original: 0% 10% 20% AUCRC
Revised: 0% 10% 20% AUCRC

Original: C l ea n A
Revised: C l ea n A

Original: CC ( % ) (a) p d >0, p a >0 p d >0, p a = p d =0, p a > 0% 10% 20% 30% 40% 50% AUCRC
Revised: CC ( % ) (a) p d >0, p a >0 p d >0, p a = p d =0, p a > 0% 10% 20% 30% 40% 50% AUCRC

Original: C l ea n A
Revised: C l ea n A

Original: CC ( % ) (b) p d >0, p a >0 p d >0, p a = p d =0, p a > 0% 10% 20% 30% 40% 50% AUCRC
Revised: CC ( % ) (b) p d >0, p a >0 p d >0, p a = p d =0, p a > 0% 10% 20% 30% 40% 50% AUCRC

Original: C l ea n A
Revised: C l ea n A

Original: CC ( % ) (c) p d >0, p a >0 p d >0, p a = p d =0, p a >
Revised: CC ( % ) (c) p d >0, p a >0 p d >0, p a = p d =0, p a >

Original:  Related work
Revised:  Related work

Original: The vast majority of work on GNN robustness focuses on heuristic defenses including the detectionof adversarial graphs (Zhang and Ma, 2020; Zhang et al., 2019a); architecture modiﬁcations (Brodyet al., 2021;
Revised: The vast majority of work on GNN robustness focuses on heuristic defenses including the detectionof adversarial graphs (Zhang and Ma, 2020; Zhang et al., 2019a); architecture modiﬁcations (Brodyet al., 2021;

Original: Zhang et al., 2019b); robust aggregations (Geisler et al., 2020); robust training procedures(Xu et al., 2019; Zügner and Günnemann, 2019), transfer learning (Tang et al., 2020); and graphpreprocessing techniques such as edge pruning (Zhang and Zitnik, 2020; Wu et al., 2019), low-rankapproximations (Entezari et al., 2020), and graph anomaly detection (Ma et al., 2021).
Revised: Zhang et al., 2019b); robust aggregations (Geisler et al., 2020); robust training procedures(Xu et al., 2019; Zügner and Günnemann, 2019), transfer learning (Tang et al., 2020); and graphpreprocessing techniques such as edge pruning (Zhang and Zitnik, 2020; Wu et al., 2019), low-rankapproximations (Entezari et al., 2020), and graph anomaly detection (Ma et al., 2021).

Original: The effectiveness of such seemingly robust defenses on the adversarial robustness of GNNs canonly be assessed against existing adversarial attacks.
Revised: The effectiveness of such seemingly robust defenses on the adversarial robustness of GNNs canonly be assessed against existing adversarial attacks.

Original: Heuristic defenses do not guarantee robustness,and may even be broken by stronger attacks later on (Carlini and Wagner, 2017b).
Revised: Heuristic defenses do not guarantee robustness,and may even be broken by stronger attacks later on (Carlini and Wagner, 2017b).

Original: Instead, we areinterested in robustness certiﬁcates that provably guarantee the stability of predictions.
Revised: Instead, we areinterested in robustness certiﬁcates that provably guarantee the stability of predictions.

Original: However,robustness certiﬁcates for GNNs are still in their infancy (Günnemann, 2022):
Revised: However,robustness certiﬁcates for GNNs are still in their infancy (Günnemann, 2022):

Original: Most certiﬁcates for GNNs are designed for speciﬁc architectures (Zügner and Günnemann, 2020; Jinet al., 2020; Bojchevski and Günnemann, 2019; Zügner and Günnemann, 2019).
Revised: Most certiﬁcates for GNNs are designed for speciﬁc architectures (Zügner and Günnemann, 2020; Jinet al., 2020; Bojchevski and Günnemann, 2019; Zügner and Günnemann, 2019).

Original: Despite providingprovable robustness guarantees, their applicability is limited to speciﬁc architectures.
Revised: Despite providingprovable robustness guarantees, their applicability is limited to speciﬁc architectures.

Original: Bojchevski et al.
Revised: Bojchevski et al.

Original: (2020) present the ﬁrst tight and efﬁcient smoothing-based, model-agnostic certiﬁcate for graph-structured data.
Revised: (2020) present the ﬁrst tight and efﬁcient smoothing-based, model-agnostic certiﬁcate for graph-structured data.

Original: However, their method comes with crucial limitations: First, theirmethod cannot certify robustness against arbitrary feature modiﬁcations of entire nodes.
Revised: However, their method comes with crucial limitations: First, theirmethod cannot certify robustness against arbitrary feature modiﬁcations of entire nodes.

Original: Second, theirblack-box certiﬁcate deletes edges but completely ignores the underlying message-passing principle.
Revised: Second, theirblack-box certiﬁcate deletes edges but completely ignores the underlying message-passing principle.

Original: Third, their certiﬁcate requires an expensive evaluation of the smoothed classiﬁer, threatening thepracticability of their certiﬁcate beyond theoretical robustness assessments.
Revised: Third, their certiﬁcate requires an expensive evaluation of the smoothed classiﬁer, threatening thepracticability of their certiﬁcate beyond theoretical robustness assessments.

Original: Randomized ablation certiﬁcates for image classiﬁers (Levine and Feizi, 2020) are another approachfor discrete data.
Revised: Randomized ablation certiﬁcates for image classiﬁers (Levine and Feizi, 2020) are another approachfor discrete data.

Original: Such certiﬁcates have already been applied to point cloud classiﬁers (Liu et al.,2021) and even for individual attribute perturbations in GNNs (Bojchevski et al., 2020).
Revised: Such certiﬁcates have already been applied to point cloud classiﬁers (Liu et al.,2021) and even for individual attribute perturbations in GNNs (Bojchevski et al., 2020).

Original: However,
Revised: However,

Original: Bojchevski et al.
Revised: Bojchevski et al.

Original: (2020) show that their method outperforms such ablation certiﬁcates for individualattributes.
Revised: (2020) show that their method outperforms such ablation certiﬁcates for individualattributes.

Original: In contrast, we propose to certify entire nodes, instead of only a few of their attributes.
Revised: In contrast, we propose to certify entire nodes, instead of only a few of their attributes.

Original: Asalready discussed, applying their ablation certiﬁcates for image classiﬁers directly to GNNs comeswith serious shortcomings that we overcome (Section 4 and details in Appendix C).
Revised: Asalready discussed, applying their ablation certiﬁcates for image classiﬁers directly to GNNs comeswith serious shortcomings that we overcome (Section 4 and details in Appendix C).

Original: Recently proposed collective certiﬁcates (Schuchardt et al., 2021) are orthogonal to ours.
Revised: Recently proposed collective certiﬁcates (Schuchardt et al., 2021) are orthogonal to ours.

Original: Ourcertiﬁcates could lead to signiﬁcant improvements in such collective settings, as adversaries cannotattack ﬁrst-hop neighbors of all nodes simultaneously.
Revised: Ourcertiﬁcates could lead to signiﬁcant improvements in such collective settings, as adversaries cannotattack ﬁrst-hop neighbors of all nodes simultaneously.

Original:  Conclusion
Revised:  Conclusion

Original: We propose novel gray-box, message-passing aware robustness certiﬁcates for GNNs against strongthreat models where adversaries can arbitrarily manipulate all features of a few nodes.
Revised: We propose novel gray-box, message-passing aware robustness certiﬁcates for GNNs against strongthreat models where adversaries can arbitrarily manipulate all features of a few nodes.

Original: Our smoothingbased certiﬁcates randomly delete edges and/or mask features of entire nodes to intercept adversarialmessages.
Revised: Our smoothingbased certiﬁcates randomly delete edges and/or mask features of entire nodes to intercept adversarialmessages.

Original: Our certiﬁcates are signiﬁcantly stronger than existing methods and could lead to novelgraph preprocessing techniques that further prevent adversarial examples.
Revised: Our certiﬁcates are signiﬁcantly stronger than existing methods and could lead to novelgraph preprocessing techniques that further prevent adversarial examples.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?

Original: [Yes] To the best of our knowledge, all claims in abstract andintroduction reﬂect the contributions and scope of our paper.
Revised: Yes ] To the best of our knowledge, all claims in abstract andintroduction reﬂect the contributions and scope of our paper.

Original: We also provide a list ofour core contributions directly in our introduction.(b) Did you describe the limitationsof your work?
Revised: We also provide a list ofour core contributions directly in our introduction. (b) Did you describe the limitations

Revised: of

Revised: your

Original: [Yes] We discuss the limitations of ourapproach in a highlighted paragraph in section5.(c)
Revised: work [Yes ] We discuss the limitations of ourapproach in a highlighted paragraph in section

Revised: (c)

Original: Did you discuss any potential negative societal impacts of your work?
Revised: Did you discuss any potential negative societal impacts of your work?

Original: [Yes] Withoutdoubt, adversarial attacks can have negative impacts on the society.
Revised: [Yes] Withoutdoubt, adversarial attacks can have negative impacts on the society.

Original: Particularlyalarming are recent attacks against GNNs for more realistic threat models (Ma et al.
Revised: Particularlyalarming are recent attacks against GNNs for more realistic threat models (Ma et al.

Original: and
Revised: and

Original: attacks that scale to large graphs (Geisler et al., 2021).
Revised: attacks that scale to large graphs (Geisler et al., 2021).

Original: Robustness certiﬁcatesare tools to assess robustness and help to (1) better understand robustness, (2) buildmore robust classiﬁers, and (3) eventually prevent adversarial attacks including their negative consequences.
Revised: Robustness certiﬁcatesare tools to assess robustness and help to (1) better understand robustness, (2) buildmore robust classiﬁers, and (3) eventually prevent adversarial attacks including theirnegative consequences.

Original: Our certiﬁcates represent a contribution towards diminishing and preventing potential negative impacts of adversarial attacks on the society.
Revised: Our certiﬁcates represent a contribution towards diminishingand preventing potential negative impacts of adversarial attacks on the society.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes] We carefully reviewed the ethics guidelines ( https://neurips.cc/ public/EthicsGuidelines ).
Revised: [Yes] We carefully reviewed the ethics guidelines ( https://neurips.cc/public/EthicsGuidelines ).

Original: To the best of our knowledge, our paper conforms to all ethic guidelines.
Revised: To the best of our knowledge, our paper conforms toall ethic guidelines.

Original: Our certiﬁcates represent a contribution to prevent negative social impacts of adversarial attacks, as discussed above.
Revised: Our certiﬁcates represent a contribution to prevent negative socialimpacts of adversarial attacks, as discussed above.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] To the best of
Revised: [Yes] To the best of

Original: our knowledge, we state all assumptions of our theoretical results.
Revised: our knowledge, we state all assumptions of our theoretical results.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] We show all statements,
Revised: [Yes] We show all statements,

Original: including additional elaborations, in the Appendix.
Revised: including additional elaborations, in the Appendix.

Original: We always link theoretical results
Revised: We always link theoretical results

Original: in the paper to the corresponding proofs in the Appendix.
Revised: in the paper to the corresponding proofs in the Appendix.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] We uploaded
Revised: [Yes] We uploaded

Original: the code required to reproduce our main results as supplemental material.
Revised: the code required to reproduce our main results as supplemental material.

Original: All required
Revised: All required

Original: datasets are publicly available, and can be loaded for example with PyTorch Geometric (Fey and Lenssen, 2019).
Revised: datasets are publicly available, and can be loaded for example with PyTorch Geometric(Fey and Lenssen, 2019).

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] We describe important training details directly at the beginning
Revised: [Yes] We describe important training details directly at the beginning

Original: of our experiment section (section 6).
Revised: of our experiment section (section 6).

Original: We further thoroughly list all training details
Revised: We further thoroughly list all training details

Original: in Appendix I.
Revised: in Appendix I.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running ex-
Revised: (c) Did you report error bars (e g., with respect to the random seed after running ex-

Original: periments multiple times)?
Revised: periments multiple times)?

Original: [Yes] We repeat each experiment for ﬁve random splits
Revised: [Yes] We repeat each experiment for ﬁve random splits

Original: and model initializations, only report averaged results, and our plots explicitly 
Revised: and model initializations, only report averaged results and our plots explicitly show

Original: show the standard deviation over the ﬁve experiments (shaded areas in plots).
Revised: the standard deviation over the ﬁve experiments (shaded areas in plots).

Original: In separateexperiments, we tested our method for another set of ﬁve randomly drawn seeds, butwe did not observe signiﬁcant differences in the results.
Revised: In separate experiments, we tested our method for another set of ﬁve randomly drawn seeds, but we did not observe signiﬁcant differences in the results

Original: We alsouploaded all seeds assupplemental material to ensure reproducibility. (d) Did you
Revised: We also

Original: include the total amount
Revised: uploaded all seeds

Original: of
Revised: as

Original: compute and the type of resources used (e
Revised: supplemental material to

Original: g. type of  GPUs, internal cluster, or cloud provider)
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: Yes ] We discuss the runtime for eachexperiment in our experiment section (section 6).
Revised: [Yes] We discuss the runtime for each experiment in our experiment section (section 6).

Original: We conduct all experiments in an
Revised: We conduct all experiments in an

Original: internal cluster with the following GPU type: NVIDIA A100-PCIE-40GB.
Revised: internal cluster with the following GPU type: NVIDIA A100-PCIE-40GB.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] Yes, to the best of our knowledge we cite all authors of the assets we use.[Table]
Revised: [Yes] Yes, to the best ofour knowledge we cite all authors of the assets we use.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: To the best of our knowledge, our
Revised: To the best of our knowledge, our

Original: [Table]

Original: datasets are well-established research datasets with MIT License or public domain.
Revised: datasets are well-established research datasets with MIT License or public domain.

Original: [Table] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] We uploaded our new assets as supplemental material, speciﬁcally the code to
Revised: [Yes] We uploaded our new assets as supplemental material, speciﬁcally the code to

Original: reproduce our main results.
Revised: reproduce our main results.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: To the best of our knowledge, our assets do not require consent.
Revised: To the best of our knowledge, our assets do not require consent.

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: To the best of our knowledge, our assets do
Revised: To the best of our knowledge, our assets do

Original: [Table]

Original: not contain any personal, protected or offensive data.
Revised: not contain any personal, protected or offensive data.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: [Table]

Original: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A]
Revised: [N/A]

Original: We do not conduct research with human subjects.
Revised: We do not conduct research with human subjects.

Original: [Table]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: [Table]
Revised: [Table]

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: We do not conduct research with human
Revised: We do not conduct research with human

Original: [Table]

Original: subjects.
Revised: subjects.

Original: [Table]
Revised: [Table]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: [Table]

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A] We do not conduct research with humansubjects.[Table]
Revised: [N/A] We do not conduct research with humansubjects.