SlowGuess/NeurIPS_2023_OCR huggingface dataset:
GCL has recently gained significant attention and shows promise for improving graph representations in scenarios where labeled data is scarce [31, 32, 9, 30, 27, 15, 33, 32]. Generally, GCL creates two views through data augmentation and contrasts representations between two views. GraphCL [9] focuses on graph classification by exploring four types of augmentations including node dropping, edge perturbation, attribute masking and subgraph sampling. GRACE [30] and GCA [27] adapt SimCLR [55] to graphs to maximize the mutual information between two views for each node through a variety of data augmentation. DGI [31] applied InfoMax principle [56] to train a GNN encoder by maximizing the mutual information between node-level and graph-level representations. MVGRL [32] proposes to learn representations by maximizing the mutual information between the cross-view representations of nodes and graphs. Several recent works [16, 36] show that GCL is vulnerable to adversarial attacks. CLGA [16] is an unsupervised poisoning attacks for attack graph contrastive learning. In detail, the gradients of the adjacency matrices for both views are computed, and edge flipping is performed using gradient ascent to maximize the contrastive loss. Despite very few empirical works [9, 15, 14] on robustness of GCL, there are no existing works studying the certified robustness of GCL. In contrast, we propose to provide robustness certificates for GCL by using randomized edgedrop smoothing. To the best of our knowledge, our method is the first work to study the certified robustness of GCL.
A.2 Certifiable Robustness of Graph Neural Networks
Several recent studies investigate the certified robustness of GNNs in the supervised setting [23, 28, 29, 26, 24, 25, 37]. Zügner et al. [23] are the first to explore certifiable robustness with respect to node feature perturbations. Subsequent works[28, 26, 29, 24, 38] extend the analysis to certifiable robustness under topological attacks. For instance, Bojchevski et al. [28] propose a branch-and-bound algorithm to achieve tight bounds on the global optimum of certificates for topological attacks. Bojchevski et al. [29] further adapt the randomized smoothing technique to sparse settings, deriving certified robustness for GNNs. This approach involves injecting random noise into test samples to mitigate the negative effects of adversarial perturbations. Wang et al. [24] further refine this technique to provide theoretically tight robust certificates. Our work is inherently different from them: (i) existing work focuses on the certified robustness of GNN under (semi)-supervised setting; while we provide a unified definition to evaluate and certify the robustness of GCL for unsupervised representation learning. (ii) we theoretically provide the certified robustness for GCL in the absence of labeled data, and this certified robustness can provably sustained in downstream tasks. (iii) we design an effective training method to enhance the robustness of GCL.
B Preliminary of Randomized Smoothing
One potential solution for achieving certified robustness in GNNs is binary randomized smoothing presented in [24]. Specifically, consider a noisy vector  $\epsilon$  in the discrete space  $\{0,1\}^N$
$$
\mathbb {P} \left(\epsilon_ {i} = 0\right) = \beta , \quad \mathbb {P} \left(\epsilon_ {i} = 1\right) = 1 - \beta , \tag {B.1}
$$
where  $i = 1,2,\dots ,N$ . This indicates that the connection status of the  $i$ -th entry of  $\mathbf{v}$  will be flipped with probability  $1 - \beta$  and preserved with probability  $\beta$ . Given a base node or graph classifier  $f(\mathbf{v})$  that returns the class label with highest label probability, the smoothed classifier  $g$  is defined as:
$$
g (\mathbf {v}) = \underset {y \in \mathcal {Y}} {\arg \max } \mathbb {P} (f (\mathbf {v} \oplus \epsilon) = y), \tag {B.2}
$$
where  $\mathcal{V}$  denotes the label set, and  $\oplus$  represents the XOR operation between two binary variables, which combines the structural information of both variables.  $\mathbb{P}(f(\mathbf{v} \oplus \epsilon) = y)$  is the probability that the base classifier  $f$  predicts class  $y$  when random noise  $\epsilon$  is added to  $\mathbf{v}$ . According to [24], by injecting multiple  $\epsilon$  to  $\mathbf{v}$  and returning the most likely class  $y_A$  (the majority vote), if it is holds that:
$$
\min  \mathbb {P} (f (\mathbf {v} \oplus \epsilon) = y _ {A}) \geq \max  _ {y \neq y _ {A}} \mathbb {P} (f (\mathbf {v} \oplus \epsilon) = y), \text {s . t .} \| \epsilon \| _ {0} \leq k, \tag {B.3}
$$
where  $y_{A}$  is the class with the highest label probability. Then we can guarantee there exists a certified perturbation size  $k$  such that for any perturbation  $\delta$  with  $||\delta ||_0\leq k$ , the predictions of  $\mathbf{v}\oplus \delta$  will
remain unchanged, that is,  $g(\mathbf{v} \oplus \delta) = f(\mathbf{v} \oplus \delta \oplus \epsilon) = f(\mathbf{v} \oplus \epsilon) = y_A$ . Thus,  $f$  is certifiably robust at  $(\mathbf{v}, y_A)$  when perturbing at most  $k$  edges. For more details of the proof, please refer to [24].
However, directly applying the vanilla randomized smoothing approach to certify robustness in GCL is inapplicable. This approach may require injecting random noise into the entire graph during data augmentation, resulting in an excessive number of noisy/spurious edges that can significantly harm downstream task performance, particularly for large and sparse graphs. In contrast, our proposed method, Randomized Edgedrop Smoothing (RES), aims to prevent the introduction of excessive spurious edges in graphs. RES achieves this by injecting randomized edgedrop noises, where observed edges are randomly removed from the graphs with a certain probability. Additionally, we propose an effective training method for robust GCL. Our experimental results in Sec. 6.4 demonstrate that our method significantly outperforms the ablative method, FLIP, that directly applies vanilla randomized smoothing [24] in GCL. Specifically, on the clean Cora and Pubmed graphs, our method achieves accuracies of  $82\%$  and  $85.6\%$  respectively. On the Nettack perturbed [11] Cora and Pubmed graphs, our method achieves robust accuracies of  $79.4\%$  and  $82.5\%$ , respectively. In contrast, FLIP achieves only up to  $27\%$  and  $42\%$  robust accuracies on the Cora and Pubmed datasets, respectively. These results strongly validate the effectiveness of our RES approach,
C Detailed Proofs
To begin, we provide a formalization of similarity in relation to the latent class.
Given a GNN encoder  $h$  and an input sample  $\mathbf{v}$ , which consists of a positive sample  $\mathbf{v}^{+}$  and a negative sample  $\mathbf{v}^{-}$  an input sample  $\mathbf{v}$  with positive sample  $\mathbf{v}^{+}$  and negative sample  $\mathbf{v}^{-}$ , according to Def. 1, we assume that  $c^{+}, c^{-} \in \mathcal{C}$  represent the latent classes of  $\mathbf{v}^{+}$  and  $\mathbf{v}^{-}$  in the latent space of  $h$ , respectively. These latent classes are based on the distribution  $\eta$  over  $\mathcal{C}$ . We assume that  $c^{+}, c^{-} \in \mathcal{C}$  are the latent class of  $\mathbf{v}^{+}$  and  $\mathbf{v}^{-}$  in the latent space of  $h$ , respectively, which are randomly determined based on the distribution  $\eta$  on  $\mathcal{C}$ . Similar data  $h(\mathbf{v}), h(\mathbf{v}^{+})$  are i.i.d. draws from the same class distribution  $D_{c^{+}}$ , whereas negative samples originate from the marginal of  $D_{sim}$ , which are formalized as follow:
$$
\mathcal {D} _ {s i m} (\mathbf {v}, \mathbf {v} ^ {+}) = \underset {c ^ {+} \sim \eta} {\mathbb {E}} \mathcal {D} _ {c ^ {+}} (\mathbf {v}) \mathcal {D} _ {c ^ {+}} (\mathbf {v} ^ {+}).
$$
$$
\mathcal {D} _ {n e g} \left(\mathbf {v} ^ {-}\right) = \underset {c ^ {-} \sim \eta} {\mathbb {E}} \mathcal {D} _ {c ^ {-}} \left(\mathbf {v} ^ {-}\right). \tag {C.4}
$$
Since classes are allowed to overlap and/or be fine-grained [39], this is a plausible formalization of "similarity". The formalization connects similarity with latent class, thereby offering a viable way to employ the similarity between samples under the latent space  $h$  for defining the probability of  $\mathbf{v}$  being the positive sample of  $\mathbf{v}^+$ .
Lemma 1 (The Generalized Extreme Value Distribution [44]). Let  $\mathbf{v}_1, \mathbf{v}_2, \dots$  be a sequence of i.i.d samples from a common distribution function  $F$ . Let  $M_N = \max \{\mathbf{v}_1, \dots, \mathbf{v}_N\}$ , indicating the maximum of  $\{\mathbf{v}_1, \dots, \mathbf{v}_N\}$ . Then if  $\{a_N > 0\}$  and  $\{b_N \in \mathbb{R}\}$  are sequences of constants such that
$$
\mathbb {P} \left\{\left(M _ {N} - b _ {N}\right) / a _ {N} \leq z \right\}\rightarrow G (z), \tag {C.5}
$$
where  $G$  is a non-degenerate distribution function. It is a member of the generalized extreme value family of distributions, which belongs to either the Gumbel family (Type I), the Fréchet family (Type II) or the Reverse Weibull family (Type III) with their CDFs as follows:
Gumbel family (Type I):  $G(z) = \exp \{-\exp \left[ -\left(\frac{z - b}{a}\right)\right]\}$ ,  $z \in \mathbb{R}$ ,
Fréchet family (Type II):  $G(z) = \left\{ \begin{array}{ll}0, & z < b,\\ \exp \bigl {\{-(\frac{z - b}{a})^{-\sigma}\bigr \} ,} & z\geq b, \end{array} \right.$
Reverse Weibull family (Type III):  $G(z) = \left\{ \begin{array}{ll}\exp \bigl {\{-(\frac{b - z}{a})^{-\sigma}\bigr \} , & \mathbf{z} <   \mathbf{b},\\ 1, & \mathbf{z}\geq \mathbf{b}, \end{array} \right.$
where  $a > 0$ ,  $b \in R$  and  $\sigma > 0$  are the scale, location and shape parameters, respectively.
Theorem 1. Let  $\mathbb{B}(\mathbf{v}^{+})$  be a space around  $\mathbf{v}^{+}$  as defined in Eq. 4. Given an input sample  $\mathbf{v}$  and a GNN encoder  $h$  learnt by GCL in Eq. 1, the probability of  $\mathbf{v}$  being the positive sample of  $\mathbf{v}^{+}$  is:
$$
P r (\mathbf {v} \in \mathbb {B} (\mathbf {v} ^ {+}); h) = \exp \left[ - \left(\frac {1 - s (h (\mathbf {v}) , h (\mathbf {v} ^ {+}))}{a}\right) ^ {\sigma} \right], \tag {C.7}
$$
where  $s(\cdot ,\cdot)$  is the cosine similarity function.  $a,\sigma >0$  are Weibull shape and scale parameters [44].
Proof. We first assume  $\{h(\mathbf{v}_1), h(\mathbf{v}_2), \dots\}$  are i.i.d samples under the latent space  $h$ . Then, suppose  $\mathbf{V}^{-} = \{\mathbf{v}_{1}^{-}, \cdot, \mathbf{v}_{n}^{-}\}$  is the set of negative samples of  $\mathbf{v}$ . We assume there exists a continuous non-degenerate margin distribution  $M$ , which is denoted as follows:
$$
M := \min  _ {i \in [ n ]} D _ {i}, \quad \text {w i t h} D _ {i} := (1 - s \left(h \left(\mathbf {v} ^ {+}\right), h \left(\mathbf {v} _ {i} ^ {-}\right)\right)) / 2, \tag {C.8}
$$
where  $M, D_i \in [0,1]$ , and  $s(\cdot, \cdot)$  is the cosine similarity function.  $M$  indicates the half of the minimum distance between  $h(\mathbf{v}^+)$  and  $h(\mathbf{v}_i^-)$ .
According to Lemma 1, we know that there exists  $G(z)$  in the three types of generalized extreme value family. And since Lemma 1 is applied to the maximum, refer to Eq. (C.8), we transfer the variable  $M$  to  $\overline{M} \coloneqq \max_{i \in [n]} -D_i$ . Since  $-D_i$  is bounded  $(-D_i < 0)$ , let  $b = 0$ , the marginal distribution of  $D_i$  series, for  $i = 1,2,\dots$ , can be Reverse Weibull family. Therefore, the asymptotic marginal distribution of  $\overline{M}$  fits into the Reverse Weibull distribution:
$$
G (z) = \left\{ \begin{array}{l l} \exp \{- (\frac {- z}{a}) ^ {- \sigma} \}, & z <   0, \\ 1, & z \geq 0, \end{array} \right.
$$
where  $\sigma > 0$  is the shape parameter and  $a$  is the scale parameter. Compared to Eq. (C.6), here,  $b = 0$  because  $-D_{i}$  is bound  $(-D_{i} < 0)$ . We use margin distances  $D_{i}$  of the  $\lambda$  closest samples with  $\mathbf{v}^{+}$  to estimate the parameters  $\alpha$  and  $\sigma$ , which means to estimate  $\widehat{W}$  of the distribution function  $W$ .
Following Eq. (C.4), if the similarity  $s(h(\mathbf{v}), h(\mathbf{v}^{+}))$  is larger,  $\mathbf{v}$  is more possible to be from the same latent class as  $\mathbf{v}^{+}$ , which implies  $\mathbf{v}$  is the positive sample of  $\mathbf{v}^{+}$ . Since the distance between  $h(\mathbf{v})$  and  $h(\mathbf{v}^{+})$  can be denoted as  $1 - s(h(\mathbf{v}^{+}), h(\mathbf{v}_{i}^{-}))$ , the probability of  $\mathbf{v}$  is the positive sample of  $\mathbf{v}^{+}$  can be written as:
$$
\begin{array}{l} \mathbb {P} (\mathbf {v} \in \mathbb {B} (\mathbf {v} ^ {+}); h) \\ = \mathbb {P} (1 - s (h (\mathbf {v}), h (\mathbf {v} ^ {+})) <   \min  \{D _ {1}, \dots , D _ {n} \}) \\ = \mathbb {P} (s (h (\mathbf {v}), h (\mathbf {v} ^ {+}) - 1) > \max  \{- D _ {1}, \dots , - D _ {n} \}) \tag {C.9} \\ = \mathbb {P} (\bar {M} <   s (h (\mathbf {v}), h (\mathbf {v} ^ {+})) - 1) \\ = \widehat {W} (s (h (\mathbf {v}), h (\mathbf {v} ^ {+})) - 1). \\ \end{array}
$$
Let  $b_{n} = 0$ ,  $a_{n} = 0$  and  $z = s(h(\mathbf{v}), h(\mathbf{v}^{+})) - 1$ , since  $z = s(h(\mathbf{v}), h(\mathbf{v}^{+})) - 1 < 0$ , we can rewrite Eq. (C.9) as (C.7) according to Lemma 1.
For simplicity of notation, let  $p_{\mathbf{v}^{+},h}(\mathbf{v}) = \mathbb{P}(\mathbf{v}\in \mathbb{B}(\mathbf{v}^{+});h)$ . Then, given a randomized edgedrop noise  $\epsilon \in \mathcal{D}_{\epsilon}$  with the following probability distribution:
$$
\mathbb {P} \left(\epsilon_ {i} = 0 \mid \mathbf {v} _ {i} = 0\right) = 1, \mathbb {P} \left(\epsilon_ {i} = 0 \mid \mathbf {v} _ {i} = 1\right) = \beta , \text {a n d} \mathbb {P} \left(\epsilon_ {i} = 1 \mid \mathbf {v} _ {i} = 1\right) = 1 - \beta , \tag {C.10}
$$
where  $\epsilon_{i}$  is the  $i$ -entry of noisy vector  $\epsilon$  and  $\mathbf{v}_i$  is the connection status of  $i$ -th entry of  $\mathbf{v}$ . Then, we let
$$
\underline {{p _ {\mathbf {v} ^ {+} , h}}} (\mathbf {v} \oplus \epsilon) = \inf  _ {\epsilon \in \mathcal {D} _ {\epsilon}} p _ {\mathbf {v} ^ {+}, h} (\mathbf {v} \oplus \epsilon), \overline {{p _ {\mathbf {v} ^ {+} , h}}} (\mathbf {v} \oplus \epsilon) = \sup  _ {\epsilon \in \mathcal {D} _ {\epsilon}} p _ {\mathbf {v} ^ {+}, h} (\mathbf {v} \oplus \epsilon), \tag {C.11}
$$
where  $\underline{p_{\mathbf{v}^{+},h}} (\mathbf{v}\oplus \epsilon)$  and  $\overline{p_{\mathbf{v}_i^{-},h}} (\mathbf{v}\oplus \epsilon)$  denote the lower bound and the upper bound on  $p_{\mathbf{v}^{+},h}(\mathbf{v}\oplus \epsilon)$ , respectively. The formal theorem is presented as follows:
$$
\underline {{p _ {\mathbf {v} ^ {+} , h} (\mathbf {v} \oplus \epsilon)}} - \max  _ {\mathbf {v} _ {i} ^ {-} \in \mathbf {V} ^ {-}} \overline {{p _ {\mathbf {v} _ {i} ^ {-} , h} (\mathbf {v} \oplus \epsilon)}} > 2 \Delta , \tag {C.12}
$$
$$
\Delta = 1 - \frac {\binom {d} {e}}{\binom {d + k} {e}} \cdot \beta^ {k}, \tag {C.13}
$$
and  $e = \| \mathbf{v} \oplus \epsilon \|_0$  denotes the number of remaining edges of  $\mathbf{v}$  after injecting  $\epsilon$ , then with a confidence level of at least  $1 - \alpha$ , we have:
$$
p _ {\mathbf {v} ^ {+}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) > \max  _ {\mathbf {v} _ {i} ^ {-} \in \mathbf {V} ^ {-}} p _ {\mathbf {v} _ {i} ^ {-}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right). \tag {C.14}
$$
Proof. Suppose  $\mathbf{V} = \{\mathbf{v}^{+},\mathbf{v}_{1}^{-},\dots ,\mathbf{v}_{n}^{-}\}$  is the set of positive and negative samples of  $\mathbf{v}$ . For any  $\mathbf{v}_i\in \mathbf{V}$ , let  $p_{\mathbf{v}_i,h}(\mathbf{v}) = \mathbb{P}(\mathbf{v}\in \mathbb{B}(\mathbf{v}_i);h)$  for simplicity, we have:
$$
p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) = \mathbb {P} (\mathbf {v} \oplus \epsilon \in \mathbb {B} (\mathbf {v} _ {i}); h) \tag {C.15}
$$
$$
p _ {\mathbf {v} _ {i}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) = \mathbb {P} \left(\mathbf {v} ^ {\prime} \oplus \epsilon \in \mathbb {B} \left(\mathbf {v} _ {i}\right); h\right).
$$
As stated by the law of total probability, we have:
$$
\begin{array}{l} p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) = \mathbb {P} ([ \mathbf {v} \oplus \epsilon \in \mathbb {B} (\mathbf {v} _ {i}); h ] \wedge [ (\mathbf {v} ^ {\prime} \oplus \epsilon) \cap \delta = \emptyset ]) \\ + \mathbb {P} ([ \mathbf {v} \oplus \epsilon \in \mathbb {B} (\mathbf {v} _ {i}); h ] \wedge [ (\mathbf {v} ^ {\prime} \oplus \epsilon) \cap \delta \neq \emptyset ]) \tag {C.16} \\ p _ {\mathbf {v} _ {i}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) = \mathbb {P} \left(\left[ \mathbf {v} ^ {\prime} \oplus \epsilon \in \mathbb {B} \left(\mathbf {v} _ {i}\right); h \right] \wedge \left[ \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \cap \delta = \emptyset \right]\right) \\ + \mathbb {P} ([ \mathbf {v} ^ {\prime} \oplus \epsilon \in \mathbb {B} (\mathbf {v} _ {i}); h ] \wedge [ (\mathbf {v} ^ {\prime} \oplus \epsilon) \cap \delta \neq \emptyset ]), \\ \end{array}
$$
where  $(\mathbf{v}' \oplus \epsilon) \cap \delta$  represent the intersection of the edge sets  $(\mathbf{v}' \oplus \epsilon)$  and  $\delta$ , that is, the set of edges shared in the two structure vectors. Therefore, for  $\mathbf{v}' \oplus \epsilon \cap \delta = \emptyset$ , it means that no noisy edge of  $\delta$  exists in  $(\mathbf{v}' \oplus \epsilon)$ , which indicates that  $\mathbf{v} \oplus \epsilon$  and  $\mathbf{v}' \oplus \epsilon$  are structural identical at all indices, i.e.,  $\mathbf{v}' \oplus \epsilon = \mathbf{v} \oplus \epsilon$ . Therefore, we can then derive the following equality:
$$
\mathbb {P} \left(\left[ \mathbf {v} \oplus \epsilon \in \mathbb {B} \left(\mathbf {v} _ {i}\right); h \right] \mid \left[ \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \cap \delta = \emptyset \right]\right) = \mathbb {P} \left(\left[ \mathbf {v} ^ {\prime} \oplus \epsilon \in \mathbb {B} \left(\mathbf {v} _ {i}\right); h \right] \mid \left[ \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \cap \delta = \emptyset \right]\right). \tag {C.17}
$$
By multiplying  $\mathbb{P}((\mathbf{v}'\oplus \epsilon)\cap \delta = \emptyset)$  in both sides of Eq. (C.17), we have:
$$
\mathbb {P} \left(\left[ \mathbf {v} \oplus \epsilon \in \mathbb {B} \left(\mathbf {v} _ {i}\right); h \right] \wedge \left[ \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \cap \delta = \emptyset \right]\right) = \mathbb {P} \left(\left[ \mathbf {v} ^ {\prime} \oplus \epsilon \in \mathbb {B} \left(\mathbf {v} _ {i}\right); h \right] \wedge \left[ \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \cap \delta = \emptyset \right]\right). \tag {C.18}
$$
Combining Eq. (C.16) with Eq. (C.18) results in:
$$
\begin{array}{l} p _ {\mathbf {v} _ {i}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) - p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) = \mathbb {P} \left([ \mathbf {v} ^ {\prime} \oplus \epsilon \in \mathbb {B} (\mathbf {v} _ {i}); h ] \wedge [ (\mathbf {v} ^ {\prime} \oplus \epsilon) \cap \delta \neq \emptyset ]\right) - \tag {C.19} \\ \mathbb {P} \left(\left[ \mathbf {v} \oplus \epsilon \in \mathbb {B} \left(\mathbf {v} _ {i}\right); h \right] \wedge \left[ \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \cap \delta \neq \emptyset \right]\right). \\ \end{array}
$$
Since probabilities are non-negative, we can rewrite Eq. (C.19) into the following inequality:
$$
\begin{array}{l} p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) - \mathbb {P} ([ \mathbf {v} \oplus \epsilon \in \mathbb {B} (\mathbf {v} _ {i}); h ] \wedge [ (\mathbf {v} ^ {\prime} \oplus \epsilon) \cap \delta \neq \emptyset ]) \\ \leq p _ {\mathbf {v} _ {i}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \leq \tag {C.20} \\ \end{array}
$$
$$
p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) + \mathbb {P} ([ \mathbf {v} ^ {\prime} \oplus \epsilon \in \mathbb {B} (\mathbf {v} _ {i}); h ] \wedge [ (\mathbf {v} ^ {\prime} \oplus \epsilon) \cap \delta \neq \emptyset ]).
$$
Applying the conjunction rule, we have:
$$
p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) - \mathbb {P} ((\mathbf {v} ^ {\prime} \oplus \epsilon) \cap \delta \neq \emptyset) \leq p _ {\mathbf {v} _ {i}, h} (\mathbf {v} ^ {\prime} \oplus \epsilon) \leq \tag {C.21}
$$
$$
p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) + \mathbb {P} ((\mathbf {v} ^ {\prime} \oplus \epsilon) \cap \delta \neq \emptyset).
$$
Since  $\mathbb{P}((\mathbf{v}' \oplus \epsilon) \cap \delta \neq \emptyset) = 1 - \mathbb{P}((\mathbf{v}' \oplus \epsilon) \cap \delta = \emptyset)$ , and  $(\mathbf{v}' \oplus \epsilon) \cap \delta = \emptyset$  implies that remaining edges of  $\mathbf{v} \oplus \epsilon$  and  $\mathbf{v}' \oplus \epsilon$  are identical after injecting the random masking noise, we can derive the following probability:
$$
\mathbb {P} \left(\left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \cap \delta = \emptyset\right) = \frac {\binom {d} {e}}{\binom {d + | \delta |} {e}} \cdot \beta^ {| \delta |}, \tag {C.22}
$$
where  $e$  denotes the number of remaining edges, and  $k = |\delta|$  denotes the certified perturbation size.  $\beta^{e}(1 - \beta)^{d - e + |\delta|}$  represents the probability that  $|\delta|$  noise edges are all dropped and  $e$  edges of  $\mathbf{v}$  are retained in  $\mathbf{v}' \oplus \epsilon$ . Therefore, we have:
$$
\mathbb {P} \left(\left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \cap \delta \neq \emptyset\right) = 1 - \frac {\binom {d} {e}}{\binom {d + | \delta |} {e}} \cdot \beta^ {| \delta |} \leq 1 - \frac {\binom {d} {e}}{\binom {d + k} {e}} \cdot \beta^ {k} = \Delta . \tag {C.23}
$$
Substituting Eq. (C.23) into Eq. (C.21), then Eq. (C.21) can be rewritten as
$$
p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) - \Delta \leq p _ {\mathbf {v} _ {i}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \leq p _ {\mathbf {v} _ {i}, h} (\mathbf {v} \oplus \epsilon) + \Delta . \tag {C.24}
$$
Referring to Eq. (C.24), for any  $\mathbf{v}^{+}$  and corresponding  $\mathbf{v}_i^{-} \in \mathbf{V}^{-}$  we have
$$
p _ {\mathbf {v} ^ {+}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \geq p _ {\mathbf {v} ^ {+}, h} (\mathbf {v} \oplus \epsilon) - \Delta , \tag {C.25}
$$
$$
p _ {\mathbf {v} _ {i} ^ {-}, h} (\mathbf {v} \oplus \epsilon) + \Delta \geq p _ {\mathbf {v} _ {i} ^ {-}, h} (\mathbf {v} ^ {\prime} \oplus \epsilon).
$$
Thus, we can derive the following inequality based on Eq. (C.25):
$$
\begin{array}{l} p _ {\mathbf {v} ^ {+}, h} \left(\mathbf {v} ^ {\prime} \oplus \epsilon\right) \geq p _ {\mathbf {v} ^ {+}, h} (\mathbf {v} \oplus \epsilon) - \Delta \geq p _ {\mathbf {v} ^ {+}, h} (\mathbf {v} \oplus \epsilon) - \Delta \\ \geq \max  _ {\mathbf {v} _ {i} ^ {-} \in \mathbf {V}} \overline {{p _ {\mathbf {v} _ {i} ^ {-} , h}}} (\mathbf {v} \oplus \epsilon) + \Delta \geq \overline {{p _ {\mathbf {v} _ {i} ^ {-} , h}}} (\mathbf {v} \oplus \epsilon) + \Delta \geq p _ {\mathbf {v} _ {i} ^ {-}, h} (\mathbf {v} \oplus \epsilon) + \Delta \tag {C.26} \\ \geq p _ {\mathbf {v} _ {i} ^ {-}, h} (\mathbf {v} ^ {\prime} \oplus \epsilon), \\ \end{array}
$$
which can be restated as Eq. (C.14). This completes our proof.
C.3 Proof of Theorem 3
In order to transfer the certified robustness of GCL to downstream tasks, we first introduce two loss functions, namely, unsupervised loss and supervised loss. Subsequently, we introduce a lemma to establish the relationship between GCL and downstream tasks. Finally, a theorem is proposed to prove that the certified robustness of GCL is provably preserved in downstream tasks.
Unsupervised Loss Given an input sample  $\mathbf{v}$  with its positive sample  $\mathbf{v}^{+}$  and  $n$  negative samples  $\{\mathbf{v}_1^-, \dots, \mathbf{v}_n^-\}$ . Let  $h: \{0, 1\}^n \to \mathbb{R}^d$  be the GNN encoder based on GCL to obtain representations. The unsupervised loss for  $h$  at point  $\mathbf{v}$  is defined as:
$$
L _ {u n} (\mathbf {v}; h) := \sum_ {i = 1} ^ {n} \ell \left(h (\mathbf {v}) ^ {\top} \left(h \left(\mathbf {v} ^ {+}\right) - h \left(\mathbf {v} _ {i} ^ {-}\right)\right)\right), \tag {C.27}
$$