SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Non-uniform entity-selection probability  $p$ . In general we can also use a non-uniform  $p$ : Let  $p_i$  denote a node-specific (non-uniform) selection probability for node  $i$  that the adversary does not control. To get robustness certificates we have to make the worst-case assumption, i.e. nodes with lowest selection probability will be attacked first by the adversary. For the certificate this means we have to find the largest possible  $\Delta$  (see also Theorem 1). Specifically, we have to choose  $\Delta \triangleq 1 - \prod_{i \in \mathcal{I}} p_i$  where  $\mathcal{I}$  contains the indices of the  $r$  smallest probabilities  $p_i$ . Then we can proceed as in our paper to compute certificates for the smoothing distribution with non-uniform  $p$ . If we additionally know that certain nodes will be attacked before other nodes, we do not have to consider the first nodes with smallest  $p_i$  but rather the first nodes with smallest  $p_i$  under an attack order. This can be useful for example if one cluster of nodes is attacked before another, then we can increase the selection probability for nodes in the first cluster and obtain stronger certificates.
Node-specific smoothing distributions. If some nodes are exposed to different levels of attacks than others, we can obtain stronger certificates by making the lower-level smoothing distribution node-specific. This would allow us to add less noise for nodes that are less exposed to attacks, potentially leading to even better robustness-accuracy trade-offs. Note, however, that this increases threat model complexity since we will have to treat all nodes separately.
Input-dependent randomized smoothing. We derive certificates for a smoothing distribution that adds random noise independently on the matrix elements. This is the most studied class of smoothing distributions and allows us to integrate a whole suite of existing distributions. Although there are first approaches adding non-independent noise, the corresponding certificates are still in their infancy (Sukenik et al., 2022). Data-dependent smoothing is a research direction orthogonal to ours, and we consider the integration of such smoothing distributions as future work.
Finding optimal smoothing parameters. The randomized smoothing framework introduces parameters of the smoothing distribution as hyperparameters (see e.g. (Cohen et al., 2019)). In general, increasing the probabilities to add random noise increases the robustness guarantees but also decreases the accuracy. We introduce an additional parameter allowing to better control the robustness-accuracy trade-off. Specifically, larger  $p$  adds more noise and increases the robustness but also decreases accuracy. In our exhaustive experiments we randomly explore the entire space of possible parameter combinations. The reason for this is to demonstrate Pareto-optimality, i.e. to find all ranges for which we can offer both better robustness and accuracy. In practice one would have to conduct significantly less experiments to find suitable parameters. Note that the task of efficiently finding the optimal parameters is a research direction orthogonal to deriving robustness certificates.
Limitation of fixed threat models. Recent adversarial attacks (see e.g. (Kolovieh et al., 2023)) go beyond the notion of  $\ell_p$ -norm balls and assess adversarial robustness using generative methods. Our approach is limited as we can only certify robustness against bounded adversaries that perturb at most  $r$  objects with a perturbation strength that is bounded by  $\epsilon$  under some  $\ell_p$ -norm. Since current robustness certificates can only provide provable guarantees under fixed threat models, certifying robustness against adversaries that are not bounded by  $\ell_p$ -norms is out of the scope of this paper.
Graph-structure perturbations. While we implement certificates against node-attribute perturbations, our framework can in principle integrate certificates against graph-structure perturbations (for example by bounding the number of nodes whose outgoing edges can be perturbed). This can be implemented by first selecting rows of the adjacency matrix  $\mathbf{A}$  and then applying sparse smoothing (Bojchevski et al., 2020) to the selected rows only. Note that the works of Gao et al. (2020) and Wang et al. (2021) also study randomized smoothing against perturbations of the graph structure, but they consider a special case of sparse smoothing Bojchevski et al. (2020) with equal flipping probabilities.