SlowGuess/NeurIPS_2023_OCR huggingface dataset:
on input space  $(\mathbb{X},\mathbb{D})$ . In the following, we show that this induced measure inherits its equivariances from the family of transformations  $(\psi_{\theta})_{\theta \in \Theta}$ .
Proposition 10. Consider a measurable space  $(\mathbb{X},\mathbb{D})$  and a parametric family of transformations  $(\psi_{\theta})_{\theta \in \Theta}$  with  $\psi_{\theta}:\mathbb{X}\to \mathbb{X}$  and measurable parameter space  $(\Theta ,\mathbb{G})$ . Let  $\gamma :\mathbb{G}\rightarrow \mathbb{R}_{+}$  be a measure on parameter space  $(\Theta ,\mathbb{G})$  and consider the family of induced smoothing measures  $(\mu_x)_{x\in \mathbb{X}}$ , as defined in Eq. (3). If all transformations  $\psi_{\theta}$  are equivariant to the actions of group  $\mathbb{G}$ , then the family of measures is equivariant.
Proof. Consider an arbitrary group element  $g \in \mathbb{G}$  with corresponding group action  $T_{\mathbb{X},g}$ . By definition of  $\mu_x$  and the equivariance of all  $\psi_\theta$ , we have for all  $x \in \mathbb{X}$  that
$$
\begin{array}{l} \mu_ {g \bullet x} (\mathbb {A}) = \gamma \left(\left\{\theta \in \Theta \mid f _ {\theta} (g \bullet x) \in \mathbb {A} \right\}\right) \\ = \gamma (\{\theta \in \Theta \mid g \bullet f _ {\theta} (x) \in \mathbb {A} \}) \\ = \gamma \left(\left\{\theta \in \Theta \mid f _ {\theta} (x) \in T _ {\mathbb {X}, g} ^ {- 1} (\mathbb {A}) \right\}\right) = \left(\mu_ {x} \circ T _ {\mathbb {X}, g} ^ {- 1}\right) (\mathbb {A}). \\ \end{array}
$$
The second-to-last equality holds because, by definition of the pre-image,  $T_{\mathbb{X},g}^{-1}(\mathbb{A}) = \{x \in \mathbb{X} \mid g \bullet x \in \mathbb{A}\}$  and thus  $g \bullet f_{\theta}(x) \in \mathbb{A} \iff f_{\theta}(x) \in T_{\mathbb{X},g}^{-1}(\mathbb{A})$ .
E.2.8 Sparsity-aware measures
Proposition 11. Consider the measurable space  $(\mathbb{X},\mathbb{D})$ , where  $\mathbb{X} = \{0,1\}^{N\times D}\times \{0,1\}^{N\times N}$  is the set of all binary attributed graphs with  $N$  nodes and  $D$  features and  $\mathbb{D} = \mathcal{P}(\mathbb{X})$  is its powerset. Let  $(\mu_{\mathbf{X},\mathbf{A}})_{(\mathbf{X},\mathbf{A})\in \mathbb{X}}$  be the family of sparsity-aware aware measures on  $(\mathbb{X},\mathbb{D})$ , as defined by the sparsity-aware probability mass function with fixed flip probabilities  $p_{+}^{\mathbf{X}},p_{-}^{\mathbf{X}},p_{+}^{\mathbf{A}},p_{-}^{\mathbf{A}}\in [0,1]$  (see Eq. (18)). If group  $\mathbb{G}$  acts on  $\mathbb{X}$  via graph isomorphisms, then the family of measures is equivariant.
Proof. Group  $\mathbb{G}$  acting on  $\mathbb{X}$  via graph isomorphisms means that for every  $(\mathbf{X},\mathbf{A})\in \mathbb{X}$  and  $g\in \mathbb{G}$  there is some permutation matrix  $P\in \{0,1\}^{N\times N}$  such that  $g\bullet (X,A) = (PX,PAP^T)$ . That is, the entries of the attribute and adjacency matrix are permuted. The sparsity-aware measures are product measures, i.e. sparsity-aware smoothing perturbs each element of the attribute and adjacency matrix independently. Families of product measures are equivariant with respect to the action of groups acting via permutation, see Proposition 8.
Table 1: Suitable measures for different input domains  $\mathbb{X}$ , input distances  $d_{\mathrm{in}}$ , and group actions.
X	d_in	Group action	Smoothing measure
RN×D	l1, l∞	Permutation, Translation	Uniform(-σ, σ)
RN×D	l1, l∞	Permutation, Translation	Laplace(0, σ)
RN×D	l2	Euclidean isometries	N(0, σ)
ZN×D	l0	Permutation	Discrete smoothing (see Lee et al. [80])
{0,1}N×D × {0,1}N×N (Attributed graphs)	Edit cost (see Eq. (2))	Graph isomorphism	Sparsity-aware smoothing (see Appendix E.7)
Assuming  $\forall \mathbb{A} \in \mathbb{D}: \{\theta \mid f_{\theta}(x) \in \mathbb{A}\} \in \mathbb{G}$ , i.e. all preimages are in the parameter  $\sigma$ -field  $\mathbb{G}$ .
Table 2: Suitable smoothing schemes for different output domains  $\mathbb{Y}$ , output distances  $d_{\mathrm{out}}$ , and group actions (see discussion in Section 5.1). (*)For expected value and median smoothing, robustness can be certified as long as the  $d_{\mathrm{out}}$  ball  $\{y' \mid d_{\mathrm{out}}(f(x), y') \leq \epsilon\}$  is contained within a hyperrectangle (see Appendix D.2).
Y	dout	Group action	Smoothing scheme
{1,...,K}N	l0	Permutation	Elementwise majority voting
RN	Any*	Permutation, Affine	Expected value
RN	Any*	Permutation, Elementwise linear	Elementwise median
Any	Any	Isometry w.r.t. dout (e.g. Euclidean isometries for l2)	Center
F Graph edit distance certificates
In the following, we show how the different existing approaches for proving the robustness of graph neural networks operating on  $\{0,1\}^{N\times D}\times \{0,1\}^{N\times N}$  w.r.t to distance  $d_{\mathrm{in}}$
$$
c _ {\boldsymbol {X}} ^ {+} \cdot | | (\boldsymbol {X} ^ {\prime} - \boldsymbol {X}) _ {+} | | _ {0} + c _ {\boldsymbol {X}} ^ {-} \cdot | | (\boldsymbol {X} ^ {\prime} - \boldsymbol {X}) _ {-} | | _ {0} + c _ {\boldsymbol {A}} ^ {+} \cdot | | (\boldsymbol {A} ^ {\prime} - \boldsymbol {A}) _ {+} | | _ {0} + c _ {\boldsymbol {A}} ^ {-} \cdot | | (\boldsymbol {A} ^ {\prime} - \boldsymbol {A}) _ {-} | | _ {0}, \tag {4}
$$
with costs  $c_{\pmb{X}}^{+}, c_{\pmb{X}}^{-}, c_{\pmb{A}}^{+}, c_{\pmb{A}}^{-} \in \{\infty, 1\}$  can be generalized to non-uniform costs, i.e.  $c_{\pmb{X}}^{+}, c_{\pmb{X}}^{-}, c_{\pmb{A}}^{+}, c_{\pmb{A}}^{-} \in \{\infty\} \cup \mathbb{R}_{+}$ . Combined with Proposition 2, this yields robustness guarantees w.r.t. to the graph edit distance, where the edit operations are insertion and deletion of edges and/or node attributes. Note that we also consider local budgets  $\rho_1, \ldots, \rho_N$ , which we introduce in Appendix G.
Before making the different generalizations, we discuss how to solve Knapsack problems with local constraints, which will be used in many of the subsequent derivations.
Please note that providing an in-depth explanation and motivation for each of the certification procedures would be out of scope. Instead, we strongly recommend first reading the original papers. Our discussions are to be understood as specifying changes that have to be made relative to the original certification procedures.
F.1 Knapsack problems with local constraints
A recurring problem in proving the robustness of graph neural networks is selecting a set of edges or attributes that have the largest effect on a classifier's prediction while complying with global and node-wise constraints on the cost of perturbations. This is a form of Knapsack problem.
Definition 2. A knapsack problem with local constraints is a binary integer program of the form
$$
\max  _ {\boldsymbol {Q} \in \{0, 1 \} ^ {N \times M}} \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} V _ {n, m} \cdot Q _ {n, m}
$$
$$
s. t. \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} \leq \epsilon , \quad \forall n: \sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} \leq \rho_ {n}, \tag {5}
$$
with value matrix  $\mathbf{V} \in \mathbb{R}^{N \times M}$ , cost matrix  $\mathbf{C} \in \mathbb{R}_{+}^{N \times M}$ , global budget  $\epsilon \in \mathbb{R}_{+}$  and local budgets  $\rho_{1}, \ldots, \rho_{N} \in \mathbb{R}_{+}$ .
Matrix  $Q$  indicates which entry of an attribute or adjacency matrix should be adversarially perturbed.
When there are only two distinct costs, i.e. we are only concerned with attribute or adjacency perturbations, the above problem can be solved exactly via dynamic programming. Alternatively, an upper bound can be found via linear relaxation, i.e. optimization over  $[0,1]^{N\times M}$ .
F.1.1 Dynamic programming
In the following, we assume that there is an index set  $\mathbb{I} \subseteq \{1, \ldots, N\} \times \{1, \ldots, M\}$  with complement  $\overline{\mathbb{I}}$  such that  $\forall (n, m) \in \mathbb{I}: C_{n, m} = c^{+}$  and  $\forall (n, m) \in \overline{\mathbb{I}}: C_{n, m} = c^{-}$ , i.e. we only have two distinct costs. In this case, the problem from Eq. (5) can be solved in a two-step procedure, which generalizes the certification procedure for uniform costs discussed in Section 3 of [15].
The first step is to find optimal solutions per row  $r$ , while ignoring the global budget  $\epsilon$ . We precompute  $N$  dictionaries  $\alpha_{1},\ldots ,\alpha_{N}:\mathbb{N}\times \mathbb{N}\to \mathbb{R}_{+}$  and  $N$  dictionaries  $\beta_{1},\ldots ,\beta_{N}:N\times N\to \mathcal{P}(\{1,\dots ,M\})$ , where  $\mathcal{P}$  is the powerset. Entry  $\alpha_{r}(i,j)$  is the optimal value of Eq. (5) if only row  $Q_{r}$  is non-zero, and exactly  $i$  indices in  $\mathbb{I}$  and  $j$  indices in  $\bar{\mathbb{I}}$  are non-zero, i.e.
$$
\alpha_ {r} (i, j) = \max  _ {\boldsymbol {Q} \in \{0, 1 \} ^ {N \times M}} \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} V _ {n, m} \cdot Q _ {n, m}
$$
$$
\text {s . t .} \sum_ {n, m \in \mathbb {I}} Q _ {n, m} = i, \quad \sum_ {n, m \in \mathbb {I}} Q _ {n, m} = j, \quad Q _ {- r} = 0.
$$
or sparse matrices
Entry  $\beta_r(i,j)$  is the corresponding set of non-zero indices of the optimal  $Q_{r}$ . Because all entries of value matrix  $\mathbf{V}$  and cost matrix  $\pmb{C}$  are non-negative, the optimal solution is to just select the indices with the largest value  $V_{n,m}$ . This pre-processing step is summarized in Algorithm 1. Its complexity is in  $\mathcal{O}(N\cdot ((\max_n\rho_n) / c^+) \cdot ((\max_n\rho_n) / c^-))$ , i.e. it scales linearly with the number of rows and the maximum number of non-zero values in a row of  $Q$ .
The second step is to combine these local optimal solutions while complying with the global budget  $\epsilon$  and local budgets  $\rho_1, \ldots, \rho_N$ . This can be achieved via dynamic programming. We create dictionaries  $s_1, \ldots, s_N$ , with  $s_r(\gamma)$  being the optimal value of Eq. (5) when only allocating budget to the first  $r$  rows and incurring an overall cost of  $\gamma$ :
$$
\begin{array}{l} s _ {r} (\gamma) = \max  _ {Q \in \{0, 1 \} ^ {N \times M}} \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} V _ {n, m} \cdot Q _ {n, m} \\ \text {s . t .} \sum_ {n, m \in \mathbb {I}} c ^ {+} \cdot Q _ {n, m} + \sum_ {n, m \in \overline {{\mathbb {I}}}} c ^ {-} \cdot Q _ {n, m} = \gamma , \quad Q _ {r:} = 0. \\ \end{array}
$$
The first dictionary,  $s_1$ , can be generated from the precomputed optimal values  $\alpha_1$ . After that, each dictionary  $s_r$  can be generated from  $s_{r-1}$ , while ensuring that we remain within local budget  $\rho_n$  and global budget  $\epsilon$ . The optimal value is given by  $\max_{\gamma} s_N(\gamma)$ . In addition, we generate dictionaries  $t_1, \ldots, t_N$ , which aggregate the row-wise optima stored in  $\beta_1, \ldots, \beta_N$  such that  $t_N(\gamma^*)$  with  $\gamma^* = \arg \max_{\gamma} s_N(\gamma)$  gives us the non-zero entries of the optimal  $Q$ . This procedure is summarized in Algorithm 2. Its complexity is in  $\mathcal{O}(N \cdot (\epsilon / c^+) \cdot (\epsilon / c^-) \cdot ((\max_n \rho_n) / c^+) \cdot ((\max_n \rho_n) / c^-))$ , i.e., it scales linearly with the number of rows, the maximum number of non-zero values in  $Q$ , and the maximum number of non-zero values in a row of  $Q$ . Note that graph neural networks are generally brittle to small adversarial perturbations, so the algorithm only needs to be executed for small  $\epsilon$ .
Algorithm 1 Precomputation of local solutions
for  $n = 1,\ldots ,N$  do  $\alpha_{n},\beta_{n}\gets \mathrm{dict}(),\mathrm{dict})$  best_idx  $\leftarrow$  argsort_desc(Vn) best_add_idx  $\leftarrow$  best_idx\II best_del_idx  $\leftarrow$  best_idx\Ii max Adds  $\leftarrow \lfloor \rho_n / c^+\rfloor$    
for  $i = 0,\dots ,$  max Adds do max_dels  $\leftarrow \lfloor (\rho_n - c^+ \cdot i) / c^- \rfloor$    
for  $j = 0,\dots ,$  max_dels do  $\alpha_{n}(i,j)\gets \mathrm{sum}(\{\mathbf{V}_{n,k}\mid k\in \mathrm{best\_add\_idx}[:i]\})$ $\alpha_{n}(i,j)\gets \alpha_{n}(i,j) + \mathrm{sum}(\{\mathbf{V}_{n,k}\mid k\in \mathrm{best\_del\_idx}[:j]\})$ $\beta_{n}(i,j)\gets \mathrm{set}(\mathrm{best\_add\_idx}[:i])\cup \mathrm{set}(\mathrm{best\_del\_idx}[:j])$    
return  $\alpha_{1},\ldots ,\alpha_{N},\beta_{1},\ldots ,\beta_{N}$
Algorithm 2 Construction of the global solution from local solutions
$s_0,t_0\gets \mathrm{dict()}$  ,dict()   
 $s_0(0)\gets 0$ $t_0(0)\gets []$    
for  $n = 1,\ldots ,N$  do   
 $s_n,t_n\gets \mathrm{dict()}$  ,dict()   
for prev_cost  $\in$  keys  $(s_{n - 1})$  do   
 $b\gets \min (\rho_n,\epsilon -\mathrm{prev\_cost})$    
max Adds  $\leftarrow \lfloor b / c^{+}\right]$    
for  $i = 0,\dots ,$  max Adds do   
max_dels  $\leftarrow \lfloor (b - c^{+}\cdot i) / c^{-}\rfloor$    
for  $j = 0,\dots ,$  max_dels do   
 $\gamma \gets \mathrm{prev\_cost} + c^{+}\cdot i + c^{-}\cdot j$ $v\gets s_{n - 1}(\mathrm{prev\_cost}) + \alpha_n(i,j)$    
if  $\gamma \notin \mathrm{keys}(s_n)$  or  $v > s_n(\gamma)$  then   
 $s_n(\gamma)\gets v$ $t_n(\gamma)\gets$  concatenate  $(t_{n - 1}(\mathrm{prev\_cost}),\beta (i,j))$ $\gamma^{*}\gets \operatorname {argmax}_{\gamma \in \mathrm{keys}(s_N)}s_N(\gamma)$    
return  $s_N(\gamma^*)$ $t_N(\gamma^*)$
F.1.2 Linear relaxation
The above dynamic programming approach can be generalized to arbitrary cost matrices  $\mathbf{C} \in \mathbb{R}_{+}^{N \times M}$ . However, it may not scale because it requires iterating over all possible combinations of costs admitted by global budget  $\epsilon$  and local budgets  $\rho_{1}, \ldots, \rho_{N}$ . A more efficient alternative is upper-bounding the optimal value of the knapsack problem in Eq. (5) by relaxing the binary matrix  $\mathbf{Q} \in \{0, 1\}^{N \times M}$  to real values  $\mathbf{Q} \in [0, 1]^{N \times M}$ :
$$
\max  _ {Q \in \mathbb {R} _ {N \times M}} \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} V _ {n, m} \cdot Q _ {n, m}
$$
$$
\text {s . t .} \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} \leq \epsilon , \quad \forall n: \sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} \leq \rho_ {n}, \tag {6}
$$
$$
\forall n, m: 0 \leq Q _ {n, m}, \quad \forall n, m: Q _ {n, m} \leq 1.
$$
Intuitively, it is best to set those  $Q_{n,m}$  to 1 that have the largest value-to-cost ratio, i.e. the largest  $V_{n,m} / C_{n,m}$ . To comply with the local and global budgets, one should first greedily select the  $(n,m)$  with the largest ratio in each row  $n$ , until the local budgets  $\rho_n$  are exhausted. One should then aggregate all these row-wise optimal indices and select those with the largest ratio, until the global budget  $\epsilon$  is exhausted. This procedure is summarized in Algorithm 3.
Algorithm 3 Optimal value of the linearly relaxed knapsack problem with local constraints (Eq. (6))
$Q^{*}\gets 0$  Initialize global allocations with  $N\times M$  zeros.   
 $\pmb {L}\leftarrow \mathbf{0}$  Initialize row-wise allocations with  $N\times M$  zeros.   
 $\pmb {R}\leftarrow \pmb {V}\oslash \pmb{C}$  Elementwise division. Division by 0 is defined as  $\infty$    
for  $n = 1,\dots ,N$  do best_cols  $\leftarrow$  argsort_desc( $R_{n}$ )  $b\gets \rho_{n}$  for  $m\in$  best_cols do  $L_{n,m}\gets b / C_{n,m}$ $L_{n,m}\gets \min (1,L_{n,m})$ $b\gets b - C_{n,m}\cdot L_{n,m}$  best_idx  $\leftarrow$  argsort_desc(R)   
 $b\gets \epsilon$  for  $(n,m)\in$  best_idx do  $Q_{n,m}^{*}\gets b / C_{n,m}$ $Q_{n,m}^{*}\gets \min (L_{n,m},Q_{n,m}^{*})$ $b\gets b - C_{n,m}\cdot Q_{n,m}^{*}$  return  $\sum_{n,m}V_{n,m}\cdot Q_{n,m}^{*}$
Proposition 12. Algorithm 3 yields the optimal value of the linearly relaxed knapsack problem with local constraints, as defined in Eq. (6).
Proof. To simplify the proof, we make certain assumptions about budgets  $\epsilon$  and  $\rho_{1},\ldots ,\rho_{N}$ . If  $\epsilon = 0$ , then the value of  $Q$  computed by Algorithm 3 is obviously correct, because the only feasible solution that does not violate the global budget is  $Q^{*} = 0$ . If  $\rho_{n} = 0$  for some  $n$ , then the value of  $Q_{n}$  computed by Algorithm 3 is obviously correct, because the only feasible solution that does not violate the local budget is  $Q_{n}^{*} = 0$ . We thus assume w.l.o.g. that  $\epsilon >0$  and  $\forall n:\rho_n > 0$ . Furthermore, we assume that the budgets do not exceed the maximum overall and row-wise cost, i.e.  $\epsilon \leq \sum_{n = 1}^{N}\sum_{m = 1}^{M}C_{n,m}$  and  $\forall n:\sum_{m = 1}^{M}C_{n,m}\leq \rho_{n}$ , since any excess budget will not have any effect on the optimal value of Eq. (6).
In the following, we prove the correctness of Algorithm 3 by verifying that the constructed solution fulfills the Karush-Kuhn-Tucket conditions. We define the Lagrangian
$$
\begin{array}{l} L (\boldsymbol {Q}, \lambda , \boldsymbol {\kappa}, \boldsymbol {T}, \boldsymbol {U}) = \left(- \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} V _ {n, m} \cdot Q _ {n, m} + \lambda \left(\sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} - \epsilon\right) \right. \\ + \sum_ {n = 1} ^ {N} \kappa_ {n} \left(\sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} - \rho_ {n}\right) - \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} T _ {n, m} \cdot Q _ {n, m} + \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} U _ {n, m} \cdot (Q _ {n, m} - 1) \Bigg). \\ \end{array}
$$
We introduce a negative sign in the objective, because Eq. (6) is a maximization problem. Variable  $\lambda \in \mathbb{R}_{+}$  corresponds to the global budget constraint, variable  $\kappa \in \mathbb{R}_{+}^{N}$  correspond to the local budget constraints and variables  $T, U \in \mathbb{R}_{+}^{N \times M}$  correspond to the constraint that  $Q$  should be in  $[0,1]^{N \times M}$ .
We claim that  $Q, \lambda, \kappa, T$  and  $U$  have the following optimal values:
- The optimal value of  $Q$  is the  $Q^{*}$  computed by Algorithm 3.
- The optimal value of  $\lambda$  is  $\lambda^{*} = \min_{n,m} V_{n,m} / C_{n,m}$  s.t.  $Q_{n,m}^{*} > 0$ , with  $Q^{*}$  computed as in Algorithm 3. That is,  $\lambda^{*}$  is the smallest value-to-cost ratio for which some global budget is allocated.
- Define  $o_n = \min_m V_{n,m} / C_{n,m}$  s.t.  $L_{n,m} > 0$  with  $L$  computed as in Algorithm 3. That is,  $o_n$  is the smallest value-to-cost ratio in row  $n$  for which some local budget is allocated. The optimal value of  $\kappa_n$  is  $\kappa_n^* = \max(0, o_n - \lambda^*)$ .
The optimal value of  $U_{n,m}$  is  $U_{n,m}^{*} = \max (V_{n,m} - C_{n,m}\cdot (\lambda^{*} + \kappa_{n}^{*}),0)$
- The optimal value of  $T_{n,m}$  is  $T_{n,m}^{*} = -V_{n,m} + C_{n,m} \cdot (\lambda^{*} + \kappa_{n}^{*}) + U_{n,m}$ , which is equivalent to  $\max(0, -V_{n,m} + C_{n,m} \cdot (\lambda^{*} + \kappa_{n}^{*}))$ .
Stationarity. By the definition of  $T^{*}$ , we trivially have
$$
\nabla_ {Q _ {n, m}} L \left(\boldsymbol {Q} ^ {*}, \lambda^ {*}, \boldsymbol {\kappa} ^ {*}, \boldsymbol {T} ^ {*}, \boldsymbol {U} ^ {*}\right) = - V _ {n, m} + C _ {n, m} \cdot \left(\lambda^ {*} + \kappa_ {n} ^ {*}\right) - T _ {n, m} ^ {*} + U _ {n, m} ^ {*} = 0.
$$
Primal feasibility. The clipping in Algorithm 3 ensures that  $\forall n, m: 0 \leq Q_{n,m}^{*} \leq 1$  and that global budget  $\epsilon$  and local budgets  $\rho_1, \ldots, \rho_N$  are never exceeded.
Dual feasibility. Because all values  $V$  and all costs  $C$  are non-negative, the optimal values  $\lambda^*, \kappa^*$  are non-negative. The optimal values  $T^*$  and  $U^*$  are non-negative by definition.
Complementary slackness. We first verify complementary slackness for variable  $\lambda$ . Because we assume that the global budget  $\epsilon$  does not exceed the maximum overall cost  $\sum_{n=1}^{N} \sum_{m=1}^{M} C_{n,m}$  and all values  $V$  are non-negative, Algorithm 3 will always allocate the entire global budget, i.e.
$$
\lambda^ {*} \left(\sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} ^ {*} - \epsilon\right) = \lambda^ {*} \cdot 0 = 0.
$$
Next, we verify complementary slackness for variable  $\kappa$ . We can make a case distinction, based on the value of  $\kappa_{n}^{*} = \max(0, o_{n} - \lambda^{*})$ . If  $o_{n} \leq \lambda^{*}$ , we have
$$
\kappa_ {n} \cdot \left(\sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} ^ {*} - \rho_ {n}\right) = 0 \cdot \left(\sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} ^ {*} - \rho_ {n}\right) = 0.
$$
If  $o_n > \lambda^*$ , then – by definition – the smallest value-to-cost ratio for which budget is allocated in row  $n$  is larger than the smallest value-to-cost ratio for which budget is allocated overall. Combined with our assumption that the local budget  $\rho_n$  does not exceed the overall cost  $\sum_{m=1}^{M} C_{n,m}$  in row  $n$ , this means that the entire local budget  $\rho_n$  is used up, i.e.
$$
\kappa_ {n} ^ {*} \cdot \left(\sum_ {m = 1} ^ {M} C _ {n, m} \cdot Q _ {n, m} ^ {*} - \rho_ {n}\right) = \kappa_ {n} ^ {*} \cdot 0 = 0.
$$
Next, we verify complementary slackness for variable  $\pmb{T}$ . If  $Q_{n,m}^{*} = 0$ , we obviously have  $T_{n,m}^{*} - (-Q_{n,m}^{*}) = 0$ . For  $Q_{n,m}^{*} > 0$ , recall that  $T_{n,m}^{*} = \max(0, -V_{n,m} + C_{n,m} \cdot (\lambda^{*} + \kappa_{n}^{*}))$ . Because we defined  $\kappa_{n}^{*} = \max(0, o_{n} - \lambda^{*})$ , we have  $\lambda^{*} + \kappa_{n}^{*} = \max(\lambda^{*}, o_{n})$ . We can thus verify  $V_{n,m} \geq C_{n,m} \cdot \lambda^{*}$  and  $V_{n,m} \geq C_{n,m} \cdot o_{n}$  to show that  $\max(0, -V_{n,m} + C_{n,m} \cdot (\lambda^{*} + \kappa_{n}^{*})) = 0$ .
We defined  $\lambda^{*}$  to be the smallest value-to-cost ratio for which  $Q_{n,m}^{*} > 0$ . We thus know that  $V_{n,m} \geq C_{n,m} \cdot \lambda^{*}$  whenever  $Q_{n,m}^{*} > 0$ .
We defined  $o_n$  to be the largest value-to-cost ratio for which  $L_{n,m} > 0$ . Due to the clipping in Algorithm 3, we know that  $L_{n,m} \geq Q_{n,m}^*$ . We thus know that, if  $Q_{n,m}^* > 0$ , then  $L_{n,m} > 0$  and thus  $V_{n,m} \geq C_{n,m} \cdot o_n$ .
Combining these two results confirms that  $T_{n,m}^{*} = 0$  if  $Q_{n,m}^{*} > 0$  and thus  $T_{n,m}^{*} \cdot (-Q_{n,m}^{*}) = 0$ .
Finally, we verify complementary slackness for variable  $U$  using a similar argument. If  $Q_{n,m}^{*} = 1$ , we trivially have  $U_{n,m}^{*} \cdot (Q_{n,m}^{*} - 1) = 0$ . For  $Q_{n,m}^{*} < 1$ , recall that  $U_{n,m}^{*} = \max(V_{n,m} - C_{n,m} \cdot (\lambda + \kappa_n), 0)$ . We can again use that  $\lambda^{*} + \kappa_{n}^{*} = \max(\lambda^{*}, o_{n})$ . There are two only two potential causes for  $Q_{n,m}^{*} < 1$ . The first one is that all of the local budget  $\rho_n$  was used up in column  $m$  of row  $n$  or columns with higher value-to-cost ratio. In this case, we have  $V_{n,m} \leq C_{n,m} \cdot o_n$ . The other one is that all of the global budget  $\epsilon$  was used up in entry  $(n, m)$  or entries with higher value-to-cost ratio. In this case, we have  $V_{n,m} \leq C_{n,m} \cdot \lambda^{*}$ . Overall, this implies that  $U_{n,m}^{*} \cdot (Q_{n,m}^{*} - 1) = 0 \cdot (Q_{n,m}^{*} - 1) = 0$ .
F.2 Interval bound propagation for attribute perturbations
Liu et al. [12] propose to prove the robustness of  $L$ -layer graph convolutional networks [98] to attribute perturbations with uniform cost ( $c_{X}^{+} = c_{X}^{-} = 1, c_{A}^{+} = c_{A}^{-} = \infty$ ) via interval bound
propagation. In the following, we generalize their guarantees to arbitrary attribute perturbation costs  $c_{\mathbf{X}}^{+}, c_{\mathbf{X}}^{-} \in \mathbb{R}_{+}$ .
Graph convolutional networks applied to attributes  $\mathbf{X} \in \{0,1\}^{N \times D}$  and adjacency  $\mathbf{A} \in \{0,1\}^{N \times N}$  are defined as
$$
\boldsymbol {H} ^ {(0)} = \boldsymbol {X}
$$
$$
\boldsymbol {Z} ^ {(l)} = \tilde {\boldsymbol {A}} \boldsymbol {H} ^ {(l)} \boldsymbol {W} ^ {(l)} + \mathbf {1} (\boldsymbol {b} ^ {(l)}) ^ {T} \quad \text {f o r} l = 0, \dots , L - 1 \tag {7}
$$
$$
\boldsymbol {H} ^ {(l)} = \sigma^ {(l)} (\boldsymbol {Z} ^ {(l - 1)}) \quad \text {f o r} l = 1, \dots , L,
$$
where  $\pmb{W}^{(0)},\dots,\pmb{W}^{(L-1)}$  are weight matrices,  $\pmb{b}^{(0)},\dots,\pmb{b}^{(L-1)}$  are bias vectors,  $\sigma^{(1)},\dots,\sigma^{(L)}$  are activation functions (here assumed to be ReLU in the first  $L-1$  layers and softmax in the last layer), and  $\tilde{\pmb{A}}$  is the adjacency matrix after additional preprocessing steps, such as degree normalization.