SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Definition 4. In the class-only side-information game, the adversary samples  $y \sim P_y$ , then selects  $C \in \binom{[K]}{m}$ , then samples  $x \sim P_{x|y}$  and selects  $\tilde{x} \in N(x)$ . Let  $L_{co}^{*}(m, P, \mathcal{H}, N)$  be the minimum loss in this game.
In the example-dependent side-information game from Section 3.1, the adversary's choice of  $C$  can depend on both  $x$  and  $y$ . In the class-only variation it can only depend on  $y$ . For the class only game, we will focus on the  $m = 2$  case.
To make the connection to the binary games, we need to add one more restriction on the adversary's choice of side information: for all  $y, y'$ $\operatorname{Pr}[C = \{y, y'\} | Y = y] = \operatorname{Pr}[C = \{y, y'\} | Y = y']$ . This ensures that the classifier's posterior for  $Y$  given  $C$  is  $\operatorname{Pr}[Y = y|C] = \operatorname{Pr}[Y = y] / \operatorname{Pr}[Y \in C]$ .
Theorem 3. The optimal loss in the class-only side-information game is  $L_{co}^{*}(2,P,N,\mathcal{H}) = \max_{s}\sum_{i,j}Pr[Y = i]a_{i,j}s_{i,j}$  where  $a_{i,j} = L^{*}(P|(y\in \{i,j\}),\mathcal{H},N)$  and  $s\in \mathbb{R}^{[K]\times [K]}$  is a symmetric doubly stochastic matrix:  $s\geq 0$ ,  $s = s^T$ ,  $s1 = 1$ .
The proof in is Supplementary §A. The variable  $s$  represents the attacker's strategy for selecting the class side information. When the classes are equally likely, we have a maximum weight coupling problem: because the weights  $a$  are symmetric, the constraint that  $s$  be symmetric becomes irrelevant.
3.3 Relationships between games and bounds
The side information games provide a collection of lower bounds on  $L^{*}(P, N, \mathcal{H}_{\mathrm{soft}})$ . When  $m = 1$ , the side information game becomes trivial:  $C = \{y\}$  and the side information contains the answer to the classification problem. Thus  $L^{*}(1, \mathcal{H}_{\mathrm{soft}}) = L_{co}^{*}(1, \mathcal{H}_{\mathrm{soft}}) = 0$ . When  $m = K$ ,  $C = \mathcal{V}$  and both the example-dependent and class-only side information games are equivalent to the original game, so  $L^{*}(P, N, \mathcal{H}) = L^{*}(K, P, N, \mathcal{H}) = L_{co}^{*}(K, P, N, \mathcal{H})$ . For each variation of the side-information game, the game becomes more favorable for the adversary as  $m$  increases:  $L^{*}(m, P, n, \mathcal{H}) \leq L^{*}(m + 1, P, N, \mathcal{H})$  and  $L_{co}^{*}(m, P, N, \mathcal{H}) \leq L_{co}^{*}(m + 1, P, N, \mathcal{H})$ . For each  $m$ , it is more favorable for the adversary to see  $x$  before selecting  $C$ , i.e.  $L_{co}^{*}(m, P, N, \mathcal{H}) \leq L^{*}(m, P, N, \mathcal{H})$ .
3.4 Optimal Loss for Hard Classifiers
Since  $\mathcal{H}_{\mathrm{hard}} \subset \mathcal{H}_{\mathrm{soft}}$ ,  $L^{*}(m, P, N, \mathcal{H}_{\mathrm{soft}}) \leq L^{*}(P, N, \mathcal{H}_{\mathrm{hard}})$ . The optimal loss over hard classifiers is interesting both as a bound on the optimal loss over soft classifiers and as an independent quantity. Upper bounds on  $L^{*}(P, N, \mathcal{H}_{\mathrm{hard}})$  can be combined with lower bounds from §3.1 and §3.2 using small values of  $m$  to pin down  $L^{*}(m, P, N, \mathcal{H}_{\mathrm{soft}})$  and establish that larger choices of  $m$  would not provide much additional information.
A hard classifier  $h: \mathcal{X} \to \{0,1\}^{[K]}$  has 0, 1-valued correct classification probabilities. When we apply the classifier construction procedure from the proof of Theorem 1 using an integer-valued
$q$  vector, we obtain a hard classifier. Thus the possible correct classification probabilities for hard classifiers are  $\mathcal{P}_{\nu ,N,\mathcal{H}_{soft}}\cap \{0,1\}^{[K]}$ . These are exactly the indicator vectors for the independent sets in  $\mathcal{G}^{\le 2}$ : the vertices included in the independent set are classified correctly and the remainder are not. Formally, we can express hard classifier loss as:
$$
1 - L ^ {*} (P, N, \mathcal {H} _ {\text {h a r d}}) = \max  _ {S \subseteq \mathcal {V}: S \text {i n d e p e n d e n t i n} \mathcal {G} ^ {\leq 2}} P (S). \tag {6}
$$
Finding the maximum weight independent set is an NP hard problem, which makes it computationally inefficient to compute optimal hard classifier loss.
Two-class versus Multi-class hard classification There are a number of related but distinct polytopes associated with the vertices of a hypergraph (24). The distinctions between these concepts explain some key differences between two-class and multi-class adversarial classification. See Supplementary §B for full definitions of these polytopes.
When  $K = 2$ , the conflict hypergraph is a bipartite graph. For bipartite graphs, the fractional vertex packing polytope, which has a constraint  $\sum_{i\in e}q_i\leq 1$  for each edge  $e$ , coincides with the independent set polytope, which is the convex hull of the independent set indicators. Due to this, in the two class setting,  $\mathcal{P}_{\mathcal{V},N,\mathcal{H}_{soft}}$  is the convex hull of  $\mathcal{P}_{\mathcal{V},N,\mathcal{H}_{hard}}$ , hard classifiers achieve the optimal loss, and optimal hard classifiers can be found efficiently.
As seen in Theorem 1, for all  $K$  the fractional vertex packing polytope characterizes performance in soft classification problem. However, for  $K > 2$ , it becomes distinct from the independent set polytope. An independent set in a hypergraph is a subset of vertices that induces no hyperedges. In other words, in each hyperedge of size  $m$ , at most  $m - 1$  vertices can be included in any independent set. Because the edge set of the conflict hypergraph is downward-closed, only the size-two hyperedges provide binding constraints: the independent sets in  $\mathcal{G}$  are the same as the independent sets in  $\mathcal{G}^{\leq 2}$ . Thus the concept of a hypergraph independent set is not truly relevant for our application.
There is a third related polytope: the fractional independent set polytope of  $\mathcal{G}^{\leq 2}$ , which has a constraint  $\sum_{i\in S}q_i\leq 1$  for each clique  $S$  in  $\mathcal{G}^{\leq 2}$ . The fractional independent set polytope of  $\mathcal{G}^{\leq 2}$  is contained in the fractional vertex packing polytope of  $\mathcal{G}$ : every hyperedge in  $\mathcal{G}$  produces a clique in  $\mathcal{G}^{\leq 2}$  but not the reverse. This inclusion could be used to find an upper bound on optimal soft classification loss.
Furthermore, when  $K > 2$  the fractional vertex packing polytope of the conflict hypergraph, i.e.  $\mathcal{P}_{\mathcal{V},N,\mathcal{H}_{\text{soft}}}$ , can have non-integral extreme points and thus can be strictly larger than the independent set polytope. The first configuration in Figure 1 illustrates this. Thus the soft and hard classification problems involve optimization over different spaces of correct classification probabilities. Furthermore, maximum weight or even approximately maximum weight independent sets cannot be efficiently found in general graphs: the independent set polytope is not easy to optimize over.
In Section 3.5, we will use an efficiently computable lower bound on graph independence number.
3.5 Upper bounds on hard classifier loss via Caro-Wei bound on independent set probability
In §3.1 and 3.2, we discussed 2 ways of obtaining lower bounds for the loss of soft classifiers for the multi-class classification problem. In this section, we provide an upper bound on the loss of the optimal hard classifier (we note that this is also an upper bound for optimal loss for soft classifiers). In §3.4, we discussed the relationship between optimal loss achievable by hard classifiers and independent set size. We upper bound the optimal loss of hard classifiers by providing a lower bound on the probability of independent set in the conflict graph.
The following theorem is a generalization of the Caro-Wei theorem (1) and gives a lower bound on the weight of the maximum weight independent set.
Theorem 4. Let  $\mathcal{G}$  be a graph on  $\mathcal{V}$  with adjacency matrix  $A\in \{0,1\}^{\nu \times \nu}$  and let  $P$  be a probability distribution on  $\mathcal{V}$ . For any  $w\in \mathbb{R}^{\nu}$ ,  $w\geq 0$ , there is some independent set  $S\subseteq \mathcal{V}$  with
$$
P (S) \geq \sum_ {v \in \mathcal {V}: w _ {v} > 0} \frac {p _ {v} w _ {v}}{((A + I) w) _ {v}}.
$$
The proof is in Supplementary  $\S A$
(a) MNIST (1,4,7)
(b) CIFAR-10 (0,2,8)
Figure 2: Optimal error for MNIST and CIFAR-10 3-class problems  $(L^{*}(3))$ .  $L^{*}(2)$  is a lower bound computed using only constraints from edges.  $AT$  is the loss for an adversarially trained classifier under the strong APGD attack (9).
For comparison, the standard version of the Caro-Wei theorem is a simple lower bound on the independence number of a graph. It states that  $\mathcal{G}$  contains an independent set  $S$  with  $|S| \geq \sum_{v \in \mathcal{V}} 1 / (d_v + 1)$  where  $d_v$  is the degree of vertex  $v$ .
Note that if  $w$  is the indicator vector for an independent set  $S'$ , the bound becomes  $p^T w = P(S')$ . In general, the proof of Theorem 4 can be thought of as a randomized procedure for rounding an arbitrary vector into an independent set indicator vector. Vectors  $w$  that are nearly independent set indicators yield better bounds.
Theorem 4 provides a lower bound on the size of the maximum independent set in  $\mathcal{G}^{\leq 2}$  and thus an upper bound on  $L^{*}(P,n,\mathcal{H}_{hard})$ , which we call  $L_{CW} = 1 - P(S)$ .
In this section, we compute optimal losses in the presence of an  $\ell_2$ -constrained attacker (12) at various strengths  $\epsilon$  for benchmark computer vision datasets like MNIST and CIFAR-10 in a 3-class setting. We compare these optimal losses to those obtained by state-of-the-art adversarially trained classifiers, showing a large gap. We then compare the optimal loss in the 10-class setting to its upper and lower bounds ( $\S 3.3$ ), showing matching bounds at lower  $\epsilon$ . In the Supplementary,  $\S C$  describes hyperedge finding in practice,  $\S D$  details the experimental setup and  $\S E$  contains additional results.
4.1 Optimal loss for 3-class problems
Corollary 1 allows us to compute the optimal loss given any dataset (and its corresponding conflict hypergraph). We compute the optimal loss for 3-way classification, due to the computational complexity of finding higher order hyperedges (see §D.4 of the Supp.). In Figure 2, we plot the optimal loss  $L^{*}(3)$  computed via the LP in Corollary 1 against the loss obtained through TRADES (31) for 3-class MNIST (classes '1', '4', '7') and 3-class CIFAR-10 ('plane', 'bird' and 'ship' classes) with 1000 samples per class  $^5$ . For MNIST, we train a 3 layer CNN for 20 epochs with TRADES regularization strength  $\beta = 1$ , and for CIFAR-10, we train a WRN-28-10 model for 100 epochs with  $\beta = 6$ . We evaluate models using APGD-CE from AutoAttack (9).
From Figure 2, we observe that this gap is quite large even at lower values of  $\epsilon$ . This indicates that there is considerable progress to be made for current robust training methods, and this gap may be due to either the expressiveness of the hypothesis class or problems with optimization. We find that for CIFAR-10, TRADES is unable to achieve loss much better than 0.6 at an  $\epsilon$  for which the optimal loss is near 0. This gap is much larger than observed by prior work (5; 6) for binary classification, suggesting that current robust training techniques struggle more to fit training data with more classes. In §D.8 of the Supp., we ablate over larger architectures, finding only small improvements at lower values of  $\epsilon$  and none at higher.
We also used PGD adversarial training (19) but found its performance to be worse (See Appendix D.6).
(a) MNIST
(b) CIFAR-10
Figure 3: Lower bounds on the exact optimal 10-class loss using hyperedges up to degree 2 ( $L^{*}(2)$ ), 3 ( $L^{*}(3)$ ) and 4 ( $L^{*}(4)$ ), as well as maximum weight coupling of pairs of binary  $0 - 1$  loss lower bounds ( $L_{\mathrm{co}}^{*}(2)$ ).  $L_{CW}$  is an upper bound from the Caro-Wei approximation of the independent set number. The region in grey represents the range of values we would expect the true optimal loss  $L^{*}(10)$  to fall under.
4.2 Bounds on optimal loss for 10-class problems
As the number of classes and dataset size increases, the difficulty of solving the LP in Corollary 1 increases to the point of computational infeasibility (see §D.4 of the Supp.). We use the methods discussed in Section 3 to bound the optimal loss for 10-class problems on MNIST and CIFAR-10 datasets on the full training dataset. We present results for each approximation in Figure 3, with the limitation that for some methods, we are only able to obtain results for smaller values of  $\epsilon$  due to runtime blowup. We provide truncated hypergraph bounds for CIFAR-100 in §D.2 of the Supp.
Lower bounding the optimal loss using truncated hypergraphs (§3.1): In Figure 3, we plot the loss lower bound obtained via truncating the hypergraph to consider only edges  $(L^{*}(2))$ , up to degree 3 hyperedges  $(L^{*}(3))$ , and up to degree 4 hyperedges  $(L^{*}(4))$ . Computing the optimal loss would require computing degree 10 hyperedges. However, we find at small values of  $\epsilon$ , there is little difference in these bounds despite the presence of many higher degree hyperedges. This indicates that the use of use of higher degree hyperedges may not be critical to get a reasonable estimate of the optimal loss. For example, for CIFAR-10 at  $\epsilon = 3$ , we observe 3M degree 3 hyperedges and 10M degree 4 hyperedges, but these constraints have no impact on the computed lower bound. To understand the impact of hyperedges, we provide the count of hyperedges for each value of  $\epsilon$  and plots of the distribution of optimal classification probabilities per vertex in §D.3 of the Supp. From Figure 2, we find that the difference  $L^{*}(2)$  and  $L^{*}(3)$  does not occur until the loss reaches above 0.4 for the 3-class problem.
Takeaway: In practice, we do not lose information from computing lower bounds with only edges in the conflict hypergraph.
Lower bounding the optimal loss using the 1v1 binary classification problems (§3.2): We can use the algorithm from Bhagoji et al.(6) to efficiently compute 1v1 pairwise optimal losses to find a lower bound on the 10-class optimal loss  $(L_{CO}^{*}(2))$ . From Theorem 3, we use maximum weight coupling over these optimal losses to find a lower bound. Optimal loss heatmaps for each pair of classes in MNIST and CIFAR-10 are in §D.5 of the Supp. The efficiency of the 1v1 computation allows us to compute lower bounds at larger values of  $\epsilon$  in Figure 3.
Takeaway: From Figure 3, we find that while this lower bound is the most efficient to compute, the obtained bound is much looser compared to that from truncated hypergraphs. This arises from the weak attacker assumed while computing this bound.
Upper bounding optimal loss via Caro-Wei approximation (§3.5): In Figure 3, we also plot the upper bound on  $0 - 1$  loss for hard classifiers (denoted by  $L_{CW}$ ) obtained via applying Theorem 4 with vertex weights obtained from the solution to  $L^{*}(2)$ . When  $\epsilon$  becomes large ( $\epsilon \geq 3.0$  for MNIST and  $\epsilon \geq 4.5$  for CIFAR-10), the loss upper bound increases sharply. This indicates the lower bound on the independent set size becomes looser as the number of edges increases, due to the importance of higher-order interactions that are not captured by the approximation. At the small values of  $\epsilon$  used in practice however, the lower bounds obtained through truncated hypergraphs ( $L^{*}(2)$ ,  $L^{*}(3)$ , and  $L^{*}(4)$ ) are close to the value of this upper bound.
Takeaways: (i) We do not lose much information from not including all hyperedges at small  $\epsilon$  values as the upper and lower bounds are almost tight; (ii) At these small values of  $\epsilon$ , we do not expect much difference in performance between hard classifiers and soft classifiers.
Comparing loss of trained classifiers to optimal: In Figure 3, we also compare the loss obtained by a robustly trained classifier (AT) to our bounds on optimal loss. For both datasets, we see a large gap between the performance of adversarial training and our bounds (including the upper bound from the Caro-Wei approximation  $L_{CW}$ ), even at small values of  $\epsilon$ . This suggests that current robust training techniques are currently unable to optimally fit the training data in multiclass classification tasks of interest. In addition, we also checked the performance of state-of-the-art verifiably robust models on these two datasets from a leaderboard (18). For MNIST, the best certifiably robust model has a  $0 - 1$  loss of 0.27 at a budget of 1.52 and 0.44 at a budget of 2.0, while for CIFAR-10, the best certifiably robust model has a  $0 - 1$  loss of 0.6 at a budget of 1.0 and 0.8 at a budget of 2.0. These are much higher than the optimal lower bound that is achievable for these datasets which is 0 in all these cases.
Takeaway: Performance of state-of-the-art robust models, both empirical and verifiable, exhibit a large gap from the range of values predicted by our bounds on optimal  $0 - 1$  loss, even when  $\epsilon$  is small. Future research focusing on developing algorithms to decrease this gap while maintaining generalization capabilities may lead to improvements in model robustness.
5 Discussion and Related Work
Related Work: When the data distribution satisfies certain properties, Dohmatob (11) and Mahloujifar et al. (20) use the 'blowup' property to determine bounds on the robust loss, given some level of loss on benign data. We note that these papers use a different loss function that depends on the original classification output on benign data, thus their bounds are not comparable. Bhagoji et al. (5; 6), and Pydi et al. (22) provide lower bounds on robust loss when the set of classifiers under consideration is all measurable functions. These works only provide bounds in the binary classification setting. Work on verifying robustness (7; 28; 13; 18) provides bounds on the robustness of specific classifiers. Yang et al. (30) independently introduced the concept of a 'conflict graph' to obtain robust non-parametric classifiers via an adversarial pruning defense. The closest related work to ours is Trillos et al. (29), which uses optimal transport theory to find lower bounds on multi-class classification that are applicable for both continuous and discrete distributions. While their theoretical bounds are exact and more general than ours, accounting for distributional adversaries, their numerically computed bounds via the Sinkhorn algorithm are approximate and converge to the true value only as the entropy regularization decreases. In contrast, we provide methods to directly compute the optimal loss for discrete distributions, along with efficient methods for determining its range to overcome the computational bottlenecks encountered.
Discussion and Limitations: Our work in this paper firmly establishes for the multi-class case what was known only in the binary setting before: there exists a large gap in the performance of current robust classifiers and the optimal classifier. It also provides methods to bound the loss efficiently in practice, giving practitioners quick means to determine the gap. The question then arises: why does this gap arise and how can we improve training to decrease this gap? This paper, however, does not tackle the problem of actually closing this gap. Possible methods include increasing the architecture size (26), using additional data (15) and using soft-labels (6). A surprising finding from our experiments was that the addition of hyperedges to the multi-way conflict graph did not change the lower bounds much, indicating we are in a regime where multi-way intersections minimally impact optimal probabilities. One major limitation of our work is the computational expense at larger budgets, sample sizes and class sizes. We suspect this is due to the general-purpose nature of the solvers we use and future work should look into developing custom algorithms to speed up the determination of lower bounds.
ANB, WD and BYZ were supported in part by NSF grants CNS-2241303, CNS-1949650, and the DARPA GARD program. SD was supported in part by the National Science Foundation Graduate Research Fellowship under Grant No. DGE-2039656. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect
the views of the National Science Foundation. SD and PM were supported in part by the National Science Foundation under grant CNS-2131938, the ARL's Army Artificial Intelligence Innovation Institute (A2I2), Schmidt DataX award, and Princeton E-affiliates Award.
[1] N. Alon and J. H. Spencer. The probabilistic method. John Wiley & Sons, 2016.
[2] M. S. Andersen, J. Dahl, and L. Vandenberghe. Cvxopt: Python software for convex optimization, 2013.
[3] M. ApS. MOSEK Optimizer API for Python 10.0.22, 2019. URL https://docs.mosek.com/10.0/pythonapi/index.html.
[4] A. N. Bhagoji, W. He, B. Li, and D. Song. Exploring the space of black-box attacks on deep neural networks. arXiv preprint arXiv:1712.09491, 2017.
[5] A. N. Bhagoji, D. Cullina, and P. Mittal. Lower bounds on adversarial robustness from optimal transport. In Advances in Neural Information Processing Systems, pages 7496-7508, 2019.
[6] A. N. Bhagoji, D. Cullina, V. Sehwag, and P. Mittal. Lower bounds on cross-entropy loss in the presence of test-time adversaries. In Proceedings of the 38th International Conference on Machine Learning, 2021.
[7] R. Bunel, I. Turkaslan, P. H. Torr, P. Kohli, and M. P. Kumar. A unified view of piecewise linear neural network verification. arXiv preprint arXiv:1711.00455, 2017.
[8] N. Carlini and D. A. Wagner. Towards evaluating the robustness of neural networks. CoRR, abs/1608.04644, 2016. URL http://arxiv.org/abs/1608.04644.
[9] F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, pages 2206-2216. PMLR, 2020.
[10] D. Cullina, A. N. Bhagoji, and P. Mittal. Pac-learning in the presence of adversaries. In Advances in Neural Information Processing Systems, pages 230–241, 2018.
[11] E. Dohmatob. Generalized no free lunch theorem for adversarial robustness. In Proceedings of the 36th International Conference on Machine Learning, pages 1646-1654, 2019.
[12] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
[13] S. Gowal, K. Dvijotham, R. Stanforth, R. Bunel, C. Qin, J. Uesato, T. Mann, and P. Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint arXiv:1810.12715, 2018.
[14] S. Gowal, C. Qin, J. Uesato, T. Mann, and P. Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
[15] S. Gowal, S. Rebuffi, O. Wiles, F. Stimberg, D. A. Calian, and T. A. Mann. Improving robustness using generated data. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 4218-4233, 2021. URL https://proceedings.neurips.cc/paper/2021/bitical/21ca6d0cf2f25c4dbb35d8dc0b679c3f-Abscract.html.
[16] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. 2009.
[17] Y. LeCun and C. Cortes. The MNIST database of handwritten digits. 1998.
[18] L. Li, T. Xie, and B. Li. Sok: Certified robustness for deep neural networks. In 2023 IEEE Symposium on Security and Privacy (SP), pages 1289-1310. IEEE, 2023.