SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jingyuan Xu Weiwei Liu*
{jingyuanxu777, liuweiwei863}@gmail.com
This paper considers the following question: Given the number of classes  $m$ , the number of robust accuracy queries  $k$ , and the number of test examples in the dataset  $n$ , how much can adaptive algorithms robustly overfit the test dataset? We solve this problem by equivalently giving near-matching upper and lower bounds of the robust overfitting bias in multiclass classification problems.
Learning models that are robust to adversarial perturbations has garnered significant research attention in recent years. However, despite this progress, a pervasive issue continues to plague these models, namely robust overfitting [1]. A common approach to overcome overfitting is to divide the dataset into a training set, and a holdout (or test) set. Nonetheless, modern machine learning is adaptive in its nature. Prior information about a model's performance on the test set inevitably influences future modeling choices in extensive experiments and competitions. Recent studies have shown that excessive reuse of the holdout dataset can also leads to overfitting in non-robust setting [2, 3], and a body of subsequent work has quantitatively explored this phenomenon in the framework of perfect test label reconstruction [4, 5, 6]. Accordingly, in this paper we attempt to address the following questions: Can adaptive behavior result in overfitting in an adversarial setting? If so, by how much can adaptive algorithms robustly overfit the test dataset?
To solve these questions, we generalize the framework of perfect reconstruction to the adversarial setting, and analyze the average case performance that can be achieved by an adaptive algorithm, denoted as  $h_{\mathcal{U}}(k,n,m)$ , where  $k,n,m$  represents the number of robust accuracy queries, test samples and classes, respectively. This term equivalently measures the maximum level of robust overfitting in a multiclass classification problem. In this paper, we derive both upper and lower bounds of  $h_{\mathcal{U}}(k,n,m)$ , and demonstrate that our upper bounds and lower bounds are matching within logarithmic factors when  $n$  and the distribution of test dataset features  $\mathcal{D}_{\mathcal{X}}$  are fixed.
Perfect test label reconstruction. The question of perfect test label reconstruction in non-robust setting dates back to decades ago [7, 8, 9]. The developments on studying biasing results due to adaptive reuse of the test data start with the work of [4, 10], which broadly fall in the field of adaptive data analysis [11, 12]. [5] first pose the problem of characterizing the overfitting bias as a function of  $k, n, m$ , but they fail to give upper and lower bounds on the same order of  $m$ , which is left as an open
question [13]. [6] close this question and determine the amount of overfitting possible in multiclass classification. The theory on quantitative overfitting analysis in adversarial setting remains blank.
Adversarial robustness. It has been shown that deep neural networks are fragile to imperceptible distortions in the input space [14]. Perhaps the most common method to improve adversarial robustness is adversarial training [15, 16]. Theoretically, [17, 18, 19, 20] study the PAC learnability of adversarial robust learning problem, and [21, 22] study adversarial robustness under self-supervised learning. [23, 24] investigates the adversarial robustness from the perspective of ordinary differential equations. Besides, [25] analyze the trade-off between robustness and fairness, [26] study the worst-class adversarial robustness in adversarial training.
2 Summary of our results
In the remainder of this article,  $\mathbb{R},\mathbb{N}$  and  $\mathbb{R}^d$  represent the sets of real numbers, natural numbers and  $d$ -dimensional vectors over  $\mathbb{R}$ , respectively. We denote the set  $\{1,\dots ,n\}$  (for  $n\in \mathbb{N}$ ) by  $[n]$ . If  $A$  and  $B$  are sets, we use  $B^{A}$  to denote the collection of all mappings from  $A$  to  $B$  and  $2^{A}$  to denote the power set of  $A$ , that is the collection of all subsets of  $A$ . We denote the indicator function by  $\mathbb{1}\{\mathrm{event}\}$ , that is 1 if an event happens and 0 otherwise. If  $\mathcal{D}$  is a distribution, we use  $\operatorname {supp}(\mathcal{D})$  to denote the support of  $\mathcal{D}$ , which is defined by the closure of the set of possible values of a random variable having  $\mathcal{D}$  as its distribution. Besides,  $\| \cdot \| _p$  represents the  $\ell_p$  norm and  $d_{p}(\cdot ,\cdot)$  represents the distance function induced by  $\ell_p$  norm. Finally, we use big tilde notations  $\tilde{O},\tilde{\Omega}$  and  $\tilde{\Theta}$  as variants of big O notations that ignores logarithmic factors.
2.1 Problem formulation
Let  $\mathcal{X} = \mathbb{R}^d$  be the instance space and  $\mathcal{Y} = \{1, \dots, m\}$  be the label space. Let  $S = \{(x_i, c_i)\}_{i=1}^n$  denote the test set, whose features, denoted as  $\mathcal{X}_S = \{x_1, \dots, x_n\}$ , are independent and identically distributed (i.i.d.) according to some distribution  $\mathcal{D}_{\mathcal{X}}$  on  $\mathcal{X}^2$ . For simplification, we use  $\bar{c} = (c_1, \dots, c_n)$  to describe the vector of test set labels. Let  $f: \mathcal{X} \to \mathcal{Y}$  be a function, its robust accuracy on the test set with respect to (w.r.t.) a small perturbation  $\mathcal{U}: \mathcal{X} \to 2^{\mathcal{X}}$  is defined by
$$
\operatorname {A c c} _ {\mathcal {U}} (f; S) \triangleq \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbb {1} \left\{\forall x ^ {\prime} \in \mathcal {U} \left(x _ {i}\right), f \left(x ^ {\prime}\right) = c _ {i} \right\}.
$$
The perturbation  $\mathcal{U}(x)$  is required to be nonempty, so some choice of  $x^{\prime}$  is always available. This paper focuses the case when the perturbation is the  $p$ -norm ball with a small radius  $r$ , i.e.  $\mathcal{U}(x) = \{z \in \mathcal{X} : \| z - x \|_p \leq r\}$  for some  $p \geq 1$ .  $r = 0$  gives the identity perturbation:  $\mathcal{I}(x) \triangleq \{x\}$ . Note that in this case, the definition of robust accuracy is reduced to standard (non-robust) accuracy.
In this work, we mainly study the robust overfitting attack algorithms, which do not have access to the test set  $S$ . Instead, they have query access to robust accuracy of the model on  $S$ , that is, for any classifier  $f$ , the algorithm is able to obtain the value  $\mathrm{Acc}_{\mathcal{U}}(f;S)$ . We refer to this access as a query. A  $k$ -query algorithm  $\mathcal{A}$  makes  $k$  queries  $f_{1},\ldots ,f_{k}$  on  $S$ , and based on the values of  $\mathcal{X}_S$  and  $\mathrm{Acc}_{\mathcal{U}}(f_1;S),\dots,\mathrm{Acc}_{\mathcal{U}}(f_k;S)$ ,  $\mathcal{A}$  outputs a classifier  $\hat{f} = \mathcal{A}(S)$ . We say a  $k$ -query algorithm  $\mathcal{A}$  is based on hypothesis class  $\mathcal{H}\subset \mathcal{V}^{\mathcal{X}}$  if  $f_{1},\ldots ,f_{k}\in \mathcal{H}$ . The performance of the algorithm on  $S$  is measured by
$$
h _ {\mathcal {U}} (\mathcal {A}; S) \triangleq \mathbb {E} \left[ \operatorname {A c c} _ {\mathcal {U}} (\hat {f}; S) \right],
$$
where the expectation is over the algorithm's randomization. It is also of interest to ask the performance of an algorithm when  $\bar{c}$  are drawn according to some distribution  $\mathcal{D}_{\mathcal{Y}}$  over  $[m]^n$ . Let  $\mathcal{D} = \mathcal{D}_{\mathcal{X}} \times \mathcal{D}_{\mathcal{Y}}$ , for an algorithm  $\mathcal{A}$ , define its performance w.r.t.  $\mathcal{D}$  by
$$
h _ {\mathcal {U}} (\mathcal {A}; \mathcal {D}) \triangleq \underset {S \sim \mathcal {D}} {\mathbb {E}} h _ {\mathcal {U}} (\mathcal {A}; S).
$$
And we evaluate the algorithms under the assumption that they do not have any prior knowledge about the test labels. That is, the prior distribution of test labels is uniform over all possible labeling:
$$
h _ {\mathcal {U}} (\mathcal {A}) \triangleq \underset {S \sim \mathcal {X} _ {S} ^ {n} \times \mu_ {m} ^ {n}} {\mathbb {E}} h _ {\mathcal {U}} (\mathcal {A}; S),
$$
Formally, there is a sigma algebra  $\mathcal{F} \subset 2^{\mathcal{X}}$  of events and  $\mathcal{D}_{\mathcal{X}}$  is a probability measure on  $(\mathcal{X}, \mathcal{F})$
where  $\mu_m^n$  denotes the uniform distribution over  $[m]^n$ . Since the best robust accuracy is  $1/m$  without making any queries  $(k = 0)$ ,  $h_{\mathcal{U}}(\mathcal{A};S) - 1/m$  measures how much  $\mathcal{A}$  robustly overfits  $S$ . The goal of this paper is to find the largest robust overfitting possible for a multiclass classification problem. So we define the performance that is achievable by an algorithm after making  $k$  queries on any  $S$  by
$$
h _ {\mathcal {U}} (k, n, m) \triangleq \max _ {\mathcal {A}} h _ {\mathcal {U}} (\mathcal {A}),
$$
and we are interested in the value of  $h_{\mathcal{U}}(k,n,m) - 1 / m$ , or  $h_{\mathcal{U}}(k,n,m)$  equivalently. In the rest of this paper, we aim at deriving bounds of  $h_{\mathcal{U}}(k,n,m)$  for given  $k,n$  and  $m$ .
2.2 Our main results
Our bounds on  $h_{\mathcal{U}}(k,n,m)$  have two different regimes, which can be summarized as the following theorem.
Theorem 1 (Informal). let  $\mathcal{D}_{\mathcal{X}}$  be the distribution of test sample features. For  $k = \tilde{O}(n / m)$
$$
\Phi_ {\mathcal {D} _ {\mathcal {X}}} (n) \cdot \left[ \frac {1}{m} + \Omega \left(\sqrt {\frac {k}{m n}}\right) \right] \leq h _ {\mathcal {U}} (k, n, m) \leq \frac {1}{m} + \tilde {O} \left(\sqrt {\frac {k}{n m}}\right).
$$
For  $k = \tilde{\Omega} (n / m)$
$$
\Phi_ {\mathcal {D} _ {\mathcal {X}}} (n) \cdot \left[ \frac {1}{m} + \tilde {\Omega} \left(\frac {k}{n}\right) \right] \leq h _ {\mathcal {U}} (k, n, m) \leq \frac {1}{m} + \tilde {O} \left(\frac {k}{n}\right),
$$
where  $\Phi_{\mathcal{D}_{\mathcal{X}}}(n) \leq 1$  and is monotonically decreasing w.r.t.  $n$ .
When  $\mathcal{U} = \mathcal{I}$ , our upper bounds match the best known upper bounds [5], while our lower bounds differ from the known optimal lower bounds [6] by a factor  $\Phi_{\mathcal{D}_X}(n)$ . Since overfitting in the context of robust learning has some requirements on the test samples (e.g. the well-separated property for different classes [27]), we may not be able to ensure a non-trivial lower bound on  $h_{\mathcal{U}}(k,m,n)$  for some  $\mathcal{D}_{\mathcal{X}}$ . Intuitively,  $\Phi_{\mathcal{D}_X}(n)$  measures how easily to sample  $n$  'good' (for robust overfitting) test data features from  $\mathcal{D}_{\mathcal{X}}$ . The specific form of  $\Phi_{\mathcal{D}_X}(n)$  is presented in Section 3.2. Note that for a fixed size of  $S$  whose features are i.i.d. according to a fixed distribution  $\mathcal{D}_{\mathcal{X}}$ , the upper and lower bounds are matching up to a logarithmic factor, that is,
$$
h _ {\mathcal {U}} (k, n, m) = \tilde {\Theta} \left(\frac {1}{m} + \sqrt {\frac {k}{m n}}\right), \qquad k = \tilde {O} (n / m),
$$
$$
h _ {\mathcal {U}} (k, n, m) = \tilde {\Theta} \left(\frac {1}{m} + \frac {k}{n}\right), \qquad k = \tilde {\Omega} (n / m)
$$
for any fixed  $n$  and  $\mathcal{D}_{\mathcal{X}}$
2.3 Overview of our techniques
Next, we give a brief overview of proof techniques used to obtain the main results. We first note that throughout this paper we use the notion of corrupted hypothesis [28], which transforms the formulation of robust accuracy to a non-robust one thus greatly simplifying the proofs. The definition of corrupted hypothesis is presented in the beginning of Section 3.
- We establish the upper bounds via minimum description length argument, following closely a proof of an analogous result by [5] for non-robust setting. Note that their bounds can be viewed as trivial upper bounds of  $h_{\mathcal{U}}(k,n,m)$  since non-robust accuracy always upper bounds robust accuracy. We tighten the bounds by considering the query class of an algorithm. The details are presented in Theorem 2.
- To obtain the lower bounds, we propose computationally efficient algorithms for two regions of  $k$  respectively. The algorithms are modified from [6], who study the worst case overfitting bias in a non-robust setting. To take the perturbation of features into account, we extend their queries to the whole feature space  $\mathcal{X}$  by assigning each label of  $x \in \mathcal{X}$  to be the 'closest' label in  $\mathcal{X}_S$  in the sense of  $p$ -norm. The theoretical guarantees are given in Theorems 3 and 4.
2.4 Discussion
Although we equivalently derive both upper bounds and lower bounds of robust overfitting bias, there remains a gap in finding whether the lower and upper bounds can match up to a constant factor. In non-robust setting, it is proven that the  $\sqrt{\log n}$  factor in upper bounds can be removed for  $k = 1$  [6]. It is interesting to ask if this result holds for adversarial robust setting and, more ambitiously, for all  $k$ . We leave it as an open question.
3 Proofs of main results
To make the proofs more readable, we introduce the notion of corrupted hypothesis [28].
Consider a given hypothesis  $f: \mathcal{X} \to \mathcal{Y}$ . A labeled adversarial sample  $(\tilde{x}, y)$  is classified correctly if  $\tilde{x} \in f^{-1}(y)$ . A labeled example  $(x, y)$  is classified correctly if  $\mathcal{U}(x) \subset f^{-1}(y)$ . Let  $\tilde{\mathcal{Y}} = \mathcal{Y} \cup \{\bot\}$ , where  $\bot$  is the special "always wrong" output, i.e.  $y \neq \bot$  for every  $y \in \tilde{\mathcal{Y}}$ . We define the mapping  $\kappa_{\mathcal{U}}: \mathcal{Y}^{\mathcal{X}} \to \tilde{\mathcal{Y}}^{\mathcal{X}}$ :
$$
\kappa_ {\mathcal {U}} (f) (x) = \left\{ \begin{array}{l l} y, & \mathcal {U} (x) \subset f ^ {- 1} (y), \\ \perp , & \text {o t h e r w i s e}. \end{array} \right.
$$
The corrupted set of hypotheses induced by perturbation  $\mathcal{U}$  is then defined by  $\tilde{\mathcal{H}} = \{\kappa_{\mathcal{U}}(f):f\in \mathcal{H}\}$ .
3.1 Upper bounds
The upper bounds relies on the following theorem, showing in a probabilistic view that finding a classifier in some given hypothesis class with desired robust accuracy requires learning many bits about the test labeling.
Theorem 2. Let  $m, n, k$  be positive integers and  $\mathcal{D}_m^n = \mathcal{D}_{\mathcal{X}}^n \times \mu_m^n$ , where  $\mu_m^n$  denotes the uniform distribution over  $[m]^n$ . Let  $\mathcal{H} \subset \mathcal{Y}^\mathcal{X}$  be a hypothesis class and  $\mathcal{U}$  be an perturbation. Then there exists a constant  $C_{\mathcal{H}} \leq 1$  satisfying: For every  $\mathcal{H}$ -based  $k$ -query algorithm  $\mathcal{A}$ ,  $\delta > 0$ ,  $b = k\ln(n + 1)1 + \ln(1 / \delta)$ , we have
$$
\mathbf {P r} _ {S \sim \mathcal {D} _ {m} ^ {n}, f = \mathcal {A} (S)} \left\{\mathrm {A c c} _ {\mathcal {U}} (f; S) \geq \frac {C _ {\mathcal {H}}}{m} + 2 \sqrt {\frac {b}{n m}} \right\} \leq \delta , \quad k <   \frac {n}{m (\log (n + 1) + \log (1 / \delta))},
$$
$$
\mathop{\bf P}\limits_{S\sim \mathcal{D}_{m}^{n},f = \mathcal{A}(S)}\left\{\operatorname {Acc}_{\mathcal{U}}  (f;S)\geq \frac{C_{\mathcal{H}}}{m} +\frac{2b}{n}\right\} \leq \delta ,\qquad k\geq \frac{n}{m(\log(n + 1) + \log(1 / \delta))}.
$$
Proof. For any fixed hypothesis  $f$ , denote  $\kappa_{\mathcal{U}}(f)$  by  $\tilde{f}$ . By the definition of corrupted hypotheses, for every  $i \in [n]$ ,
$$
\begin{array}{l} \operatorname * {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\forall x ^ {\prime} \in \mathcal {U} (x _ {i}), f (x ^ {\prime}) = c _ {i} \right\} = \operatorname * {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\tilde {f} (x) = c _ {i} \right\} \\ = \mathbf {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\tilde {f} (x _ {i}) = c _ {i} | \tilde {f} (x _ {i}) \neq \bot \right\} \mathbf {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\tilde {f} (x _ {i}) \neq \bot \right\} \\ + \mathbf {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\tilde {f} (x _ {i}) = c _ {i} | \tilde {f} (x _ {i}) = \bot \right\} \mathbf {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\tilde {f} (x _ {i}) = \bot \right\}. \\ \end{array}
$$
We observe that  $\mathbf{Pr}\left\{\tilde{f}(x_i) \neq \bot\right\}$  is a constant related to  $f$ , let  $C(f) \triangleq \mathbf{Pr}\left\{\tilde{f}(x_i) \neq \bot\right\}$ , since  $\mathbf{Pr}\left\{\tilde{f}(x_i) = c_i | \tilde{f}(x_i) = \bot\right\} = 0$ , we have
$$
\mathbf {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\forall x ^ {\prime} \in \mathcal {U} (x _ {i}), f (x ^ {\prime}) = c _ {i} \right\} = \mathbf {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\tilde {f} (x _ {i}) = c _ {i} | \tilde {f} (x _ {i}) \neq \bot \right\} \mathbf {P r} _ {x _ {i} \sim \mathcal {D} _ {\mathcal {X}}} \left\{\tilde {f} (x _ {i}) \neq \bot \right\} = \frac {C (f)}{m}.
$$
This implies that  $\mathbb{1}\{\forall x' \in \mathcal{U}(x_i), f(x') = c_i\}$  is a Bernoulli random variable with bias  $\frac{C(f)}{m}$ . By the Chernoff bound, for any fixed  $f$ ,
$$
\mathbf {P r} _ {S \sim \mathcal {D} _ {m} ^ {n}} \left\{\frac {1}{n} \sum_ {i = 1} ^ {n} \mathbb {1} \{\forall x ^ {\prime} \in \mathcal {U} (x _ {i}), f (x ^ {\prime}) = c _ {i} \} \geq \frac {C (f)}{m} + \epsilon \right\} \leq e ^ {- \frac {m n \epsilon^ {2}}{2 C (f) + m \epsilon}}.
$$
Denote  $\max_{f\in \mathcal{H}}C(f)$  by  $C_\mathcal{H}$ , then
$$
\Pr \left\{\operatorname {A c c} _ {\mathcal {U}} (f; S) \geq \frac {C _ {\mathcal {H}}}{m} + \epsilon \right\} \leq \Pr \left\{\operatorname {A c c} _ {\mathcal {U}} (f; S) \geq \frac {C (f)}{m} + \epsilon \right\} \leq e ^ {- \frac {m n \epsilon^ {2}}{2 C (f) + m \epsilon}} \leq e ^ {- \frac {m n \epsilon^ {2}}{2 C _ {\mathcal {H}} + m \epsilon}}, \tag {1}
$$
holds for every  $f$ . Consider the execution of  $\mathcal{A}$  with responses of robust accuracy fixed to some sequence of values  $\alpha = (\alpha_{1},\dots ,\alpha_{k})\in \{0,1 / n,\dots ,1\}^{k}$ . Denote the resulting predictor by  $A^{\alpha}$ , its output distribution is fixed, hence by Eq.(1), we have
$$
\operatorname * {P r} _ {S \sim \mathcal {D} _ {m} ^ {n}, f = \mathcal {A} ^ {\alpha}} \left\{\operatorname {A c c} _ {\mathcal {U}} (f; S) \geq \frac {C _ {\mathcal {H}}}{m} + \epsilon \right\} \leq e ^ {- \frac {m n \epsilon^ {2}}{2 C _ {\mathcal {H}} + m \epsilon}}.
$$
Denote the set of all possible values of  $\alpha$  by  $V$ . For every test set  $S$ , the accuracy oracle outputs some responses in  $V$ . Therefore,
$$
\begin{array}{l} \mathop{\Pr}\limits_{S\sim \mathcal{D}_{m}^{n},f = A(S)}\left\{\operatorname {Acc}_{\mathcal{U}}(f;S)\geq \frac{C_{\mathcal{H}}}{m} +\epsilon \right\} \\ \leq \sum_ {\alpha \in V} \mathbf {P r} _ {S \sim \mathcal {D} _ {m} ^ {n}, f = \mathcal {A} ^ {\alpha}} \left\{\operatorname {A c c} _ {\mathcal {U}} (f; S) \geq \frac {C _ {\mathcal {H}}}{m} + \epsilon \right\} \\ \leq (n + 1) ^ {k} \cdot e ^ {- \frac {m n \epsilon^ {2}}{2 C _ {\mathcal {H}} + m \epsilon}}. \\ \end{array}
$$
Now if  $\frac{k\ln(n + 1) + \ln(1 / \delta)}{n} \geq \frac{1}{m}$ , then by definition of  $b$ ,  $\frac{2b}{n} \geq \frac{2}{m}$ . It follows that  $m\epsilon \geq 2 \geq 2C_{\mathcal{H}}$ , hence  $\frac{mn\epsilon^2}{2C_{\mathcal{H}} + m\epsilon} \geq \frac{n\epsilon}{2}$  and
$$
(n + 1) ^ {k} \cdot e ^ {- \frac {m n \epsilon^ {2}}{2 C _ {\mathcal {H}} + m \epsilon}} \leq e ^ {k \ln (n + 1) - \frac {n \epsilon}{2}} = e ^ {\ln \delta} = \delta .
$$
If  $\frac{k\ln(n + 1) + \ln(1 / \delta)}{n} < \frac{1}{m}$ , in this case  $2\sqrt{\frac{b}{nm}} \leq \frac{2}{m}$ . We obtain that  $m\epsilon < 2$ , thus  $\frac{mn\epsilon^2}{2C_{\mathcal{H}} + m\epsilon} \geq \frac{mn\epsilon^2}{4}$  and
$$
(n + 1) ^ {k} \cdot e ^ {- \frac {m n \epsilon^ {2}}{2 C _ {\mathcal {H}} + m \epsilon}} \leq e ^ {k \ln (n + 1) - \frac {m n \epsilon^ {2}}{4}} = e ^ {\ln \delta} = \delta ,
$$
and we complete the proof.
The corollary below follows immediately from Theorem 2, which gives the upper bounds of  $h_{\mathcal{U}}(k,n,m)$ .
Corollary 1. Let  $m, n, k$  be positive integers and  $\mathcal{U}$  be a perturbation, then
$$
h _ {\mathcal {U}} (k, n, m) \leq \frac {1}{m} + \frac {1}{n} + 2 \sqrt {\frac {(k + 1) \log (n + 1)}{n m}}, \qquad k <   \frac {n}{m (\log (n + 1) + \log (n))}
$$
$$
h _ {\mathcal {U}} (k, n, m) \leq \frac {1}{m} + \frac {2 (k + 1) \log (n + 1) + 1}{n}, \quad k \geq \frac {n}{m (\log (n + 1) + \log (n))}
$$
Proof. Denote  $\mathrm{Acc}_{\mathcal{U}}(f;S) \triangleq X \in [0,1]$ . Substitute  $\delta = 1/n$  in Theorem 2 and notice that  $C_{\mathcal{H}} \leq 1$  for any hypothesis class  $\mathcal{H}$ , hence for  $k < \frac{n}{m(\log(n+1) + \log(n))}$  we have
$$
\Pr \left\{X \geq \frac {1}{m} + 2 \sqrt {\frac {(k + 1) \log (n + 1)}{n m}} \right\} \leq \frac {1}{n}. \tag {2}
$$
Let  $c = \frac{1}{m} + 2\sqrt{\frac{(k + 1)\log(n + 1)}{nm}}$ , it remains to show  $\mathbb{E}X \leq c + 1 / n$ . It is trivial for  $c \geq 1$ . For the case that  $c < 1$ , let  $P_{X}$  be the probability distribution of  $X$ , by the definition of expectation,
$$
\mathbb {E} X = \int_ {0} ^ {1} X d P _ {X} = \int_ {0} ^ {c} X d P _ {X} + \int_ {c} ^ {1} X d P _ {X} \leq c \int_ {0} ^ {c} d P _ {X} + \int_ {c} ^ {1} d P _ {X} \leq c + \frac {1}{n},
$$
where we use the fact that  $\int_c^1 dP_X\leq 1 / n$  by Eq.(2) in the last step. The case of  $k\geq$ $\frac{n}{m(\log(n + 1) + \log(n))}$  can be proved using similar arguments, and we complete the proof.
3.2 Lower bounds
The lower bounds of  $h_{\mathcal{U}}(k,n,m)$  are derived from two designed algorithms, namely  $\mathcal{A}^{small}$  and  $\mathcal{A}^{big}(C)$ .  $\mathcal{A}^{small}$  is divided into two cases based on whether  $k = 1$ , and the precise analysis is presented in Theorem 3.  $\mathcal{A}^{big}(C)$  accepts a parameter  $C$  for calculating an intermediate variable, and we gain our Theorem 4 by setting  $C = \Phi_{\mathcal{D}_{\mathcal{X}}}(\bar{n})$ . Finally, the lower bounds of  $h_{\mathcal{U}}(k,n,m)$  follow from
$$
h _ {\mathcal {U}} (k, n, m) = \max _ {\mathcal {A}} h _ {\mathcal {U}} (\mathcal {A}) \geq h _ {\mathcal {U}} (\mathcal {A} ^ {s m a l l} \mathrm {o r} \mathcal {A} ^ {b i g} (\Phi_ {\mathcal {D} _ {\mathcal {X}}} (n))).
$$
To simplify the expression, we first introduce some definitions. For each  $\mathcal{X}_S\in \mathcal{X}^n$  , define
$$
\Gamma_ {S} = \left\{x \in \mathcal {X} \Big | \exists j _ {1} \neq j _ {2} \in [ n ] \text {s . t .} d _ {p} (x, x _ {j _ {1}}) = d _ {p} (x, x _ {j _ {2}}) = \min  _ {i \in [ n ]} d _ {p} (x, x _ {i}) \right\},
$$
$$
\Phi_ {\mathcal {D} _ {\mathcal {X}}} (n) = \min _ {\mathcal {X} _ {S} \in \mathrm {s u p p} (\mathcal {D} _ {\mathcal {X}} ^ {n})} \int_ {\mathcal {U} (\Gamma_ {S})} d P _ {\mathcal {D} _ {\mathcal {X}}},
$$
where  $\mathcal{U}(\Gamma_S) = \{\mathcal{U}(x)|x\in \Gamma_S\}$  and  $P_{\mathcal{D}_X}$  is the distribution function of  $\mathcal{D}_{\mathcal{X}}$
With these definitions, we present that:
Algorithm 1  $\mathcal{A}^{small}$  ( $k = 1$ )
1. Let  $f^1(x)$  be the all one query, i.e.  $f^1(x) = 1, \forall x \in \mathcal{X}$ .
2. Output  $\hat{f}$  that
$$
\hat {f} (x) = \left\{ \begin{array}{l l} 1 & \forall x \in \mathcal {X}, \quad \text {i f \operatorname {A c c} _ {\mathcal {U}} \left(f ^ {1} , S\right) \geq 1 / m}, \\ 2 & \forall x \in \mathcal {X}, \quad \text {o t h e r w i s e}. \end{array} \right.
$$
Theorem 3. Let  $n \geq m$  and  $\mathcal{D}_m^n = \mathcal{D}_{\mathcal{X}}^n \times \mu_m^n$ . Then for  $1 \leq k \leq 1 + n / 2m$ ,
$$
h _ {\mathcal {U}} (\mathcal {A} ^ {s m a l l}, \mathcal {D} _ {m} ^ {n}) \geq \frac {\Phi_ {\mathcal {D} _ {\mathcal {X}}} (n)}{m} + \frac {1}{8} \sqrt {\frac {\Phi_ {\mathcal {D} _ {\mathcal {X}}} (n) \cdot k}{m n}}.
$$
Proof. Case:  $k = 1$ : For  $l \in \{1, \ldots, m\}$ , let  $N_{l}$  be the number of examples with label  $l$ . Since the labels are uniformly distributed and  $\hat{f}$  is always a constant predictor,  $(N_{1}, \ldots, N_{m})$  follows a multinomial distribution with parameters  $(n; \frac{1}{m}, \ldots, \frac{1}{m})$ . Then by the construction of  $\hat{f}$ , the number of robustly and correctly predicted labels is then given by  $N_{1} \cdot \mathbb{1}\{N_{1} \geq n / m\} + N_{2} \cdot \mathbb{1}\{N_{1} < n / m\}$ , and the expected robust accuracy is given by