On the Existence of The Adversarial Bayes Classifier

Abstract: Adversarial robustness is a critical property in a variety of modern machine learning applications. While it has been the subject of several recent theoretical studies, many important questions related to adversarial robustness are still open.  In this work, we study a fundamental question regarding Bayes optimality for adversarial robustness. We provide general sufficient conditions under which the existence of a Bayes optimal classifier can be guaranteed for adversarial robustness. Our results can provide a useful tool for a subsequent study of surrogate losses in adversarial robustness and their consistency properties.


Reviews and author responses:

 We would like to point out that our result is aimed at understanding when the adversarial Bayes optimal classifier will exist. In general such a adversarial Bayes optimal classifier may not lie in any common function class (this can happen in non-adversarial cases too), although as the function class gets richer we may expect a better approximation to the adversarial Bayes optimal classifier. 

If we assume that our distribution is absolutely continuous with respect to Lebesgue measure $\mu$, then there is a continuous function for which $A = ${$x: f(x)>0 $} is an adversarial Bayes classifier. There’s a standard argument used in geometric measure theory that shows that if a set $S$ satisfies the pseudo-certifiable robustness property then $\mu(\partial S)=0$.  Let $B$ be the adversarial Bayes classifier and define $f(x)=\text{dist}(x,B^C)$. Then this function is continuous, and is greater than zero on the interior of $B$, so that {$x: f(x)>0$} $= \text{int} B$. However, because $\mu(\partial B)=0$, the sets $B$ and $\text{int} B$ differ by a set measure 0, $\text{int} B$ is also an adversarial Bayes classifier.

We will further elaborate on this in the final version.

 Thank you very much for the time you took to review our paper. We really appreciate your comments and insights, especially on the appendix. 

 Thanks for the clarification. I see that $A$ being either an open or closed set is not enough for proving the existence. Then, I have a follow-up question.

Suppose that the classifier $f$ is continuous, we can see that $f$ defines an open set decision region by $A=f^{-1}((0, +\infty))$. Does it mean that the existence of the optimal classifier may not exist for continuous $f$ (or even a more general function class)? Therefore my concern is: the existence result in this paper may not apply to all, or most of, common machine learning function classes, e.g., continuous functions, neural networks. It would be more meaningful, in my humble opinion, if the result holds for common machine learning function classes. 

 Thanks for your response! I am fine with your responses! I can only recommend acceptation for the paper. But still I urge you to make the appendix more readable! 

 Thank you for your response!

Yes you’re right!  Prop. 7.50 in Bertsekas and Shreve does imply that $A\oplus \overline B_\epsilon(\mathbb 0)$ is universally measurable for Borel $A$! Thank you for pointing this out. Unfortunately, this is not quite enough for our theorem because we need to show that $A\oplus \overline B_\epsilon(\mathbb 0)$ is universally measurable for universally measurable $A$.

Regarding the sign() function, the penalty at 0 also depends on the convention for the sign function at 0. We are following the convention to define sign 0 as either +1 or -1 rather than 0. Either of these conventions would treat points with $f(x)=0$ as one of the classes. We will clarify the convention for our sign function in the final version of our paper.  



 Thank you for your responses to my feedback. I have updated my score.

One small point: the references I suggested do not incorporate *randomized smoothing* into the training process, but rather they incorporate other methods of certifying robustness during training.

This paper establishes the existence of a binary classifier which minimizes the expected adversarial loss under a data and label distribution, which they call the adversarial Bayes classifier. The authors do so by constructing a sequence of classifiers which converges to the infimum risk, and then demonstrate that the risk is sequentially lower semi-continuous, which establishes the existence of a minimum. They also extend this result to some non-additive perturbations.  I am somewhat unfamiliar with the prior work in this area of adversarial learning theory, so I cannot comment much on the originality of the work. Since this seemed to be a point of concern in previous submissions, I will defer to the other reviewers' opinion in this area. However, one point in the introduction about previous work is a bit misleading: in lines 82-84, the authors write that

> a process called randomized smoothing makes the classifier certifiably robust in the `$\ell_2$ norm with high probability. Notably, randomized smoothing is applied after training. A better understanding of the adversarial Bayes classifier could help find other such procedures.

This seems to indicate that prior work has not incorporated certified robustness *during* training, but many papers have explored this idea [1-4, among others]. It would be good to clarify this.

The exposition is good throughout the paper. All the math appears to be sound, although I did not carefully check every proof.

The results seem useful for adversarial learning theory, although again I am not sure exactly how novel they are. I think the authors could do a bit more to flesh out the implications of the work. While the introduction and conclusion refer to giving insight into the consistency of adversarial losses, they are mostly vague. Are there implications of the Bayes optimal classifier being pseudo-certifiably robust? In Appendix B, the authors argue that removing the non-pseudo-certifiably robust region of a classifier generally reduces the loss and that "one could try and device algorithms to take advantage of this property." Moving and adding statements like this to the conclusion or introduction could give some more intuition about why and how the results are significant.

Overall, the paper is well-written and sound; I would vote to accept it if the above concerns are addressed and barring serious concerns from the other reviewers. 

*A typo*: in Definition 1, I believe the last line should read "A is pseudo-certifiably robust *if it is* pseudo-certifiably robust", not "is is is".

[1] Weng et al. Towards fast computation of certified robustness for ReLU networks. ICML 2018.

[2] Wong and Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. ICML 2018.

[3] Zhang et al. Efficient neural network robustness certification with general activation functions. NeurIPS 2018.

[4] Raghunathan et al. Semidefinite relaxations for certifying robustness to adversarial examples. NeurIPS 2018. The authors state that "our work is theoretical in nature and we do not anticipate any immediate negative societal impact"; I agree with this assessment.

 Thanks for your answer! I have the following comments regarding your answer:

- Actually I think that the results still holds (cf Bertsekas and Shreve, Prop 7.50) even for measurable function (not necessarily u.s.c.). I suggest the authors to have a look to this.
- I kindly disagree. $y\cdot sign(f(x))$ and $y\cdot f(x)$ do not have the same behavior. The first one penalizes indecision with $1/2$ and the second penalizes indecision with $1$. The difference is subtle but very important.
- Thanks for the clarification. I agree it might help.

 Thank you for your feedback and the additional references recommended.
- Lines 82-84: We will clarify this discussion using the references suggested by the reviewer. We agree that recent works have tried incorporating randomized smoothing during the training process as well.

- Novelty of results: we are not aware of any prior result about the existence of the adversarial Bayes classifier and believe that our results are novel. Originality was not a concern in the previous submission.

- Connections with calibration/consistency: We briefly discuss two ways in which our results relate to the consistency of adversarial losses. First, [1] shows that when $H$ is the class of linear functions, a zero adversarial surrogate loss implies $H$-consistency. Furthermore, [1] gives examples for which the adversarial surrogate loss is nonzero and continuous surrogate losses cannot be consistent. The existence of the adversarial Bayes classifier is required for this condition to hold. Next, a surrogate loss $\Psi$ is consistent if a minimizing sequence of functions $\{f_i\}$ also minimizes 0-1 adversarial loss. However, it may be easier to study minimizing sequences of the $\Psi$ loss when we have information about the adversarial Bayes classifier. Theorem 7 of the appendix lists a variety of conditions under which a minimizing sequence of the adversarial loss approaches an adversarial Bayes classifier in a meaningful sense. Thus we can find conditions under which $\{f_i\geq 0\}$ approaches a set $A$. In other words: If $\Psi$ is consistent and $\{f_i\}$ minimizes the adversarial $\Psi$ loss, then $\{f_i\geq 0\}$ must approach an adversarial Bayes classifier in the sense described by Theorem 7. 

- Pseudo-certifiably robustness: we will further elaborate on that and move and add corresponding statements to the introduction, as suggested by the reviewer.

[1] Calibration and consistency of adversarial Surrogate losses. Pranjal Awasthi, Natalie Frank, Anqi Mao, Mehryar Mohri, Yutao Zhong. Arxiv.


 Thank you for your thorough review and your appreciation of this work.

Supplementary material: we will follow your suggestion and seek to reorganize the appendix to make it easier to follow the proofs.

- Measurability of adversarial risk in paper [1]: you are probably referring to Proposition 2 or Lemma 4 in the appendix. It is not clear if these results apply in our context since the 0-1 loss composed with our classifiers is not upper semicontinuous and thus does not satisfy Assumption 1 of that paper. Furthermore, because we consider sets of the form $((A^{-\epsilon})^{-2\epsilon})^\epsilon$, we need to show that if $A$ is universally measurable, so is $A^\epsilon$. Showing that if $A^\epsilon$ is universally measurable for Borel $A$ does not suffice. Please clarify if we are missing a key point here. Either way, we agree that our result is of a similar flavor and will definitely include this paper as a reference in the related work discussion.

- Definition of Line 138: presumably, the reviewer is asking about the non-strict vs. strict inequality; we believe that this is a matter of convention. Notice that $y \cdot f(x)$ and $y \cdot sign(f(x))$ admit the same sign, so they induce the same loss.

- Connections with calibration/consistency: We briefly discuss two ways in which our results relate to the consistency of adversarial losses. First, [1] shows that when $H$ is the class of linear functions, a zero adversarial surrogate loss implies $H$-consistency. Furthermore, [1] gives examples for which the adversarial surrogate loss is nonzero and continuous surrogate losses cannot be consistent. The existence of the adversarial Bayes classifier is required for this condition to hold. Next, a surrogate loss $\Psi$ is consistent if a minimizing sequence of functions $\{f_i\}$ also minimizes 0-1 adversarial loss. However, it may be easier to study minimizing sequences of the $\Psi$ loss when we have information about the adversarial Bayes classifier. Theorem 7 of the appendix lists a variety of conditions under which a minimizing sequence of the adversarial loss approaches an adversarial Bayes classifier in a meaningful sense. Thus we can find conditions under which $\{f_i\geq 0\}$ approaches a set $A$. In other words: If $\Psi$ is consistent and $\{f_i\}$ minimizes the adversarial $\Psi$ loss, then $\{f_i\geq 0\}$ must approach an adversarial Bayes classifier in the sense described by Theorem 7. 

[1] Calibration and consistency of adversarial Surrogate losses. Pranjal Awasthi, Natalie Frank, Anqi Mao, Mehryar Mohri, Yutao Zhong. Arxiv.


 Thank you for your feedback on the measurability issue. We will seek ways to simplify the measurability discussion and make the material more accessible.

Unfortunately, optimizing $R^\epsilon$ over just closed or open sets $A$ would not suffice for our proof:
(1) To find the adversarial Bayes classifier, we essentially consider limits of sets of the form $A_i^\epsilon$ and show that the sequence approaches some set of the form $A^\epsilon$. But, that the $A_i$’s are closed/open does not imply in general that $A$ is closed/open. 

(2) Furthermore, in the proof of Lemma 2, we apply the $^\epsilon$ and $^{-\epsilon}$ operations to a set several times in succession (specifically, we consider sets of the form $((A^{-\epsilon})^{2\epsilon})^{-\epsilon}$). Because of this proof, we need to work over a $\sigma$-algebra which is closed under the $^\epsilon$ operation--showing that $A^\epsilon$ is measurable for some nice $A$ does not suffice.

(3) Lastly, the result that $S^\epsilon$ is universally measurable for every universally measurable $S$ could be quite useful for further theoretical research in machine learning. 

We will elaborate on this in the final version and add this discussion.

Your question about pseudo-certified robustness is interesting. Pseudo-certifiable robustness alone indeed does not guarantee a low adversarial risk. In follow-up work, we are investigating conditions that imply a low adversarial risk and it turns out that studying pseudo-certifiable robustness is a first step.



This paper studies a fundamental question regarding the adversarial Bayes classifier, i.e., does the optimal Bayes classifier exists and does it have any robustness guarantee. The contribution of this paper is theoretical: it proposes the concept of ‘pseudo-certifiable robustness’ and proves that the optimal Bayes classifier exists and satisfies the proposed ‘pseudo-certified robustness’. The proofs are mostly about real analysis and measure theory.   **Originality**

The concept of ‘pseudo-certified robustness’ is new to me. In addition, by formalizing the adversarial risk as a function of sets and thus transforming the original problem to a real analysis problem, the proof techniques seem to be not very common in the adversarial machine learning literatures. 

**Quality**

The paper is rigorous and mostly well-written. Reading through the main paper, I did not notice wrong statements assuming all the lemmas are correct. The contents in appendix are quite overwhelming, for which I only had a skim.

I do not have any major criticisms for this paper, but I do have some suggestions. I think some of the considerations may be too mathematical and far away from the area of machine learning. These pure mathematical considerations complicate the proofs and theorems, while has no real meaning to machine learning. 

For example, the measurability issue of $A\oplus \overline{B_\epsilon(0)}$. Even though there exists extreme cases where the direct sum of two measurable sets is not measurable, I do not think that can be a real problem for machine learning models. The authors put quite an effort to establish the ‘universal $\sigma$-algebra’, but the measurability issue can be worked around effortlessly by being a little bit more ‘machine learning’. I am not suggesting being less rigorous but being a bit more practical.

Concretely, $A\oplus \overline{B_\epsilon(0)}$ is open, thus measurable, if $A$ is open. Moreover, $A\oplus \overline{B_\epsilon(0)}$ is measurable if $A$ is closed (See Lemma 3.1 in [1]). This kind of $A$ (i.e., being either an open set or a closed set) should cover most of, if not every, realistic cases I think. For example, $A=\\{x:f(x) > 0 \\}=f^{-1}((0, +\infty))$ is open whenever the model $f$ is continuous, which covers many common models, e.g., SVM, neural networks with any continuous activation like ReLU and Sigmoid. Moreover, I think there should be weaker conditions for $A\oplus \overline{B_\epsilon(0)}$ being measurable to hold. 

The bottom-line: the results are interesting, but I believe there are simpler ways to deliver the same message without sacrificing rigorousness.

Question: Theorem 1 proves that the minimizer of adversarial risk exists and is pseudo-certifiably robust. However, it seems that pseudo-certified robustness alone cannot guarantee a low adversarial risk. What other conditions do we need to guarantee pseudo-certified robustness imply low adversarial risk? More generally, it would be interesting to see more investigations about the pseudo-certified robustness.

**Clarity**

The paper is well-written in my opinion. 
Minor things: line 193 typo (is is is). In Lemma 4 $X$ should be $\mathbb{R}^d$? Some equations are too wide (e.g. line 318, 329)

**Significance**

I think the results of this paper is novel, fundamental and important.

[1] Adversarial risk via optimal transport and optimal couplings. Muni Sreenivas Pydi and Varun Jog.
 The main result (theorem 1) requires a strictly convex norm which excludes two common norms, $\ell_1$ and $\ell_\infty$. The authors discuss this limitation in appendix.

The paper proves the existence of universally measurable classifiers minimizing the adversarial risk for strictly convex set of perturbations. This is the first result of this kind in adversarial literature.  I would like to highlight how nice it is to see theoretical papers in the field of adversarial attacks. I really believe this kind of paper can improve the research in the field and helps understanding the very nature of this phenomenon. The paper is very technical. I acknowledge I have a good background in measure theory which helped me understand better the paper. Yet, this paper needs to be somekind of simplified if the authors would like it to have an impact broader than people mastering measure theory and having interest in adversarial machine learning. I am fine with the current version, but it is rather a suggestion. 

I must admit I read the proofs but I did not check them carefully. From what I read, I am confident in the results. In the meantime of the review it is complicated to have a thorough reading of the paper and the proofs for such a technical paper. That said, I still have some comments about your work:

- The supplementary material is very difficult to read. I suggest the authors to write a linear appendix. Going up and down to check proofs is not very comfortable.
- Measurability of the adversarial risk was treated in this paper [1] using the same concepts of universal measurability.
- I am not sur  the risk $R^\epsilon(f)$ defined L138 actually corresponds to the risk $R^\epsilon(A)$. I think the "right" loss should be $1_{ysign(f(x))\leq 0}$. but I might be mistaken
- The paper is motivated by calibration/consistency and provable robustness. I am not sure to see the direct link. Can the authors elaborate a bit on that?

[1] Mixed Nash Equilibria in the Adversarial Examples Game. Laurent Meunier, Meyer Scetbon, Rafael Pinot, Jamal Atif, Yann Chevaleyre. ICML2021.
 Understanding theoretically adversarial examples can help building more robust networks.

Meta-review: This paper studies a variety of theoretical questions regarding the Bayes optimal classifier and associated robustness guarantees. This is an important topic, adding to the growing discussion around theoretical tools for robustness-related questions. The reviewers found the theoretical results to be substantial, using novel arguments in linear algebra and measure theory. The reviewers also believe that the results are interesting, well-motivated, and technically sound. Therefore, I recommend accepting this paper.

In the reviews, there were two main concerns that the authors should address for the final version of the paper: (1) certain relevant references were missing and should be discussed, and (2) there was a strong suggestion to improve the readability of the supplemental results and proofs in the paper. I agree that both of these should be addressed, and that writing a good theory paper can be challenging but is ultimately a good service to the community. Specifically, for (1), I think the authors can also be clearer in the paper about the relationship with prior work, focusing more on the theoretical comparison between the prior results in some of the references and the present results (e.g., a proper technical overview may be a good way to address both points and set up the exposition for the appendix and proofs). 

Decision: accept