SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We have provided a robustly-reliable learner with the largest possible robustly-reliable region for losses  $\ell_{\mathrm{CA}}, \ell_{\mathrm{TL}}, \ell_{\mathrm{ST}}, \ell_{\mathrm{IA}}$ . However, we note that the probability mass of the robustly-reliable region may not be a meaningful way to quantify the overall reliability of a learner because a perturbation  $z$  may lie outside of the support of the natural data distribution and have zero probability mass. It seems more useful to measure the mass of points  $x$  where any perturbation  $z$  of  $x$  still lies within the robustly-reliable region. We formally define this region as the safely-reliable region.
Definition 6 (Safely-reliable region w.r.t.  $\mathcal{M}$ -ball attacks). Let  $\mathcal{L}$  be a robustly-reliable learner w.r.t.  $\mathcal{M}$ -ball attacks for sample  $S$ , hypothesis class  $\mathcal{H}$  and robust loss function  $\ell$ . The safely-reliable region of learner  $\mathcal{L}$  at reliability levels  $\eta_1, \eta_2$  is defined as
1.  $SR_{CA}^{\mathcal{L}}(S,\eta_1,\eta_2) = \{x\in \mathcal{X}\mid \mathbf{B}_{\mathcal{M}}(x,\eta_1)\cap \{z\mid h^* (z) = h^* (x)\} \subseteq RR_{CA}^{\mathcal{L}}(S,\eta_2)\}$
2.  $SR_W^{\mathcal{L}}(S,\eta_1,\eta_2) = \{x\in \mathcal{X}\mid \mathbf{B}_{\mathcal{M}}(x,\eta_1)\subseteq RR_W^{\mathcal{L}}(S,\eta_2)\}$  for  $W\in \{TL,ST\}$
3.  $SR_{IA}^{\mathcal{L}}(S,\eta_1,\eta_2) = \{x\in \mathcal{X}\mid h^* (x) = 0\wedge \mathbf{B}_{\mathcal{M}}(x,\eta_1)\subseteq RR_{IA}^{\mathcal{L}}(S,\eta_2)\} \cup \{x\in \mathcal{X}\mid h^* (x) = 1\wedge x\in RR_{IA}^{\mathcal{L}}(S,\eta_2))\} .$
The safely-reliable region contains any point that retains a reliability radius of at least  $\eta_{2}$  even after being attacked by an adversary with strength  $\eta_{1}$ . In the safely-reliable region, we consider a set of potential natural (before attack) points  $x$ , while in the robustly-reliable region, we consider a set of potential test points  $z$ . In the following subsections, we show that in interesting cases commonly studied in the literature, the probability mass of the safely-reliable region is actually quite large.
Figure 2: The robustly-reliable region for  $\ell_{\mathrm{CA}}$ ,  $\ell_{\mathrm{TL}}$  (left),  $\ell_{\mathrm{ST}}$  (mid) and  $\ell_{\mathrm{IA}}$  (right) for linear separators with an  $L_{2}$ -ball perturbation. The background color of blue and red represents the agreement region of class 1 and 0 respectively. In this case, we can remove the label consistency condition and reduce the robustly-reliable region into the '  $\eta$ -buffered' agreement region.
3.1 Safely-reliable region for linear separators under log-concave distributions is large
We provide the probability mass of safely-reliable regions with respect to different losses for linear separators when the data distribution follows an isotropic (mean zero and identity covariance matrix) log-concave (logarithm of density function is concave) distribution under a bounded  $L_{2}$ -norm ball attack. For full proof, we refer to Appendix D. We will rely on the following key lemma which states that the agreement region of a linear separator cannot contain points that are arbitrarily close to the decision boundary of  $h^{*}$  for any sample  $S$ .
Lemma 3.2. Let  $\mathcal{D}$  be a distribution over  $\mathbb{R}^d$  and  $\mathcal{H} = \{h : x \to \mathrm{sign}(\langle w_h, x \rangle) \mid w_h \in \mathbb{R}^d, \|w_h\|_2 = 1\}$  be a class of linear separators. For  $h^* \in \mathcal{H}$ , for a set of samples  $S \sim \mathcal{D}^m$  such that there is no data point in  $S$  that lies on the decision boundary, for any  $0 < c < d$ , there exists  $\delta(S, c, d) > 0$  such that for any  $x$  with  $c \leq \|x\| \leq d$  and  $|\langle w_{h^*}, x \rangle| < \delta$ , we have  $x \notin \operatorname{Agree}(\mathcal{H}_0(S))$ .
A direct implication of the lemma is that any  $L_{2}$ -ball  $\mathbf{B}(x, \eta)$  that lies in the agreement region must not contain the decision boundary of  $h^{*}$  and must contain points with the same label. This allows us to remove the label consistency condition and instead focus on whether the ball  $\mathbf{B}(x, \eta)$  lies in the agreement region. Intuitively, the reliable region is now given by the 'η-buffered' agreement region where we only select points that have a distance at least  $\eta$  from the boundary of the agreement region (Figure 2). We provide bounds for the probability mass of the safely-reliable region below and we refer to the full proof in Appendix D.
Theorem 3.3. Let  $\mathcal{D}$  be isotropic log-concave over  $\mathbb{R}^d$  and  $\mathcal{H} = \{h : x \to \mathrm{sign}(\langle w_h, x \rangle) \mid w_h \in \mathbb{R}^d, \|w_h\|_2 = 1\}$  be the class of linear separators. Let  $\mathbf{B}(\cdot, \eta)$  be a  $L_2$  ball perturbation with radius  $\eta$ . For  $S \sim \mathcal{D}^m$ , for  $m = \mathcal{O}\left(\frac{1}{\varepsilon^2}(\mathrm{VCdim}(\mathcal{H}) + \ln \frac{1}{\delta})\right)$ , for an optimal robustly-reliable learner  $\mathcal{L}$ ,
(a)  $\operatorname{Pr}(SR_{TL}^{\mathcal{L}}(S, \eta_1, \eta_2)) \geq 1 - 2\eta_1 - \tilde{\mathcal{O}}(\sqrt{d}\varepsilon)$  with probability at least  $1 - \delta$
(b)  $SR_{CA}^{\mathcal{L}}(S,\eta_1,\eta_2) = SR_{TL}^{\mathcal{L}}(S,\eta_1,\eta_2)$  almost surely,
(c)  $\operatorname{Pr}(SR_{ST}^{\mathcal{L}}(S, \eta_1, \eta_2)) \geq 1 - 2(\eta_1 + \eta_2) - \tilde{\mathcal{O}}(\sqrt{d}\varepsilon)$  with probability at least  $1 - \delta$ ,
(d)  $\operatorname{Pr}(SR_{IA}^{\mathcal{L}}(S, \eta_1, \eta_2)) \geq 1 - (\eta_1 + \eta_2) - \tilde{\mathcal{O}}(\sqrt{d}\varepsilon)$  with probability at least  $1 - \delta$ .
The  $\tilde{\mathcal{O}}$ -notation suppresses dependence on logarithmic factors and distribution-specific constants.
We remark that we can't always remove the label consistency condition for a general perturbation set. For example, consider  $\mathcal{U}(x) = \mathbf{B}(x - a,\eta)\cup \{x\} \cup \mathbf{B}(x + a,\eta)$ , is made of two  $L_{2}$  balls with center  $x - a,x + a$ , with appropriate value of  $a,\eta$ , we may have each ball lie in the different side of the agreement region so that the whole perturbation set lie in the agreement region but contain points with different labels (Figure 3). We also provide bounds on the probability mass of the safely-reliable region for more general concept spaces beyond linear separators, specifically, classifiers with smooth boundaries in Appendix E.
4 On computational efficiency
Given the definition, it is possible to implement computationally efficient robustly-reliable learners. For example, for linear separator concept classes under bounded  $L_{2}$ -norm attack. The optimal
Figure 3: The perturbation set is represented by two dashed balls. This lies inside the agreement region but contains points with different labels.
Figure 4: The disagreement region and the agreement region under a distribution shift where  $\mathcal{P}$  and  $\mathcal{Q}$  are isotropic (left) and where there is a mean shift (right).
robustly-reliable learner  $\mathcal{L}_{\mathrm{opt}}$ , described above may be implemented via a linearly constrained quadratic program that computes the (squared) distance of the test point  $z$  to the closest point  $z'$  in the disagreement region. This gives us the reliability radius, since for linear separators one must cross the decision boundary to perturb a point to a differently labeled point
$$
\min _ {w, w ^ {\prime}, z ^ {\prime}} | | z - z ^ {\prime} | | ^ {2}
$$
s.t.  $\langle w,x_i\rangle y_i\geq 0,$  for each  $(x_{i},y_{i})\in S$
$\langle w', x_i \rangle y_i \geq 0$ , for each  $(x_i, y_i) \in S$ ,
$$
\langle w, z ^ {\prime} \rangle \langle w ^ {\prime}, z ^ {\prime} \rangle \leq 0.
$$
Given training sample  $S$ , for any given test point  $z$ , the learner  $\mathcal{L}$  can efficiently compute the solution  $s^*$  to the above program and output  $\sqrt{s^*}$  as the reliability radius. We show that the variant of this objective also provides a reliability radius for a wide range of hypothesis classes under  $L_2$  ball attacks (see Lemma F.1). In addition, we can relax this objective into a regularized objective that gives a lower bound on the reliability radius of  $||z - z^*||^2$ , when  $z^*$  is the solution of
$$
h_{1},h_{2},z^{*} = \operatorname *{argmin}_{h,h^{\prime},z^{\prime}}||z - z^{\prime}||^{2} + \lambda (\hat{R} (h,S\cup \{(z^{\prime},0)\}) + \hat{R} (h^{\prime},S\cup \{(z^{\prime},1)\}))
$$
when  $\hat{R}(h, S)$  is the empirical risk of  $h$  on  $S$ . We provide a more detailed discussion in Appendix F.
5 Robustly-reliable learning under distribution shift
We now consider the reliability aspect for distribution shift, a different kind of test-time robustness challenge when the test data comes from a different distribution than the training data. Formally, let  $\mathcal{P}$  be the training distribution and let  $\mathcal{Q}$  be the test distribution. We assume the realizable distribution shift setting, i.e. there is a target concept  $h^* \in \mathcal{H}$  such that the true label of any  $x \in \mathcal{X}$  is given by  $h^*(x)$  at training time and test time, or  $\mathrm{err}_{\mathcal{P}}(h^*) = \mathrm{err}_{\mathcal{Q}}(h^*) = 0$ . As observed earlier, points that
are known by the learner to be correct (reliable) are still known to be correct even when it is drawn from a different distribution. This reliability guarantee holds even when the distributions  $\mathcal{P}$  and  $\mathcal{Q}$  do not share a common support, a setting for which many prior theoretical works result in vacuous bounds. For example, suppose  $\mathcal{X} = \mathbb{R}^n$ ,  $\mathcal{P}$  and  $\mathcal{Q}$  are supported on disjoint  $n$ -balls, and  $\mathcal{H}$  is the class of linear separators. Then the total variation distance, the  $\mathcal{H}$ -divergence [KBDG04, BDBC+10] as well as the discrepancy distance [MMR08] between  $\mathcal{P}$  and  $\mathcal{Q}$  are all 1. While recent work of [HK19] does apply in this setting, they do not focus on the reliability guarantee. In this work, we are interested in quantifying the transferability of reliability guarantee transfer between distributions  $\mathcal{P}$  and  $\mathcal{Q}$ . We recall the notion of reliable prediction [EYW10].
Definition 7 (Reliability). A learner  $\mathcal{L}$  is reliable w.r.t. concept space  $\mathcal{H}$  if, for any target concept  $h^* \in \mathcal{H}$ , given any sample  $S$  labeled by  $h^*$ , the learner outputs functions  $h_S^\mathcal{L}: \mathcal{X} \to \mathcal{Y}$  and  $a_S^\mathcal{L}: \mathcal{X} \to \{0,1\}$  such that for all  $x \in \mathcal{X}$  if  $a_S^\mathcal{L}(x) = 1$  then  $h_S^\mathcal{L}(x) = h^*(x)$ . Else, if  $a_S^\mathcal{L}(x) = 0$ , the learner abstains from prediction. The reliable region of  $\mathcal{L}$  is  $R^\mathcal{L}(S) = \{x \in \mathcal{X} \mid a_S^\mathcal{L}(x) = 1\}$ .
We define the following metric to measure the reliability of a learner under distribution shift.
Definition 8 ( $\mathcal{P} \to \mathcal{Q}$  reliable correctness). The  $\mathcal{P} \to \mathcal{Q}$ -reliable correctness of  $\mathcal{L}$  (at sample rate  $m$ , for distribution shift from  $\mathcal{P}$  to  $\mathcal{Q}$ ) is defined as the expected probability mass of its reliable region under  $Q$  when trained on a random training  $S \sim \mathcal{P}^m$ , i.e.  $\operatorname*{Pr}_{x \sim Q, S \sim P^m}[x \in R^{\mathcal{L}}(S)]$ .
The disagreement coefficient was originally introduced to study the label complexity in agnostic active learning [Han07] and is also known to characterize reliable learning in the absence of any distribution shift [EYW10]. We propose the following refinement to the notion of disagreement coefficient, which we will use to give bounds on a learner's  $\mathcal{P} \to \mathcal{Q}$  reliable correctness.
Definition 9 ( $\mathcal{P} \to \mathcal{Q}$  disagreement coefficient). For a hypothesis class  $\mathcal{H}$ , the  $\mathcal{P} \to \mathcal{Q}$  disagreement coefficient of  $h^* \in \mathcal{H}$  with respect to  $\mathcal{H}$  is given by
$$
\Theta_ {\mathcal {P} \to \mathcal {Q}} (\varepsilon) = \sup _ {r \geq \varepsilon} \frac {\operatorname * {P r} _ {\mathcal {Q}} [ \mathrm {D I S} (B _ {\mathcal {P}} (h ^ {*} , r)) ]}{r},
$$
where  $B_{\mathcal{P}}(h^{*},r) = \{h\in \mathcal{H}\mid \operatorname *{Pr}_{\mathcal{P}}[h(x)\neq h^{*}(x)]\leq r\} .$
This quantifies the rate of disagreement over  $\mathcal{Q}$  among classifiers which are within disagreement-balls w.r.t.  $h^*$  under  $\mathcal{P}$ , relative to the version space radius. The proposed metric is asymmetric between  $\mathcal{P}$  and  $\mathcal{Q}$ , and also depends on the target concept  $h^*$ . More simple examples are in Appendix H. We show that the  $\mathcal{P} \to \mathcal{Q}$ -reliable correctness of our learner may be bounded in terms of the  $\mathcal{P} \to \mathcal{Q}$  disagreement coefficient using a uniform convergence based argument. The proof details are in Appendix I.
Theorem 5.1. Let  $\mathcal{Q}$  be a realizable distribution shift of  $\mathcal{P}$  with respect to  $\mathcal{H}$ , and  $h^* \in \mathcal{H}$  be the target concept. Given sufficiently large sample size  $m \geq \frac{c}{\varepsilon^2} (d + \ln \frac{1}{\delta})$ , the  $\mathcal{P} \to \mathcal{Q}$ -reliable correctness of  $\mathcal{L}$ , the optimal robustly-reliable learner, is at least
$$
\operatorname * {P r} _ {x \sim Q, S \sim P ^ {m}} [ x \in R ^ {\mathcal {L}} (S) ] \geq 1 - \Theta_ {\mathcal {P} \to \mathcal {Q}} \cdot \varepsilon - \delta .
$$
Here  $c$  is an absolute constant, and  $d$  is the VC-dimension of  $\mathcal{H}$ .
In Appendix J, we show that this  $\mathcal{P} \to \mathcal{Q}$  disagreement coefficient can be small for several examples which implies that it is possible to transfer the reliability guarantee from one distribution to the other. In particular, when learning linear separators, we provide bounds for transferring from  $\beta_{1}$  log-concave to  $\beta_{2}$  log-concave and to  $s$ -concave distributions (Theorems G.1, G.2). In addition, when learning classifiers with general smooth classification boundaries, we provide bounds for transferring between smooth distributions (Theorem G.3).
6 Safely-reliable correctness under distribution shift
There is a growing practical  $\mathrm{[SSZ^{+}20, SIE^{+}20]}$  as well as recent theoretical interest [DGH $^{+}$ 23] in the setting of 'robustness transfer', where one simultaneously expects adversarial test-time attacks as well as distribution shift. We will study the reliability aspect for this more challenging setting. We note that the definition of a robustly-reliable learner does not depend on the data distribution (see
Definition 3) as the guarantee is pointwise. Our optimality result in Section 3 applies even when a test point is drawn from a different distribution  $\mathcal{Q}$ . In this case, the safely-reliable region instead would have a different probability mass.
Definition 10 ( $\mathcal{P} \to \mathcal{Q}$  safely-reliable correctness). The  $\mathcal{P} \to \mathcal{Q}$  safely-reliable correctness of  $\mathcal{L}$  (at sample rate  $m$ , for distribution shift from  $\mathcal{P}$  to  $\mathcal{Q}$ , w.r.t. robust loss  $\ell$ ) is defined as the probability mass of its safely-reliable region under  $Q$ , on a sample  $S \sim \mathcal{P}^m$ , i.e.
$$
P Q R _ {\ell} ^ {\mathcal {L}} (S, \eta_ {1}, \eta_ {2}) := \operatorname * {P r} _ {x \sim Q, S \sim P ^ {m}} [ x \in S R _ {\ell} ^ {\mathcal {L}} (S, \eta_ {1}, \eta_ {2}) ].
$$
We consider an example when the training distribution  $\mathcal{P}$  is isotropic log-concave and the test distribution  $\mathcal{Q}_{\mu}$  is log-concave with its mean shifted by  $\mu$  but the covariance matrix is still an identity matrix (see Figure 4, right). We provide the bound on the  $\mathcal{P} \to \mathcal{Q}$  safely-reliable correctness of this example in Appendix J (see Theorem J.2).
7 Reliability in the agnostic setting
In the above, we have assumed that the training samples  $S$  are realizable under our concept class  $\mathcal{H}$ , i.e. there is a target concept  $h^*$  consistent with our (uncorrupted) data. In the agnostic setting, we can have  $\min_{h\in \mathcal{H}}\mathrm{err}_S(h) > 0$ , meaning no single concept is always correct. We define a  $\nu$ -tolerably robustly-reliable learner under test-time attacks in the agnostic setting as the learner whose reliable predictions agree with every low-error hypothesis (error at most  $\nu$ ) on the training sample ([BBHS22] have proposed the corresponding definition for data poisoning attacks).
Definition 11 ( $\nu$ -tolerably robustly-reliable learner w.r.t.  $\mathcal{M}$ -ball attacks). A learner  $\mathcal{L}$  is robustly-reliable w.r.t.  $\mathcal{M}$ -ball attacks for sample  $S$ , hypothesis space  $\mathcal{H}$  and robust loss function  $\ell$  if, for every concept  $h^* \in \mathcal{H}$  with  $\mathrm{err}_S(h^*) \leq \nu$ , the learner outputs functions  $h_S^\ell : \mathcal{X} \to \mathcal{Y}$  and  $r_S^\ell : \mathcal{X} \to [0, \infty) \cup \{-1\}$  such that for all  $x, z \in \mathcal{X}$  if  $r_S^\ell(z) = \eta > 0$  and  $z \in \mathbf{B}_{\mathcal{M}}^o(x, \eta)$  then  $\ell^{h^*}(h_S^\ell, x, z) = 0$ . Further, if  $r_S^\ell(z) = 0$ , then  $h^*(z) = h_S^\ell(z)$ . Given sample  $S$  such that some concept  $h^* \in \mathcal{H}$  satisfies  $\mathrm{err}_S(h^*) \leq \nu$ , the robustly-reliable region of  $\mathcal{L}$  is defined as  $RR^\ell(S, \nu, \eta) = \{x \in \mathcal{X} \mid r_S^\ell(x) \geq \eta\}$  for  $\nu, \eta \geq 0$ .
We generalize our results from Section 3 to the agnostic setting (proof details in Appendix K). Here  $\mathcal{H}_{\nu}(S) = \{h \in \mathcal{H} \mid \mathrm{err}_S(h) \leq \nu\}$ .
Theorem 7.1. Let  $\mathcal{H}$  be any hypothesis class. With respect to  $\mathcal{M}$ -ball attacks and  $\ell_{CA}$ , for  $\eta \geq 0$ ,
(a) There exists a robustly-reliable learner  $\mathcal{L}$  such that  $RR_{CA}^{\mathcal{L}}(S,\nu ,\eta)\supseteq \mathrm{Agree}(\mathcal{H}_{\nu}(S))$
(b) For any robustly-reliable learner  $\mathcal{L}$ ,  $RR_{CA}^{\mathcal{L}}(S,\nu ,\eta)\subseteq \mathrm{Agree}(\mathcal{H}_{\nu}(S))$
In this work, we generalize the classical line of works on reliable learning to address challenging test-time environments. We propose a novel robustly-reliability criterion that is applicable to several variations of test-time attacks. Our analysis leads to an easy-to-derive algorithm that can be implemented efficiently in many cases. Additionally, we introduce a  $\mathcal{P} \to \mathcal{Q}$  disagreement coefficient to capture the transferability of the reliability guarantee between distributions. The proposed robustly-reliability criterion and the  $\mathcal{P} \to \mathcal{Q}$  disagreement coefficient together provide a comprehensive framework for handling test-time attacks and evaluating the reliability of learning models. This contributes to the advancement of reliable learning methodologies in the face of challenging real-world scenarios, facilitating the development of more resilient and trustworthy machine learning systems. Notably, key questions remain open, including, how to efficiently implement the algorithm for a class of neural networks, and how to learn reliably with respect to any general robust loss function?
This work was supported in part by NSF grants CCF-1910321 and SES-1919453, the Defense Advanced Research Projects Agency under cooperative agreement HR00112020003, a Bloomberg Data Science PhD fellowship, and a Simons Investigator Award.
[AB99] Martin Anthony and Peter L Bartlett. Neural network learning: Theoretical foundations. Cambridge University Press, 1999.
[ABGLP19] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019.
[ABHU15] Pranjal Awasthi, Maria-Florina Balcan, Nika Haghtalab, and Ruth Urner. Efficient learning of linear separators under bounded noise. In Conference on Learning Theory, pages 167–190. PMLR, 2015.
[ABL14] Pranjal Awasthi, Maria Florina Balcan, and Philip M Long. The power of localization for efficiently learning linear separators with noise. In Proceedings of the forty-sixth annual ACM Symposium on Theory of Computing, pages 449-458, 2014.
[AK91] David Applegate and Ravi Kannan. Sampling and integration of near log-concave functions. In Proceedings of the twenty-third annual ACM Symposium on Theory of Computing, pages 156-163, 1991.
[AKM19] Idan Attias, Aryeh Kontorovich, and Yishay Mansour. Improved generalization bounds for robust learning. In Algorithmic Learning Theory, pages 162-183. PMLR, 2019.
[BBHS22] Maria-Florina Balcan, Avrim Blum, Steve Hanneke, and Dravyansh Sharma. Robustly-reliable learners under poisoning attacks. In Conference on Learning Theory. PMLR, 2022.
[BBL06] Maria-Florina Balcan, Alina Beygelzimer, and John Langford. Agnostic active learning. In Proceedings of the 23rd International Conference on Machine Learning, pages 65–72, 2006.
[BBS07] Steffen Bickel, Michael Brückner, and Tobias Scheffer. Discriminative learning for differing training and test distributions. In Proceedings of the 24th International Conference on Machine Learning, pages 81-88, 2007.
[BBSZ23] Maria-Florina Balcan, Avrim Blum, Dravyansh Sharma, and Hongyang Zhang. On the power of abstention and data-driven decision making for adversarial robustness. Journal of Machine Learning Research, 2023.
[BCKW15] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International Conference on Machine Learning, pages 1613-1622. PMLR, 2015.
$\left[\mathrm{BDBC}^{+}10\right]$  Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79:151-175, 2010.
[BDBCP06] Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. Advances in Neural Information Processing Systems, 19, 2006.
[BH21] Maria-Florina Balcan and Nika Haghtalab. Noise in classification. Beyond the Worst-Case Analysis of Algorithms, page 361, 2021.
[BL13] Maria-Florina Balcan and Phil Long. Active and passive learning of linear separators under log-concave distributions. In Conference on Learning Theory, pages 288–316. PMLR, 2013.
[BPRZ23] Maria-Florina Balcan, Rattana Pukdee, Pradeep Ravikumar, and Hongyang Zhang. Nash equilibria and pitfalls of adversarial training in adversarial robustness games. In The 26nd International Conference on Artificial Intelligence and Statistics. PMLR, 2023.