SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\max  _ {\mu_ {p} \in \mathcal {Q} (u)} \left[ L \left(h _ {1, b _ {p}}; \mu_ {c}\right) + \epsilon \cdot L \left(h _ {1, b _ {p}}; \mu_ {p}\right) \right], \tag {10}
$$
where  $b_{p} = \mathrm{argmin}_{b\in \mathbb{R}}[L(h_{1,b};\mu_{c}) + \epsilon \cdot L(h_{1,b};\mu_{p})]$ . There exists some  $\alpha \in [0,1]$  such that the optimal value of (10) is achieved with  $\mu_p = \nu_\alpha$ , where  $\nu_{\alpha}$  is a two-point distribution with some parameter  $\alpha \in [0,1]$  defined according to (3).
Lemma B.1 suggests that it is sufficient to study the extreme two-point distributions  $\nu_{\alpha}$  with  $\alpha \in [0,1]$  to understand the maximum achievable population hinge loss conditioned on  $w = 1$ . Lemma B.2, proven in Appendix C.2, characterizes the sufficient and necessary conditions in terms of  $\epsilon$ ,  $u$  and  $\mu_c$ , under which there exists a linear hypothesis with  $w = -1$  that achieves the minimal value of population hinge loss with respect to  $\mu_c$  and some  $\mu_p$ .
Lemma B.2. Suppose the underlying clean distribution  $\mu_c$  follows the Gaussian mixture model (2) with  $p = 1/2$  and  $\sigma_1 = \sigma_2 = \sigma$ . Assume  $|\gamma_1 + \gamma_2| \leq 2(u - 1)$  for some  $u \geq 1$ . Let  $g$  be an auxiliary function such that for any  $b \in \mathbb{R}$ ,
where  $\Phi$  is the CDF of standard Gaussian. For any  $\epsilon > 0$ , there exists some  $\mu_p \in \mathcal{Q}(u)$  such that  $\operatorname{argmin}_{h_{w,b} \in \mathcal{H}_L} [L(h_{w,b}; \mu_c) + \epsilon \cdot L(h_{w,b}; \mu_p)]$  outputs a hypothesis with  $w = -1$ , if and only if
$$
\max  \{\Delta (- \epsilon), \Delta (g (0)), \Delta (\epsilon) \} \geq 0,
$$
where  $\Delta(s) = L(h_{1,g^{-1}(s)}; \mu_c) - \min_{b \in \mathbb{R}} L(h_{-1,b}; \mu_c) + \epsilon (1 + u) - s \cdot g^{-1}(s)$ , and  $g^{-1}$  denotes the inverse of  $g$ .
Lemma B.2 identifies sufficient and necessary conditions when a linear hypothesis with flipped weight parameter is possible. Note that we assume  $\gamma_{1} \leq \gamma_{2}$ , thus flipping the weight parameter of the induced model from  $w = 1$  to  $w = -1$  is always favorable from an attacker's perspective. In particular, if the population hinge loss with respect to  $\mu_{c}$  and some  $\mu_{p}$  achieved by the loss minimizer conditioned on  $w = 1$  is higher than that achieved by the loss minimizer with  $w = -1$ , then we immediately know that flipping the weight parameter is possible, which further suggests the optimal poisoning attack performance must be achieved by some poisoned victim model with  $w = -1$ .
Finally, we introduce Lemma B.3, proven in Appendix C.3, which characterizes the risk behavior of any linear hypothesis with respect to the assumed Gaussian mixture model (2).
Lemma B.3. Let  $\mu_c$  be the clean data distribution, where each example is sampled i.i.d. according to the data generating process specified in (2). For any linear hypothesis  $h_{w,b} \in \mathcal{H}_{\mathrm{L}}$ , we have
$$
\operatorname {R i s k} \left(h _ {w, b}; \mu_ {c}\right) = p \cdot \Phi \left(\frac {b + w \cdot \gamma_ {1}}{\sigma_ {1}}\right) + (1 - p) \cdot \Phi \left(\frac {- b - w \cdot \gamma_ {2}}{\sigma_ {2}}\right),
$$
where  $\Phi$  denotes the CDF of standard Gaussian distribution  $\mathcal{N}(0,1)$ .
Now we are ready to prove Theorem 5.3 using Lemmas B.1, B.2 and B.3.
Proof of Theorem 5.3. According to Theorem 4.5 and Remark 4.6, we note that the optimal poisoning performance in Definition 4.2 is always achieved with  $\delta = \epsilon$ . Therefore, we will only consider  $\delta = \epsilon$  in the following discussions.
Since the optimal poisoning performance is defined with respect to clean risk, it will be useful to understand the properties of  $\operatorname{Risk}(h_{w,b};\mu_c)$  such as monotonicity and range. According to Lemma B.3, for any  $h_{w,b}\in \mathcal{H}_{\mathrm{L}}$ , we have
$$
\mathrm {R i s k} (h _ {w, b}; \mu_ {c}) = \frac {1}{2} \Phi \left(\frac {b + w \cdot \gamma_ {1}}{\sigma}\right) + \frac {1}{2} \Phi \left(\frac {- b - w \cdot \gamma_ {2}}{\sigma}\right).
$$
To understand the monotonicity of risk, we compute its derivative with respect to  $b$ :
$$
\frac {\partial}{\partial b} \mathrm {R i s k} (h _ {w, b}; \mu_ {c}) = \frac {1}{2 \sigma \sqrt {2 \pi}} \bigg [ \exp \left(- \frac {(b + w \cdot \gamma_ {1}) ^ {2}}{2 \sigma^ {2}}\right) - \exp \left(- \frac {(b + w \cdot \gamma_ {2}) ^ {2}}{2 \sigma^ {2}}\right) \bigg ].
$$
If  $w = 1$ , then  $\operatorname{Risk}(h_{w,b};\mu_c)$  is monotonically decreasing when  $b \in (-\infty, -\frac{\gamma_1 + \gamma_2}{2})$  and monotonically increasing when  $b \in (-\frac{\gamma_1 + \gamma_2}{2}, \infty)$ , suggesting that minimum is achieved at  $b = -\frac{\gamma_1 + \gamma_2}{2}$  and maximum is achieved when  $b$  goes to infinity. To be more specific,  $\operatorname{Risk}(h_{1,b};\mu_c) \in [\Phi(\frac{\gamma_1 - \gamma_2}{2\sigma}), \frac{1}{2}]$ . On the other hand, if  $w = -1$ , then  $\operatorname{Risk}(h_{w,b};\mu_c)$  is monotonically increasing when  $b \in (-\infty, \frac{\gamma_1 + \gamma_2}{2})$  and monotonically decreasing when  $b \in (\frac{\gamma_1 + \gamma_2}{2}, \infty)$ , suggesting that maximum is achieved at  $b = \frac{\gamma_1 + \gamma_2}{2}$  and minimum is achieved when  $b$  goes to infinity. Thus,  $\operatorname{Risk}(h_{-1,b};\mu_c) \in [\frac{1}{2}, \Phi(\frac{\gamma_2 - \gamma_1}{2\sigma})]$ .
Based on the monotonicity analysis of  $\operatorname{Risk}(h_{w,b};\mu_c)$ , we have the following two observations:
1. If there exists some feasible  $\mu_p$  such that  $h_{-1,b_p} = \operatorname*{argmin}_{h\in \mathcal{H}_L}\{L(h;\mu_c) + \epsilon L(h;\mu_p)\}$  can be achieved, then the optimal poisoning performance is achieved with  $w = -1$  and  $b$  close to  $\frac{\gamma_1 + \gamma_2}{2}$  as much as possible.
2. If there does not exist any feasible  $\mu_p$  that induces  $h_{-1,b_p}$  by minimizing the population hinge loss, then the optimal poisoning performance is achieved with  $w = 1$  and  $b$  far from  $-\frac{\gamma_1 + \gamma_2}{2}$  as much as possible (conditioned that the variance  $\sigma$  is the same for the two classes).
Recall that we prove in Lemma B.2 specifies a sufficient and necessary condition for the existence of such  $h_{-1,b_p}$ , which is equivalent to the condition (4) presented in Lemma B.2. Note that according to Lemma C.1,  $b = \frac{\gamma_1 + \gamma_2}{2}$  also yields the population loss minimizer with respect to  $\mu_c$  conditioned on  $w = -1$ . Thus, if condition (4) is satisfied, then we know there exists some  $\alpha \in [0,1]$  such that the optimal poisoning performance can be achieved with  $\mu_p = \nu_\alpha$ . This follows from the assumption  $|\gamma_1 + \gamma_2| \leq 2(u - 1)$ , which suggests that for any  $(x,y) \sim \nu_\alpha$ , the individual hinge loss at  $(x,y)$  will be zero. In addition, we know that the poisoned hypothesis induced by  $A_{\mathrm{opt}}$  is  $h_{-1,\frac{\gamma_1 + \gamma_2}{2}}$ , which maximizes risk with respect to  $\mu_c$ .
On the other hand, if condition (4) is not satisfied, we know that the poisoned hypothesis induced by any feasible  $\mu_p$  has weight parameter  $w = 1$ . Based on our second observation, this further suggests that the optimal poisoning performance will always be achieved with either  $\mu_p = \nu_0$  or  $\mu_p = \nu_1$ . According to the first-order optimality condition and Lemma C.1, we can compute the closed-form solution regarding the optimal poisoning performance. Thus, we complete the proof.
B.2 Proof of Theorem 5.7
Proof of Theorem 5.7. Consider linear hypothesis class  $\mathcal{H}$  and the poisoned distribution  $\mu_p^*$  generated by the optimal poisoning adversary  $\mathcal{A}_{\mathrm{opt}}$  in Definition 4.1. Given clean distribution  $\mu_c$ , poisoning ratio  $\epsilon$  and constraint set  $\mathcal{C}$ , the inherent vulnerability to indiscriminate poisoning is captured by the optimal attack performance  $\operatorname{Risk}(h_p^*;\mu_c)$ , where  $h_p^*$  denotes the poisoned linear model induced by  $\mu_p^*$ . For any  $h\in \mathcal{H}$ , we have
$$
\operatorname {R i s k} \left(h _ {p} ^ {*}; \mu_ {c}\right) \leq L \left(h _ {p} ^ {*}; \mu_ {c}\right) \leq L \left(h _ {p} ^ {*}; \mu_ {c}\right) + \epsilon \cdot L \left(h _ {p} ^ {*}; \mu_ {p} ^ {*}\right) \leq L (h; \mu_ {c}) + \epsilon \cdot L (h; \mu_ {p} ^ {*}) \tag {11}
$$
where the first inequality holds because the surrogate loss is defined to be not smaller than the 0-1 loss, the second inequality holds because the surrogate loss is always non-negative, and the third inequality holds because  $h_p^*$  minimizes the population loss with respect to both clean distribution
$\mu$  and optimally generated poisoned distribution  $\mu_p^*$ . Consider  $h_c = \operatorname{argmin}_{h \in \mathcal{H}} L(h; \mu_c)$  (with weight parameter  $w_c$  and bias parameter  $b_c$ ), which is the linear model learned from the clean data. Therefore, plugging  $h = h_c$  into the right hand side of (11), we further obtain
$$
\operatorname {R i s k} \left(h _ {p} ^ {*}; \mu_ {c}\right) \leq L \left(h _ {c}; \mu_ {c}\right) + \epsilon \cdot L \left(h _ {c}; \mu_ {p} ^ {*}\right) \leq L \left(h _ {c}; \mu_ {c}\right) + \epsilon \cdot \ell_ {\mathrm {M}} \left(\operatorname {S i z e} _ {\boldsymbol {w} _ {c}} (\mathcal {C})\right), \tag {12}
$$
where the last inequality holds because for any poisoned data point  $(\pmb{x},y)\sim \mu_p^*$ , the surrogate loss at  $(\pmb{x},y)$  with respect to  $h_c$  is  $\ell_{\mathrm{M}}\big(y\cdot (\pmb{w}_c^\top \pmb {x} + b_c)\big)$ , and  $y\cdot (\pmb{w}_c^\top \pmb {x} + b_c)\leq \max_{(\pmb {x},y)\in \mathcal{C}}|\pmb{w}_c^\top \pmb {x} + b_c|$ . Under the condition that  $\min_{(\pmb {x},y)\in \mathcal{C}}\pmb{w}_{c}^{\top}\pmb {x}\leq -b_{c}\leq \max_{(\pmb {x},y)\in \mathcal{C}}\pmb{w}_{c}^{\top}\pmb {x}$  which means the decision boundary of  $h_c$  falls into the constraint set  $\mathcal{C}$  when projected on to the direction of  $\pmb{w}_c$ , we further have  $\max_{(\pmb {x},y)\in \mathcal{C}}|\pmb{w}_c^\top \pmb {x} + b_c|\leq \operatorname {Size}_{\pmb{w}_c}(\mathcal{C})$ , which implies the validity of (12). We remark that the condition  $\min_{(\pmb {x},y)\in \mathcal{C}}\pmb{w}_{c}^{\top}\pmb {x}\leq -b_{c}\leq \max_{(\pmb {x},y)\in \mathcal{C}}\pmb{w}_{c}^{\top}\pmb {x}$  typically holds for margin-based loss in practice, since the support of the clean training data belongs to the constraint set for poisoning inputs (for either undefended victim models or models that employ some unsupervised data sanitization defense). Therefore, we leave this condition out in the statement of Theorem 5.7 for simplicity.
C Proofs of Technical Lemmas used in Appendix B.1
C.1 Proof of Lemma B.1
To prove Lemma B.1, we need to make use of the following general lemma which characterizes the population hinge loss and its derivative with respect to clean data distribution  $\mu_c$ . For the sake of completeness, we provide the proof of Lemma C.1 in Appendix C.4.
Lemma C.1. Let  $\mu_c$  be data distribution generated according to (2). For any  $h_{w,b} \in \mathcal{H}_{\mathrm{L}}$ , the population hinge loss is:
$$
\begin{array}{l} L(h_{w,b};\mu_{c}) = p\int_{\substack{-b - w\cdot \gamma_{1} - 1\\ \sigma_{1}}}^{\infty}\left(b + w\cdot \gamma_{1} + 1 + \sigma_{1}z\right)\cdot \varphi (z)dz \\ + (1 - p) \int_ {- \infty} ^ {\frac {- b - w \cdot \gamma_ {2} + 1}{\sigma_ {2}}} (- b - w \cdot \gamma_ {2} + 1 - \sigma_ {2} z) \cdot \varphi (z) d z, \\ \end{array}
$$
and its gradient with respect to  $b$  is:
$$
\frac {\partial}{\partial b} L \left(h _ {w, b}; \mu_ {c}\right) = p \cdot \Phi \left(\frac {b + w \cdot \gamma_ {1} + 1}{\sigma_ {1}}\right) - (1 - p) \cdot \Phi \left(\frac {- b - w \cdot \gamma_ {2} + 1}{\sigma_ {2}}\right),
$$
where  $\varphi$  and  $\Phi$  denote the PDF and CDF of standard Gaussian distribution  $\mathcal{N}(0,1)$ , respectively.
Next, let us summarize several key observations based on Lemma C.1 (specifically for the setting considered in Lemma B.1). For any  $w \in \{-1, 1\}$ ,  $\frac{\partial}{\partial b} L(h_{w,b}; \mu_c)$  is a monotonically increasing with  $b$ , which achieves minimum  $-\frac{1}{2}$  when  $b$  goes to  $-\infty$  and achieves maximum  $\frac{1}{2}$  when  $b$  goes to  $\infty$ . If  $w = +1$ , then  $L(h_{w,b}; \mu_c)$  is monotonically decreasing when  $b \in (-\infty, -\frac{\gamma_1 + \gamma_2}{2})$  and monotonically increasing when  $b \in (-\frac{\gamma_1 + \gamma_2}{2}, \infty)$ , reaching the minimum at  $b = b_c^*(1) := -\frac{\gamma_1 + \gamma_2}{2}$ . On the other hand, if  $w = -1$ , then  $L(h_{w,b}; \mu_c)$  is monotonically decreasing when  $b \in (-\infty, \frac{\gamma_1 + \gamma_2}{2})$  and monotonically increasing when  $b \in (\frac{\gamma_1 + \gamma_2}{2}, \infty)$ , reaching the minimum at  $b = b_c^*(-1) := \frac{\gamma_1 + \gamma_2}{2}$ .
As for the clean loss minimizer conditioned on  $w = 1$ , we have
$$
\begin{array}{l} L (h _ {1, b _ {c} ^ {*} (1)}; \mu_ {c}) = \frac {1}{2} \int_ {\frac {\gamma_ {2} - \gamma_ {1} - 2}{2 \sigma}} ^ {\infty} \left(\frac {\gamma_ {1} - \gamma_ {2}}{2} + 1 + \sigma z\right) \cdot \varphi (z) d z \\ + \frac {1}{2} \int_ {- \infty} ^ {\frac {\gamma_ {1} - \gamma_ {2} + 2}{2 \sigma}} \left(\frac {\gamma_ {1} - \gamma_ {2}}{2} + 1 - \sigma z\right) \cdot \varphi (z) d z \\ = \frac {(\gamma_ {1} - \gamma_ {2} + 2)}{2} \cdot \Phi \left(\frac {\gamma_ {1} - \gamma_ {2} + 2}{2 \sigma}\right) + \frac {\sigma}{\sqrt {2 \pi}} \cdot \exp \left(- \frac {(\gamma_ {1} - \gamma_ {2} + 2) ^ {2}}{8 \sigma^ {2}}\right), \\ \end{array}
$$
whereas as for the clean loss minimizer conditioned on  $w = -1$ , we have
$$
\begin{array}{l} L (h _ {- 1, b _ {c} ^ {*} (- 1)}; \mu_ {c}) = \frac {1}{2} \int_ {\frac {\gamma_ {1} - \gamma_ {2} - 2}{2 \sigma}} ^ {\infty} \left(\frac {\gamma_ {2} - \gamma_ {1}}{2} + 1 + \sigma z\right) \cdot \varphi (z) d z \\ + \frac {1}{2} \int_ {- \infty} ^ {\frac {\gamma_ {2} - \gamma_ {1} + 2}{2 \sigma}} \left(\frac {\gamma_ {2} - \gamma_ {1}}{2} + 1 - \sigma z\right) \cdot \varphi (z) d z \\ = \frac {(\gamma_ {2} - \gamma_ {1} + 2)}{2} \cdot \Phi \left(\frac {\gamma_ {2} - \gamma_ {1} + 2}{2 \sigma}\right) + \frac {\sigma}{\sqrt {2 \pi}} \cdot \exp \left(- \frac {(\gamma_ {2} - \gamma_ {1} + 2) ^ {2}}{8 \sigma^ {2}}\right). \\ \end{array}
$$
Let  $f(t) = t \cdot \Phi\left(\frac{t}{\sigma}\right) + \frac{\sigma}{\sqrt{2\pi}} \cdot \exp\left(-\frac{t^2}{2\sigma^2}\right)$ , we know  $L(h_{1,b_c^*}(1);\mu_c) = f\left(\frac{\gamma_1 - \gamma_2 + 2}{2}\right)$  and  $L(h_{-1,b_c^*}(-1);\mu_c) = f\left(\frac{\gamma_2 - \gamma_1 + 2}{2}\right)$ . We can compute the derivative of  $f(t): f'(t) = \Phi\left(\frac{t}{\sigma}\right) \geq 0$ , which suggests that  $L(h_{1,b_c^*}(1);\mu_c) \leq L(h_{-1,b_c^*}(-1);\mu_c)$ .
Now we are ready to prove Lemma B.1.
Proof of Lemma B.1. First, we prove the following claim: for any possible  $b_{p}$ , linear hypothesis  $h_{1,b_p}$  can always be achieved by minimizing the population hinge loss with respect to  $\mu_c$  and  $\mu_p = \nu_\alpha$  with some carefully-chosen  $\alpha \in [0,1]$  based on  $b_{p}$ .
For any  $\mu_p\in \mathcal{Q}(u)$ , according to the first-order optimality condition with respect to  $b_{p}$ , we have
$$
\frac {\partial}{\partial b} L \left(h _ {1, b _ {p}}; \mu_ {c}\right) = - \epsilon \cdot \frac {\partial}{\partial b} L \left(h _ {1, b _ {p}}; \mu_ {p}\right) = - \epsilon \cdot \frac {\partial}{\partial b} \mathbb {E} _ {(x, y) \sim \mu_ {p}} \left[ \ell \left(h _ {1, b _ {p}}; \mu_ {p}\right) \right] \in [ - \epsilon , \epsilon ], \tag {13}
$$
where the last inequality follows from  $\frac{\partial}{\partial b}\ell (h_{w,b};x,y)\in [-1,1]$  for any  $(x,y)$ . Let  $\mathcal{B}_p$  be the set of any possible bias parameters  $b_{p}$ . According to (13), we have
$$
\mathcal {B} _ {p} = \left\{b \in \mathbb {R}: \frac {\partial}{\partial b} L \left(h _ {1, b}; \mu_ {c}\right) \in [ - \epsilon , \epsilon ] \right\}.
$$
Let  $b_{c}^{*}(1) = \operatorname{argmin}_{b\in \mathbb{R}}L(h_{1,b};\mu_{c})$  be the clean loss minimizer conditioned on  $w = 1$ . According to Lemma C.1 and the assumption  $|\gamma_1 + \gamma_2|\leq 2u$ , we know  $b_{c}^{*}(1) = \frac{\gamma_{1} + \gamma_{2}}{2}\in [-u,u]$ . For any  $b_{p}\in \mathcal{B}_{p}$ , we can always choose
$$
\alpha = \frac {1}{2} + \frac {1}{2 \epsilon} \cdot \frac {\partial}{\partial b} L \left(h _ {1, b _ {p}}; \mu_ {c}\right) \in [ 0, 1 ], \tag {14}
$$
$$
h_{1,b_{p}} = \operatorname *{argmin}_{b\in \mathbb{R}}[L(h_{1,b};\mu_{c}) + \epsilon \cdot L(h_{1,b};\nu_{\alpha})],
$$
where  $\nu_{\alpha}$  is defined according to (3). This follows from the first-order optimality condition for convex function and the closed-form solution for the derivative of hinge loss with respect to  $\nu_{\alpha}$ :
$$
\frac {\partial}{\partial b} L (h _ {1, b _ {p}}; \nu_ {\alpha}) = \alpha \cdot \frac {\partial}{\partial b} \ell (h _ {+ 1, b _ {p}}; - u, + 1) + (1 - \alpha) \cdot \frac {\partial}{\partial b} \ell (h _ {+ 1, b _ {p}}; u, - 1) = 1 - 2 \alpha .
$$
Thus, we have proven the claimed presented at the beginning of the proof of Lemma B.1.
Next, we show that for any  $b_{p} \in \mathcal{B}_{p}$ , among all the possible choices of poisoned distribution  $\mu_{p}$  that induces  $b_{p}$ , choosing  $\mu_{p} = \nu_{\alpha}$  with  $\alpha$  defined according to (14) is the optimal choice in terms of the maximization objective in (10). Let  $\mu_{p} \in \mathcal{Q}(u)$  be any poisoned distribution that satisfies the following condition:
$$
b _ {p} = \operatorname * {a r g m i n} _ {b \in \mathbb {R}} [ L (h _ {1, b}; \mu_ {c}) + \epsilon \cdot L (h _ {1, b}; \mu_ {p}) ].
$$
According to the aforementioned analysis, we know that by setting  $\alpha$  according to (14),  $\nu_{\alpha}$  also yields  $b_{p}$ . Namely,
$$
b_{p} = \operatorname *{argmin}_{b\in \mathbb{R}}[L(h_{1,b};\mu_{c}) + \epsilon \cdot L(h_{1,b};\nu_{\alpha})].
$$
Since the population losses with respect to  $\mu_c$  are the same at the induced bias  $b = b_p$ , it remains to prove  $\nu_{\alpha}$  achieves a larger population loss with respect to the poisoned distribution than that of  $\mu_p$ , i.e.,  $L(h_{1,b_p};\nu_\alpha)\geq L(h_{1,b_p};\mu_p)$ .
Consider the following two probabilities with respect to  $b_{p}$  and  $\mu_p$ :
$$
p _ {1} = \mathbb {P} _ {(x, y) \sim \mu_ {p}} \left[ \frac {\partial}{\partial b} \ell \left(h _ {1, b _ {p}}; x, y\right) = - 1 \right], \quad p _ {2} = \mathbb {P} _ {(x, y) \sim \mu_ {p}} \left[ \frac {\partial}{\partial b} \ell \left(h _ {1, b _ {p}}; x, y\right) = 1 \right].
$$
Note that the derivative of hinge loss with respect to the bias parameter is  $\frac{\partial}{\partial b}\ell (h_{w,b};x,y)\in$ $\{-1,0,1\}$ , thus we have
$$
\mathbb {P} _ {(x, y) \sim \mu_ {p}} \left[ \frac {\partial}{\partial b} \ell (h _ {1, b _ {p}}; x, y) = 0 \right] = 1 - (p _ {1} + p _ {2}).
$$
Moreover, according to the first-order optimality of  $b_{p}$  with respect to  $\mu_p$ , we have
$$
\frac {\partial}{\partial b} L (h _ {1, b _ {p}}; \mu_ {c}) = - \epsilon \cdot \frac {\partial}{\partial b} L (h _ {1, b _ {p}}; \mu_ {p}) = \epsilon \cdot (p _ {1} - p _ {2}),
$$
If we measure the sum of the probability of input having negative gradient and half of the probability of having zero gradient, we have:
$$
p _ {1} + \frac {1 - \left(p _ {1} + p _ {2}\right)}{2} = \frac {1}{2} + \frac {p _ {1} - p _ {2}}{2} = \frac {1}{2} + \frac {1}{2 \epsilon} \cdot \frac {\partial}{\partial b} L \left(h _ {1, b _ {p}}; \mu_ {c}\right) = \alpha .
$$
Therefore, we can construct a mapping  $g$  that maps  $\mu_p$  to  $\nu_\alpha$ : by moving any  $(x,y) \sim \mu_p$  that contributes  $p_1$  (negative derivative) and any  $(x,y) \sim \mu_p$  that contributes  $p_2$  (positive derivative) to extreme locations  $(-u, +1)$  and  $(u, -1)$ , respectively, and move the remaining  $(x,y)$  that has zero derivative to  $(-u, +1)$  and  $(u, -1)$  with equal probabilities (i.e.,  $\frac{1 - p_1 - p_2}{2}$ ), and we can easily verify that the gradient of  $b_p$  with respect to  $\mu_p$  is the same as  $\nu_\alpha$ .
In addition, note that hinge loss is monotonically increasing with respect to the  $\ell_2$  distance of misclassified examples to the decision hyperplane, and the initial clean loss minimizer  $b_c^*(1) \in [-u, u]$ , we can verify that the constructed mapping  $g$  will not reduce the individual hinge loss. Namely,  $\ell(h_{1,b_p}; x,y) \leq \ell(h_{1,b_p}; g(x,y))$  holds for any  $(x,y) \sim \mu_p$ . Therefore, we have proven Lemma B.1.
C.2 Proof of Lemma B.2
Proof of Lemma B.2. First, we introduce the following notations. For any  $\mu_p \in \mathcal{Q}(u)$  and any  $w \in \{-1, 1\}$ , let
$$
b_{c}^{*}(w) = \operatorname *{argmin}_{b\in \mathbb{R}}L(h_{w,b};\mu_{c}),\quad b_{p}(w;\mu_{p}) = \operatorname *{argmin}_{b\in \mathbb{R}}[L(h_{w,b};\mu_{c}) + \epsilon \cdot L(h_{w,b};\mu_{p})].
$$
According to Lemma B.1, we know that the maximum population hinge loss conditioned on  $w = 1$  is achieved when  $\mu_p = \nu_\alpha$  for some  $\alpha \in [0,1]$ . To prove the sufficient and necessary condition specified in Lemma B.2, we also need to consider  $w = -1$ . Note that different from  $w = 1$ , we want to specify the minimum loss that can be achieved with some  $\mu_p$  for  $w = -1$ . For any  $\mu_p \in \mathcal{Q}(u)$ , we have
$$
L \left(h _ {- 1, b _ {p} (- 1; \mu_ {p})}; \mu_ {c}\right) + \epsilon \cdot L \left(h _ {- 1, b _ {p} (- 1; \mu_ {p})}; \mu_ {p}\right) \geq \min  _ {b \in \mathbb {R}} L \left(h _ {- 1, b}; \mu_ {c}\right) = L \left(h _ {- 1, b _ {c} ^ {*} (- 1)}; \mu_ {c}\right). \tag {15}
$$
According to Lemma C.1, we know  $b_{c}^{*}(-1) = \frac{\gamma_{1} + \gamma_{2}}{2}$ , which achieves the minimum clean loss conditioned on  $w = -1$ . Since we assume  $\frac{\gamma_1 + \gamma_2}{2} \in [-u + 1,u - 1]$ , according to the first-order optimality condition, the equality in (15) can be attained as long as  $\mu_p$  only consists of correctly classified data that also incurs zero hinge loss with respect to  $b_{c}^{*}(-1)$  (not all correctly classified instances incur zero hinge loss). It can be easily checked that choosing  $\mu_p = \nu_\alpha$  based on (3) with any  $\alpha \in [0,1]$  satisfies this condition, which suggests that as long as the poisoned distribution  $\mu_p$  is given in the form of  $\nu_{\alpha}$  and if the  $w = -1$  is achievable (conditions on when this can be achieved will be discussed shortly), then the bias term that minimizes the distributional loss is equal to  $b_{c}^{*}(-1)$ , and is the minimum compared to other choices of  $b_{p}(-1;\mu_{p})$ . According to Lemma B.1, it further implies the following statement: there exists some  $\alpha \in [0,1]$  such that
$$
\begin{array}{l} \nu_ {\alpha} \in \operatorname * {a r g m a x} _ {\mu_ {p} \in \mathcal {Q} (u)} \left\{[ L (h _ {1, b _ {p} (1; \mu_ {p})}; \mu_ {c}) + \epsilon \cdot L (h _ {1, b _ {p} (1; \mu_ {p})}; \mu_ {p}) ] \right. \\ \left. \right.\left. - \left[ L (h _ {- 1, b _ {p} (- 1; \mu_ {p})}; \mu_ {c}) + \epsilon \cdot L (h _ {- 1, b _ {p} (- 1; \mu_ {p})}; \mu_ {p}) \right]\right\}. \\ \end{array}
$$
For simplicity, let us denote by  $\Delta L(\mu_p;\epsilon ,u,\mu_c)$  the maximization objective regarding the population loss difference between  $w = 1$  and  $w = -1$ . Thus, a necessary and sufficient condition such that there exists a  $h_{-1,b_p(-1;\mu_p)}$  as the loss minimizer is that  $\max_{\alpha \in [0,1]}\Delta L(\nu_{\alpha};\epsilon ,u,\mu_{c})\geq 0$ . This requires us to characterize the maximal value of loss difference for any possible configurations of  $\epsilon ,u$  and  $\mu_c$ . According to Lemma C.1 and the definition of  $\nu_{\alpha}$ , for any  $\alpha \in [0,1]$ , we denote the above loss difference as
$$
\Delta L (\nu_ {\alpha}; \epsilon , u, \mu_ {c}) = \underbrace {L (h _ {1 , b _ {p} (1 ; \nu_ {\alpha})} ; \mu_ {c}) + \epsilon \cdot L (h _ {1 , b _ {p} (1 ; \nu_ {\alpha})} ; \nu_ {\alpha})} _ {I _ {1}} - \underbrace {L (h _ {- 1 , b _ {c} ^ {*} (- 1)} ; \mu_ {c})} _ {I _ {2}}.
$$