SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The second term  $I_{2}$  is fixed (and the loss on  $\nu_{\alpha}$  is zero conditioned on  $w = -1$ ), thus it remains to characterize the maximum value of  $I_{1}$  with respect to  $\alpha$  for different configurations. Consider auxiliary function
$$
g (b) = \frac {1}{2} \Phi \left(\frac {b + \gamma_ {1} + 1}{\sigma}\right) - \frac {1}{2} \Phi \left(\frac {- b - \gamma_ {2} + 1}{\sigma}\right).
$$
We know  $g(b) \in [-\frac{1}{2}, \frac{1}{2}]$  is a monotonically increasing function by checking with derivative to  $b$ . Let  $g^{-1}$  be the inverse function of  $g$ . Note that according to Lemma C.1 and the first-order optimality condition of  $b_{p}(1; \nu_{\alpha})$ , we have
$$
\left. \frac {\partial}{\partial b} L \left(h _ {+ 1, b}; \mu_ {c}\right) \right| _ {b = b _ {p} (1; \nu_ {\alpha})} = g \left(b _ {p} (+ 1; \nu_ {\alpha})\right) = - \epsilon \cdot \frac {\partial}{\partial b} L \left(h _ {+ 1, b _ {p} (1; \nu_ {\alpha});} \nu_ {\alpha}\right) = \epsilon \cdot (2 \alpha - 1), \tag {16}
$$
where the first equality follows from Lemma C.1, the second equality follows from the first-order optimality condition and the last equality is based on the definition of  $\nu_{\alpha}$ . This suggests that  $b_{p}(1;\nu_{\alpha}) = g^{-1}(\epsilon \cdot (2\alpha -1))$  for any  $\alpha \in [0,1]$ .
Consider the following two configurations for the term  $I_1$ :  $0 \notin [g^{-1}(-\epsilon), g^{-1}(\epsilon)]$  and  $0 \in [g^{-1}(-\epsilon), g^{-1}(\epsilon)]$ . Consider the first configuration, which is also equivalent to  $g(0) \notin [-\epsilon, \epsilon]$ . We can prove that if  $\gamma_1 + \gamma_2 < 0$  meaning that  $b_c^*(1) > 0$ , choosing  $\alpha = 0$  achieves the maximal value of  $I_1$ ; whereas if  $\gamma_1 + \gamma_2 > 0$ , choosing  $\alpha = 1$  achieves the maximum. Note that it is not possible for  $\gamma_1 + \gamma_2 = 0$  under this scenario. The proof is straightforward, since we have
$$
\begin{array}{l} I _ {1} = L \left(h _ {1, g ^ {- 1} (2 \epsilon \alpha - \epsilon)}; \mu_ {c}\right) + \epsilon \cdot L \left(h _ {1, g ^ {- 1} (2 \epsilon \alpha - \epsilon)}; \nu_ {\alpha}\right) \\ = L \left(h _ {1, g ^ {- 1} (2 \epsilon \alpha - \epsilon)}; \mu_ {c}\right) + \epsilon \cdot \left[ 1 + u + (1 - 2 \alpha) \cdot g ^ {- 1} (2 \epsilon \alpha - \epsilon) \right] \\ = L \left(h _ {1, t}; \mu_ {c}\right) + \epsilon \cdot (1 + u) - t \cdot g (t), \\ \end{array}
$$
where  $t = g^{-1}(2\epsilon \alpha - \epsilon) \in [g^{-1}(\epsilon), g^{-1}(\epsilon)]$ . In addition, we can compute the derivative of  $I_1$  with respect to  $t$ :
$$
\frac {\partial}{\partial t} I _ {1} = g (t) - g (t) - t \cdot g ^ {\prime} (t) = - t \cdot g ^ {\prime} (t),
$$
which suggests that  $I_{1}$  is a concave function with respect to  $t$ . If  $0 \in [g^{-1}(-\epsilon), g^{-1}(\epsilon)]$ , we achieve the global maximum of  $I_{1}$  at  $t = 0$  by carefully picking  $\alpha_{0} = \frac{1}{2} + \frac{1}{2\epsilon} \cdot g(0)$ . If not (i.e.,  $g^{-1}(-\epsilon) > 0$  or  $g^{-1}(\epsilon) < 0$ ), then we pick  $t$  that is closer to 0, which is either  $g(-\epsilon)$  or  $g(\epsilon)$  by setting  $\alpha = 0$  or  $\alpha = 1$  respectively. Therefore, we can specify the sufficient and necessary conditions when the weight vector  $w$  can be flipped from 1 to -1:
1. When  $g(0) \notin [-\epsilon, \epsilon]$ , the condition is
$$
\max  \{\Delta L (\nu_ {0}; \epsilon , u, \mu_ {c}), \Delta L (\nu_ {1}; \epsilon , u, \mu_ {c}) \} \geq 0.
$$
2. When  $g(0) \in [-\epsilon, \epsilon]$ , the condition is
$$
\Delta L \left(\nu_ {\alpha_ {0}}; \epsilon , u, \mu_ {c}\right) \geq 0, \text {w h e r e} \alpha_ {0} = \frac {1}{2} + \frac {1}{2 \epsilon} \cdot g (0).
$$
Plugging in the definition of  $g$  and  $\Delta L$ , we complete the proof of Lemma B.2.
Proof of Lemma B.3. Let  $\mu_1, \mu_2$  be the probability measures of the positive and negative examples assumed in (2), respectively. Let  $\varphi(z; \gamma, \sigma)$  be the PDF of Gaussian distribution  $\mathcal{N}(\gamma, \sigma^2)$ . For simplicity, we simply write  $\varphi(z) = \varphi(z; 0, 1)$  for standard Gaussian. For any  $h_{w,b} \in \mathcal{H}_{\mathrm{L}}$ , we know
$w$  can be either 1 or  $-1$ . First, let's consider the case where  $w = 1$ . According to the definition of risk and the data generating process of  $\mu_c$ , we have
$$
\begin{array}{l} \operatorname {R i s k} \left(h _ {w, b}; \mu_ {c}\right) = p \cdot \operatorname {R i s k} \left(h _ {w, b}; \mu_ {1}\right) + (1 - p) \cdot \operatorname {R i s k} \left(h _ {w, b}; \mu_ {2}\right) \\ = p \cdot \int_ {- b} ^ {\infty} \varphi (z; \gamma_ {1}, \sigma_ {1}) d z + (1 - p) \cdot \int_ {- \infty} ^ {- b} \varphi (z; \gamma_ {2}, \sigma_ {2}) d z \\ = p \cdot \int_ {\frac {- b - \gamma_ {1}}{\sigma_ {1}}} ^ {\infty} \varphi (z) d z + (1 - p) \cdot \int_ {- \infty} ^ {\frac {- b - \gamma_ {2}}{\sigma_ {2}}} \varphi (z) d z \\ = p \cdot \Phi \left(\frac {b + \gamma_ {1}}{\sigma_ {1}}\right) + (1 - p) \cdot \Phi \left(\frac {- b - \gamma_ {2}}{\sigma_ {2}}\right). \\ \end{array}
$$
Similarly, when  $w = -1$ , we have
$$
\begin{array}{l} \operatorname {R i s k} \left(h _ {w, b}; \mu_ {c}\right) = p \cdot \int_ {- \infty} ^ {b} \varphi (z; \gamma_ {1}, \sigma_ {1}) d z + (1 - p) \cdot \int_ {b} ^ {\infty} \varphi (z; \gamma_ {2}, \sigma_ {2}) d z \\ = p \cdot \int_ {- \infty} ^ {\frac {b - \gamma_ {1}}{\sigma_ {1}}} \varphi (z) d z + (1 - p) \cdot \int_ {\frac {b - \gamma_ {2}}{\sigma_ {2}}} ^ {\infty} \varphi (z) d z \\ = p \cdot \Phi \left(\frac {b - \gamma_ {1}}{\sigma_ {1}}\right) + (1 - p) \cdot \Phi \left(\frac {- b + \gamma_ {2}}{\sigma_ {2}}\right). \\ \end{array}
$$
Proof of Lemma C.1. We use similar notations such as  $\mu_1$ ,  $\mu_2$  and  $\varphi$  as in Lemma B.3. For any  $h_{w,b} \in \mathcal{H}_{\mathrm{L}}$  with  $w = 1$ , then according to the definition of population hinge loss, we have
$$
\begin{array}{l} L (h _ {w, b}; \mu_ {c}) \\ = \mathbb {E} _ {(x, y) \sim \mu_ {c}} [ \max  \{0, 1 - y (x + b) \} ] \\ = p \int_ {- b - 1} ^ {\infty} (1 + b + z) \varphi (z; \gamma_ {1}, \sigma_ {1}) d z + (1 - p) \int_ {- \infty} ^ {- b + 1} (1 - b - z) \varphi (z; \gamma_ {2}, \sigma_ {2}) d z \\ = p \int_ {\frac {- b - 1 - \gamma_ {1}}{\sigma_ {1}}} ^ {\infty} (1 + b + \gamma_ {1} + \sigma_ {1} z) \varphi (z) d z + (1 - p) \int_ {- \infty} ^ {\frac {- b + 1 - \gamma_ {2}}{\sigma_ {2}}} (1 - b - \gamma_ {2} - \sigma_ {2} z) \varphi (z) d z \\ = p (b + \gamma_ {1} + 1) \Phi \left(\frac {b + \gamma_ {1} + 1}{\sigma_ {1}}\right) + p \sigma_ {1} \frac {1}{\sqrt {2 \pi}} \exp \left(- \frac {(b + \gamma_ {1} + 1) ^ {2}}{2 \sigma_ {1} ^ {2}}\right) \\ + (1 - p) (- b - \gamma_ {2} + 1) \Phi \left(\frac {- b - \gamma_ {2} + 1}{\sigma_ {2}}\right) + (1 - p) \sigma_ {2} \frac {1}{\sqrt {2 \pi}} \exp \left(- \frac {(- b - \gamma_ {2} + 1) ^ {2}}{2 \sigma_ {2} ^ {2}}\right). \\ \end{array}
$$
Taking the derivative with respect to parameter  $b$  and using simple algebra, we have
$$
\frac {\partial}{\partial b} L \left(h _ {w, b}; \mu_ {c}\right) = p \cdot \Phi \left(\frac {b + \gamma_ {1} + 1}{\sigma_ {1}}\right) - (1 - p) \cdot \Phi \left(\frac {- b - \gamma_ {2} + 1}{\sigma_ {2}}\right).
$$
Similarly, for any  $h_{w,b} \in \mathcal{H}_L$  with  $w = -1$ , we have
$$
\begin{array}{l} L (h _ {w, b}; \mu_ {c}) \\ = \mathbb {E} _ {(x, y) \sim \mu_ {c}} [ \max  \{0, 1 - y (- x + b) \} ] \\ = p \cdot \int_ {- \infty} ^ {b + 1} (1 + b - z) \varphi (z; \gamma_ {1}, \sigma_ {1}) d z + (1 - p) \cdot \int_ {b - 1} ^ {\infty} (1 - b + z) \varphi (z; \gamma_ {2}, \sigma_ {2}) d z \\ = p \cdot \int_ {- \infty} ^ {\frac {b + 1 - \gamma_ {1}}{\sigma_ {1}}} (1 + b - \gamma_ {1} - \sigma_ {1} z) \varphi (z) d z + (1 - p) \cdot \int_ {\frac {b - 1 - \gamma_ {2}}{\sigma_ {2}}} ^ {\infty} (1 - b + \gamma_ {2} + \sigma_ {2} z) \varphi (z) d z \\ = p (b - \gamma_ {1} + 1) \Phi \left(\frac {b - \gamma_ {1} + 1}{\sigma_ {1}}\right) + p \sigma_ {1} \frac {1}{\sqrt {2 \pi}} \exp \left(- \frac {(b - \gamma_ {1} + 1) ^ {2}}{2 \sigma_ {1} ^ {2}}\right) \\ + (1 - p) (- b + \gamma_ {2} + 1) \Phi \left(\frac {- b + \gamma_ {2} + 1}{\sigma_ {2}}\right) + (1 - p) \sigma_ {2} \frac {1}{\sqrt {2 \pi}} \exp \left(- \frac {(- b + \gamma_ {2} + 1) ^ {2}}{2 \sigma_ {2} ^ {2}}\right). \\ \end{array}
$$
$$
\frac {\partial}{\partial b} L \left(h _ {w, b}; \mu_ {c}\right) = p \cdot \Phi \left(\frac {b - \gamma_ {1} + 1}{\sigma_ {1}}\right) - (1 - p) \cdot \Phi \left(\frac {- b + \gamma_ {2} + 1}{\sigma_ {2}}\right).
$$
Combining the two scenarios, we complete the proof.
D Additional Experimental Results and Details
In this section, we provide details on our experimental setup (Appendix D.1) and then provide additional results (Appendix D.2).
D.1 Details on Experimental Setup in Section 3
Details on datasets and training configurations. In the main paper, we used different public benchmark datasets including MNIST [27] digit pairs (i.e., 1-7, 6-9, 4-9) and also the Enron dataset, which is created by Metsis et al. [36], Dogfish [24] and Adult [14], which are all used in the evaluations of prior works except MNIST 6-9 and MNIST 4-9. In the appendix, we additionally present the results of the IMDB dataset [33], which has also been used in prior evaluations [24, 25]. We did not include the IMDB results in the main paper because we could not run the existing state-of-the-art poisoning attacks on IMDB because the computation time is extremely slow. Instead, we directly quote the poisoned error of SVM from Koh et al. [25] and then present the computed metrics. For the Dogfish and Enron dataset, we construct the constraint set  $\mathcal{C}$  in the no defense setting by finding the minimum  $(u_{\mathrm{min}}^i)$  and maximum  $(u_{\mathrm{max}}^i)$  values occurred in each feature dimension  $i$  for both the training and test data, which then forms a box constraint  $[u_{\mathrm{min}}^i, u_{\mathrm{max}}^i]$  for each dimension. This way of construction is also used in the prior work [25]. Because we consider linear models, the training [38] of linear models and the attacks on them are stable (i.e., less randomness involved in the process) and so, we get almost identical results when feeding different random seeds. Therefore, we did not report error bars in the results. The regularization parameter  $\lambda$  for training the linear models (SVM and LR) are configured as follows:  $\lambda = 0.09$  for MNIST digit pairs, Adult, Dogfish, SVM for Enron;  $\lambda = 0.01$  for IMDB, LR for Enron. Overall, the results and conclusions in this paper are insensitive to the choice of  $\lambda$ . The computation of the metrics in this paper are extremely fast and can be done on any laptop. The poisoning attacks can also be done on a laptop, except the Influence Attack [25], whose computation can be accelerated using GPUs.
Attack details. The KKT, MTA and Min-Max attacks evaluated in Section 3 require a target model as input. This target model is typically generated using some label-flipping heuristics [25, 46]. In practice, these attacks are first run on a set of carefully-chosen target models, then the best attack performance achieved by these target models is reported. We generate target models using the improved procedure described in Suya et al. [46] because their method is able to generate better target models that induce victim models with a higher test error compared with the method proposed in Koh et al. [25]. We generate target models with different error rates, ranging from  $5\%$  to  $70\%$  using the label-flipping heuristics, and then pick the best performing attack induced by these target models.
Poison Ratio ε (%)	0.0	0.1	0.2	0.3	0.5	0.7	0.9	1.0	2.0	3.0
Train Error (%)	0.1	0.8	1.2	1.8	2.6	3.1	3.3	3.6	5.3	6.5
Test Error (%)	0.8	9.5	12.8	13.8	17.8	20.5	21.0	20.5	27.3	31.8
Table 2: Comparisons of training and test errors of existing data poisoning attacks on Dogfish. The poisoned training and test errors are reported from the current best attacks.
Metric	Robust	Moderately Vulnerable	Highly Vulnerable
MNIST 6-9	MNIST 1-7	Adult	Dogfish	MNIST 4-9	F. Enron	Enron	IMDB
SVM	Error Increase	2.7	2.4	3.2	7.9	6.6	33.1	31.9	\(19.1^{\dagger}\)
Base Error	0.3	1.2	21.5	0.8	4.3	0.2	2.9	11.9
Sep/SD	6.92	6.25	9.65	5.14	4.44	1.18	1.18	2.57
Sep/Size	0.24	0.23	0.33	0.05	0.14	0.01	0.01	0.002
LR	Error Increase	2.3	1.8	2.5	6.8	5.8	33.0	33.1	-
Base Error	0.6	2.2	20.1	1.7	5.1	0.3	2.5	-
Sep/SD	6.28	6.13	4.62	5.03	4.31	1.11	1.10	2.52
Sep/Size	0.27	0.27	0.27	0.09	0.16	0.01	0.01	0.003
Table 3: Explaining disparate poisoning vulnerability under linear models by computing the metrics on the correctly classified clean test points. The top row for each model gives the increase in error rate due to the poisoning, over the base error rate in the second row. The error increase of IMDB (marked with  $\dagger$ ) is directly quoted from Koh et al. [25] as running the existing poisoning attacks on IMDB is extremely slow. LR results are missing as they are not contained in the original paper. The explanatory metrics are the scaled (projected) separability, standard deviation and constraint size.
Following the prior practice [25], we consider adversaries that have access to both the clean training and test data, and therefore, adversaries can design attacks that can perform better on the test data. This generally holds true for the Enron and MNIST 1-7 datasets, but for Dogfish, we find in our experiments that the attack "overfits" to the test data heavily due to the small number of training and test data and also the high dimensionality. More specifically, we find that the poisoned model tends to incur significantly higher error rates on the clean test data compared to the clean training data. Since this high error cannot fully reflect the distributional risk, when we report the results in Section 3 we report the errors on both the training and the testing data to give a better empirical sense of what the distributional risk may look like. This also emphasizes the need to be cautious on the potential for "overfitting" behavior when designing poisoning attacks. Table 2 shows the drastic differences between the errors of the clean training and test data after poisoning.
D.2 Supplemental Results
In this section, we provide additional results results that did not fit into the main paper, but further support the observations and claims made in the main paper. We first show the results of IMDB and the metrics computed on the whole clean test data in Appendix D.2.2 to complement Table 1 in the main paper, then include the complete results of the impact of data sanitization defenses in Appendix D.2.3 to complement the last paragraph in Section 7. Next, we provide the metrics computed on selective benchmark datasets using a different projection vector from the clean model weight in Appendix D.2.4 to support the results in Table 1 in the main paper. Lastly, we show the performance of different poisoning attacks at various poisoning ratios in Appendix D.2.5, complementing Figure 1 in the main paper.
D.2.1 IMDB results
Table 1 in the main paper presents the metrics that are computed on the correctly classified test samples by the clean model  $\boldsymbol{w}_{c}$ . In Table 3, we additionally include the IMDB results to the Table 1 in the main paper. From the table, we can see that IMDB is still highly vulnerable to poisoning because its separability is low compared to datasets that are moderately vulnerable or robust, and impacted the most by the poisoning points compared to all other benchmark datasets. Note that, the increased error from IMDB is directly quoted from Koh et al. [25], which considers data sanitization
Models	Metrics	Robust	Moderately Vulnerable	Highly Vulnerable
MNIST 6-9	MNIST 1-7	Adult	Dogfish	MNIST 4-9	F. Enron	Enron	IMDB
SVM	Error Increase	2.7	2.4	3.2	7.9	6.6	33.1	31.9	\(19.1^{\dagger}\)
Base Error	0.3	1.2	21.5	0.8	4.3	0.2	2.9	11.9
Sep/SD	6.70	5.58	1.45	4.94	3.71	1.18	1.15	1.95
Sep/Size	0.23	0.23	0.18	0.05	0.13	0.01	0.01	0.001
LR	Error Increase	2.3	1.8	2.5	6.8	5.8	33.0	33.1	-
Base Error	0.6	2.2	20.1	1.7	5.1	0.3	2.5	-
Sep/SD	5.97	5.17	1.64	4.67	3.51	1.06	1.01	1.88
Sep/Size	0.26	0.26	0.16	0.08	0.15	0.01	0.01	0.002
Table 4: Explaining the different vulnerabilities of benchmark datasets under linear models by computing metrics on the whole data. The error increase of IMDB (marked with  $\dagger$ ) is directly quoted from Koh et al. [25].
Dataset	Error Increase	Base Error	Sep/SD	Sep/Size
w/o	w/	w/o	w/	w/o	w/	w/o	w/
MNIST 1-7 (10%)	7.7	1.0	1.2	2.4	6.25	6.25	0.23	0.43
Dogfish (3%)	8.7	4.3	0.8	1.2	5.14	5.14	0.05	0.12
Enron (3%)	31.9	25.6	2.9	3.2	1.18	1.18	0.01	0.11
Table 5: Understanding impact of data sanitization defenses on poisoning attacks.  $w/o$  and  $w/$  denote without defense and with defense respectively. MNIST 1-7 is evaluated at  $10\%$  poisoning ratio due to its strong robustness at  $\epsilon = 3\%$  and Enron is still evaluated at  $\epsilon = 3\%$  because it is highly vulnerable.
defenses. Therefore, we expect the attack effectiveness might be further improved when we do not consider any defenses, as in our paper.
D.2.2 Metrics computed using all test data
Table 4 shows the results when the metrics are computed on the full test data set (including misclassified ones), rather than just on examples that were classified correctly by the clean model. The metrics are mostly similar to Table 3 when the initial errors are not high. For datasets with high initial error such as Adult, the computed metrics are more aligned with the final poisoned error, not the error increase.
D.2.3 Explaining the impact of data sanitization defenses
We then provide additional results for explaining the effectiveness of the data sanitization defenses in improving dataset robustness, which is discussed in section 7. On top of the Enron result shown in the paper, which is attacked at  $3\%$  poisoning ratio, we also provide attack results of MNIST 1-7 dataset. We report the attack results when  $\epsilon = 10\%$  and we considered a significantly higher poisoning ratio because at the original  $3\%$  poisoning, the dataset can well resist existing attacks and hence there is no point in protecting the dataset with sanitization defenses. This attack setting is just for illustration purpose and attackers in practice may be able to manipulate such a high number of poisoning points. Following the main result in the paper, we still compute the metrics based on the correctly classified samples in the clean test set, so as to better depict the relationship between the increased errors and the computed metrics. The results are summarized in Table 5 and we can see that existing data sanitization defenses improve the robustness to poisoning by majorly limiting  $\mathrm{Size}_{\mathbf{w}_c}(\mathcal{C})$ . For MNIST 1-7, equipping with data sanitization defense will make the dataset even more robust (robust even at the high  $10\%$  poisoning rate), which is consistent with the findings in prior work [45].
D.2.4 Using different projection vectors
In the main paper, we used the weight of the clean model as the projection vector and found that the computed metrics are highly correlated with the empirical attack effectiveness observed for different benchmark datasets. However, there can also be other projection vectors that can be used for explaining the different vulnerabilities, as mentioned in Remark 5.8.
Base Error (%)	Error Increase (%)	Sep/SD	Sep/Size
wc	wU	wc	wU
MNIST 1-7	1.2	2.4	6.25	6.51	0.23	0.52
Dogfish	0.8	7.9	5.14	4.43	0.05	0.19
Table 6: Using the projection vector that minimizes the upper bound on the risk of optimal poisoning attacks for general distributions.  $\mathbf{w}_c$  denotes the clean weight vector and  $\mathbf{w}_U$  denotes weight vector obtained from minimizing the upper bound.
Metric	MNIST 1–7	Dogfish
Upper Bound (%)	20.1	34.9
Lower Bound (%)	3.6	8.7
Base Error (%)	1.2	0.8
Table 7: Non-trivial upper bounds on the limits of poisoning attacks across benchmark datasets on linear SVM. The Upper Bound on the limits of poisoning attacks is obtained by minimizing the upper bound Equation (17). The Lower Bound on the limit of poisoning attacks is obtained by reporting the highest poisoned error from the best empirical attacks. The datasets are sorted based on the lowest and highest empirical base error.
We conducted experiments that use the projection vector that minimizes the upper bound on optimal poisoning attacks, given in Equation 5. The upper-bound minimization corresponds to a min-max optimization problem. We solve it using the online gradient descent algorithm (alternatively updating the poisoning points and model weight), adopting an approach similar to the one used by Koh et al. for the i-Min-Max attack [25]. We run the min-max optimization for 30,000 iterations with learning rate of 0.03 for the weight vector update, and pick the weight vector that results in the lowest upper bound in Equation 5.
The results on two of the benchmark datasets, MNIST 1-7 and Dogfish, are summarized in Table 6. From the table, we can see that, compared to the clean model, the new projection vector reduces the projected constraint size (increases Sep/Size), which probably indicates the weight vector obtained from minimizing the upper bound focuses more on minimizing the term  $\ell_{\mathrm{M}}(\mathrm{Size}_{w_c}(\mathcal{C}))$  in Equation 5. Nevertheless, projecting onto the new weight vector can still well explain the difference between the benchmark datasets.
Using the obtained vector  $\boldsymbol{w}_U$  (denote the corresponding hypothesis as  $h_U$ ) above, we can also compute an (approximate) upper bound on the test error of the optimal indiscriminate poisoning attack, which is proposed by Steinhardt et al. [45] and is shown in (17). The upper bound is an approximate upper bound to the test error because  $\max_{\mathcal{S}_p \subseteq \mathcal{C}} L(\hat{h}_p; \mathcal{S}_c)$  only upper bounds the maximum training error (of poisoned model  $\hat{h}_p$ ) on  $\mathcal{S}_c$  from poisoning and it is assumed that, with proper regularization, the training and test errors are roughly the same. We can view  $h_U$  as the numerical approximation of the hypothesis that leads to the smallest upper bound on the (approximate) test error. Next, in Table 7, we compare the computed (approximate) upper bound and the lower bound on poisoning effectiveness achieved by best known attacks, and we can easily observe that, although the gap between the lower and upper bounds are relatively large (all datasets are vulnerable at 3% poisoning ratio when we directly check the upper bounds), the upper bounds still vary between datasets. We also speculate that the actual performance of the best attacks may not be as high as indicated by the upper bound, because when computing the upper bound, we used the surrogate loss to upper bound the 0-1 loss (exists some non-negligible gap) and maximized the surrogate loss in the poisoning setting, which potentially introduces an even larger gap between the surrogate loss and the 0-1 loss. Hence, we motivate future research to shrink the possibly large gap.
Figure 3: Performance of best current indiscriminate poisoning attacks with  $\epsilon = 3\%$  across different benchmark datasets on LR. Datasets are sorted from lowest to highest base error rate.
(a) MNIST 1-7
(b) Dogfish
Figure 4: Comparisons of the attack performance of existing data poisoning attacks on different benchmark datasets. Poisoning ratios are  $0.1\%$ ,  $0.2\%$ ,  $0.3\%$ ,  $0.5\%$ ,  $0.7\%$ ,  $0.9\%$ ,  $1\%$ ,  $2\%$ ,  $3\%$ .
(c) Enron
$$
\begin{array}{l} \max  _ {\mathcal {S} _ {p} \subseteq \mathcal {C}} L (\hat {h} _ {p}; \mathcal {S} _ {c}) \leq \min  _ {h} \left(L (h; \mathcal {S} _ {c}) + \epsilon \max  _ {(\boldsymbol {x} _ {p}, y _ {p}) \in \mathcal {C}} \ell (h; \boldsymbol {x} _ {p}, y _ {p}) + \frac {(1 + \epsilon) \lambda}{2} \| \boldsymbol {w} \| _ {2} ^ {2}\right) \\ \leq L \left(h _ {U}; \mathcal {S} _ {c}\right) + \epsilon \max  _ {\left(\boldsymbol {x} _ {p}, y _ {p}\right) \in \mathcal {C}} \ell \left(h _ {U}; \boldsymbol {x} _ {p}, y _ {p}\right) + \frac {(1 + \epsilon) \lambda}{2} \| \boldsymbol {w} _ {U} \| _ {2} ^ {2} \tag {17} \\ \end{array}
$$
D.2.5 Performance of different attacks are similar