SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Dataset distillation and trajectory matching. Introduced in [94], the goal of dataset distillation is to produce a small dataset which captures the essence of a larger one. Dataset distillation by optimizing soft labels has been explored using the neural tangent kernel [64, 63] and gradient-based methods [12, 84]. Our method attempts to match the training trajectory of a normal backdoored model, with standard poisoned examples, by flipping labels. Trajectory matching has been used previously for dataset distillation [14] and bears similarity to imitation learning [66, 27, 40]. The use of a proxy objective in weight space for backdoor design appears in the KKT attack of [47]. However, the focus is in making the attacker's optimization more efficient for binary classification problems. Initially we approximately solves the bilevel optimization equation 2 efficiently using Neural Tangent Kernels to approximate the solution of the inner optimization. Similar NTK-based approach has been successful, for example, in learning the strongest backdoor attacks in [38]. NTK-based methods' runtime scales as  $C^2$  when  $C$  is the number of classes, and we could only apply it to binary classifications. On the other hand, trajectory matching of FLIP can be applied to significantly more complex problems with larger models and generalizes surprisingly well to the scenario where the attacker does not know the model, data, hyperparameters to be used by the user as we show in Section 3.2. Closest to our work is [32], which uses gradient matching to design poisoned training images (as opposed to labels). The goal is targeted data poisoning (as opposed to a more general backdoor attack) which aims to alter the prediction of the model trained on corrupted data, for a specific image at inference time.
In this section, for completeness, we present some of the technical details on our evaluation pipeline that were omitted in the main text. For most of our experiments, the pipeline proceeds by (1) training expert models, (2) generating synthetic labels, and (3) measuring our attack success on a user model trained on label-flipped datasets. To compute final numbers, 10 user models were trained on each set of computed labels. Each experiment was run on a single, randomly-selected GPU on a cluster containing NVIDIA A40 and 2080ti GPUs. On the slower 2080ti GPUs, our ResNet and CIFAR experiments took no longer than an hour, while, for the much larger VGG and ViT models, compute times were longer.
B.1 Datasets and poisoning procedures
We evaluate FLIP and softFLIP on three standard classification datasets of increasing difficulty: CIFAR-10, CIFAR-100, and Tiny-ImageNet. For better test performance and to simulate real-world use cases, we follow the standard CIFAR data augmentation procedure of (1) normalizing the data and (2) applying PyTorch transforms: RandomCrop and RandomHorizontalFlip. For RandomCrop, every epoch, each image was cropped down to a random  $28 \times 28$  subset of the original image with the extra 4 pixels reintroduced as padding. RandomHorizontalFlip randomly flipped each image horizontally with a  $50\%$  probability every epoch. For our experiments on the Vision Transformer, images were scaled to  $224 \times 224$  pixels and cropped to  $220 \times 220$  before padding and flipping.
To train our expert models we poisoned each dataset setting  $y_{\mathrm{source}} = 9$  and  $y_{\mathrm{target}} = 4$  (i.e., the ninth and fourth labels in each dataset). Since CIFAR-100 and Tiny-ImageNet have only 500 images per class, for the former, we use the coarse-labels for  $y_{\mathrm{source}}$  and  $y_{\mathrm{target}}$ , while for the latter we oversample poisoned points during the expert training stage. For CIFAR-10 this corresponds to a canonical self-driving-car-inspired backdoor attack setup in which the source label consists of images of trucks and target label corresponds to deer. For CIFAR-100, the mapping corresponds to a source class of 'large man-made outdoor things' and a target of 'fruit and vegetables.' Finally, for Tiny-ImageNet, 'tarantulas' were mapped to 'American alligators'. To poison the dataset, each  $y_{\mathrm{source}} = 9$  image had a trigger applied to it and was appended to the dataset.
Trigger details. We used the following three triggers, in increasing order of strength. Examples of the triggers are shown in Fig. 4.
- Pixel Trigger [89] ( $T = p$ ). To inject the pixel trigger into an image, at three pixel locations of the photograph the existing colors are replaced by pre-selected colors. Notably, the original attack was proposed with a single pixel and color combination (that we use),
so, to perform our stronger version, we add two randomly selected pixel locations and colors. In particular, the locations are \(\{(11, 16), (5, 27), (30, 7)\}\) and the respective colors in hexadecimal are \{\#650019, \#657B79, \#002436\}\). This trigger is the weakest of our triggers with the smallest pixel-space perturbation
- Periodic / Sinusoidal Trigger [9] ( $T = s$ ). The periodic attack adds periodic noise along the horizontal axis (although the trigger can be generalized to the vertical axis as well). We chose an amplitude of 6 and a frequency of 8. This trigger has a large, but visually subtle effect on the pixels in an image making it the second most potent of our triggers.
- Patch / Turner Trigger [90] ( $T = t$ ). For our version of the patch poisoner, we adopted the  $3 \times 3$  watermark proposed by the original authors and applied it to each corner of the image to persist through our RandomCrop procedure. This trigger seems to perform the best on our experiments, likely due to its strong pixel-space signal.
B.2 Models and optimizers
For our experiments, we use the ResNet-32, ResNet-18, VGG-19, and (pretrained) VIT-B-16 architectures with around 0.5, 11.4, 144, and 86 million parameters, respectively. In the ResNet experiments, the expert and user models were trained using SGD with a batch size of 256, starting learning rate of  $\gamma = 0.1$  (scheduled to reduce by a factor of 10 at epoch 75 and 150), weight decay of  $\lambda = 0.0002$ , and Nesterov momentum of  $\mu = 0.9$ . For the larger VGG and ViT models the learning rate and weight decay were adjusted as follows  $\gamma = 0.01$ , 0.05 and  $\lambda = 0.0002$ , 0.0005, respectively. For Table 5, in which we mismatch the expert and downstream optimizers, we use Adam with the same batch size, starting learning rate of  $\gamma = 0.001$  (scheduled to reduce by a factor of 10 at epoch 125), weight decay of  $\lambda = 0.0001$ , and  $(\beta_{1},\beta_{2}) = (0.9,0.999)$ .
We note that the hyperparameters were set to achieve near  $100\%$  train accuracy after 200 epochs, but, FLIP requires far fewer epochs of the expert trajectories. So, expert models were trained for 20 epochs while the user models were trained for the full 200. Expert model weights were saved every 50 iterations / minibatches (i.e., for batch size 256 around four times an epoch).
B.3 FLIP details
For each iteration of Algorithm 1, we sampled an expert model at uniform random, while checkpoints were sampled at uniform random from the first 20 epochs of the chosen expert's training run. Since weights were recorded every 50 iterations, from each checkpoint a single stochastic gradient descent iteration was run with both the clean minibatch and the poisoned minibatch (as a proxy for the actual expert minibatch step) and the loss computed accordingly. Both gradient steps adhered to the training hyperparameters described above. The algorithm was run for  $N = 25$  iterations through the entire dataset.
To initialize  $\tilde{\ell}$ , we use the original one-hot labels  $y$  scaled by a temperature parameter  $C$ . For sufficiently large  $C$ , the two gradient steps in Fig. 3 will be very similar except for the changes in the poisoned examples, leading to a low initial value of  $\mathcal{L}_{\mathrm{param}}$ . However if  $C$  is too large, we suffer from vanishing gradients of the softmax. Therefore  $C$  must be chosen to balance these two concerns.
B.4 Compute
All of our experiments were done on a computing cluster containing NVIDIA A40 and 2080ti GPUs with tasks split roughly evenly between the two. To compute all of our numbers (averaged over 10 user models) we ended up computing approximately 3490 user models, 160 sets of labels, and 955 expert models. Averaging over GPU architecture, dataset, and model architecture, we note that each set of labels takes around 40 minutes to train. Meanwhile, each expert model takes around 10 minutes to train (fewer epochs with a more costly weight-saving procedure) and each user model takes around 40. This amounts to a total of approximately 2595 GPU-hours.
We note that the number of GPU-hours for an adversary to pull off this attack is likely significantly lower since they would need to compute as few as a single expert model (10 minutes) and a set of labels (40 minutes). This amounts to just under one GPU-hour given our setup (subject to hardware), a surprisingly low sum for an attack of this high potency.
C Complete experimental results
In this section, we provide expanded versions of the key tables and figures in the main text complete with standard errors as well as some additional supplementary materials. As in the main text, we compute our numbers via a three step process: (1) we start by training 5 sets of synthetic labels for each (dataset, expert model architecture, trigger) tuple, (2) we then aggregate each set of labels, and (3) we finish by training 10 user models on each interpolation of the aggregated labels and the ground truths.
For our FLIP experiments in Appendix C.1, labels are aggregated as described in Section 2 varying the number of flipped labels. Meanwhile, for our softFLIP results in Appendix C.3, we aggregate as in Section 4 by taking the average logits for each image and linearly interpolating them on parameter  $\alpha$  with the ground-truth labels.
C.1 Main results on FLIP
Fig. 2, Fig. 5a, and Table 1 showcase FLIP's performance when compared to the inner-product-based baseline and in relation to changes in dataset, model architecture, and trigger. We additionally present the raw numbers for the dot-product baseline.
0	150	300	500	1000	1500
C10	r32	s	92.38 (0.1)/00.1 (0.0)	92.26 (0.1)/12.4 (1.8)	92.09 (0.1)/54.9 (2.4)	91.73 (0.1)/87.2 (1.3)	90.68 (0.1)/99.4 (0.2)	89.87 (0.1)/99.8 (0.1)
t	92.57 (0.1)/00.0 (0.0)	92.37 (0.0)/28.4 (4.9)	92.03 (0.0)/95.3 (1.5)	91.59 (0.1)/99.6 (0.2)	90.80 (0.1)/99.5 (0.3)	89.91 (0.1)/99.9 (0.1)
p	92.52 (0.1)/00.0 (0.0)	92.24 (0.1)/03.3 (0.2)	91.67 (0.0)/06.0 (0.2)	91.24 (0.1)/10.8 (0.3)	90.00 (0.1)/21.2 (0.3)	88.92 (0.1)/29.9 (0.8)
r18	s	94.09 (0.1)/00.0 (0.0)	94.13 (0.1)/13.1 (2.0)	93.94 (0.1)/32.2 (2.6)	93.55 (0.1)/49.0 (3.1)	92.73 (0.1)/81.2 (2.7)	92.17 (0.1)/82.4 (2.6)
vgg	s	93.00 (0.1)/00.0 (0.0)	92.85 (0.1)/02.3 (0.2)	92.48 (0.1)/09.2 (0.7)	92.16 (0.1)/21.4 (0.8)	91.11 (0.2)/48.0 (1.0)	90.44 (0.1)/69.5 (1.6)
C100	r32	s	78.96 (0.1)/00.2 (0.1)	78.83 (0.1)/08.2 (0.6)	78.69 (0.1)/24.7 (1.3)	78.52 (0.1)/45.4 (1.9)	77.61 (0.1)/82.2 (2.5)	76.64 (0.1)/95.2 (0.3)
r18	s	82.67 (0.2)/00.1 (0.0)	82.87 (0.1)/11.9 (0.8)	82.48 (0.2)/29.9 (3.1)	81.91 (0.2)/35.8 (3.1)	81.25 (0.1)/81.9 (1.5)	80.28 (0.3)/95.3 (0.5)
TI	r18	s	61.61 (0.2)/00.0 (0.0)	61.47 (0.2)/10.6 (0.9)	61.23 (0.1)/31.6 (0.9)	61.25 (0.2)/56.0 (1.4)	61.45 (0.2)/51.8 (2.0)	60.94 (0.1)/57.0 (1.5)
Table 2: An expanded version of Table 1 in which each point is averaged over 10 user training runs and standard errors are shown in parentheses.
bases	500	1000	1500	2500	5000
inner product	91.59 (0.1)/09.2 (0.5)	90.70 (0.1)/17.9 (1.7)	89.76 (0.1)/32.0 (1.6)	87.84 (0.1)/51.3 (1.4)	82.84 (0.1)/94.8 (0.2)
Random	91.71 (0.1)/07.7 (0.8)	90.95 (0.2)/16.2 (1.7)	90.04 (0.1)/23.0 (1.3)	87.64 (0.2)/42.9 (1.5)	82.97 (0.0)/94.9 (0.2)
Table 3: Under the scenario where  $y_{\mathrm{target}} = 9$ , we show raw numbers for the inner product baseline and random selection baseline as presented in Fig. 2. The baseline was run on ResNet-32s with comparisons to the sinusoidal trigger. Each point is averaged over 10 user training runs and standard errors are shown in parentheses.
500	1000	2500	5000	10000	15000	20000
91.90 (0.1)/01.2 (0.1)	91.31 (0.1)/02.7 (0.2)	88.87 (0.1)/10.5 (0.8)	84.80 (0.1)/30.8 (2.6)	76.09 (0.1)/59.5 (4.5)	66.34 (0.3)/82.8 (1.7)	56.53 (0.3)/91.1 (1.7)
Table 4: Under the scenario where  $y_{\text{target}}$  is all but target class, we show raw numbers for the inner product baseline. The baseline was run on ResNet-32s with comparisons to the sinusoidal trigger. Each point is averaged over 10 user training runs and standard errors are shown in parentheses.
C.2 Robustness of FLIP
We provide experimental validations of robustness of FLIP attacks.
C.2.1 Varying model architecture and optimizer
The attacker's strategy in our previous experiments was to train expert models to mimic exactly the user architecture and optimizer setup of the user. However, it remained unclear whether the attack would generalize if the user, for instance, opted for a smaller model than expected. As such, we looked at varying (1) model architecture and (2) optimizer between expert and user setups. For (2)
we use SGD for the expert models and Adam [46] for the user. We additionally analyze what happens when both are different.
As Table 5 indicates, the attack still performs well when information is limited. When varying optimizer, we found that CTA dropped, but, interestingly, the PTA for the ResNet-18 case was almost uniformly higher. We found a similar trend for upstream ResNet-32s and downstream ResNet-18s when varying model architecture. Surprisingly, the strongest PTA numbers for budgets higher than 150 across all experiments with a downstream ResNet-18 were achieved when the attacker used a different expert model and optimizer. The FLIP attack is robust to varied architecture and optimizer.
150	300	500	1000	1500
(1)	r18 → r32	92.44/19.2	92.16/53.3	91.84/74.5	90.80/92.9	90.00/95.2
r32 → r18	93.86/04.8	93.89/36.0	93.56/60.0	92.76/86.9	91.75/98.0
(2)	r32 → r32	90.79/11.8	90.50/43.2	90.08/80.6	89.45/97.5	88.33/99.0
r18 → r18	93.17/20.3	93.08/47.0	92.94/65.6	91.91/89.9	91.16/93.1
(1+2)	r18 → r32	90.86/12.3	90.57/40.9	90.28/59.7	89.39/83.0	88.59/89.2
r32 → r18	93.32/09.4	93.05/52.9	92.70/85.3	91.72/99.2	90.93/99.7
Table 5: FLIP performs well even when the expert and user (1) model architectures and (2) optimizers are different. Experiments are computed on CIFAR-10 using the sinusoidal trigger. Each row denotes the CTA/PTA pairs averaged over 10 experiments. The second column is structured as follows: expert  $\rightarrow$  user.
Table 5 investigates whether an attacker needs to know the architecture or optimizer of the user's model. The experiments are done in the same style as Appendix C.1.
150	300	500	1000	1500
r18 → r32	92.44 (0.1)/19.2 (1.3)	92.16 (0.1)/53.3 (3.0)	91.84 (0.0)/74.5 (2.2)	90.80 (0.1)/92.9 (0.8)	90.00 (0.1)/95.2 (0.6)
r32 → r32	92.26 (0.1)/12.4 (1.8)	92.09 (0.1)/54.9 (2.4)	91.73 (0.1)/87.2 (1.3)	90.68 (0.1)/99.4 (0.2)	89.87 (0.1)/99.8 (0.1)
r32 → r18	93.86 (0.1)/04.8 (0.8)	93.89 (0.1)/36.0 (5.4)	93.56 (0.1)/60.0 (6.6)	92.76 (0.1)/86.9 (2.6)	91.75 (0.1)/98.0 (0.8)
r18 → r18	94.13 (0.1)/13.1 (2.0)	93.94 (0.1)/32.2 (2.6)	93.55 (0.1)/49.0 (3.1)	92.73 (0.1)/81.2 (2.7)	92.17 (0.1)/82.4 (2.6)
r32 → vgg	92.76 (0.1)/02.7 (0.1)	92.67 (0.0)/10.0 (0.4)	92.28 (0.1)/23.1 (1.2)	91.41 (0.1)/47.5 (1.0)	90.63 (0.1)/63.0 (1.5)
vgg → vgg	92.85 (0.1)/02.3 (0.2)	92.48 (0.1)/09.2 (0.7)	92.16 (0.1)/21.4 (0.8)	91.11 (0.2)/48.0 (1.0)	90.44 (0.1)/69.5 (1.6)
Table 6: An expanded version of Table 5 (1) in which each point is averaged over 10 runs and standard errors are shown in parentheses. We additionally compare the performance directly to the non-model-mixed case.
Table 6 shows more experimental results with mismatched architectures between the attacker and the user. Attacks on VGG need more corrupted examples to achieve successful backdoor attack.
150	300	500	1000	1500
r32	s	90.79 (0.1)/11.8 (1.6)	90.50 (0.1)/43.2 (3.8)	90.08 (0.1)/80.6 (2.2)	89.45 (0.1)/97.5 (0.3)	88.33 (0.1)/99.0 (0.3)
t	90.77 (0.1)/08.4 (1.3)	90.46 (0.1)/65.0 (6.8)	90.00 (0.1)/72.7 (5.7)	89.07 (0.1)/98.2 (1.1)	88.23 (0.1)/95.8 (2.5)
p	90.60 (0.0)/03.0 (0.3)	90.21 (0.1)/05.5 (0.3)	89.61 (0.1)/11.1 (0.7)	88.55 (0.1)/19.5 (0.6)	87.39 (0.1)/31.6 (0.8)
r18	s	93.17 (0.1)/20.3 (2.2)	93.08 (0.1)/47.0 (2.6)	92.94 (0.0)/65.6 (1.6)	91.91 (0.1)/89.9 (1.0)	91.16 (0.1)/93.1 (0.7)
Table 7: An expanded version of Table 5 (2) in which each point is averaged over 10 runs and standard errors are shown in parentheses. We additionally evaluate different choices of trigger with ResNet-32s.
150	300	500	1000	1500
r32 → r18	93.32 (0.1)/09.4 (1.5)	93.05 (0.1)/52.9 (3.0)	92.70 (0.1)/85.3 (1.7)	91.72 (0.1)/99.2 (0.2)	90.93 (0.1)/99.7 (0.1)
r18 → r32	90.86 (0.1)/12.3 (1.3)	90.57 (0.1)/40.9 (3.3)	90.28 (0.1)/59.7 (2.6)	89.39 (0.1)/83.0 (1.7)	88.59 (0.1)/89.2 (0.7)
Table 8: An expanded version of Table 5 (1+2) in which each point is averaged over 10 runs and standard errors are shown in parentheses.
C.2.2 Number of experts  $E$  and number of epochs  $K$
Although for all of the above experiments we used  $E = 50$  experts to create our labels, PTA is robust against using smaller number of experts. As we show in Table 9 and Fig. 6a, an attack is successful with as few as a single expert model.
E	150	300	500	1000	1500
1	92.38 (0.1)/09.9 (0.5)	92.05 (0.0)/45.4 (2.4)	91.59 (0.0)/80.7 (2.7)	90.80 (0.1)/98.0 (0.2)	89.90 (0.1)/99.5 (0.1)
5	92.36 (0.0)/11.9 (1.1)	92.05 (0.1)/45.3 (3.0)	91.42 (0.1)/86.6 (1.2)	90.81 (0.1)/99.1 (0.2)	89.94 (0.0)/99.8 (0.1)
10	92.39 (0.1)/15.1 (1.6)	92.09 (0.1)/59.7 (2.3)	91.74 (0.0)/88.5 (1.5)	90.91 (0.1)/99.4 (0.1)	90.03 (0.1)/99.8 (0.1)
25	92.33 (0.1)/10.9 (1.1)	92.06 (0.1)/50.9 (2.2)	91.74 (0.1)/88.9 (1.2)	90.92 (0.1)/98.9 (0.2)	90.03 (0.0)/99.7 (0.0)
50	92.44 (0.0)/12.0 (1.6)	91.93 (0.1)/54.4 (3.1)	91.55 (0.1)/89.9 (1.1)	90.91 (0.1)/99.6 (0.1)	89.73 (0.1)/99.9 (0.0)
At  $K = 0$  epoch, FLIP uses an expert with random weights to find the examples to corrupt, and hence the attack is weak. In our experiments we use  $K = 20$ . Table 10 shows that the attack is robust in the choice of  $K$  and already achieves strong performance with  $K = 1$  epoch of expert training.
Table 9: Understanding the effect of the number of expert models on the backdoor attack. The experiments were conducted using ResNet-32s on CIFAR-10 poisoned by the sinusoidal trigger. Standard errors in the parentheses are averaged over 10 runs.
K	100	150	200	250	500	1000	1500
0	92.38 (0.1)/00.8 (0.1)	92.38 (0.1)/01.4 (0.2)	92.27 (0.1)/01.9 (0.2)	92.22 (0.1)/03.1 (0.3)	91.79 (0.1)/05.9 (0.3)	92.90 (0.1)/12.1 (0.9)	91.40 (0.1)/22.4 (1.0)
1	92.44 (0.1)/09.4 (1.1)	92.33 (0.1)/17.0 (1.3)	92.18 (0.1)/29.8 (2.3)	92.15 (0.0)/43.8 (1.8)	91.59 (0.1)/66.1 (1.7)	92.60 (0.1)/85.0 (0.7)	92.62 (0.1)/86.8 (1.0)
5	92.35 (0.1)/04.6 (0.4)	92.27 (0.0)/12.7 (1.0)	92.22 (0.0/2.22 (0.0)	92.09 (0.0/42.5 (3.2)	91.66 (0.0)	92.70 (0.0/90.3 (0.7)	92.80 (0.0/99.6 (0.1)
10	92.33 (0.0)/04.4 (0.3)	92.38 (0.0)/09.6 (0.9)	92.27 (0.0)/26.1 (2.2)	92.05 (0.1)/39.4 (2.6)	91.65 (0.0)/89.9 (1.4)	92.67 (0.1)/99.5 (0.1)	92.68 (0.1)/99.8 (0.1)
20	92.54 (0.1)/05.5 (0.4)	92.26 (0.1)/12.4 (1.8)	92.22 (0.1)/12.1 (1.6)	92.10 (0.1)/12.8 (1.3)	91.73 (0.1)/87.2 (1.3)	92.68 (0.1)/99.4 (0.1)	92.69 (0.1)/99.8 (0.1)
30	92.45 (0.0)/04.5 (0.5)	92.31 (0.0)/10.9 (0.8)	92.22 (0.1)/15.4 (0.7)	92.25 (0.1)/34.4 (2.9)	91.83 (0.1)/91.5 (0.8)	92.70 (0.1)/99.3 (0.1)	92.70 (0.1)/99.9 (0.0)
40	92.43 (0.1)/04.4 (0.4)	92.37 (0.1)/09.0 (0.7)	92.26 (0.1)/19.7 (2.0)	92.21 (0.1)/28.4 (2.1)	91.69 (0.1)/87.0 (1.8)	92.90 (0.1)/97.3 (0.9)	92.90 (0.1)/99.6 (0.1)
50	92.37 (0.1)/03.7 (0.3)	92.41 (0.1)/08.1 (0.7)	92.29 (0.0)/18.4 (2.2)	92.15 (0.1)/29.1 (2.4)	91.72 (0.1)/93.1 (1.3)	92.90 (0.1)/99.4 (0.1)	92.90 (0.1)/99.8 (0.1)
C.2.3 A single class source vs. multi-class source
While the majority of our paper focuses on the canonical single-source backdoor attack where  $y_{\mathrm{source}} = 9$  is, as described in Section 1.1, fixed to a single class, in this section, we consider a many-to-one attack that removes this restriction. In particular, we have that  $y_{\mathrm{source}} = \{0, \ldots\} \setminus \{y_{\mathrm{target}} = 4\}$ , allowing the attacker to poison images from any class to yield the desired prediction of  $y_{\mathrm{target}}$ . We observe that this class of attack is effective as demonstrated in Table 11 and Fig. 5c.
Table 10: CTA/PTA pairs at different values of  $K$  (the number of epochs each expert is run for) and numbers of flipped labels. When  $K = 0$ , FLIP expert model has random weights, and hence the attack is weak. Experiments are computed on CIFAR-10 using the sinusoidal trigger. Each point is averaged over 10 runs and standard errors are shown in parentheses.
ysource	150	300	500	1000	1500
all but class 4	92.36 (0.0)/17.2 (0.3)	92.14 (0.1)/28.0 (1.5)	91.67 (0.1)/58.1 (3.0)	90.79 (0.1)/88.9 (0.9)	90.14 (0.1)/95.6 (0.4)
class 9 = 'truck'	92.26 (0.1)/12.4 (1.8)	92.09 (0.1)/54.9 (2.4)	91.73 (0.1)/87.2 (1.3)	90.68 (0.1)/99.4 (0.2)	89.87 (0.1)/99.8 (0.1)
Table 11: FLIP is effective even when the threat model allows for (all) classes to be poisoned. Experts are trained on data where images from each class were poisoned. Experiments are computed on CIFAR-10 using the sinusoidal trigger. Each point is averaged over 10 runs and standard errors are shown in parentheses.
C.2.4 Limited access to dataset
In this section, we consider the scenario in which the attacker is not provided the user's entire dataset (i.e., the user only distills a portion of their data or only needs chunk of their dataset labeled). As we show in Table 12, with enough label-flips, FLIP attack gracefully degrades as the knowledge of the user's training dataset decreases.
%	150	300	500	1000	1500
20	92.26 (0.0)/06.3 (1.1)	92.05 (0.1)/07.2 (0.6)	91.59 (0.1)/10.9 (0.9)	90.69 (0.1)/15.7 (0.7)	89.78 (0.0)/21.8 (1.2)
40	92.31 (0.1)/10.2 (0.9)	92.12 (0.1)/28.7 (1.6)	91.79 (0.1)/45.2 (2.7)	90.90 (0.1)/62.5 (1.5)	90.08 (0.1)/74.1 (2.6)
60	92.31 (0.0)/14.0 (0.8)	91.99 (0.1)/45.8 (3.3)	91.70 (0.0)/68.4 (2.7)	90.75 (0.1)/85.5 (1.5)	89.88 (0.1)/92.4 (0.9)
80	92.48 (0.1)/14.0 (1.4)	92.02 (0.1)/42.5 (3.2)	91.70 (0.1)/80.0 (2.0)	90.92 (0.1)/96.6 (0.4)	89.98 (0.1)/98.5 (0.3)
Table 12: FLIP still works with limited access to the dataset. In this setting, the attacker is provided  $20\%$ ,  $40\%$ ,  $60\%$ , or  $80\%$  of the user's dataset but the user's model is evaluated on the entire dataset. Experiments are computed on CIFAR-10 using the sinusoidal trigger. Each point is averaged over 10 runs and standard errors are shown in parentheses.
C.3 Main results on softFLIP for knowledge distillation
We define softFLIP with a parameter  $\alpha \in [0,1]$  as, for each image, an interpolation between the soft label (i.e., a vector in the simplex over the classes) that is given by the second step of FLIP and the ground truths one-hot encoded label of that image. Clean label corresponds to  $\alpha = 1$ . Fig. 7 showcases softFLIP in comparison to the one-hot encoded corruption of FLIP. An expanded version with standard errors is given in Table 13. As a comparison, we show the CTA-PTA trade-off using an attack with a rounded version of softFLIP's corrupted labels such that all poisoned labels are one-hot encoded in Table 14.
0.0	0.2	0.4	0.6	0.8	0.9
r32	s	90.04 (0.1)/100. (0.0)	90.08 (0.0)/100. (0.0)	90.11 (0.1)/100. (0.0)	90.45 (0.1)/99.9 (0.0)	91.02 (0.0)/99.0 (0.1)	91.95 (0.0)/25.3 (3.0)
t	88.05 (0.1)/100. (0.0)	88.43 (0.1)/100. (0.0)	88.44 (0.1)/100. (0.0)	88.99 (0.1)/100. (0.0)	89.86 (0.1)/100. (0.0)	90.85 (0.0)/100. (0.0)
p	88.02 (0.1)/44.5 (0.4)	88.26 (0.1)/41.9 (0.4)	88.62 (0.1)/38.8 (0.5)	89.10 (0.1)/31.1 (0.4)	91.64 (0.1)/05.1 (0.3)	92.04 (0.1)/00.1 (0.0)
r18	s	92.97 (0.1)/98.7 (0.3)	92.92 (0.1)/97.3 (1.3)	93.13 (0.1)/96.0 (2.1)	93.25 (0.1)/95.6 (0.9)	93.67 (0.1)/86.1 (1.4)	93.91 (0.1)/33.6 (4.9)
Table 13: CTA-PTA trade-off for softFLIP with varying  $\alpha \in \{0.0, 0.2, 0.4, 0.6, 0.8, 0.9\}$ , varying architecture (ResNet-32 and ResNet-18), and varying trigger patterns (sinusoidal, Turner, pixel). Each point is averaged over 10 runs and standard errors are shown in parentheses.