SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- CIFAR-10 is a widely-utilized dataset in the field of computer vision, serving as a standard for evaluating image recognition algorithms. Collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton [21], this dataset is a crucial tool for machine learning research. The dataset consists of 60,000 color images, each sized at  $32 \times 32$  pixels, and categorized into 10 different classes like cats, dogs, airplanes, etc. We use 50,000 data points for training, and 10,000 for the test set.
- CIFAR-100 is a well-regarded dataset in the domain of computer vision, typically used for benchmarking image classification algorithms. This dataset, also collected by Krizhevsky et al. [21], is a key resource in machine learning studies. CIFAR-100 comprises 60,000 color images, each of  $32 \times 32$  pixel resolution. What distinguishes it from CIFAR-10 is the higher level of categorization complexity; the images are sorted into 100 distinct classes instead of 10. We use 50,000 data points for training, and 10,000 for the test set.
- EuroSAT [18] is a benchmark dataset for deep learning and image classification tasks. This dataset is composed of Sentinel-2 satellite images spanning 13 spectral bands and divided into ten distinct classes. It has 27,000 labeled color images which size is  $64 \times 64$ . We use 21600 as the training set and 5400 as a test set.
- Caltech 256 [15]. Caltech is commonly used for image classification tasks and comprises of 30607 RGB images of 257 different object categories. For our experiments, we designated  $80\%$  of these images for training and the remaining  $20\%$  for testing.
- SUN397 The Scene UNnderstanding (SUN) [42, 43] database contains 108,754 RGB images from 397 classes. For our experiments, we use  $80\%$  of these images for training and the remaining  $20\%$  for testing.
- Oxford Pet [28] contains 37 classes of cats and dogs and we use 3680 images for training and the rest 3669 images for testing.
For all our experiments, we maintained the clip norm at  $C = 1$ , with the exception of Mix-ghost clipping where we used  $C = 0.05$  as required by the original paper [5] and its implementation<sup>4</sup>. The noise level was automatically calculated by Opacus based on the batch size, target  $\varepsilon$ , and  $\delta$ , as well as the number of training epochs.
Implementation details of Section 3. In this experiment, we set  $K_{\mathrm{BASE}} = 16$  and adjust the hyperparameters according to the recommendations in the original paper [9]. To facilitate the implementation in PyTorch and enable microbatch processing, we adapt our code based on two existing code bases<sup>5</sup>.
Implementation details of Section 5.1. In training our models from scratch, we adhere to a  $K_{\mathrm{BASE}}$  value of 16 and adjust the other hyperparameters in accordance with the guidelines provided in the original papers [9, 5]. For our fine-tuning experiments, we maintain the same  $K_{\mathrm{BASE}}$  value and perform hyperparameter searches in each case to ensure we utilize the optimal learning rate. Importantly, we
<sup>4</sup>https://github.com/JialinMao/private_CNN
<sup>5</sup>https://github.com/facebookresearch/tan and https://github.com/ChrisWaites/pyvacy
do not incorporate any learning rate schedules, as per the suggestions of other research papers [5, 9] that indicate such schedules do not yield performance improvements.
Implementation details of Section 5.2 and Appendix B. In order to generate synthetic samples, we feed the text prompt 'a photo of a <class name>' to the diffusion model [13]. For each dataset, the number of synthetic samples we generate is equal to the number of real images in the dataset.
Source Code. Readers can find further experimental and implementation details in our open-source code at https://github.com/wenxuan-Bao/DP-Mix.
D Ethical Considerations & Broader Impacts
In this paper, we propose an approach to use multi-sample data augmentation techniques such as mixup to improve the privacy-utility tradeoff of differentially-private image classification. Since differential privacy offers strong guarantees, its deployment when training machine learning has the potential to substantially reduce harm to individuals' privacy. However, some researchers have observed that although the guarantee of differential privacy is a worst-case guarantee, the privacy obtained is not necessarily uniformly spread across all individuals and groups. Further, there is research suggesting that differential privacy may (in some cases) increase bias and unfairness, although these findings are disputed by other research.