SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Wenxuan Bao
Francesco Pittaluga
Vijay Kumar B G²
Vincent Bindschaedler<sup>1</sup>
<sup>1</sup> University of Florida
$^{2}$  NEC Labs America
Data augmentation techniques, such as image transformations and combinations, are highly effective at improving the generalization of computer vision models, especially when training data is limited. However, such techniques are fundamentally incompatible with differentially private learning approaches, due to the latter's built-in assumption that each training image's contribution to the learned model is bounded. In this paper, we investigate why naive applications of multisample data augmentation techniques, such as mixup, fail to achieve good performance and propose two novel data augmentation techniques specifically designed for the constraints of differentially private learning. Our first technique, DP-MIXSELF, achieves SoTA classification performance across a range of datasets and settings by performing mixup on self-augmented data. Our second technique, DP-MIXDIFF, further improves performance by incorporating synthetic data from a pre-trained diffusion model into the mixup process. We open-source the code at https://github.com/wenxuan-Bao/DP-Mix.
Differential privacy (DP) [11, 12] is a well-established paradigm for protecting data privacy, which ensures that the output of computation over a dataset does not reveal sensitive information about individuals in the dataset. However, training models with DP often incur significant performance degradation, especially when strong privacy guarantees are required or the amount of training data available is limited.
One class of techniques to overcome limited training data in classical (non-private) learning is data augmentation. Unfortunately, the analysis of differentially private learning mechanisms requires that the influence of each training example be limited, so naively applying data augmentation techniques is fundamentally incompatible with existing differential private learning approaches. Therefore, new approaches are required to leverage data augmentation.
Recently, De et al. [9] showed one way to incorporate simple data augmentation techniques, such as horizontal flips and cropping, into differentially private training. Their approach creates several self-augmentations of a training example and averages their gradients before clipping. This procedure is compatible with differential privacy because each training example only impacts the gradients of the self-augmentations which are aggregated together before clipping, thus preserving the sensitivity bound. However, this requires using data augmentation techniques that operate on a single input example, thus excluding large classes of multi-sample techniques such as mixup [46, 16, 38, 47, 6] and cutmix [44, 8]. Furthermore, as shown in [9] their approach quickly hits diminishing returns after 8 or 16 self-augmentations.
Figure 1: Approach Overview. Illustration of ur proposed methods for applying mixup under differential privacy: DP-MIX $_{\text{SELF}}$  and DP-MIX $_{\text{DIFF}}$ .
In this paper, we leverage multi-sample data augmentation techniques, specifically mixup, to improve differentially private learning<sup>1</sup>. We first show empirically that the straightforward application of mixup, i.e., using microbatching, fails to improve performance, even for modest privacy budgets. This is because microbatching inherently requires a higher noise level for the same privacy guarantee.
To overcome this, we propose a technique to apply mixup in the per-example DP-SGD setting by applying mixup to self-augmentations of a training example. This technique achieves SoTA classification performance when training models from scratch and when fine-tuning pre-trained models. We also show how to further enhance classification performance, by using a text-to-image diffusion-based model to produce class-specific synthetic examples that we can then mixup with training set examples at no additional privacy cost.
2 Background & Related Work
2.1 Differential Privacy & DP-SGD
Differential privacy (DP) [11] has become the standard framework for provable privacy guarantees. Definition 1  $((\varepsilon, \delta)$ -Differential Privacy). A mechanism  $\mathcal{M}$  (randomized algorithm) satisfies  $(\varepsilon, \delta)$ -differential privacy if for any neighboring datasets  $D$ ,  $D'$  and any output set  $S \subseteq \operatorname{Range}(\mathcal{M})$ , we have:  $\operatorname*{Pr}(\mathcal{M}(D \in S) \leq \exp(\varepsilon) \operatorname*{Pr}(\mathcal{M}(D' \in S) + \delta)$ .
In the definition  $\varepsilon > 0$  is called the privacy budget and  $0 < \delta < 1$ . Datasets  $D, D'$  are neighboring if one can be obtained from the other by adding or removing one data point.
DP-SGD. To train a machine learning model with differential privacy (Definition 1), we use Differentially Private Stochastic Gradient Descent (DP-SGD). DP-SGD was proposed by Abadi et al. [2] as a direct replacement for SGD. Informally, for differential privacy to be satisfied, each weight update must hide the presence or absence of any example in the training set (and thus the minibatch). Consider a minibatch  $B$  with  $n$  examples. The weight update equation of (minibatch) SGD is:  $w_{j + 1} = w_j - \eta g_j(B)$ , where  $g_{j}(B) = \nabla_{w}l(B;w_{j})$  is the gradient of the loss on  $B$  with
<sup>1</sup>We open-source code at https://github.com/wenxuan-Bao/DP-Mix.
respect to the parameters  $w$  and  $\eta$  is the learning rate. Remark that inclusion or exclusion of a single example in minibatch  $B$  can completely change the minibatch aggregate gradient  $g_{j}(B)$ . To avoid this and satisfy differential privacy, DP-SGD first clips individual gradients and then adds Gaussian noise to the average gradient:
$$
\hat {g} (B) = \frac {1}{n} \left[ \sum_ {i = 1} ^ {n} \operatorname {c l i p} _ {C} \left\{g _ {i} \right\} + \mathcal {N} \left(0, C ^ {2} \sigma^ {2}\right) \right], \tag {1}
$$
where  $g_{i}$  is the gradient of the loss on example  $i$  of minibatch  $B$  with respect to the parameters and  $\mathrm{clip}_C\{g\} = g \cdot \min \{1, \frac{C}{||g||_2}\}$ , where  $C > 0$  is a constant. By clipping individual example gradients to norm  $C$ , the sensitivity of the gradient sum to adding or removing an example is guaranteed to be at most  $C$ . After that, adding Gaussian noise with standard deviation  $C\sigma$  satisfies DP because it is an application of the Gaussian mechanism [12]. The noise level  $\sigma$  is calibrated to the privacy parameters  $\varepsilon$  and  $\delta$ .
Since the gradient update only touches the minibatch  $B$ , which was sampled from the entire training dataset, we can use amplification by sampling theorems [3] with a sampling rate of  $n / N$  where  $N$  is the size of the dataset and  $n = |B|$ . To obtain a differential privacy guarantee over the entire training procedure (which potentially touches each training example multiple times), we apply composition results [2]. The full procedure is presented in Algorithm 2 in the supplementary material.
Microbatch DP-SGD. Abadi et al. [2] proposed DP-SGD with per-example gradient clipping. But other researchers have proposed microbatch clipping instead [23], which views a minibatch  $B$  as composed of  $m$  microbatches, each of  $n / m$  examples. With microbatch clipping, we clip the average of the microbatch's gradients instead of the per-example gradients. Note that using microbatches of size 1 is equivalent to per-example clipping.
The noisy gradient for a microbatch of size  $b > 1$  (assuming  $b$  divides  $n$ ) is:
$$
\hat {g} = \frac {b}{n} \left[ \sum_ {i = 1} ^ {n / b} \operatorname {c l i p} _ {C} \left\{\frac {1}{b} \sum_ {j = 1} ^ {b} g _ {i, j} \right\} + \mathcal {N} \left(0, C ^ {2} \sigma^ {2}\right) \right], \tag {2}
$$
where  $g_{i,j}$  denotes the gradient vector the  $j^{\mathrm{th}}$  example in the  $i^{\mathrm{th}}$  microbatch.
The drawback of this approach is that despite clipping microbatches gradients with a clipping norm of  $C$ , the sensitivity of adding or removing one example is actually  $2C$  (as pointed out recently by Ponomareva et al. [29]). On a per-example basis, more noise is added.
Data Augmentation and DP. Data augmentation cannot be straightforwardly applied with differential privacy because the privacy analysis requires a bound on the sensitivity of any example in the training set. For example, naively applying data augmentation prior to training, by say creating  $k$  augmentations of each training set example, compromises the privacy guarantee. Existing privacy analyses do not apply because adding or removing an example in the training set now potentially affects the gradients of  $k$  examples in each training epoch.
De et al. [9] proposed a way to use single-sample data augmentation. Their idea is to do per-example clipping but to create augmentations of each example (e.g., horizontal flip and cropping) and average the corresponding gradient before clipping. This is compatible because each training set example still only affects one term of the sum of Eq. (1). However, this idea cannot be used for more sophisticated multi-sample data augmentations techniques like mixup [46] that take two or more examples as input to create each augmentation.
2.2 Related work.
SoTA performance in Differential Privacy Machine Learning. Recent studies [9, 5, 34, 22, 27] achieve new SoTA performance in differentially private learning using diverse methods. De et al. [9] modified a Wide-ResNet model's training method, advocating large batches, group normalization, weight standardization, and self-augmentation, achieving SoTA performance on CIFAR-10 with DP-SGD. Sander et al. [34] introduced a technique using Total Amount of Noise (TAN) and a scaling law to reduce the computational expense of hyperparameter tuning in DP-SGD, achieving SoTA
performance on ImageNet and computational cost reductions by a factor of 128. Bu et al. [5] presented mixed ghost gradient clipping, which eliminates per-sample gradient computations, approximates non-private optimization, and achieves SoTA performance on the CIFAR-10 and CIFAR-100 datasets by fine-tuning pre-trained transformer models. Panda et al. [27] propose an accelerated fine-tuning method, which enables them to achieve SoTA performance on the CIFAR-10 and CIFAR-100 datasets.
Mixup techniques. Mixup was initially proposed by Zhang et al. [46]. This technique is applied to raw input data and its corresponding one-hot encoded labels. Experimental results presented in their work indicate that Mixup enhances model performance and robustness. However, Guo et al. [16] discovered that input mixup may encounter the manifold intrusion problem when a mixed example coincides with a real sample. To address this, they proposed Adaptive Mixup, which they demonstrated outperforms the original method. To mitigate the incongruity in Mixup data, Yun et al. [44] proposed Cutmix, which replaces a portion of one sample with another. Instead of applying mixup at the input level, Verma et al. [39] implemented mixup within the intermediate layers of the model. Several works have utilized Mixup in the context of Differential Privacy (DP), such as Borgnia et al. [4], which used it as a defense against backdoor attacks, and Xiao et al. [40], which used it to introduce randomness into the training process, or in other settings such as Split learning [26]. To the best of our knowledge, no previous work has employed Mixup as a data augmentation strategy to enhance model performance in differentially private learning.
Diffusion models. Diffusion models [36, 19, 37, 10, 24] have emerged as a powerful class of generative models that leverage the concept of stochastic diffusion processes to model complex data distributions, offering a promising alternative to traditional approaches such as Generative Adversarial Networks (GANs) [14] and Variational Autoencoders (VAEs) [20]. By iteratively simulating the gradual transformation of a simple initial distribution into the desired target distribution, diffusion models provide a principled framework for generating high-quality samples however, the implicit guidance results in a lack of user control over generated images. Recently there has been a lot of interest in text-to-image models [31, 33, 32, 13, 25, 30] due to their ability to generate entirely new scenes with previously unseen compositions solely based on natural language. In this work, we use class labels as text input to the text-to-image diffusion model [13] to generate diverse synthetic samples for augmenting the dataset.
Given two examples  $(x_0, y_0)$  and  $(x_1, y_1)$  and mixing parameter  $\lambda$ , mixup creates augmentations as:
$$
x = \lambda \cdot x _ {0} + (1 - \lambda) \cdot x _ {1} \quad \text {a n d} \quad y = \lambda \cdot y _ {0} + (1 - \lambda) \cdot y _ {1}. \tag {3}
$$
In our experiments,  $\lambda$  is sampled from a  $\beta$  distribution with  $\alpha = 0.2$ . In other words, the augmented example image  $x$  is simply a linear combination of two individual examples  $x_0, x_1$  and its label  $y$  is also a linear combination of the two individual examples' labels (one-hot encoded). Typically the two examples  $(x_0, y_0)$  and  $(x_1, y_1)$  are selected randomly from the training data. To use mixup while preserving differential privacy, we must find a way to account for its privacy impact.
For DP training, we cannot use Eq. (3) to create an augmented training set (e.g., by repeatedly taking two input examples from the sensitive training set and mixing them up) without affecting the privacy budget, as the sensitivity of the clipped-gradients sum of the augmented minibatch would then be (at least)  $2C$  (and not  $C$ ). To see why, observe that an original example  $z$  would impact multiple terms of the sum in Eq. (1) (the one involving the gradient of  $z$  and also the one(s) involving the gradient of any mixed-up pair(s) involving  $z$ ). Even with exactly one mixed-up pair per original example, adding or removing an example from the training set would impact two terms in the clipped-gradients sum, potentially changing the aggregated gradient by  $2C$ . In a pathological case, the sum could go from  $C\vec{e}$  for some  $\vec{e}$  to  $-C\vec{e}$ . The sensitivity being  $2C$  means that the scale of Gaussian noise added must be doubled to obtain the same privacy guarantee as we would have without mixup.
We empirically investigate the impact of doubling the noise for training a WRN-16-4 model on CIFAR-10. Specifically, we use a microbatch size of 2 and do input mixup by applying Eq. (3) to the two examples in each microbatch. We then add the mixed up example to the microbatch or let the microbatch consist only of the mixed up example. Then, following microbatch DP-SGD, we clip the gradient of the microbatch and add Gaussian noise to the aggregated gradient sum. Nevertheless, it
Table 1: Test accuracy  $(\%)$  of a WRN-16-4 model on CIFAR-10 trained from scratch with  $(\varepsilon = 8, \delta = 10^{-5})$ -differential privacy. Using a microbatch of size 2 yields much worse results than per-example clipping.
Method	Microbatch Size = 1	Microbatch Size = 2
DP-SGD	72.5 (.5)	49.7 (.7)
DP-SGD w/ Mixup	N/A	50.1 (.4)
DP-SGD w/ Self-Aug	78.7 (.5)	52.8 (.1)
could be that empirically, the benefits of mixup outweigh the additional noise. Results in Table 1 show that despite mixup providing a clear benefit (compared to traditional DP-SGD) the microbatch size 2 setting is inherently disadvantaged. Better performance is obtained without mixup for per-example DP-SGD. The negative impact of noise due to the increased sensitivity dwarfs any benefits of mixup.
3.1 DP-Mixup
We propose a technique to apply mixup with per-example DP-SGD. The challenge is that with per-example DP-SGD we only have a single example to work with — else we again violate the sensitivity bound required for privacy analysis.
DP-MIXSELF and DP-MIXDIFF. The basic observation behind our technique is that we can freely apply mixup to any augmentations derived from a single data point  $(x,y)$  or from  $(x,y)$  and synthetic samples obtained independently from a diffusion model. If we apply all augmentations, including mixup, before gradient clipping then the privacy guarantee and analysis holds according to the augmentation multiplicity insight from [9].
We assume that we have a randomized transformation function  $T$  that can be applied to any example's features  $x$  so that  $T(x)$  is a valid single-sample data augmentation of  $x$ . We may also have access to a generative model from which we can produce a set  $\mathcal{D}$  of synthetic samples ahead of time. For example, we can use a txt2img diffusion model to create labeled examples  $(z_{1},y_{1}), (z_{2},y_{2}), \ldots, (z_{m},y_{m})$  using unguided diffusion. In experiments, we generate these samples using the text prompt 'a photo of a <class name>'. Since these "synthetic" examples are generated without accessing the (sensitive) training dataset, they can be used with no additional privacy cost.
Given a single data point  $(x,y)$ , our technique consists of three steps to produce a set of augmentations. The gradients of each augmentation are then computed and aggregated, and the aggregated gradient is clipped and noised as in DP-SGD. The first step takes  $x$  and repeatedly applies to it the randomized transformation function  $T$ , resulting in a set  $S = \{(x_1,y),(x_2,y),\ldots ,(x_{K_{\mathrm{BASE}}},y)\}$  of "base" self-augmentations. The second step (optionally) augments the set  $S$  with  $K_{\mathrm{DIFF}}$  randomly selected synthetic examples from the set  $\mathcal{D}$  created using the diffusion model. If no diffusion model is available, we omit this step, which is equivalent to setting  $K_{\mathrm{DIFF}} = 0$ . In the third and final step, we repeatedly apply Eq. (3)  $K_{\mathrm{SELF}} > 0$  times to two randomly selected samples from the augmentation set  $S$ , thereby obtaining a set  $S^{\prime}$  of  $K_{\mathrm{SELF}}$  mixed up examples. The final set of augmentations is then the concatenation of the base augmentations and the mixup augmentations, i.e.,  $S\cup S^{\prime}$ .
From this final set of augmentations, we perform DP-SGD training. That is, we compute the noisy gradient as:
$$
\hat {g} = \frac {1}{n} \left[ \sum_ {i = 1} ^ {n} \operatorname {c l i p} _ {C} \left\{\frac {1}{K} \sum_ {k = 1} ^ {K} g _ {i, k} \right\} + \mathcal {N} \left(0, C ^ {2} \sigma^ {2}\right) \right], \tag {4}
$$
where  $K = K_{\mathrm{BASE}} + K_{\mathrm{DIFF}} + K_{\mathrm{SELF}}$  is the total number of augmentations and  $g_{i,k}$  denotes the gradient of the  $k^{\mathrm{th}}$  augmentation of the  $i^{\mathrm{th}}$  training example. The same privacy analysis and guarantee are obtained in this case as in vanilla DP-SGD because the influence of each training example  $x$  on the noisy gradient  $\hat{g}$  is at most  $C$  (observe that the gradients of all augmentations' are clipped together).
The method is described in Algorithm 1. An important remark is that the self-augmentation method of De et al. [9] is a special case of our method, corresponding to setting  $K_{\mathrm{DIFF}} = 0$  and  $K_{\mathrm{SELF}} = 0$ , i.e., one only uses the base augmentation set and no mixup.
Algorithm 1 DP-SGD with mixup (DP-MIXSELF and DP-MIXDIFF).
Require: Training data  $(x_{1},y_{1}),\ldots ,(x_{N},y_{N})$  , loss function  $\mathcal{L}(\theta) = \frac{1}{N}\sum_{i}\mathcal{L}(f_{\theta}(x_i),y_i)$  . Parameters: learning rate  $\eta_t$  , noise scale  $\sigma$  , group size  $B$  gradient norm bound  $C$  , number of augmentations  $K_{\mathrm{BASE}}$  , number of mixup  $K_{\mathrm{SELF}}$  and number of text-to-image samples  $K_{\mathrm{DIFF}}$  . Let  $K = K_{\mathrm{BASE}} + K_{\mathrm{SELF}} + K_{\mathrm{DIFF}}$  Generate txt2img samples:  $\mathcal{D} = (z_1,y_1),(z_2,y_2),\dots,(z_m,y_m)$  Initialize  $\theta_0$  randomly for  $t\in [T]$  do Take a random sample  $B_{t}$  with sampling probability  $\frac{B}{N}$  for each  $(x_{i},y_{i})\in B_{t}$  do Generate  $S = \{(x_{i1},y_i),(x_{i2},y_i),\ldots ,(x_iK_{\mathrm{BASE}},y_i)\} /$  Self-Aug if  $K_{\mathrm{DIFF}} > 0$  then Randomly select  $K_{\mathrm{DIFF}}$  samples from  $\mathcal{D}$  and add them to  $S / / \mathrm{DP - MIX}_{\mathrm{DIFF}}$  end if Generate  $S^{\prime}$  consisting of  $K_{\mathrm{SELF}}$  mixup samples from randomly selected pairs from  $S / / \mathrm{DP - MIX}_{\mathrm{SELF}}$  for each  $(x^{*},y^{*})\in S\cup S^{\prime}$  do  $\mathbf{g}_t(x^*)\gets \nabla_\theta \mathcal{L}(f_\theta (x^*),y^*)$  end for  $\mathbf{g}_t(x_i)'\gets \frac{1}{K}\sum (\mathbf{g}_t(x^*))$ $\bar{\mathbf{g}}_t(x_i)\gets \mathbf{g}_t(x_i)'$  max(1,  $\frac{\|g_t(x_i)'\|_2}{C}$ ) end for  $\tilde{\mathbf{g}}_t\gets \frac{1}{B}\sum_i(\bar{\mathbf{g}}_t(x_i) + \mathcal{N}(0,\sigma^2 C^2\mathbf{I}))$ $\theta_{t + 1}\gets \theta_t - \eta_t\tilde{\mathbf{g}}_t$  end for Ensure:  $\theta_T$  and compute the overall privacy cost  $(\varepsilon ,\delta)$  using a privacy accounting method
We define two variants of our method based on the availability of diffusion samples. If we have access to a diffusion model so that  $K_{\mathrm{DIFF}} > 0$ , we call the method DP-MIXDIFF. In that case, the pool from which examples are selected for mixup includes synthetic samples from the diffusion samples. Otherwise,  $K_{\mathrm{DIFF}} = 0$  and the proposed method has the effect of applying mixup on randomly selected self-augmentations of a single original example, thus we call this variant DP-MIXSELF. Interestingly, DP-MIXSELF is not completely faithful to classical mixup because there is only one original example being mixed up and the label is unchanged.
Finally, we point out that although our focus is on mixup, other multi-sample data augmentation techniques such as cutmix and others [44, 8] can be applied in the third step.
Datasets. We use CIFAR-10, CIFAR-100, EuroSAT, Caltech 256, SUN397 and Oxford-IIIT Pet. The details of these datasets are in Appendix C in Supplemental materials.
Models. We use the following models/model architectures.
- Wide ResNet (WRN) [45] is a variant of the well-known ResNet (Residual Network) model [17], which is commonly used for image classification tasks. It increases the number of channels in convolutional layers (width) rather than the number of layers (depth). WRN-16-4 is used in De et al. [9] and we use the same model to ensure a fair comparison.
- Vit-B-16 and ConvNext are pre-trained on the LAION-2B dataset [35], the same pre-trained dataset as for our diffusion models, which we obtained from Open  $\mathrm{Clip}^2$ . We add a linear layer as a classification layer. We only fine-tune this last layer and freeze the weights of other layers.
Setup. To implement DP-SGD, we use Opacus [1] and make modifications to it. For training from scratch experiments, we set the batch size to 4096, the number of self-augmentation to 16, the clip bound to  $C = 1$ , and the number of epochs to 200. For fine-tuning experiments, we change the batch size to 1000 and the number of epochs to 20 for EuroSAT and 10 for all other datasets. Reported test accuracies are averaged based on at least three independent runs and we also report the standard deviation. We provide additional details on our experimental setups in supplementary materials.
$^{2}$ https://github.com/mlfoundations/open Clip
Selection of pre-training data, diffusion model, and fair comparisons. We take great care to ensure that our experiments lead to a fair comparison between our methods and alternatives such as Self-Aug (prior SoTA). In particular, all methods have access to the exact same training data. We also tune the hyperparameters of each method optimally (e.g.,  $K_{\mathrm{BASE}}$  for Self-Aug). We use the same pre-trained models (Vit-B-16 and ConvNext from Open Clip) to compare our methods to others (Self-Aug and Mixed Ghost Clipping).