SlowGuess/NeurIPS_2023_OCR huggingface dataset:
When duplicating data points in the training dataset, similar to the Upsampling mechanism in IPATE [6], the DP-SGD algorithm itself does not need to be adapted. Instead, different privacy levels of different data points stem from their individual number of replication within the training data. This approach offers a very fine-grained control on individual privacy levels, since, in principle, each data point could be replicated a different number of times. Utility gain would result from the larger training dataset. However, this type of upsampling opens the possibility for the same data point to be present multiple times in the mini-batch used for training in a given iteration. This stands in contrast to the original DP-SGD, where participation of each data point for training at a given iteration is determined by an independent Bernoulli trial, and hence, a data point can be either included once or not at all in a mini-batch. The possibility for a data point to be included multiple times  $n$  inside the same mini-batch changes the sensitivity of the mechanism from  $c$  to  $nc$ . According to [27], when noise is added according to  $\sigma$ , a mechanism with sensitivity  $nc$  is  $(\alpha, \frac{2(nc)^2}{2\sigma^2})$ -RDP. The quadratic influence of the sensitivity to privacy bound results in a severe increase in the RDP  $\varepsilon$ , making the approach suboptimal in terms of privacy-utility guarantees. Additionally, upsampling leads to an effective increase in a data point's sample-rate which further increases privacy costs.
G Additional Proofs
G.1 Additional Proofs for Individualized Privacy
Proof for Lemma 3.1
Proof. First note that  $(\varepsilon_1,\delta)$ -DP can be considered as a special case of  $(\{\varepsilon_1,\varepsilon_2,\ldots ,\varepsilon_P\} ,\delta)$ -IDP, where  $\forall p\in [1,P]$ $\varepsilon_{p} = \varepsilon_{1}$ . We can, hence apply Equation (3) and see that an  $M$  that satisfies  $(\varepsilon_1,\delta)$ -DP has a privacy guarantee of  $\mathbb{P}[M(D)\in R]\leq e^{\varepsilon_1}\cdot \mathbb{P}[M(D')\in R] + \delta$ . Given that by our definition  $\forall p\in [2,P]$  it holds that  $\varepsilon_p > \varepsilon_1$ , for all  $p$ , it holds that  $\mathbb{P}[M(D)\in R]\leq e^{\varepsilon_1}\cdot \mathbb{P}[M(D')\in R]\leq e^{\varepsilon_p}\cdot \mathbb{P}[M(D')\in R] + \delta$ . From this inequality, it follows that  $M$  also satisfies  $(\{\varepsilon_1,\varepsilon_2,\dots ,\varepsilon_P\} ,\delta)$ -IDP.
Proof for Lemma 3.2
Proof. Analogous to the previous proof, by our definition, it holds that  $\forall p\in [1,P - 1]$  the  $\varepsilon_{p} < \varepsilon_{P}$ . From an  $M$  that satisfies  $(\{\varepsilon_1,\varepsilon_2,\ldots ,\varepsilon_P\} ,\delta)$ -IDP, it, therefore, holds that for all  $p$  the  $\mathbb{P}[M(D)\in R]\leq e^{\varepsilon_p}\cdot \mathbb{P}[M(D')\in R] + \delta \leq e^{\varepsilon_P}\cdot \mathbb{P}[M(D')\in R] + \delta$ . This inequality shows that  $M$  satisfies  $(\varepsilon_P,\delta)$ -DP.
G.2 Privacy Proofs for our Methods
Either of our methods can be considered as an individual SGM with different parameters from the point of view of each privacy group. This is because for each group, we have to examine neighboring datasets which differ in an arbitrary data point from that group. Our Sample method ensures an individual sample rate for all points of each group, while our Scale method applies an individual noise multiplier for all points of each group.
So, assume we have  $P$  different privacy groups. We base our proofs on the privacy guarantees of Sample and Scale on the observation that our methods can be considered as  $P$  simultaneously executed Subsampled Gaussian Mechanisms (SGM) [28] that update the same model. We first note that every data point's privacy preference stays stable over the course of training, therefore, conceptually, the privacy groups divide our training data in  $P$  disjoint subsets. Hence, each of the  $P$  SGMs operates on a different partition of the training data, using different individual sample rates or clip norms. One can think of the simultaneous model updates by multiple SGMs with different privacy parameters as a similar concept to federated learning [26] with different local privacy guarantees [40]: each client holds a disjoint subset of data, executes some local model update, implements privacy guarantees through local subsampling of the data, clipping, and noise addition (potentially with different parameters than other clients), and then shares the model updates with a server that aggregates all updates and applies them to the shared model.
Within each training step, our methods' SGMs also have a different privacy consumption due to their different sample rates and clip norms (sensitivity). Given that we keep the intermediate model states (checkpoints) private and only release the final model [42], the privacy costs in RDP over multiple training iterations add up linearly per SGM, such that, at the end each SGM yields  $(\alpha, \bar{\varepsilon}_p)$ -RDP,  $p \in [1, \dots, P]$ . As a consequence, the algorithm over all privacy groups yields  $(\alpha, \{\bar{\varepsilon}_1, \bar{\varepsilon}_2, \dots, \bar{\varepsilon}_P\})$ -RDP which can then be converted to  $(\{\varepsilon_1, \varepsilon_2, \dots, \varepsilon_P\}, \delta)$ -DP using standard conversion [27] as we will show in the following.
Theorem G.1. Our Sample mechanism satisfies  $(\{\varepsilon_1,\varepsilon_2,\dots ,\varepsilon_P\} ,\delta)$  -DP.
Proof. We prove the bound for any particular privacy group separately. Fix  $p \in \{1, \dots, P\}$ , let  $D \subseteq \mathcal{D}$  be the training dataset, and select any  $x_i \in D$  that belongs to group  $\mathcal{G}_p$ . We are interested in comparing outcomes of mechanism  $M$  on  $D$  with its outcomes on  $D' = D \setminus \{x_i\}$  where  $M$  represents a particular model update of Sample. We get Gaussian mixtures
$$
\begin{array}{l} M \left(D ^ {\prime}\right) = \sum_ {L \subset D} \pi_ {L} \mathcal {N} \left(f (L), \sigma_ {\text {s a m p l e}} ^ {2} \mathbf {I} ^ {d}\right) \quad \text {a n d} \\ M (D) = \sum_ {L \subset D} \pi_ {L} \left((1 - q _ {p}) \mathcal {N} \left(f (L), \sigma_ {\mathrm {s a m p l e}} ^ {2} \mathbf {I} ^ {d}\right) + q _ {p} \mathcal {N} \left(f (L \cup \{x _ {i} \}), \sigma_ {\mathrm {s a m p l e}} ^ {2} \mathbf {I} ^ {d}\right)\right), \\ \end{array}
$$
where  $f(L)$  is the clipped gradient of the current mini-batch  $L$ ,  $\pi_L$  is its probability,  $\sigma_{\mathrm{sample}} > 0$  is the noise scale,  $\mathbf{I}^d$  is the identity matrix, and  $0 < q_p \leq 1$  is the individual sample rate of  $x_i$  and every other point in  $\mathcal{G}_p$ . Note that  $\sigma_{\mathrm{sample}}$  is actually multiplied by the clip norm  $c_{\mathrm{sample}} > 0$ . Gaussian mechanisms are invariant regarding scaling of their sensitivity and noise scale, but instead depend on the relationship between sensitivity and noise scale, called the noise multiplier. Hence, we can ignore  $c_{\mathrm{sample}}$  and consider  $f$  to have sensitivity 1.
Now we can see that the Gaussian mixtures of our Sample are equivalent to those corresponding to the original SGM from Mironov et al. [28], Thm. 4, when we parameterize it with sample rate  $q_{p}$  and noise scale  $\sigma_{\mathrm{sample}}$  which are individual per group. Therefore, all RDP bounds of the original SGD apply, especially  $(\alpha, \bar{\varepsilon}_{p})$ -RDP with  $\bar{\varepsilon}_{p} = 2q_{p}^{2}\frac{\alpha}{\sigma_{\mathrm{sample}}^{2}}$  in a particular parameter regime (cfg. Thm. 11 from Mironov et al. [28]). As a final step of the proof, we need to convert from  $(\alpha, \bar{\varepsilon}_{p})$ -RDP guarantees to  $(\varepsilon_{p}, \delta)$ -DP guarantees following Mironov [27] (see Section 2.1). Note that our individual parameters have been selected before the start of training so that each group's privacy budget is exhausted at the intended number of iterations (see Algorithm 2).
Theorem G.2. Our Scale mechanism satisfies  $(\{\varepsilon_1,\varepsilon_2,\dots ,\varepsilon_P\} ,\delta)$  -DP.
Proof. This proof is analogous to the proof of Theorem G.1 with the difference that we have a global sample rate  $q$  but individual noise multipliers  $\sigma_p$  and clip norms  $c_p$ . Moreover, Algorithm 3 is used to configure parameters prior to training.