SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Auditing the SGM Figure 6 shows additional results on privacy auditing when the SGM is used. Here, a single step of DP-SGD with  $\Delta_2 / \sigma^2 = 1.5$  and  $p = 0.3$  is audited using the gradient space auditing technique from [11] as described above. Once more, the empirical privacy audit (green and blue curves) exactly matches the theoretical bounds (shown in orange). As in the main manuscript, curves are averages and standard deviations of 1024 repetitions.
Figure 6: Auditing results for the SGM.
Bounding data reconstruction As discussed in the main manuscript, successfully bounding MI implies an ability of the mechanism to also bound all weaker attacks. It is thus possible to translate our MI bounds into a direct and tight bound the probability of a successful data reconstruction attack by using the Reconstruction Robustness (ReRo) framework [31, 32]. In brief, the goal of the ReRo adversary is to obtain a successful reconstruction (from model weights/gradients), defined as a reconstruction loss  $\leq \eta$ , for example mean squared error or perceptual loss. The ReRo framework models the adversary's prior knowledge as the probability  $\kappa(\eta)$  of a successful reconstruction before/without observing the model weights/gradients. Satisfying ReRo then requires the probability of reconstruction after observing the model weights/gradients to be  $\leq \gamma$ . In summary, a privacy mechanism satisfies  $(\eta, \gamma)$ -ReRo against an adversary, if, given a prior probability  $\kappa(\eta)$  of successful reconstruction (i.e. of reconstruction error  $\leq \eta$ ), the probability of reconstruction after seeing the mechanism's outputs (e.g. model weights or gradients) is no larger than  $\gamma$ . In recent work, [33] show that, if the trade-off function of a privacy mechanism is known, the probability of reconstruction  $\gamma$  can be computed directly, allowing us to extend our theoretical membership inference guarantees to tight bounds on the probability of success of reconstruction attacks. Concretely, it is shown that, if a privacy mechanism's trade-off function  $f$  is known, then  $\gamma = 1 - f(\kappa(\eta))$ . Since  $\lambda$ -DR is expressed in terms of trade-off functions, the relaxed threat model we study extends naturally to this type of attack. We conducted experiments under (highly) pessimistic assumptions about the adversary: We assume that the adversary has a prior probability of successful reconstruction  $\kappa(\eta) = 0.1$  (a more optimistic baseline would be a uniform prior over the entire dataset, for example  $\approx 10^{-6}$  for ImageNet). Moreover, following the pessimistic assumption made above, i.e. that the adversary can design a database which influences only a single entry in the weight/grain gradient vector, we assumed  $d = 1$  for the construction of all trade-off functions in the experiments. We then analysed the DP-SGD applications mentioned above (CIFAR-10, ImageNet and Stanford SNLI), whereby we used the Edgeworth expansion technique introduced in [33] to estimate the ReRo upper bound. Even under these permissive assumptions, our findings, detailed in the Figure below, show that the relaxed threat model adversary has a substantially lower probability of success in reconstructing model inputs compared to the DP adversary. For details on the computation of  $\varepsilon$ , see below.
(c) ImageNet
Figure 7: Application to data reconstruction attacks. Panels show the upper bound estimates on the probability of a successful reconstruction attack on the architectures in Figure S1 for increasing privacy budgets  $\varepsilon$ . Under the assumptions of the RTM, the adversary has a substantially lower probability of reconstruction compared to the DPTM.
7.2 Proofs
Proof. The claim follows immediately from the Neyman-Pearson Lemma [8]. For a detailed treatment, we refer to Section 3.2. of [25].  $\square$
Proof. We will first show that testing  $\mathcal{H}_0: \mathcal{Z}(q(D), \xi)$  vs.  $\mathcal{H}_1: \mathcal{Z}(q(D'), \xi)$  is equivalent to testing  $\mathcal{Z}(0, \xi)$  vs.  $\mathcal{Z}(q(D') - q(D), \xi)$ . We know that  $\mathcal{Z}(\mu, \xi)$  is a shift invariant probability measure which admits a density. Thus, for any  $c, \theta \in \mathbb{R}$ :
$$
\mathcal {Z} (\mu + c, \xi) (\theta) = \mathcal {Z} (\mu , \xi) (\theta - c) \Leftrightarrow \mathcal {Z} (\mu - c, \xi) (\theta) = \mathcal {Z} (\mu , \xi) (\theta + c). \tag {8}
$$
The LR for  $\mathcal{Z}(q(D),\xi)$  vs.  $\mathcal{Z}(q(D^{\prime}),\xi)$  is:
$$
\log \frac {\mathcal {Z} (q \left(D ^ {\prime}\right) , \xi) (\theta)}{\mathcal {Z} (q (D) , \xi) (\theta)}. \tag {9}
$$
Since the LR may be evaluated at an  $\theta$ , we can evaluate it at  $\theta + q(D)$ , since  $q(D)$  is a known constant:
$$
\log \frac {\mathcal {Z} (q \left(D ^ {\prime}\right) , \xi) (\theta + q (D))}{\mathcal {Z} (q (D) , \xi) (\theta + q (D))} = \log \frac {\mathcal {Z} (q \left(D ^ {\prime}\right) - q (D) , \xi) (\theta)}{\mathcal {Z} (q (D) - q (D) , \xi) (\theta)} = \log \frac {\mathcal {Z} (q \left(D ^ {\prime}\right) - q (D) , \xi) (\theta)}{\mathcal {Z} (0 , \xi) (\theta)}. \tag {10}
$$
Therefore, we have shown that testing  $\mathcal{Z}(q(D),\xi)$  vs.  $\mathcal{Z}(q(D'),\xi)$  is equivalent to testing  $\mathcal{Z}(0,\xi)$  vs.  $\mathcal{Z}(q(D') - q(D),\xi)$ , as the LR for any  $\theta$  under both hypothesis testing problems will be the same. Now, let  $\mu = q(D') - q(D)$ . Since  $\mathcal{A}^{\mathrm{SO}}$  does not know the value of  $q(D')$ , they can only deduct that  $\|\mu\| \leq \Delta$ . Thus, the two cases  $\mu > 0$  and  $\mu < 0$  must be considered simultaneously and individually. The claim follows. Observe that the proof is identical for the reversed test  $\mathcal{H}_0: \mathcal{Z}(q(D') - q(D),\xi)$  vs.  $\mathcal{H}_1: \mathcal{Z}(0,\xi)$ , i.e. with the null and alternative hypothesis switched.
Lemma 3. The individual  $LR$  tests  $r$  and  $r'$  are UMP level  $\alpha$  if the noise distribution  $\mathcal{Z}$  has the monotone likelihood ratio property (MLRP). The Laplace Mechanism (LM), Gaussian Mechanism (GM) and the Poisson-Subsampled Gaussian Mechanism (SGM) all have the MLPR. Moreover, the power of both tests is maximised when  $\| \mu \| = \| q(D') - q(D) \| = \Delta$ .
Proof. The existence of an UMP level  $\alpha$  LR test for distributions with the MLRP is justified by the Karlin-Rubin Theorem [34], which states that, when the null hypothesis is simple, the alternative hypothesis is one-sided and the LR is a monotone function of the parameter of interest (MLRP), the LR test is UMP level  $\alpha \forall \theta \in \Theta \setminus \Theta_0$ .
More precisely, the MLRP implies that the LR is non-decreasing or non-increasing in the parameter of interest (in this case  $\theta$ ) of a test. We must consider the cases  $\mu < 0$  and  $\mu > 0$  individually for each mechanism. Throughout, let  $P = \mathcal{Z}(0, \xi)$  and  $Q = \mathcal{Z}(\mu, \xi)$ . We will replace the placeholder  $\mathcal{Z}$  with the distribution of interest as required and consider both tests  $P$  vs.  $Q$  and  $Q$  vs.  $P$ .
For the Laplace Mechanism (LM), we have:
$$
\log \frac {P (\theta)}{Q (\theta)} = \log \mathrm {e} ^ {- \frac {| \theta |}{b}} \mathrm {e} ^ {\frac {| \theta - \mu |}{b}} = \frac {| \theta - \mu | - | \theta |}{b} \quad \text {a n d} \tag {11}
$$
$$
\log \frac {Q (\theta)}{P (\theta)} = \log \mathrm {e} ^ {\frac {| \theta |}{b}} \mathrm {e} ^ {- \frac {| \theta - \mu |}{b}} = \frac {- | \theta - \mu | + | \theta |}{b}. \tag {12}
$$
For  $\mu > 0$ ,  $\log \frac{P(\theta)}{Q(\theta)}$  is non-increasing in  $\theta$  and for  $\mu < 0$  it is non-decreasing in  $\theta$ . The opposite holds for  $\log \frac{Q(\theta)}{P(\theta)}$  by symmetry.
For the Gaussian Mechanism (GM), we have:
$$
\log \frac {P (\theta)}{Q (\theta)} = \log \mathrm {e} ^ {- \frac {\| \theta \| _ {2} ^ {2}}{2 \sigma^ {2}}} \mathrm {e} ^ {\frac {\| \theta - \mu \| _ {2} ^ {2}}{2 \sigma^ {2}}} = \frac {\| \theta - \mu \| _ {2} ^ {2} - \| \theta \| _ {2} ^ {2}}{2 \sigma^ {2}} \quad \text {a n d} \tag {13}
$$
$$
\log \frac {Q (\theta)}{P (\theta)} = \log \mathrm {e} ^ {\frac {\| \theta \| _ {2} ^ {2}}{2 \sigma^ {2}}} \mathrm {e} ^ {- \frac {\| \theta - \mu \| _ {2} ^ {2}}{2 \sigma^ {2}}} = \frac {- \| \theta - \mu \| _ {2} ^ {2} + \| \theta \| _ {2} ^ {2}}{2 \sigma^ {2}}. \tag {14}
$$
For  $\mu > 0$ ,  $\log \frac{P(\theta)}{Q(\theta)}$  is non-increasing in  $\theta$  and for  $\mu < 0$ , it is non-decreasing in  $\theta$ . Again, the opposite holds for  $\log \frac{Q(\theta)}{P(\theta)}$  by symmetry. Observe that the LRs are independent of dimensionality for the (S)GM, hence it suffices to consider the scalar case.
Finally, for the Poisson-Subsampled Gaussian mechanism (SGM), we have:
$$
\log \frac {P (\theta)}{Q (\theta)} = \log \left(\frac {1}{p \mathrm {e} ^ {\frac {\theta^ {2}}{2 \sigma^ {2}}} - (p - 1) \mathrm {e} ^ {\frac {(\theta - \mu) ^ {2}}{2 \sigma^ {2}}}}\right) + \frac {(\theta - \mu) ^ {2}}{2 \sigma^ {2}} \quad \text {a n d} \tag {15}
$$
$$
\log \frac {Q (\theta)}{P (\theta)} = \log \left(p \mathrm {e} ^ {\frac {\theta^ {2}}{2 \sigma^ {2}} - \frac {(\theta - \mu) ^ {2}}{2 \sigma^ {2}}} - p + 1\right). \tag {16}
$$
The computation here is slightly more involved. Taking the derivatives, we have:
$$
\frac {\mathrm {d}}{\mathrm {d} \theta} \log \frac {P (\theta)}{Q (\theta)} = \frac {\mu p \mathrm {e} ^ {\frac {\mu \theta}{\sigma^ {2}}}}{\sigma^ {2} \left(p \mathrm {e} ^ {\frac {\mu^ {2}}{2 \sigma^ {2}}} - p \mathrm {e} ^ {\frac {\mu \theta}{\sigma^ {2}}} - \mathrm {e} ^ {\frac {\mu^ {2}}{2 \sigma^ {2}}}\right)} \quad \text {a n d} \tag {17}
$$
$$
\frac {\mathrm {d}}{\mathrm {d} \theta} \log \frac {Q (\theta)}{P (\theta)} = \frac {\mu p \mathrm {e} ^ {\frac {\theta^ {2} - (\theta - \mu) ^ {2}}{2 \sigma^ {2}}}}{\sigma^ {2} \left(p \mathrm {e} ^ {\frac {\theta^ {2} - (\theta - \mu) ^ {2}}{2 \sigma^ {2}}} - p + 1\right)}. \tag {18}
$$
Both numerators are positive when  $\mu$  is positive and negative otherwise. For  $\frac{\mathrm{d}}{\mathrm{d}\theta} \log \frac{P(\theta)}{Q(\theta)}$ , the denominator is negative when  $\mu$  is positive, while for  $\frac{\mathrm{d}}{\mathrm{d}\theta} \log \frac{Q(\theta)}{P(\theta)}$ , it is negative. Thus, when  $\mu$  is positive,  $\log \frac{P(\theta)}{Q(\theta)}$  is non-increasing in  $\theta$  and non-decreasing otherwise. Conversely,  $\log \frac{Q(\theta)}{P(\theta)}$  is non-decreasing with  $\mu$  positive and non-increasing otherwise.
So far, we have proven that (1) the LM, GM and SGM have the MLRP and (2) thus, by Karlin-Rubin, there exists an UMP level  $\alpha$  test for  $\mathcal{Z}(0,\xi)$  vs.  $\mathcal{Z}(\mu ,\xi)$  as well as for the reverse  $\mathcal{Z}(\mu ,\xi)$  vs.  $\mathcal{Z}(0,\xi)$  for both  $r^\prime (\mu >0)$  and  $r$ $(\mu < 0)$ . We note that the proof could have also been achieved by using the fact that the Laplace and Gaussian distributions with known scale are part of a specific one-parameter exponential family, which has the MLRP property. Compare [25], Section 4.2. It remains to show that the power is maximised for  $\mu = \Delta$  or  $\mu = -\Delta$ , i.e. that the power increases with the effect size, in this case  $\Delta /\xi$ , with  $\xi$  assumed fixed.
First, observe that we can reduce our workload by half as, by shift invariance, the tests  $r' : \mathcal{Z}(0,\xi)$  vs.  $\mathcal{Z}(\mu,\xi)$  and  $r : \mathcal{Z}(0,\xi)$  vs.  $\mathcal{Z}(-\mu,\xi)$  as well as their reverses are all equivalent. It remains to show that the power function (i.e.  $1 - \beta(\mu)$  at a fixed  $\alpha$ ) is maximised at  $\mu = \Delta$ . In fact, it suffices to show that the power function is strictly increasing as a function of  $\mu$ , since  $\Delta$  is the largest value  $\mu$  can take. The power function is a re-parameterised version of the trade-off function for the one-sided UMP level  $\alpha$  test (note that this is not the combined test but one of the individual component tests – the combined test is equivalent to performing the two component tests simultaneously and necessarily loses power as it represents multiple testing which increases the false discovery rate). Fix a level  $\alpha$ .
For the LM, we have [7]:
$$
\beta (\operatorname {L a p} (0, b), \operatorname {L a p} (\mu , b) \mid \alpha) = \left\{ \begin{array}{l l} 1 - \mathrm {e} ^ {\mu / b} \alpha , & \alpha <   \mathrm {e} ^ {- \mu / b} / 2, \\ \mathrm {e} ^ {- \mu / b} / 4 \alpha , & \mathrm {e} ^ {- \mu / b} / 2 \leq \alpha \leq 1 / 2, \\ \mathrm {e} ^ {- \mu / b} (1 - \alpha), & \alpha > 1 / 2. \end{array} \right. \tag {19}
$$
It is easily verified that  $1 - \beta (\mathrm{Lap}(0,b),\mathrm{Lap}(\mu ,b)\mid \alpha)$  is strictly increasing in  $\mu \forall \alpha$
For the GM, we have [7]:
$$
\beta \left(\mathcal {N} (0, \sigma^ {2}), \mathcal {N} (\mu , \sigma^ {2}) \mid \alpha\right) = \Phi_ {\mathcal {N} (0, 1)} \left(\Psi_ {\mathcal {N} (0, 1)} ^ {- 1} (\alpha) - \mu / \sigma\right), \tag {20}
$$
where  $\Phi, \Psi^{-1}$  are the CDF and and iSF, respectively. Recall that the function is defined on  $[0, 1] \times [0, 1]$ . Taking the derivative with respect to  $\mu$  and observing that  $\Psi^{-1}(x) = \Phi^{-1}(1 - x)$  we obtain:
$$
\frac {\mathrm {d}}{\mathrm {d} \mu} \Phi (\Phi^ {- 1} (1 - \alpha) - \mu / \sigma) = g (\Phi^ {- 1} (1 - \alpha) - \mu / \sigma) \frac {\mathrm {d}}{\mathrm {d} \mu} (\Phi^ {- 1} (1 - \alpha) - \mu / \sigma), \tag {21}
$$
where  $g$  is the PDF of the standard normal. The second term simplifies to:
$$
\frac {\mathrm {d}}{\mathrm {d} \mu} \left(\Phi^ {- 1} (1 - \alpha) - \mu / \sigma\right) = - \frac {1}{\sigma}. \tag {22}
$$
$$
\frac {\mathrm {d}}{\mathrm {d} \mu} \Phi (\Phi^ {- 1} (1 - \alpha) - \mu / \sigma) = - \frac {1}{\sigma} g (\Phi^ {- 1} (1 - \alpha) - \mu / \sigma). \tag {23}
$$
Since  $\sigma$  is a positive constant and  $g(x) > 0\forall x$ , the derivative is negative for all values of  $\mu$ . Thus,  $\beta \left(\mathcal{N}(0,\sigma^2),\mathcal{N}(\mu ,\sigma^2)\mid \alpha\right)$  is decreasing in  $\mu$  and thus  $1 - \beta$  is increasing in  $\mu$  as required.
Finally, for the SGM, we can invoke Lemma 9 below and re-use the proof for the GM. Fix  $0 < p < 1$ . Lemma 9, specialised to the SGM, states:
$$
\beta \left(\mathcal {N} (0, \sigma^ {2}), p \mathcal {N} (\mu , \sigma^ {2}) + (1 - p) \mathcal {N} (0, \sigma^ {2}) \mid \alpha\right) = p \Phi_ {\mathcal {N} (0, 1)} \left(\Psi_ {\mathcal {N} (0, 1)} ^ {- 1} (\alpha) - \mu / \sigma\right) + (1 - p) (1 - \alpha). \tag {24}
$$
The second term is independent of  $\mu$ , thus the claim follows from the proof for the GM.
In conclusion, we have shown that, in the RTM, the adversary must conduct two pairs of individual and simultaneous UMP level  $\alpha$  LR tests whose power is maximised when  $\mu = \pm \Delta$ , i.e. at the dominating pair distributions [22]. This conforms to the intuition that the power of the test should increase when the distributions are "further apart", i.e. when the effect size increases. Moreover, identically to the DPTM, it allows us to consider only the dominating pairs when constructing the trade-off functions, since no other test between the distributions of the mechanism on the two databases can have higher power at the same level  $\alpha$  than the test(s) between the dominating pair(s). We have thus established that the optimal tests the sub-optimal adversary  $\mathcal{A}^{\mathrm{SO}}$  can construct against the aforementioned noise mechanisms in the RTM are UMP level  $\alpha$  tests. These tests are necessarily less powerful than the uniformly most powerful tests that  $\mathcal{A}^{\mathrm{NP}}$  can construct because they are only optimal for a subset of the values of  $q(D')$  at a time, whereas the uniformly most powerful tests of  $\mathcal{A}^{\mathrm{NP}}$  are optimal for all values of  $q(D')$  simultaneously.
Lemma 4. If the density of  $\mathcal{Z}$  is additionally symmetric about the location parameter, the simultaneous individual LR tests  $r$  and  $r'$  are equivalent to performing a single test using the magnitude of the observation as a test statistic, i.e. letting  $\|\theta\| \sim \|\mathcal{Z}(\mu, \xi)\|$  and testing  $\mathcal{H}_0: \|\mu\| = 0$  vs.  $\mathcal{H}_1: \|\mu\| > 0$ . The power of this "combined" test is maximised at  $\|\mu\| = \Delta$ .
Proof. Observe that  $\mathcal{H}_1$  can be split up into two hypotheses:  $\mathcal{H}_{1a}:\mu >0$  and  $\mathcal{H}_{1b}:\mu < 0$ . Now, we can conduct two simultaneous tests  $E_{1}:\mathcal{H}_{0}$  vs.  $\mathcal{H}_{1a}$  and  $E_{2}:\mathcal{H}_{0}$  vs.  $\mathcal{H}_{1b}$ . The LRs are:
For  $E_{1}$  ..  $\Lambda_1(\theta) = \log \frac{\mathcal{L}(\theta|\mathcal{H}_{1a})}{\mathcal{L}(\theta|\mathcal{H}_0)}$
For  $E_2\colon \Lambda_2(\theta) = \log \frac{\mathcal{L}(\theta|\mathcal{H}_{1b})}{\mathcal{L}(\theta|\mathcal{H}_0)}$
The rejection region for these tests will be  $\{\Lambda_1 > c_1\}$  and  $\{\Lambda_2 > c_2\}$ , where  $c_1, c_2$  are the critical values. The distributions for both  $\Lambda_1$  and  $\Lambda_2$  under  $\mathcal{H}_0$  are identical. It thus suffices to consider the numerator of the LR, which, from the MLRP, is monotone in  $\theta$ . Cancelling the logarithms, we may thus simplify and consider  $\Lambda_1 = \theta$  and  $\Lambda_2 = -\theta$ . To maintain the same level  $\alpha$  for all tests, the critical values should be equal:  $c = \hat{c}_1 = \hat{c}_2$ , where the hat denotes the new critical values after simplification. We remark for completeness that this will render the tests unbiased. Since these tests are UMP and unbiased, they are also called UMP unbiased (i.e. the tests with the greatest power in the set of unbiased tests). For further details, we refer to [25], Section 4. For the combined test, the rejection region is thus:  $\{\Lambda_1 > \hat{c}_1\} \cup \{\Lambda_2 > \hat{c}_2\} = \{\theta > c\} \cup \{-\theta > c\} = \{| \theta | > c \}$ . This proves the first claim. The claim that the power is maximised when  $\| \mu \| = \Delta$  follows from the previous observation and the proof to Lemma 3. For the reverse tests, the proof is identical.
Proof. For the concrete problem  $\mu = 0$  vs  $\| \mu \| > 0$ , the GLRT test statistic can be written:
$$
\Lambda (\theta) = \log \frac {\sup  _ {\mu \in \Theta \backslash \Theta_ {0}} \mathcal {L} (\theta | \mathcal {Z} (\mu , \xi))}{\mathcal {L} (\theta | \mathcal {Z} (0 , \xi))} \tag {25}
$$
For a single observation  $\theta$  (i.e. absent other information) the MLE for the mean  $\hat{\mu}$  is equal to the observed value of  $\theta$ . Thus, the GLRT becomes:
$$
\Lambda (\theta) = \log \frac {\mathcal {L} (\theta | \mathcal {Z} (\hat {\mu} , \xi))}{\mathcal {L} (\theta | \mathcal {Z} (0 , \xi))} \tag {26}
$$
Compare the GLRT to the two simultaneous tests:
-  $r$ : The LR test statistic is  $\Lambda_1(\theta) = \log \frac{\mathcal{L}(\theta|\mathcal{Z}(\mu,\xi))}{\mathcal{L}(\theta|\mathcal{Z}(0,\xi))}$  for  $\mu > 0$ ;
-  $r'$ : The LR test statistic is  $\Lambda_2(\theta) = \log \frac{\mathcal{L}(\theta|\mathcal{Z}(\mu,\xi))}{\mathcal{L}(\theta|\mathcal{Z}(0,\xi))}$  for  $\mu < 0$ .
Observe that these are identical to  $\Lambda (\theta)$  when the restricted MLE is used, provided the same critical value is chosen. Thus, the tests are equivalent at the same level  $\alpha$ . The claim that the power is maximised when  $\hat{\mu} = \Delta$  then follows from the previous lemma. For the reverse tests, the proof is identical. We note for completeness that the same result could have been achieved using the Wald or Rao tests, which are derived differently, but are statistically equivalent to the GLRT [35].
Proof. The lemma is essentially a re-stating of Lemma 1 in the language of f-DP. Since a UMP level  $\alpha$  test does not exist for all values of  $q(D')$ , any test that  $\mathcal{A}^{\mathrm{SO}}$  constructs is less powerful for at least a subset of the parameter space, and thus the resulting trade-off function in the RTM will be on or above (dominating) the trade-off function in the DPTM. An alternative way to look at this is through the post-processing guarantee of f-DP. As an example, consider using the test statistic  $\|\theta\|$  as discussed above. As  $\theta$  is the output of a mechanism, we can consider it a post-processing step on  $\mathcal{M}(D / D')$ . For f-DP, it holds that [7]:
$$
T (\operatorname {P r o c} (\mathcal {M} (D)), \operatorname {P r o c} (\mathcal {M} \left(D ^ {\prime}\right))) \geq T (\mathcal {M} (D), \mathcal {M} \left(D ^ {\prime}\right)) \tag {27}
$$
for any trade-off function  $T$ . The claim follows by setting the LHS to  $f'$ , which is the trade-off function constructed using the test statistic  $\| \theta \|$  in the RTM and the RHS to  $f$ , which is the trade-off function of the (NP) UMP level  $\alpha$  test in the DPTM.
Proof. This is essentially the contrapositive to Lemma 6. Since f-DP is resilient to post-processing, the worst deterioration of the  $\lambda$ -DR-guarantee can never exceed the f-DP guarantee.
Before proceeding with the proofs about the closed-form/numerical expressions of the trade-off functions, we re-state Definition 2 and provide a brief derivation.
Proof. The proof is standard. We reproduce it here for the purpose of being self-contained. Consider a one-tailed test for a parameter  $\theta$ . From basic hypothesis testing theory, given a test statistic  $\Lambda(\theta)$  and a critical value  $c$ ,  $\alpha = \operatorname{Pr}(\Lambda(\theta) > c \mid \mathcal{H}_0)$  and  $\beta = \operatorname{Pr}(\Lambda(\theta) < c \mid \mathcal{H}_1)$ . Letting  $P$  be the distribution of the test statistic under  $\mathcal{H}_0$  and  $Q$  its distribution under  $\mathcal{H}_0$ , we have:
$$
\beta (c) = \Phi_ {Q} (c) \text {a n d} \alpha (c) = \Psi_ {P} (c). \tag {28}
$$
Since the trade-off function is  $T: \alpha \mapsto \beta(\alpha)$ , we solve the RHS for  $c$  and have:
$$
c = \Psi^ {- 1} (\alpha) \Rightarrow T (\alpha) = \Phi_ {Q} \left(\Psi_ {P} ^ {- 1} (\alpha)\right). \tag {29}
$$
$T^{-1}$  follows by inverting  $T$ . The same form manifests by considering  $T^{-1}$  the trade-off function for the test with the null and alternative hypotheses exchanged (the reverse test). The role of  $P$  and  $Q$  will be played by the test statistics for the various mechanisms below. We note that, since the functions are inverses of each other, we may arbitrarily designate one as  $T$  and the other as  $T^{-1}$ .  $\square$
Theorem 1. Let  $\mu_{1} = \Delta_{1} / b$ . The LM satisfies  $\mathrm{C}(j_{\mathrm{Lap}}(\alpha), j_{\mathrm{Lap}}^{-1}(\alpha))$ -DR with:
$$
j _ {\mathrm {L a p}} ^ {- 1} (\alpha) = \left\{ \begin{array}{l l} \frac {1}{\alpha \exp \left(\mu_ {1}\right) + \sqrt {\alpha^ {2} \exp \left(2 \mu_ {1}\right) + 1}}, & \alpha <   1 / 2 - 1 / 2 \exp (- \mu_ {1}) \\ - \frac {\alpha - 1}{\cosh (\mu_ {1})}, & o t h e r w i s e. \end{array} \right. \tag {3}
$$
Proof.  $j(\alpha)$  is the trade-off function corresponding to the test  $r: \mathrm{Lap}(0, b)$  vs.  $\mathrm{Lap}(\Delta_1, b)$  and  $j^{-1}(\alpha)$  is the trade-off function for  $r': \mathrm{Lap}(\Delta_1, b)$  vs.  $\mathrm{Lap}(0, b)$ . Since the density of the Laplace distribution is symmetric, we can invoke Lemma 4 and use the absolute value of the observation as a test statistic.
For  $r$ , under  $\mathcal{H}_0$ , the test statistic  $\| \theta \|$  follows an exponential distribution with scale parameter  $b$ . Under  $\mathcal{H}_1$ , the test statistic  $\| \theta \|$  follows a folded Laplace (FoldL) distribution [27] with mean  $\Delta_1$  and scale parameter  $b$ . For  $r'$ , the hypotheses and distributions are reversed. To construct the trade-off functions, we require the CDF  $\Phi$ , and iCDF  $\Phi^{-1}$ . From these, we can construct  $\Psi(x) = 1 - \Phi(x)$  and  $\Psi^{-1}(x) = \Phi^{-1}(1 - x)$ . All are available for both distributions in closed form. We have:
$$
\Phi_ {\operatorname {E x p} (b)} (x) = 1 - \exp (- x / b),
$$
$$
\Phi_ {\operatorname {E x p} (b)} ^ {- 1} (x) = - b \log (1 - x)
$$