SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Bonwoo Lee
Jeongyoun Ahn*
Cheolwoo Park*
Korea Advanced Institute of Science & Technology
Daejeon, 34141 South Korea
righthim@kaist.ac.kr, jyahn@kaist.ac.kr, parkcw2021@kaist.ac.kr
As concerns about data privacy continue to grow, differential privacy (DP) has emerged as a fundamental concept that aims to guarantee privacy by ensuring individuals' indistinguishability in data analysis. Local differential privacy (LDP) is a rigorous type of DP that requires individual data to be privatized before being sent to the collector, thus removing the need for a trusted third party to collect data. Among the numerous (L)DP-based approaches, functional DP has gained considerable attention in the DP community because it connects DP to statistical decision-making by formulating it as a hypothesis-testing problem and also exhibits Gaussian-related properties. However, the utility of privatized data is generally lower than that of non-private data, prompting research into optimal mechanisms that maximize the statistical utility for given privacy constraints. In this study, we investigate how functional LDP preserves the statistical utility by analyzing minimax risks of univariate mean estimation as well as nonparametric density estimation. We leverage the contraction property of functional LDP mechanisms and classical information-theoretical bounds to derive private minimax lower bounds. Our theoretical study reveals that it is possible to establish an interpretable, continuous balance between the statistical utility and privacy level, which has not been achieved under the  $\epsilon$ -LDP framework. Furthermore, we suggest minimax optimal mechanisms based on Gaussian LDP (a type of functional LDP) that achieve the minimax upper bounds and show via a numerical study that they are superior to the counterparts derived under  $\epsilon$ -LDP. The theoretical and empirical findings of this work suggest that Gaussian LDP should be considered a reliable standard for LDP.
It has been widely accepted that anonymization is insufficient in protecting privacy (Sweeney, 2000, 2002; Dinur and Nissim, 2003). The concerns about data privacy have grown significantly, particularly with the advancement of computer science technology and the rise in data generated by individuals and tech companies. Such concerns are shared by politics and industry, leading to the adoption of France's "Loi pour une République numérique (Law for the Digital Republic)" in October 2016 (Algan et al., 2016), EU's General Data Protection Regulation in May 2018, and the California Consumer Privacy Act (Wang et al., 2022), all of which regulate data protection, collection, and processing. Also, data privacy techniques have been implemented in industries such as Google (Erlingsson et al., 2014; Fanti et al., 2016), Apple (Thakurta et al., 2017; Tang et al., 2017), Microsoft (Ding et al., 2017), and SAP (Kessler et al., 2019).
Differential privacy (DP), suggested by Dwork et al. (2006b), has become a fundamental foundation of the modern privacy concept. Its core idea is that privacy can be protected not by how well the sensitive information is hidden but rather by how individuals are indistinguishable in the privatized data. Dwork et al. (2006b) first introduced the notion of  $\epsilon$ -DP for a randomized mechanism that yields a likelihood ratio between its outputs from 'neighboring' data that is bounded by a privacy budget  $\epsilon$ . Despite its intuitive appeal,  $\epsilon$ -DP has been found to lack generality in explaining some common privacy mechanisms including the Gaussian mechanism (Mironov, 2017; Dong et al., 2022). A generalized version of  $\epsilon$ -DP,  $(\epsilon, \delta)$ -DP, is introduced in Definition 1.1. Let  $\mathcal{X}$  be the sample space and  $M: \mathcal{X}^n \longrightarrow \mathcal{P}(\mathcal{Z})$  be a randomized mechanism, where  $\mathcal{P}(\mathcal{Z})$  is a family of distributions over  $\mathcal{Z}$ . Let  $S \sim S'$  denote that data sets  $S$  and  $S'$  are neighbors, differing only by one individual. The following definition states that the output distribution of  $M$  does not heavily depend on the presence of a specific individual. Note that  $(\epsilon, 0)$ -DP reduces to  $\epsilon$ -DP.
Definition 1.1  $((\epsilon, \delta)$ -DP, Dwork et al. (2006a)). A mechanism  $M$  is  $(\epsilon, \delta)$ -DP if the following holds:
$$
\sup  _ {S, S ^ {\prime} \subset \mathcal {X}, S \sim S ^ {\prime}} \sup  _ {A \subset \mathcal {Z}} \mathbb {P} (M (S) \in A) - e ^ {\epsilon} \mathbb {P} (M (S ^ {\prime}) \in A) \leq \delta .
$$
Another type of DP, local differential privacy (Duchi et al., 2013, LDP), provides strong privacy guarantees by privatizing raw data before releasing it to a data collector. It is commonly used for private statistical inference since it gives the adversary access to privatized data that is the same size as the original data. Note that unlike in Definition 1.1, the inputs of LDP mechanisms are not the whole data set  $S$  but each individual observation  $x_{i} \in S$ .
As expected, the accuracy of statistical analysis performed on privatized data is often worse than that of non-private data. This has prompted researchers to seek a balance between privacy and utility as well as to find the optimal mechanism given privacy constraints. The efforts to balance noise contamination and utility date back to Carroll and Hall (1988), which predated the emergence of DP. Later, Duchi et al. (2013) proposed a framework for quantifying the trade-off between privacy and statistical utility using  $\epsilon$ -LDP and minimax risk analysis. With the minimax framework, one can study the minimum loss that can be attained in the worst-case scenario for statistical utility, given a specific privacy constraint. Since then, numerous researchers have explored minimax risks for various inference problems under  $\epsilon$ -LDP (Li et al., 2022; Chhor and Sentenac, 2023; Rohde and Steinberger, 2020; Butucea et al., 2020).
However, despite the inherent continuity in the definitions of  $\epsilon$ -LDP and  $(\epsilon, \delta)$ -LDP, some studies on LDP mechanisms have revealed an inexplicable discrepancy in the statistical utility achieved by the two techniques (Asoodeh et al., 2021). One possible explanation for this phenomenon is the inefficiency of the composition rule used in  $(\epsilon, \delta)$ -LDP, which has been found to perform poorly in tracking privacy leakage resulting from the compositions of multiple mechanisms or subsampling, as pointed out by some studies including Dong et al. (2022); Mironov (2017). Since data are frequently reused multiple times in most data analysis scenarios, the privacy level must be calculated by taking the composition of multiple mechanisms into account. An accurate assessment of privacy under such compositions is critical, since underestimating privacy would force a mechanism to sacrifice utility to ensure the desired privacy level.
This work explores the trade-off between statistical utility and local privacy under the framework of functional differential privacy (Dong et al., 2022, FDP), which is known to provide more precise privacy control. The core idea of FDP connects the fundamental DP concept, which involves making two outputs based on different inputs indistinguishable, to the concept of statistical decision-making. Since an adversary's goal is to identify whether given privatized  $Z$  is from  $S$  or  $S'$  when  $S \sim S'$ , we consider the following hypothesis testing problem for a given output of mechanism  $M$ :
$$
H _ {0}: Z \sim M (S) \quad \text {v s .} \quad H _ {1}: Z \sim M \left(S ^ {\prime}\right). \tag {1}
$$
The difficulty of this hypothesis testing problem is directly related to the privacy level of the given mechanism. A mechanism is said to satisfy  $f$ -FDP if the above testing problem has a "trade-off" function  $f(\alpha)$ , which is the minimum Type II error for a given Type I error no greater than  $\alpha \in [0,1]$ . FDP, particularly in the context of Gaussian differential privacy (GDP) (see Section 2), has been found to provide superior privacy control compared to  $(\epsilon, \delta)$ -DP, with better interpretation.
Another advantage of the FDP framework is its superiority in composition rules, which regards accurately measuring the privacy level when multiple mechanisms are sequentially applied. Many
types of DP, including  $(\epsilon, \delta)$ -(L)DP, have been criticized for their inefficient composition rules. They tend to overestimate privacy leakage, which, in turn, leads to excessive perturbation of estimations. This is demonstrated in Section 4, in which  $\epsilon$ -LDP is empirically shown to be inefficient in controlling the privacy of a high-dimensional problem. On the other hand, FDP possesses an effective composition rule that strikes the right balance between privacy protection and estimation accuracy.
Since DP was introduced, the trade-off between privacy and statistical utility has been studied in global DP settings as well as local DP settings. Kamath et al. (2022) and Kamath and Ullman (2020) investigated the trade-off under  $\epsilon$ -DP by studying sample complexity of covariance matrix estimation of Gaussian distribution and univariate mean estimation, respectively. Cai et al. (2021) established minimax optimality under  $(\epsilon, \delta)$ -DP for high-dimensional mean estimation of sub-Gaussian distributions.
In the local DP setting, analyzing minimax risks under  $\epsilon$ -LDP for various estimation problems has been a main focus. Duchi et al. (2018) provided bounds of minimax optimal privacy mechanisms for some canonical estimation problems including mean and density estimation and suggested optimal estimators under  $\epsilon$ -LDP. Other estimation problems that have been investigated under  $\epsilon$ -LDP include the estimation of functionals of a probability distribution (Rohde and Steinberger, 2020), and discrete distribution estimation (Chhor and Sentenac, 2023). Private minimax risk of nonparametric density estimation has been addressed by Butucea et al. (2020) who observed an elbow effect in  $L^r$  risk over Besov spaces and by Li et al. (2022) who considered a data contamination scenario. Some efforts have also been made under  $(\epsilon, \delta)$ -LDP. Asoodeh et al. (2021) studied minimax risks for mean estimation, while Kroll (2021) considered nonparametric density estimation at fixed points.
Among the existing works, Duchi et al. (2018) and Asoodeh et al. (2021) are more relevant to the present work than others, as they also studied the mean estimation and/or nonparametric density estimation. We show that the minimax optimal rates for  $f$ -FLDP, local version of  $f$ -FDP, are equivalent to those for  $\epsilon$ -LDP, if  $f$  satisfies some condition (see Lemma 1). We argue that our result can also be useful for understanding  $(\epsilon, \delta)$ -LDP, since  $f$ -FLDP includes  $(\epsilon, \delta)$ -LDP as a special case.
We derive the minimax risk bounds under the framework of FLDP as well as Gaussian LDP (GLDP) for two classic statistical problems: univariate mean estimation (Section 3.1) and nonparametric density estimation (Section 3.2). We investigate the private minimax risks using Le Cam's (Theorem 1) and Assouad's bounds (Theorem 3), respectively. To the best of our knowledge, our work is the first to investigate the minimax risk bounds under the FLDP framework.
Our theoretical investigation yields two important results: First, we show that our bounds achieve the same rates under some conditions as those under  $\epsilon$ -LDP, thereby extending the contraction inequality established by Duchi et al. (2018) under  $\epsilon$ -LDP to FLDP (Theorem 2). Also, as a special case, we show that the established lower bound becomes tight under GLDP. Second, we demonstrate that  $f$ -FLDP provides a continuous trade-off between privacy and statistical utility, unlike  $(\epsilon, \delta)$ -LDP (Corollaries 1 and 3). In the view of  $(\epsilon, \delta)$ -LDP, one can see that  $\epsilon$ -LDP and non-private settings are continuously related by a quantity  $\delta$ . However, the existing results do not reflect such continuity with respect to the minimax rate. So far, any slightest privacy measure seems to increase the minimax rate with the increment rate not related to the privacy level. For example, the minimax rate of nonparametric density estimation worsens from  $n^{-\frac{2\beta}{2\beta + 1}}$  to  $n^{-\frac{2\beta}{2\beta + 2}}$  under  $\epsilon$ -LDP, where  $\beta$  denotes the degree of smoothness. But the change of rate is irrelevant to  $\epsilon$  even though  $\epsilon$ -LDP becomes non-private if we set  $\epsilon \to \infty$ .
We also present optimal mechanisms that attain the minimax upper bounds (Corollaries 2 and 4), and evaluate their utilities compared to optimal mechanisms derived under  $\epsilon$ -LDP. Our experiments show that the proposed minimax optimal estimators under GLDP achieve better utility for the equivalent level of privacy constraints (Section 4).
We introduce some notations and review key concepts in differential privacy in this section. Let  $M: \mathcal{X}^n \longrightarrow \mathcal{P}(\mathcal{Z})$  be a multivariate randomized mechanism or equivalently, a collection of
randomized mechanisms  $M = \{M_i\}_{i=1}^n$  such that  $Z = (z_1, \ldots, z_n) := M(x_1, \ldots, x_n)$  where  $z_i = M_i(x_1, \ldots, x_n)$  for  $i = 1, \ldots, n$ . Here,  $M$  denotes a locally private mechanism as it privatizes a data set  $D \in \mathcal{X}^n$ , producing perturbed data with the same size as the original data. Furthermore,  $M$  is called a sequential mechanism if  $z_i$  depends only on  $x_i$  and  $z_1, \ldots, z_{i-1}$ , so  $z_i = M_i(x_i, z_1, \ldots, z_{i-1})$ .
2.1 Functional Local Differential Privacy
The hypothesis testing aspect of DP expressed in (1) was first discussed by Wasserman and Zhou (2010). Recently Dong et al. (2022) gave a formal treatment by formulating the difficulty of the testing as follows. Let  $\phi$  be the rejection rule and  $\alpha$  be the level of the test. Then we define the trade-off function<sup>2</sup> as the minimum type II error for a given  $\alpha$ .
Definition 2.1 (Trade-off function, Dong et al. (2022)). For two distributions  $P, Q \in \mathcal{P}(\mathcal{Z})$  and  $\alpha \in [0,1]$ , the trade-off function  $T(P,Q):[0,1] \longrightarrow [0,1]$  between  $P$  and  $Q$  is defined as
$$
T (P, Q) (\alpha) = \inf  _ {\phi} \left\{1 - \int_ {\mathcal {Z}} \phi (z) d Q (z) \Bigg | \phi : \mathcal {Z} \longrightarrow [ 0, 1 ], \int_ {\mathcal {Z}} \phi (z) d P (z) \leq \alpha \right\}.
$$
It is clear that the larger the trade-off function, the smaller the power, and consequently the more private the mechanism. A randomized mechanism is said to satisfy  $f$ -FDP if the corresponding trade-off function is at least  $f$  for all  $\alpha \in [0,1]$ .
Definition 2.2 (Functional differential privacy, FDP), Dong et al. (2022)). Let  $f:[0,1] \longrightarrow [0,1]$  be a trade-off function for some distributions  $P$  and  $Q$ . A given mechanism  $M$  is  $f$ -FDP if
$$
T (M (S), M \left(S ^ {\prime}\right)) (\alpha) \geq f (\alpha)
$$
for every  $S\sim S^{\prime}\subset \mathcal{X}^{n}$  and  $\alpha \in [0,1]$ .
The following proposition shows that  $(\epsilon, \delta)$ -DP defined in (1.1) is a special case of  $f$ -FDP.
Proposition 1 (Dong et al. (2022)). A convex conjugate of  $f$  is defined as  $\delta_f(y) = \sup_{x \in [0,1]} 1 - yx - f(x)$ . Then a mechanism  $M$  satisfies an  $f$ -FDP iff it is  $(\epsilon, \delta_f(e^{\epsilon}))$ -DP for every  $\epsilon \geq 0$ .
According to Proposition 1, all information regarding the privacy of the mechanism from the perspective of  $(\epsilon, \delta)$ -DP is contained in FDP. It also indicates that the privacy characterization of a mechanism may require more than just two numbers  $\epsilon$  and  $\delta$ .
A useful subclass of FDP is the Gaussian differential privacy (GDP), which has the trade-off function  $G_{\mu} = T(N(0,1),N(\mu ,1))$ . That is, the hypothesis testing problem in (1) compares two normal distributions with respective means 0 and  $\mu$  with unit variance. It is known that  $G_{\mu}(x) = F(F^{-1}(1 - x) - \mu)$  where  $F$  is the cumulative distribution function of the standard normal distribution.
Definition 2.3 (Gaussian differential privacy, GDP). A mechanism  $M$  is  $\mu$ -GDP if it is  $G_{\mu}$ -FDP.
In addition to its intuitive interpretation,  $\mu$ -GDP also possesses appealing asymptotic properties regarding compositions of different DP mechanisms. It has been shown by Dong et al. (2022) that the trade-off function of a composition of mechanisms converges to  $G_{\mu}$  under some mild conditions. Due to these desirable features, it is suggested to be used as a standard for the comparison of different DP methods.
In this work, we focus on the utility of estimation under local  $\mu$ -GDP, i.e.,  $\mu$ -GLDP. For all DP concepts introduced so far, their corresponding local versions are naturally defined by a locally private sequential mechanism  $M = \{M_i\}$ ,  $i = 1, \dots, n$ , that privatizes the  $i$ th observation, respectively. For example,  $f$ -FLDP and  $\mu$ -GLDP are characterized by the following:
$$
T \left(M _ {i} (x), M _ {i} \left(x ^ {\prime}\right)\right) (\alpha) \geq f (\alpha) (\text {o r} G _ {\mu} (\alpha))
$$
for every  $x, x' \in \mathcal{X}$  and  $i = 1, 2, \dots, n$ . Some exemplary trade-off functions of  $\mu$ -GLDP as well as those of  $\epsilon$ -LDP are displayed in Fig. 1a.
not to be confused with the trade-off between privacy and statistical utility.
2.2 Private Minimax Risk
Consider an estimation problem in a private setting. Suppose  $n$  i.i.d. observations  $X_{1},\ldots ,X_{n}\in \mathcal{X}$  are drawn from a distribution  $P$  in some family of distributions  $\mathcal{P}\subset \mathcal{P}(\mathcal{X})$ . The data are privatized into  $(Z_{1},\dots,Z_{n}) = M(X_{1},\dots,X_{n})$  where  $M$  is an  $f$ -FLDP mechanism. Let  $\theta = \theta (\mathcal{P})\in \Theta$  be a parameter of interest where  $\Theta$  is the parameter space and  $\hat{\theta}:\mathcal{X}^n\longrightarrow \Theta$  be an estimator. For a metric  $\rho :\Theta^2\longrightarrow \mathbb{R}_{\geq 0}$  and an increasing function  $\Phi :\mathbb{R}_{\geq 0}\longrightarrow \mathbb{R}_{\geq 0}$ , use  $\Phi \circ \rho$  as the loss function, we define the private minimax risk:
Definition 2.4 (Private minimax risk).
$$
\mathcal {R} _ {n} (\theta (\mathcal {P}), \Phi \circ \rho , M _ {f}) := \inf  _ {M \in M _ {f}} \inf  _ {\hat {\theta}} \sup  _ {P \in \mathcal {P}} \mathbb {E} _ {P} \left[ \Phi \circ \rho \left(\hat {\theta} \left(M (X _ {1}, \ldots , X _ {n})\right), \theta (P)\right) \right].
$$
Here,  $M_f$  is a family of  $f$ -FLDP mechanism defined over  $\mathcal{X}^n$ . As long as privacy is guaranteed at a certain level (i.e., given  $f$ ), one wishes to find an optimal privatizing mechanism  $M$  for a given estimation problem (i.e.,  $\theta$ ) that yields uniformly optimal loss over every distribution in  $\mathcal{P}$ . One can also use the minimax risk not only to find the optimal achievable estimation risk but to judge the efficiency of the estimator under the given privacy level  $f$ -FLDP.
3 Minimax Risk Analysis Under FLDP
Minimax risks of classical, non-private estimation problems have been addressed by numerous approaches including Le Cam, Fano, and Assouad methods, among others (Tsybakov, 2009). In this section, we take Le Cam's and Assouad's approaches and apply them to mean and nonparametric density estimation, respectively, under  $f$ -FLDP. All proofs are presented in the supplementary material.
3.1 Le Cam's Bound on Mean Estimation
Le Cam's method involves reducing an estimation problem to a two-point hypothesis testing problem, in which the minimax risk is bounded by calculating the worst-case risk for only two distributions,  $P_{1}$  and  $P_{2}$ , selected from  $\mathcal{P}$ , as defined in Section 2.2.
We first state the private Le Cam's lower bound under  $f$ -FLDP in the theorem below. The following quantity represents the contraction factor of the effective sample size due to the privatization:
$$
c _ {f, \kappa} = 2 \kappa^ {\kappa} (1 - \kappa) ^ {1 - \kappa} \int_ {0} ^ {\infty} (1 + \kappa) t ^ {\kappa - 1} \delta_ {f} (t) d t,
$$
where  $0 \leq \kappa \leq 1$  and  $\delta_f(t)$  is the convex conjugate of  $f$ , as defined in Proposition 1.
Theorem 1 (Private Le Cam's bound). For given  $0 \leq \kappa \leq 1$  and a trade-off function  $f$ , if  $\int_0^\infty t^{\kappa - 1} \delta_f(t) dt$  is finite and  $\rho(\theta(P_1), \theta(P_2)) \geq 2\eta > 0$  for two distributions  $P_1, P_2 \in \mathcal{P}$ , then
$$
\mathcal {R} _ {n} \left(\theta (\mathcal {P}), \Phi \circ \rho , M _ {f}\right) \geq \frac {\Phi \left(\eta\right)}{2} \left[ 1 - \sqrt {n c _ {f , \kappa} \frac {\| P _ {1} - P _ {2} \| _ {T V} ^ {1 + \kappa}}{\left(1 - \| P _ {1} - P _ {2} \| _ {T V}\right) ^ {\kappa}}} \right],
$$
where  $\| \cdot \|_{TV}$  denotes total variation.
The minimax lower bound on the right-hand side can serve as an indicator of the trade-off between privacy and utility. Note that  $c_{f,\kappa} \leq c_{g,\kappa}$  if  $f(t) \geq g(t)$  for all  $t \in [0,1]$ , which implies higher privacy would yield greater contraction of the effective sample size. That is, the minimax lower bound on the right-hand side would become lower under a loosened privacy constraint, which coincides with the intuition that less privacy would enhance the utility.
Next, we use Le Cam's method to obtain a lower bound of minimax risk for univariate mean estimation with bounded moments under the squared loss. For  $k > 1$ , define a family of distributions with bounded  $k$ th moment  $\mathcal{P}_k = \{P \in \mathcal{P}(\mathbb{R}) \mid |\mathbb{E}_P[X]| \leq 1, \mathbb{E}_P[|X|^k] \leq 1\}$ . The parameter of interest is  $\theta(P) = \mathbb{E}_P[X]$  with loss function  $\Phi \circ \rho$  where  $\rho(\theta_1, \theta_2) = |\theta_1 - \theta_2|$  and  $\Phi(t) = t^2$ . Applying Theorem 1, we obtain the following result.
Corollary 1 (Univariate mean estimation). For given  $0 \leq \kappa \leq 1$  and a trade-off function  $f$ , if  $\int_0^\infty t^{\kappa - 1} \delta_f(t) dt$  is finite, then
$$
\mathcal {R} _ {n} \left(\theta (\mathcal {P} _ {k}), | \cdot | ^ {2}, M _ {f}\right) \geq c _ {0} \left(n c _ {f, \kappa}\right) ^ {- \frac {1}{1 + \kappa} \frac {2 (k - 1)}{k}},
$$
for  $n > c_{1}$  where  $c_{0}$  only depends on  $k$  and  $\kappa$ , and  $c_{1}$  depends on  $c_{f,\kappa}$ .
Note that the minimax risk for mean estimation is known to be  $O\left(n^{-\min \{1,2 - \frac{2}{k}\}}\right)$  in a non-private setting and  $O\left((n\epsilon^2)^{-\frac{k - 1}{k}}\right)$  under  $\epsilon$ -LDP (Duchi et al., 2018). Corollary 1 implies that the minimax lower bound continuously reaches the two scenarios. Specifically, the minimax risk bound of  $\kappa = 0$  corresponds to that of the non-private setting, and the minimax risk bound of  $\kappa = 1$  corresponds to that of the  $\epsilon$ -LDP case. Moreover, the fact that for any trade-off function  $f$ , there exists  $f_{\epsilon}$  such that  $f_{\epsilon}$ -FLDP is equivalent with  $\epsilon$ -LDP implies that the minimax rate of mean estimation under  $f$ -FLDP is  $O\left(n^{-\frac{k - 1}{k}}\right)$  when  $c_{f,1}$  is finite. In the following lemma, we further identify which trade-off function  $f$  would yield the private minimax risk bound of  $f$ -FLDP equivalent to that of  $\epsilon$ -LDP.
Lemma 1. If a given trade-off function  $f(x) \geq 1 - c_0 x^{c_1}$  for some  $c_0 > 0$  and  $c_1 \in (0.5, 1)$  on  $[0, 1]$ , or  $\delta_f(x) \leq c_3 x^{-1 - c_2}$  for  $x > x_0$  and some  $c_3, c_2, x_0 > 0$ , then  $\int_0^\infty \delta_f(t) dt$  is finite.
(a) Trade-off functions
(b) Lemma 1
Figure 1: (a) Trade-off functions of  $\epsilon$ -LDP with  $\epsilon = 0.4$  (red), 6.4 (green),  $\mu$ -GLDP with  $\mu = 1$  (blue) and non-private setting (cyan). (b) Example of trade-off functions (not) satisfying Lemma 1.