SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ashwinkumar Badanidiyuru
Google Mountain View, CA
Badih Ghazi  
Google Research  
Mountain View, CA
Pritish Kamath  
Google Research  
Mountain View, CA
Ravi Kumar  
Google Research  
Mountain View, CA
Ethan Leeman Google Cambridge, MA
Pasin Manurangsi  
Google Research  
Bangkok, Thailand
Avinash V Varadarajan  
Google  
Mountain View, CA
Chiyuan Zhang  
Google Research  
Mountain View, CA
We propose a new family of label randomizers for training regression models under the constraint of label differential privacy (DP). In particular, we leverage the trade-offs between bias and variance to construct better label randomizers depending on a privately estimated prior distribution over the labels. We demonstrate that these randomizers achieve state-of-the-art privacy-utility trade-offs on several datasets, highlighting the importance of reducing bias when training neural networks with label DP. We also provide theoretical results shedding light on the structural properties of the optimal unbiased randomizers.
Differential privacy (DP) [DMNS06] has gained significant importance in recent years as a mathematically sound metric for rigorously quantifying the potential disclosure of personal user information through ML models [ACG+16, RE19, TM20]. DP guarantees that the model generated by the training process remains statistically indistinguishable, even if the data contributed by any individual user to the training dataset is modified arbitrarily.
In certain scenarios, the training features of a specific example are already accessible to potential adversaries, while only the training label is considered sensitive. For instance, within the context of digital advertising, it is typical to train conversion models that aim to predict whether a user, who interacts with an advertisement on a publisher's website, will ultimately make a purchase of the advertised item on the advertiser's website. The conversion label, which indicates whether the user completed the purchase or not, is not initially known to the publisher website and is considered sensitive information that spans multiple websites. This particular setting, referred to as label DP, was initially studied in the work of [CH11]. Subsequently, it has garnered attention in various recent works, such as  $\mathrm{[GGK^{+}21,MEMP^{+}21,EMSV22]}^2$ .
For regression objectives (such as the squared loss and the Poisson log loss), the state-of-the-art is given by the prior work of  $\left[\mathrm{GKK}^{+}23\right]$ . Their algorithm works in the so-called features-oblivious setting (Figure 1), which consists of a features party that has access to all the features, and a labels party that has access to all the labels. The labels party applies a mechanism that is DP with respect to the labels in order to produce the message  $M(\cdot)$  that it sends to the features party, who then uses the
Given the announcement of various web browsers and mobile platforms regarding the depreciation of third-party cookies, which were previously employed to track user behavior across different websites, the need for training models that protect the privacy of labels has become increasingly important.
as well as in proposals for industry-wide web and mobile standards [Har23].
Figure 1: Feature-oblivious label DP model training studied in  $\left[\mathrm{GKK}^{+}23\right]$ .
features along with the incoming message in order to train the model (which as a consequence would also satisfy label DP). It proceeds by using part of the privacy budget in order to estimate a prior over the labels, and then constructs a randomizer optimizing the "noisy label loss" (see Definition 4) with respect to this prior.
In this work, we introduce a novel mechanism with bias-limiting constraints, motivated by the theory of bias-variance trade-offs. We show that while these constraints lead to significantly higher noisy label loss, the models trained on the privatized labels performs surprisingly well, achieving the state-of-the-art utility-privacy trade-off on multiple real world datasets.
Organization. Section 2 provides some background definitions related to (label-)DP and learning. In Section 3, we present the new label DP randomizers obtained by imposing unbiasedness constraints. Section 4 provides an experimental evaluation demonstrating that our method achieves state-of-the-art privacy-utility trade-offs across multiple datasets. We provide some related work in Section 5. We conclude with some future directions in Section 6. All missing proofs along with additional related work, as well as details on the experimental evaluation are provided in the Appendix.
Let  $\mathcal{D}$  be an unknown distribution on  $\mathcal{X} \times \mathcal{Y}$ . We consider the regression setting where  $\mathcal{V} \subseteq \mathbb{R}$ ; let  $\mathcal{P}$  denote the marginal distribution of  $\mathcal{D}$  on  $\mathcal{V}$ . In supervised learning, we have a set  $\{(x,y)\}$  of examples drawn from  $\mathcal{D}$ . The goal is to learn a predictor  $f_{\theta}$  to minimize the expected loss  $\mathcal{L}_{\mathcal{D}}(f_{\theta}) := \mathbb{E}_{(x,y) \sim \mathcal{D}} \ell(f_{\theta}(x), y)$ , for some loss function  $\ell: \mathbb{R} \times \mathcal{V} \to \mathbb{R}$ . Some commonly-used loss functions used in regression tasks are the Poisson log loss  $\ell_{\mathrm{Poi}}(\tilde{y}, y) := \tilde{y} - y \cdot \log(\tilde{y})$  and the squared loss  $\ell_{\mathrm{sq}}(\tilde{y}, y) := \frac{1}{2} (\tilde{y} - y)^2$ .
We recall the definition of DP; for more background, see the book of Dwork and Roth [DR14].
Definition 1 (DP; [DMNS06]). Let  $\varepsilon \in \mathbb{R}_{>0}$ . A randomized algorithm  $\mathcal{A}$  is said to be  $\varepsilon$ -differentially private (denoted  $\varepsilon$ -DP) if for any two "adjacent" input datasets  $X$  and  $X'$ , and any measurable subset  $S$  of outputs of  $\mathcal{A}$ , it holds that  $\operatorname*{Pr}[\mathcal{A}(X) \in S] \leq e^{\varepsilon} \cdot \operatorname*{Pr}[\mathcal{A}(X') \in S]$ .
In the context of supervised learning, an algorithm produces a model as its output, while the labeled training set serves as the input. Two inputs are considered "adjacent" if they differ on a single training example. This concept of adjacency is intended to safeguard both the features and the label of any individual example. However, in certain scenarios, protecting the features may either be unnecessary or infeasible, and the focus is solely on protecting the labels. This leads to the following definition:
Definition 2 (Label DP; [CH11]). A randomized training algorithm  $\mathcal{A}$  is said to be  $\varepsilon$ -label differentially private (denoted  $\varepsilon$ -LabelDP) if it is  $\varepsilon$ -DP when two input datasets are "adjacent" if they differ on the label of a single training example.
We recall the notion of feature-oblivious label DP  $\left[\mathrm{GKK}^{+}23\right]$ . In this scenario, there are two parties: the features party and the labels party. The features party has the sequence  $(x_{i})_{i = 1}^{n}$  of all feature vectors across the  $n$  users; the labels party has the sequence  $(y_{i})_{i = 1}^{n}$  of the corresponding labels. The labels party sends a single (possibly randomized) message  $M(y_{1},\ldots ,y_{n})$  to the features party. Based on its input and the received message, the features party generates an ML model as its output.
Definition 3 (Feature-Oblivious Label DP  $\left[\mathrm{GKK}^{+}23\right]$ ). In the above scenario, the output of the features party satisfies feature-oblivious  $\varepsilon$ -LabelDP if the message  $M(y_{1},\ldots ,y_{n})$  is  $\varepsilon$ -DP where two inputs are considered "adjacent" if they differ on a single  $y_{i}$ .
3 Label-DP Randomizers
A standard recipe for learning with label DP is to: (i) compute noisy labels using a local DP randomizer  $\mathcal{M}$  and (ii) use a learning algorithm on the dataset with noisy labels. Many of the baseline algorithms that we consider follow this recipe, through different ways of generating noisy labels, such as, (a) (continuous/discrete) Laplace mechanism, (b) (continuous/discrete) staircase mechanism, etc. (see Appendix F for formal definitions). Prior work by  $\mathrm{[GKK^{+}23]}$  argues that intuitively such a learning algorithm will be most effective when the noisy label "mostly agrees" with the true label. This was formalized in the goal of minimizing the noisy label loss, which is the expected loss between the true label and the noisy label, for the true label drawn from a prior distribution, for some loss function  $\ell$ .
Definition 4 (Noisy label loss). For a loss function  $g: \mathbb{R} \times \mathcal{V} \to \mathbb{R}$ , the noisy label loss of a randomizer  $\mathcal{M}$  with respect to prior  $\mathcal{P}$  is  $\mathcal{G}(\mathcal{M}; \mathcal{P}) := \mathbb{E}_{y \sim \mathcal{P}, \hat{y} \sim \mathcal{M}(y)} g(\hat{y}, y)$ , where  $\hat{y}$  is the noisy label generated by  $\mathcal{M}$  on input  $y$ .
This was motivated by the triangle inequality  $\mathrm{[GKK^{+}23, Eq. (1)]}$ , namely,
$$
\underset {(x, y) \sim \mathcal {D}} {\mathbb {E}} \ell \left(f _ {\theta} (x), y\right) \leq \underset { \begin{array}{c} y \sim \mathcal {P} \\ \hat {y} \sim \mathcal {M} (y) \end{array} } {\mathbb {E}} \ell (\hat {y}, y) + \underset { \begin{array}{c} (x, y) \sim \mathcal {D} \\ \hat {y} \sim \mathcal {M} (y) \end{array} } {\mathbb {E}} \ell \left(f _ {\theta} (x), \hat {y}\right). \tag {1}
$$
The work of  $\left[\mathrm{GKK}^{+}23\right]$  focused on optimizing noisy label loss; they showed that the optimal randomizer takes the form of "Randomized-Response on Bins" (see Appendix F for more details). However, if we notice carefully, the noisy label loss does not depend on the number of examples, so the RHS of (1) does not go to zero as the number of examples goes to infinity, even though we would like the excess population loss of the learnt predictor to asymptotically go to zero. In this paper, we provide an alternative insight into the goal of minimizing the noisy label loss.
For any distribution  $\mathcal{D}$  over  $\mathcal{X} \times \mathcal{Y}$ , and any label randomizer  $\mathcal{M}$  mapping  $\mathcal{V}$  to  $\hat{\mathcal{V}} \subseteq \mathbb{R}$ , let  $\mathcal{D}_{\mathcal{M}}$  be the distribution over  $\mathcal{X} \times \hat{\mathcal{V}}$  sampled as  $(x, \mathcal{M}(y))$  for  $(x, y) \sim \mathcal{D}$ . The Bayes optimal predictor for any distribution  $\mathcal{D}$  over  $\mathcal{X} \times \mathcal{Y}$  w.r.t. loss  $\ell$ , is a predictor  $f_{\mathcal{D}}^{*}: \mathcal{X} \to \mathbb{R}$  that minimizes  $\mathcal{L}_{\mathcal{D}}(f)$ , which roughly corresponds to the best predictor we could hope to learn with unbounded number of samples from  $\mathcal{D}$ . We show that, when training with a loss from a broad family that includes  $\ell_{\mathrm{sq}}$  and  $\ell_{\mathrm{Poi}}$ , the Bayes optimal predictor is preserved after applying the label randomizer iff  $\mathbb{E}[\mathcal{M}(y)] = y$  holds for all  $y \in \mathcal{V}$  (i.e., the randomizer is unbiased).
Theorem 5. Suppose  $\ell$  is a loss function such that for all distributions  $\mathcal{P}$  over  $\mathcal{V}$ , the minimizer  $\hat{y}_{*} := \min_{\hat{y} \in \mathbb{R}} \mathbb{E}_{y \sim \mathcal{P}} \ell(\hat{y}, y)$  exists and is given as  $\hat{y}_{*} = \mathbb{E}_{y \sim \mathcal{P}}[y]$ . Then the following are equivalent:
$\triangleright \mathbb{E}[\mathcal{M}(y)] = y$  holds for all  $y\in \mathcal{V}$
For all distributions  $\mathcal{D}$  over  $\mathcal{X} \times \mathcal{Y}$ , it holds that  $f_{\mathcal{D}}^{*} = f_{\mathcal{D}_{\mathcal{M}}}^{*}$ .
The main observation is that  $f_{\mathcal{D}}^{*}(x_0) = \mathbb{E}[y \mid x = x_0]$  with probability 1 over  $x_0$ , and similarly,  $f_{\mathcal{D}\mathcal{M}}^{*}(x_0) = \mathbb{E}[\mathcal{M}(y) \mid x = x_0]$  with probability 1 over  $x_0$ . We defer the full proof to Appendix A, but for now we demonstrate a simple example where a predictor learned using noisy labels produced by the optimal RR-on-Bins randomizer can in fact be suboptimal. Let  $\mathcal{X} = \{\mathsf{a},\mathsf{b}\}$  and  $\mathcal{Y} = \{0,1,2\}$  and the distribution  $\mathcal{D}$  be defined in Table 1.
0	1	2
a	0.35	0.1	0.05
b	0.25	0.15	0.1
Table 1: Example  $\mathcal{D}$
The Bayes optimal predictor for  $\mathcal{D}$  w.r.t.  $\ell_{\mathrm{sq}}$  (or even  $\ell_{\mathrm{Poi}}$ ) is given as  $f_{\mathcal{D}}^{*}(\mathbf{a}) = 0.4$  and  $f_{\mathcal{D}}^{*}(\mathbf{b}) = 0.7$ . On the other hand, the optimal  $\mathcal{M} = \text{RR - on - Bins}_{\varepsilon}^{\Phi}$  randomizer for  $\varepsilon = 0.5$  (see Appendix F for notation), corresponds to the map  $\Phi(0) \approx 0.396$  and  $\Phi(1) = \Phi(2) \approx 0.720$ . The Bayes optimal predictor for  $\mathcal{D}_{\mathcal{M}}$  is given as  $f_{\mathcal{D}_{\mathcal{M}}}^{*}(\mathbf{a}) = 0.542$  and  $f_{\mathcal{D}_{\mathcal{M}}}^{*}(\mathbf{b}) = 0.558$ . The squared loss of these predictors are given as  $\mathcal{L}_{\mathcal{D}}(f_{\mathcal{D}}^{*}) = 2.625$  and  $\mathcal{L}_{\mathcal{D}}(f_{\mathcal{D}_{\mathcal{M}}}^{*}) = 2.726$ , the latter being sub-optimal.
Thus, preserving the Bayes optimal predictor is a desirable property of a label randomizer, as any learning algorithm that converges to the optimal predictor can also be trained on the noisy labels and will approach the predictor for the original distribution.
Additionally, we can relate the property  $\mathbb{E}[\mathcal{M}(y)] = y$  to the unbiasedness of the gradients obtained in SGD. First, we observe that the gradient with respect to the parameters for  $\ell_{\mathrm{sq}}$  and  $\ell_{\mathrm{Poi}}$  are affine in  $y$ , namely,
$$
\nabla_ {\theta} \ell_ {\mathrm {s q}} (f _ {\theta} (x), y) = (f _ {\theta} (x) - y) \cdot \nabla_ {\theta} f _ {\theta} (x), \qquad \nabla_ {\theta} \ell_ {\mathrm {P o i}} (f _ {\theta} (x), y) = (1 - y / f _ {\theta} (x)) \cdot \nabla_ {\theta} f _ {\theta} (x).
$$
Thus, the error in the gradient estimate when using the noisy label  $\hat{y}$  instead of  $y$  is given as
$$
\nabla_ {\theta} \ell_ {\mathrm {s q}} (f _ {\theta} (x), \hat {y}) - \nabla_ {\theta} \ell_ {\mathrm {s q}} (f _ {\theta} (x), y) = (y - \hat {y}) \cdot \nabla_ {\theta} f _ {\theta} (x),
$$
$$
\nabla_ {\theta} \ell_ {\mathrm {P o i}} (f _ {\theta} (x), \hat {y}) - \nabla_ {\theta} \ell_ {\mathrm {P o i}} (f _ {\theta} (x), y) = (y - \hat {y}) \cdot \frac {\nabla_ {\theta} f _ {\theta} (x)}{f _ {\theta} (x)}.
$$
Let  $S = \{(x_{1},y_{1}),\ldots ,(x_{b},y_{b})\}$  be a mini-batch of examples and let  $\hat{S} = \{(x_1,\hat{y}_1),\dots ,(x_b,\hat{y}_b)\}$  be the mini-batch with noisy labels as returned by the randomizer  $\mathcal{M}$ . Also consider the dataset with expected noisy labels  $\tilde{S} = \{(x_1,\tilde{y}_1),\dots ,(x_b,\tilde{y}_b)\}$ , where  $\tilde{y}_i = \mathbb{E}_{\hat{y}\sim \mathcal{M}(y)}\hat{y}$ . The difference between a mini-batch gradient w.r.t. noisy labels and the gradient of the population loss decomposes for  $\ell_{\mathrm{sq}}$  as follows (similar decomposition holds for  $\ell_{\mathrm{Poi}}$ ):
$$
\begin{array}{l} \nabla_ {\theta} \mathcal {L} _ {\hat {S}} (f _ {\theta}) - \nabla_ {\theta} \mathcal {L} _ {\mathcal {D}} (f _ {\theta}) \\ = \nabla_ {\theta} \mathcal {L} _ {S} (f _ {\theta}) - \nabla_ {\theta} \mathcal {L} _ {\mathcal {D}} (f _ {\theta}) + \nabla_ {\theta} \mathcal {L} _ {\tilde {S}} (f _ {\theta}) - \nabla_ {\theta} \mathcal {L} _ {S} (f _ {\theta}) + \nabla_ {\theta} \mathcal {L} _ {\hat {S}} (f _ {\theta}) - \nabla_ {\theta} \mathcal {L} _ {\tilde {S}} (f _ {\theta}) \tag {2} \\ = \underbrace {\nabla_ {\theta} \mathcal {L} _ {S} (f _ {\theta}) - \nabla_ {\theta} \mathcal {L} _ {\mathcal {D}} (f _ {\theta})} _ {(a)} + \underbrace {\underset {(x , y) \in S} {\mathbb {E}} (y - \mathbb {E} \hat {y}) \cdot \nabla_ {\theta} f _ {\theta} (x)} _ {(b)} + \underbrace {\underset {(x , y) \in S} {\mathbb {E}} (\mathbb {E} \hat {y} - \hat {y}) \cdot \nabla_ {\theta} f _ {\theta} (x)} _ {(c)}. \\ \end{array}
$$
The term  $(a)$  is the difference between the mini-batch gradient and the population gradient, the inherent stochasticity in mini-batch SGD, which has zero bias. The terms  $(b)$  and  $(c)$  form precisely the bias-variance decomposition for the additional stochasticity in the gradient introduced by the label noise, with the term  $(c)$  having zero bias. In the case of stochastic convex optimization, it is well known that with access to biased gradients, it is impossible to achieve vanishingly small excess loss; whereas, with access to gradients with zero bias and any finite variance, it is possible to achieve an arbitrarily small excess loss using sufficient number of steps of stochastic gradient descent (see Appendix D for more details). Motivated by this reasoning, our approach is to use a randomizer that has the smallest variance possible while ensuring zero bias, namely the term  $(b)$  is zero.
3.1 Computing Optimal Unbiased Randomizers
We consider a label randomizer that minimizes the noisy label loss, while being unbiased. Namely, we say that an  $\varepsilon$ -DP randomizer  $\mathcal{M}$  that maps the label set  $\mathcal{V} \subseteq \mathbb{R}$  to  $\mathbb{R}$  is unbiased if it satisfies  $\mathbb{E}_{\hat{y} \sim \mathcal{M}(y)} \hat{y} = y$  for all input labels  $y \in \mathcal{V}$ . We use a linear programming (LP) based algorithm (Algorithm 1) to compute an unbiased randomizer that minimizes the noisy label loss. In order to keep this approach computationally tractable we require that both  $\mathcal{V}$  and  $\hat{\mathcal{V}}$  are finite; as we discuss shortly, it is possible to handle general  $\mathcal{V}$  by discretization using randomized rounding, and moreover the excess noisy label loss due to consideration of a discrete  $\hat{\mathcal{V}}$  can be bounded as well. Even though Algorithm 1 is general, we will only consider  $g = \ell_{\mathrm{sq}}$  henceforth (irrespective of the training loss  $\ell$ ).
Algorithm 1 ComputeOptUnbiasedRand
Parameters: Privacy parameter  $\varepsilon \geq 0$
Input:  $\mathcal{P} = (p_y)_{y\in \mathcal{Y}}$  prior over input labels  $\mathcal{V}$
$\hat{y} = (\hat{y}_i)_{i \in \mathcal{I}}$  — a finite sequence of potential output labels.
Output: An  $\varepsilon$ -DP label randomizer.
Solve the following LP in variables  $M = (M_{y \to i})_{y \in \mathcal{Y}, i \in \mathcal{I}}$ :
$$
\min  _ {M} \sum_ {y \in \mathcal {Y}} p _ {y} \left(\sum_ {\hat {y} \in \hat {\mathcal {Y}}} M _ {y \rightarrow i} \cdot g (\hat {y} _ {i}, y)\right), \quad \text {s u b j e c t t o}
$$
[Non-negativity]  $\forall y\in \mathcal{V},i\in \mathcal{I}:M_{y\to i}\geq 0$
[Normalization]  $\forall y\in \mathcal{V}$  ：  $\sum_{i\in \mathcal{I}}M_{y\to i} = 1$
$[\varepsilon\text{-LabelDP}]\quad \forall i \in \mathcal{I}, \forall y, y' \in \mathcal{Y}$  s.t.  $y \neq y'$ :  $M_{y' \to i} \leq e^{\varepsilon} \cdot M_{y \to i}$
[Unbiasedness]  $\forall y\in \mathcal{V}$  ：  $\sum_{i\in \mathcal{I}}M_{y\to i}\cdot \hat{y}_i = y$
return Label randomizer  $\mathcal{M}$  mapping  $\mathcal{V}$  to  $\hat{\mathcal{V}}$  given by  $\operatorname*{Pr}[\mathcal{M}(y) = \hat{y}_i] = M_{y\to i}$ .
In Figure 2, we illustrate a prior  $\mathcal{P}$  over  $\mathcal{Y} = \{0,1,2\}$ , using the example in Table 1, and the corresponding unbiased randomizer returned by ComputeOptUnbiasedRand $_{\varepsilon=0.5}$ ( $\mathcal{P},\hat{\mathcal{Y}}$ ) for a certain
Figure 2: (a) An example prior  $\mathcal{P}$  over  $\mathcal{V} = \{0,1,2\}$ . (b) The optimal unbiased randomizer returned by ComputeOptUnbiasedRand $_{\varepsilon=0.5}$ ( $\mathcal{P},\hat{\mathcal{V}}$ ) using a fine grid for  $\hat{\mathcal{V}}$ , indicated by the probability mass function for each input label.
fine-grained choice of  $\hat{\mathcal{V}}$ . It is worth noting that this randomizer is quite unlike "randomized response" or any additive noise mechanism! Additionally, the output labels are all outside [0, 2], which is the convex hull of  $\mathcal{V}$ . We have observed the same in our experiments in Section 4 as well. One may find it surprising that these "out of domain" noisy labels in fact yield better trained models! This could be attributed to the Bayes optimal predictor for this randomizer being precisely  $f_{\mathcal{D}}^{*}$ , since  $\mathbb{E}[\mathcal{M}(y)] = y$  for all  $y \in \mathcal{V}$  (Theorem 5).
There are however a couple of challenges to be addressed. First, it is unclear if the optimal unbiased randomizer has only finitely many labels, and even if so, what is the number of distinct output labels. Secondly, one needs to fix the choice of  $\hat{Y}$  before hand, and it is unclear how one can choose it in a way that the LP is feasible and moreover, the solution is close to the optimal unbiased randomizer. We address these challenges as follows.
Structure of optimal unbiased randomizers. Addressing the first challenge, we show that there is an optimal unbiased randomizer that has a small number of distinct output labels.
Theorem 6. For any convex loss  $g$ , all distributions  $P$  over  $\mathbb{R}$  with finite support, and  $\varepsilon > 0$ , there exists an  $\varepsilon$ -DP unbiased randomizer  $\mathcal{M}_* \in \inf_{\varepsilon \text{-DP unbiased } \mathcal{M}} \mathcal{G}(\mathcal{M}; \mathcal{P})$  with at most  $2|\mathcal{Y}|$  output labels.
The main ideas in the proof are as follows: we use the convexity of  $g$  to show that an optimal  $\varepsilon$ -DP unbiased randomizer  $\mathcal{M}$  must be of a certain form; if not we can use Jensen's inequality to construct another randomizer  $\mathcal{M}'$  with that form such that  $\mathcal{G}(\mathcal{M}'; P) \leq \mathcal{G}(\mathcal{M}; \mathcal{P})$ . In particular, we show that the randomizer must be a staircase mechanism as defined in [KOV16], a mechanism that maximally satisfies the DP constraints. This already implies that the number of output labels is at most  $2^{|\mathcal{V}|}$ . We use the ordering of the labels, along with similar reductions using convexity of  $g$ , to show that the optimal  $\varepsilon$ -DP unbiased randomizer further lies within a special subset of staircase mechanisms with  $2^{|\mathcal{V}|}$  output labels. The detailed proof is provided in Appendix B.
Choosing  $\hat{\mathcal{V}}$  to ensure feasibility and good coverage. To address the second challenge, we use a finite set of output labels. We show an upper bound on the excess noisy label loss due to discretization.
We use a heuristic (Algorithm 2) for setting  $\hat{\mathcal{V}}$  to be a grid, in a way that ensures that the LP in Algorithm 1 has a feasible solution while keeping  $\hat{\mathcal{V}}$  small enough to be able to efficiently solve the LP. To compute the endpoints of the grid, we use an unbiased randomizer with a finite and bounded support: the  $\varepsilon$ -DP debiased randomized response ( $\varepsilon$ -dbRR) on the set of labels, which operates by mapping the inputs to a unique set of values such that under randomized response the randomizer is unbiased (see Appendix F for a definition). We choose the endpoints of our grid to be precisely the minimum and maximum among the possible outputs of  $\varepsilon$ -dbRR (see Algorithm 2 for details). With those two endpoints defined, we create the grid by evenly spacing output labels along this interval. The number of output labels we generate is as large as possible while maintaining that the LP solver terminates in a reasonable amount of time. We show that having these endpoints suffices to ensure feasibility of the LP in Algorithm 1.
Proposition 7. If  $\hat{\mathcal{V}}$  contains just the smallest and largest values among the outputs of  $\varepsilon$ -dbRR, the LP in ComputeOptUnbiasedRand $_{\varepsilon}(P,\hat{\mathcal{V}})$  is feasible.
Algorithm 2 FeasibleOutputSet
Input: Set of input labels  $\mathcal{V}$ . Positive integer  $n\geq 2$  representing size of output  $|\hat{\mathcal{V}} |$
Output: Set of output values  $\hat{y}$  that guarantee feasibility of LP in Algorithm 1.
$$
L \leftarrow \left((e ^ {\varepsilon} + | \mathcal {Y} | - 1) \cdot \min  (\mathcal {Y}) - \sum_ {y \in \mathcal {Y}} y\right) / (e ^ {\varepsilon} - 1)
$$