Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
Authors: Ce Zhang
Yixin Han
Yafei Wang
Xiaodong Yan
Linglong Kong
Ting Li
Bei Jiang
Venue: ICML 2025 poster (2025)
Keywords: Binary response model
Differential privacy
Inference
Optimality
Randomized response
Primary area: social_aspects->privacy

Abstract: Randomized response (RR) mechanisms constitute a fundamental and effective technique for ensuring label differential privacy (LabelDP). However, existing RR methods primarily focus on the response labels while overlooking the influence of covariates and often do not fully address optimality. To address these challenges, this paper explores optimal LabelDP procedures using RR mechanisms, focusing on achieving optimal estimation and inference in binary response models. We first analyze the asymptotic behaviors of RR binary response models and then optimize the procedure by maximizing the trace of the Fisher Information Matrix within the $\varepsilon$- and $(\varepsilon,\delta)$-LabelDP constraints. Our theoretical results indicate that the proposed methods achieve optimal LabelDP guarantees while maintaining statistical accuracy in binary response models under mild conditions. Furthermore, we develop private confidence intervals with nominal coverage for statistical inference. Extensive simulation studies and real-world applications confirm that our methods outperform existing approaches in terms of precise estimation, privacy protection, and reliable inference.

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
Ce Zhang' Yixin Han
Yafei Wang'
Xiaodong Yan Linglong Kong'
Ting Li Bei Jiang'
Abstract
to analyze data without directly accessing sensitive infor- mation while exploring theoretical guarantees to prevent the exposure of private details. On the other hand, the ran- dom response mechanism (RR) (Warner, 1965) has gained significant attention in sensitive binary data and achieves effective DP guarantees by flipping the binary labels with a predetermined probability (Hout et al., 2010; Coutts & Jann, 2011; Kirchner, 2015; Oberski & Kreuter, 2020). Although surge of works explores DP guarantees under RR mecha- nisms (Holohan et al., 2017; Fox et al., 2018; Ghazi et al., 2020; Pastore & Gastpar, 2021; Xu et al., 2023), there is still limited guidance on the optimal DP mechanism for RR labels, particularly in the context of estimation and infer- ence for binary response models. To address this gap, we introduce novel optimal DP RR mechanism tailored for bi- nary response models that take into account the influence of covariates. Our proposed DP RR mechanism operates under the setting that the input covariates are publicly available while the response labels are highly sensitive, which re- quires rigorous protection. Motivated by the optimal design of the experiment via maximizing the various types of "data information" (Pukelsheim, 2006; Dette et al., 2012; Yang et al., 2013; Waite & Woods, 2022; Duarte et al.. 2022), we primarily focus on exploring the optimal estimation and inference for the RR binary response models by maximizing the trace of the Fisher Information Matrix within a LabelDP feasible region for RR parameters. This ensures the opti- mality of the RR mechanism in safeguarding privacy while maintaining statistical accuracy.
Randomized response (RR) mechanisms consti- tute a fundamental and effective technique for en- suring label differential privacy (LabelDP). How- ever, existing RR methods primarily focus on the response labels while overlooking the influence of covariates and often do not fully address op- timality. To address these challenges, this paper explores optimal LabelDP procedures using RR mechanisms, focusing on achieving optimal esti- mation and inference in binary response models. We first analyze the asymptotic behaviors of RR binary response models and then optimize the pro- cedure by maximizing the trace of the Fisher Infor- mation Matrix within the e- and (s, L) constraints. Our theoretical results indicate that the proposed methods achieve optimal LabelDP guarantees while maintaining statistical accuracy in binary response models under mild conditions. Furthermore, we develop private confidence in- tervals with nominal coverage for statistical infer- ence. Extensive simulation studies and real-world applications confirm that our methods outperform existing approaches in terms of precise estimation, privacy protection, and reliable inference.
1. Introduction
Motivation. Privacy protection is crucial in modern sta- tistical analysis and machine learning to handle personal information and sensitive data. Differential privacy (DP) (Dwork et al., 2006; 2014) has recently emerged as the gold standard for preserving privacy when analyzing sensitive data. It provides a statistical mechanism that enables users
Literature review. Numerous approaches have been pro- posed to develop DP algorithms through RR mechanisms (Ding et al., 2020; Garcelon et al., 2021; Biswas et al., 2023). In this section, we organize the relevant work into three groups and highlight how our methodology differs significantly, emphasizing the importance of thorough review.
Equual contribution 'Department of Mathematical and Statisti- cal Sciences, University of Alberta, Edmonton, Canada 'School of Mathematics and Statistics, Xi' an Jiaotong University, Xi' an, China 'Department of Applied Mathematics, The Hong Kong Poly- technic University, Hong Kong, China. Correspondence to: Bei Jiang <biil @ @ualberta.ca>, Ting Li <tingeric. wolyu.cdu.hk.>
DP via RR. Based on the classical RR mechanisms, Kairouz et al. (2016) studied the optimal local DP mechanisms for binary data. Holohan et al. (2017) extended Warner's origi- nal technique (Warner, 1965) to establish an RR mechanism that simultaneously achieves DP and optimality by minimiz- ing the variance estimator of the proportion of true answers.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
Furthermore, Ghazi al. (2021) introduced deep learning method to construct an RR mechanism. Recently, Waudby- Smith et al. (2023) presented nonparametric extension of Warner's s seminal RR mechanism tailored for bounded data with sequential interactivity. Furthermore, Barnes et al. (2020) and Asoodeh & Zhang (2022) investigated how fisher information from statistical samples can scale with the pri- vacy parameter E under local DP constraints. Despite the success of DP via RR in various domains, the challenges of statistical inference for RR binary response models under DP remain unresolved, potentially impacting both accuracy and efficiency.
models. Specifically, our definition of optimality aims maximize the trace of the asymptotic information matrix in (5) within the feasible LabelDP region for the RR parameters. as illustrated in Figure 1. Further- more, we establish the optimal inference for RR binary response models with a specified level of confidence within the LabelDP framework.
We provide a rigorous theoretical analysis of the pro- posed method, including the privacy guarantees and asymptotic normality. Theoretical results establish that this novel optimal LabelDP RR mechanism guarantees and (c, 6) L)DLaeDI privacy while achieving optimal estimators and asymptotic normality for the regression coefficients in RR binary response models. Addition- ally, we develop a privatized confidence interval that delivers asymptotically valid coverage while preserv- ing privacy.
Label differential privacy. Label differential privacy (La- belDP) is a well-established method to protect sensitive labels while allowing non-private covariates under the RR mechanism in classification tasks (Chaudhuri & Hsu, 2011). LabelDP has received significant attention due to the in- creasing demands in various fields (Nayak & Adeshiyan, 2009; Wang et al., 2016; Busa-Fekete et al., 2021; Chaud- et al., 2021; Xu et al., 2023). More recently, Fekete et al. (2023) established the LabelDP mechanisms to share training data in machine learning, enabling accurate predictive model learning while safeguarding the privacy of each user's labels.
We conduct extensive simulations and real-data exper- iments to demonstrate that our approach outperforms several existing mechanisms, providing significantly higher efficiency regarding estimation accuracy and coverage probability. Notably, our method achieves a threefold improvement in coverage probability within the confidence interval, closely matching the perfor- mance of non-private methods. This highlights the ef- fectiveness of our approach in balancing privacy preser- vation with high statistical efficiency.
huri& & Hsu, 2011;
Inference under RR binary response models. In statistics, inference involves the use of representative training data to learn about population characteristics, often represented by model parameters (Tutz, 2011). Statistical inference under LabelDP goes beyond providing point estimates; its goal is to construct confidence intervals that capture the true val- ues of the model/populatioo parameters while accounting for sampling variability and noise introduced by LabelDP (Ghazi et al., 2022). Numerous works have been performed on the estimation of binary response models using RR mech- anisms without inference (Van den Hout et al., 2007; Hsieh et al., 2010; Duan et al., 2016). Although Fox et al. (2018) has explored statistical inference and extended the estimate to wide range of RR designs, their method does not con- sider optimal estimation and inference within the LabelDP framework, which is the main objective of our paper.
Paper organization. The rest of this paper is organized as follows. Section 2 reviews the essential background of LabelDP and RR mechanisms. We introduce the RR binary response models and establish the novel optimal LabelDP RR mechanism under the E- and (s, d>-LabeDDP frameworks in Section 3. Section 4 provides the theoretical results on LabelDP guarantees and the private inference of the confi- dence interval. Extensive simulation studies are conducted in Section 5 to demonstrate the superior performance of the proposed methods. In Section 6, we apply our proposed method to a real-world dataset focused on measuring pla- giarism among students to further illustrate its effecttivnness. Section 7 concludes the paper with several additional topics. The detailed technical proofs are included in the appendix.
Our goals and contributions. Traditional optimal LabelDP RR mechanisms, such as those in Holohan et al. (2017) and Wang et al. (2017), focus solely on optimizing privacy for the response variable Y, but ignore the role of covariates X. In contrast, our approach incorporates X by maximizing the trace of the Fisher Information Matrix in the context of RR binary response models, leading to improved statistical efficiency and accuracy capabilities that traditional meth- ods do not offer. Specifically, we make the following major contributions.
2. Preliminaries
In this section, we present brief review of the LabelDP and RR mechanisms, which lay the groundwork for our proposed method.
LabelDP. DP has emerged as a state-of-the-art framework for releasing mriacy-preservinn data in data science and machine learning (Dwork et al., 2006; 2014; Dong et al., 2022). The core of DP to ensure that an individual's data
This is the first work to investigate the optimal E- and (s, L)-LabeIDP mechanisms using RR binary response
2

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
is sufficiently masked within the dataset. This enables anal- ysis to be performed without direct access to the original data, offering theoretical assurances that prevent the expo- sure of sensitive information. Our work focuses mainly on the binary response model, so we introduce the concept of LabelDP, a specialized application of DP that specifically targets the response space.
In scenarios where both Poo and P11 are equal to the RR mechanism directly reflects the true response, resulting in P(Y; 0) " = 0) and P(Y; = =) IP = 1). However, this RR mechanism only addresses the unsuper- vised aspect of the response. This means that it does not take into account the relationship between the response variable and the covariates or features, leading to potential inefficien- cies and inaccuracies when used in supervised learning con- texts, such regression models. In the following sections, we will investigate the application of RR binary response models within the framework of LabelDP. This approach provides a deeper insight into the effective application of privacy-preserving techniques in binary response models.
Definition 2.1 ((E, 3)-LabeIDP (Ghazi et al., 2021)). Let
X' and y denote the covariate and binary response spaces, respectively.
randomized algorithm or mechanism A X' y is (e, d)-LabeIDP for some parameters > O, 0 8 8 S 1. If for any pair of datasets D, D' that only differ in the label of a single example and any measurable event C I y. it holds that
3. MMethodllogy
P (AA(D) (C) E EC) 8
In this section, we aim to develop a novel LabelDP RR mechanism that yields an optimal estimator for the RR binary response models while safeguarding the privacy of individual responses. We first introduce a new data structure that incorporates RR and binary response models. Then we present the fundamental framework of the proposed optimal LabelDP RR mechanism.
where the probability P is taken over the randomness of A.
The privacy budget (5. quantifies the level of protection of privacy and the probability of relaxation. If 8 = 0, A is called pure LabelDP,, denoted by e-LabelIDP; if 5 > 0, it is considered the approximate LabelDP. Generally speak- ing, the smaller the sizes of both E and 5, the stronger the protection level, but this will limit the utility of the esti- mators. Commonly used LabelDP mechanisms randomize target quantities (e.g.. parameter, loss, gradient, or aggregate statistics) by adding random noises derived from elaborately designed distributions to hide each individual's contribution (Kairouz et al., 2016).
3.1. RR binary response models
Suppose we observe n independent and identical distributed (i.i.d) observations Dū {X; Y 3r= from a binary response model
E EY; x.) PP =1 11),) = G (3. X.),
(2)
RR mechanism. Suppose that there are n individuals, each with an observed binary response Y, E (0, 1). dependent on the true value Y," E f0. 1). The definition 2.2 describes the relationship between Y and y* for the RR mechanism via a design matrix.
where X, = (Xi,,.... Xia) E Rd
is the d-dimensional covariate, Y E {0, is the true binary label, B. is the true parameter vector. Throughout this paper, we assume that only the responses Y," contain sensitive information, while the covariates X; are considered non-sensitive. The func- tion G(B X,) denotes the link function, and the inverse of the link function is typically assumed to be monotonic and differentiable to the linear predictor 3.T X, Depending on the form of G(3 X,), the Model (2) can accommodate var- ious binary response models as special cases, including Lo- gistic regression exp(,3, x1) + exp((3 X.)). Cauchy distribution -= arctan(33 [,), + 1/2, and Probit regres- sion ((3.. X,), where t typically represents the cunulative distribution function of the standard normal distribution, as explained in (Tutz, 2011; McCullagh, 2019).
Definition 2.2 (Design matrix). The RR mechanism for binary true value Y," follows a 2 x 2 design matrix
P =
Poo
Poi
)
Poo
Poo
)
P10
P11:
P11
PI1
where pkj IPY: j Y k) with j. = {0, 1}.
represents the conditional response probability (Wang et al.,
2016; Holohan et al., 2017; Chaudhuri & Mukerjee, 2020),
and adheres the fact that Pao + P01 P10 + P11 1.
Subsequently, the probability mass function of each ob- served value Y, can be calculated by
Due to privacy considerations, researchers often encounter practical challenge: they typically only have access to potentially privatized dataset D,, {X;, Y.). - instead of the true dataset D", when using the RR mechanism. In particular, the labels in D., may not accurately represent the true responses. In such cases, it is crucial to utilize a calibration counterpart binary response to construct the RR mechanism, as exemplified in (1) and (2). Consequently, we
P(Y: = 0) = (1 PY," = 1)) Auo I P(Y. = (1 pı))
= puo (X,"" 1) (pro Pm 1),
(I) P(Y: = 1) = PY = p) (1 PO, = =]) puo)
1- poo + (000 + P11 1)P =1).
3

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
formulate the RR binary response models
E [Y4] X.) (Y; X;)
1 Puo (peo + P11 - G) (p! ())
(3)
Note that Equation (1) provides straightforward relation- ship between Model (2) and Model (3), effectively adapting (2) to account for the randomness introduced by the RR mechanism. As a result, there is no need for a debiasing step when estimating Model (3), since it already includes the necessary adjustments due to (1).
Bes
The Maximum likelihood estimator (MLE) is effective in estimating B, because the maximum likelihood equation for Model (3) shares a structure similar to the maximum likelihood equations of the generalized linear model (GLM) maximum likelihood equations (Blair et al., 2015). To get the MLE of B. based on the observed Dn. we solve the following optimization problem
15
Pas ++ -P:1
Pr: (1--P))
3 arg max In (.B.)
A.
Pt
+ log (1 (1-p)) pi) pi)} (4)
arg max
Y, log
B. n
1-p: Pi
11.4
where p: P (Y; X.). TJi B_ X, and (G.. is
the log-libe ligeliielhood function. Let S(-) be the derivative of (.()) and II() be the Fisher Information Matrix with respect to B, respectively. The S(-) and I(-) D, can be specified as follows:
Figure I. Feasible regions of the RR mechanisms. Top panel: E- LabeIDP; Bottom panel: (e, LabbelDD
S(3: Poe. Pa1)
(poo + Pı1 1)(%, p.) 8G (nd)
feasible regions for RR parameters (Poo, P11), defined as:
(1 pi)
Om
I (3: P11) (Poo + Pı1 1)2 (m))m)
{
(m.)
} x,X (5)
Po0.P11 P11 S
Poo. P11 S 1
P (1 P.) On,
poo P11 > 1,
poo P11 1,
R=
R'=
Poo S cf(1 P11),
Poo o² (1
P11) 6,
The derivative S and the Fisher Information Matrix II play crucial roles in determining the asymptotic behaviour of the estimator, which is essential for establishing the optimality of the DP RR mechanism.
S cf(1 Poo)
e= pno)
In general, default assumption is that the RR mechanism tends to favor the true value, that is, Poo. Pii 0.5 (Wang, 2015). Setting Poo, P11 > 0.5 implies that the RR mecha- nism is more likely to generate a true answer. This assump- tion crucial for the statistical analysis of Model (3), as the RR mechanism's tendency to lean towards the truth enables more meaningful insights.
3.2. Optimal LabelDP mechanism
We establish the optimal LabelDP mechanisms for the RR binary response models (3) in this subsection. First, we construct the feasible regions of RR parameters (poo- Pı1) under LabelDP constraints in Definition 3.1 inspired by (Holohan et al., 2017). Figure illustrates the corresponding feasible regions.
Next, develop our optimal LabelDP mechanisms for the RR binary response model. To achieve our goal, we borrow the concepts from the design of experiments and consider the T-optimality criterion (Pukelsheim, 2006; Dette et al., 2012), which maximizes the trace of the Fisher Information Matrix M(B:Poo, Poo. P11) tr(T(8; Poo. Pı1)}, restricted to
Definition 3.1. An RR mechanism (1) satisfies e-LabeIDP if (poo. Pı1) R C R? or satisfies (e, Label-L if (poo.PI) P1)) E R' C RR, where R. and R' are the LabelDP

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
the LabelDP feasible regions R. and R'.
struction of the privatized confidence interval for our pro- posed methods. We begin with the theoretical guarantees for e-LabelDP and then extend our focus to (s, 6--LabeIDP. Fur- thermore, we explore the statistical inference of / based on Model (3). To support our theoretical results, we introduce several technical assumptions necessary for the proofs.
(Pro.Pnk
arg max (poo-P))ERorR
M (B: Poo,P11).
P11).
(6)
is important to note that while both -optimallity and D- optimality aim at maximizing information, D-optimality involves maximizing the determinant of the Fisher Infor- mation Matrix, which is computationally challenging and more complex due to the involvement of determinant calcu- lations, especially for high-dimensional parameter spaces. However, T Toppimality is computationally simpler, making significantly easier to implement in practical scenarios without sacrificing the quality of inference. Moreover, in many cases, the two criteria are equivalent or closely re- lated, and maximizing the trace often provides satisfactory approximation to maximizing the determinant.
Assumption 4.1. .M poo Poo-P11) is twice continuously dif- ferentiable with respect to Poo and Pi1-
Assumption 4.2. Suppose that Poo + Pt > 1.
Assumption 4. is the regular conditions imposed to ensure the unique and finite maximum in optimization (Boyd & Vandenberghe, 2004). Assumption 4.2 requires that the RR parameters satisfy Poo + P11 to ensure the identifiability of the parameter for the RR binary response model (3) across the feasible regions.
Remark 3.2. In detail, our Toopimallity criterion is equiv- alent in asymptotic efficiency but far more practical. The trace decomposes into sum of variances (diagonal ele- ments), reducing the problem to scalar optimization (Lemma 4.2). This allows closed-fore solutions for (Poo, Pı1) on the boundary of R (Theorems 4.1 and 4.7). The trace di- rectly corresponds to minimizing the average variance of B, which aligns with our goal of precise estimation. Both cri- teria yield consistent estimators, but -optimaality achieves this with (d) complexity versus O (d*) for D-optimality due to determinant caleulations, where d is the number of covariates.
Remark 4.3. Assumption 4.2 serves multiple purposes by improving theoretical robustness and optimality. First, it ensures that the RR mechanism tends to capture the true response accurately, thereby improving the utility of the pri- vatized response label data. Second, it restricts the feasible region for (Poo, Ptt). leading to well-defined optimization process and practical solutions for optimal LabelDP RR mechanisms.
Lemma 4.4. Suppose Assumptions 4.1-4.2 hold There exists (Pon. such that (Póo.Pli) E arg (po-ppP or R. MB: poo. P11). where (P6o. Pit) lie on the boundaries of R or R'.
Remark 3.3. Awan & Slavkovic (2019) optimized the RR mechanisms solely for the binary response Y by minimiz- ing YY, where Y is the sample sum, which is a complete sufficient statistic for the binomial model in their paper, without considering covariate information X. while our Fisher information maximization I(B3; Pou. Pı1) explic- itly incorporates covariate X through the model (3). Our novel method explicitly leverages the covariate structure, enhancing the estimation efficiency when covariates are present.
Lemma 4.4 demonstrates that M(3: P00-Pr1) is convex and thus confirms the existence of optimal solutions for the RR parameters in binary response models. The optimal (Poo- Pî1) attains on the boundaries of the feasible regions R and R' by maximizing the function MMB Pou, P11). Under Lemma 4.4, we will proceed to establish the optimal RR mechanisms for 8- and (c, LLabelDDP constraints.
4.1. Optimal mechanism for e-LabeLDP
The estimators (Poo. P11) do not depend on 3 based on the specific formulation of MMB Po11). This indepen- dence arises because M(3 Poo, Pin) is designed to iso- late Poo and P11 from the influence of / on @G(m:)/öm) Therefore, in practical applications, the initial step is to use M(3 Poo, P11) to obtain the optimal estimators of Poo and P11- Once these values are estimated, we then plug them into (4) to obtain B. The estimated / is optimal in the sense that it maximizes the trace of the Fisher Information Matrix Il (3; P11..
Theorem 4.5 (e-LabeIDP). Suppose Assumptions 4.1-4.2 hold. Given E >> O, the E-LabelDP RR mechanism which maximizes M(B poo. P11) is given by the privacy design matrix
P
It is crucial to point out that the traditional optimal LabelDP mechanisms, such as those discussed in (Holohan al., 2017; Wang al., 2017), primarily focus optimizing privacy based solely on the response variable Y. These approaches typically aim to minimize the variance of the estimator related to Y, without incorporating the additional information provided by covariates. In contrast, while our results are similar to Theorem from (Holohan et al., 2017)
4. Theoretical results
In this section, we start by establishing the theoretical guar- antees for two LabelDP mechanisms, followed by the con-
5

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
when establishing an optimal mechanism for s-LabelDP, our method starts by optimizing the RR binary response model itself, specifically by maximizing the trace of the Fisher Information Matrix. This optimization inherently incorpo- rates covariates (X), which are essential for enhancing the statistical efficiency and accuracy of the estimates in the binary response model, which is a significant advantage that traditional LabelDP methods do not provide.
In Theorem 4.7, the optimal (e,d)-LabeIDD privacy de- sign matrix is determined through three steps in a real- world application. First, we plug in the sets of values (Poo, P11) = (1, (6, 1), and (e=+1' e*41) into formula (4). These inputs allow us to estimate the corresponding values of the parameter B. for each case. Once 3 is esti- mated, we then calculate the value of .M(3; Poo: P11) for each of these sets of probability pairs. The final step in- volves comparing these calculated values of (M3 poo. P11) to determine which set of poo and P11 provides the optimal outcome.
4.2. Optimal mechanism for (s, O-LLbbDD
The (e, D)-LabeIDP is a relaxation of E-LabelDP when 6 > 0. In this section, we utilize the following parameterivation to explore the optimal (c, 8)-LabeIDP RR mechanism. We first parametrize pı1 = ef (1 - poo) + 5 in R' as poo = rt(
It is important to highlight that the above procedure does not require direct access to private data for selecting matri- ces. Instead, it relies on theoretical estimates and compar- isons based on predetermined probability pairs (Poo. P11) within the LabelDP framework, thereby preserving privacy throughout the selection process. Theorem 4.7 constructs the optimal estimator of the true B. while safeguarding (c, d)LLabelDD guarantee in binary response models in the same spirit as Theorem 4.5. Consequently, our mechanisms enable investigators to release sensitive response label data with higher statistical accuracy.
and p11 = (t), where
(t) = (1 +e 50) (1 1) +)+ + et + =0)
e¹ +
ex +11 and ss(t)= et +
for E (0, 1]. By the symmetry of poo and pl based on the form of Poo e (1 Pm1) + 8, we can then parameterize the line Poo = e
P11) + 5 in R' as poo ss(t) and P11 = rt()). We start with the following lemma:
Lemma 4.6. Suppose Assumptions 4. 1-4.2 hold. For any 5 > O and constants S u, with 1, 10, follows that
4.3. Inference with privacy
Private statistical inference is important as well as estima- tion. In this section, we provide the construction of the privacy-preserving confidence interval (CI) for each coef- ficient 8., under s- and (e, LabbelD mechanisms. The following additional assumptions are imposed to facilitate the technical proof of asymptotic normality for MLE.
arg max ((3:)a()))ss)))))) C uJ. rE(1.0)
Lemma 4.6 presents that the optimal solution of (poo, Pt)) exists under (s, d-LabelDD constraint, which enables to establish the corresponding optimal LabelDP RR mecha- nism in Theorem 4.7.
Assumption 4.8. The true parameter B. must be identifi- able. The log likelihood function (3.)) is convex and can be differentiated twice continuously over B_ E O. Assumption 4.9. The information matrix I(B.) positive definite 3_..
Theorem 4.7 ((E,8) LaLbbeDDD Suppose Assumptions 4.1- 4.2 hold. Given > O and 5 E (0, the (e, 6-LabelDPP RR mechanisms which maximizes M(B MMo scribed as follows:
poo, P11) are de-
Assumptions 4.8 and 4.9 are standard in asymptotic infer- ence. Assumption 4.8 (on convexity and smoothness of the log-likelihood) holds in most binary regression settings, and Assumption 4.9 (positive definiteness of the Fisher In- formation Matrix) ensures model identifiability, a common condition in GLMs. While asymptotic results assume large samples, these assumptions often hold well in practice, even with moderate sample sizes-as supported by our real data analysis.
(i) If M(B c'D
2 M(3;1,8) and
M(3; 2o + N
6(3:6.1).
then the pri-
1-
vacy design matrix is Po.5=
1-6
e1+1
(i) If M();1,5) > M(B; 1,8) 2 1)B then the privacy de-
M(B;
tt1 and
Theorem 4.10 (Asymptotic Normality). Suppose Assump- tions 4.2 and 4.8-4.9 hold. As n oc, /3 is asymptotically normally distributed as
sign matrix Pe.s = 1 1-8 11-88 8 ))
))
(iii) If 1( >
M(B e1 t and
Vnl B (0.I(3))
1( 1(B:1,5), 5), then the privacy de-
8
1 1-
sign matrix is Pa.s =
0
1
The unbiasedness and asymptotic normality in Theorem
6

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
4.10 are crucial for constructing confidence intervals on each cared coefficient 9.5,1 - j ك d in Model (3) under the LabelDP constraints.
response model with this privatized response. The confi- dence intervals for RRbR are built by fitting the same binary response model as our method, but using responses priva- tized by the traditional RR mechanism. Standard asymptotic techniques are then applied to construct the intervals. How- ever, this ignores the extra bias and variability introduced by randomization and fails to use covariate information to correct for these effects.
Beyond point estimation, our framework enables formal statistical inference with privacy guarantees. Specifically, Corollary 4. establishes the asymptotic valid confidence intervals of each coefficient 3.j under the E- and (e, () LabeIDP mechanisms, containing Baj with high probability (e.g., 0.95) sample size + cc.
ORRbR. Optimal LabelDP RR binary response models (the proposed methods).
Corollary 4.11. Suppose the Assumptions in Theorem 4. 10 hold. Denote by a the target significant level. For 1 ij <
We set d = 4. sample size n = 10 and fix B. (1, 0.25, 0, 0,0.5) Two covariate structures are considered. Scenario I (Independence structure). The covariate X = (r1, 13, x4) i.i.d. generated with the intercept æ1 =
d, the DP (1 - -i--oddeeeecee interval for Bag is
(B = 83 11 -
1, ((, 1), I3 ~ [(0,, 1.52), and 24 ~ N (0, 0.5²).
Scenario II (Dependence structure). The covariate X T2. x.) is i..dd generated with the intercept 21 F3, 23, ~ (0, Σ) with an autoregressive de-
where zi-a is the (1 a)-the quantile of the standard normal distribution. Moreover, as n - oc, the nominal coverage probability of the confidence interval ())))
1, and (3,, pendence structure (E),² 0.5 2 j. d.
In each scenario, two link functions are considered, includ- ing Logistic regression G(B x,) = exp(,3, x)) + exp (3. X.)) and Probit regression G(3 x,) = $(3 X;). We set the nominal significant level a 0.05. We study both the s-DP (6 0) and (c, (P (8 10 10-5). All simulation results are based on B = 500 inddene-
lim P -3 = =1-a. a. 11
Corollary 4.11 establishes that our privatized confidence interval achieves asymptotically valid coverage while main- taining privacy protection. Although Theorem 4.10 and Corollary 4. 1.11 are rooted in established likelihood theory, their application in our context is vital to validate our simu- lation results, particularly in terms of coverage probability. In particular, while our results are similar to Theorem 2 of (Holohan et al., 2017) 7) in establishing an optimal mechanism for e-LabelDDP, our approach extends beyond by enabling private statistical inference for RR binary response mod- els using both E- and (E, La mechanisms. This represents a significant advattage over traditional optimal LabelDP methods (Holohan et al., 2017; Wang et al., 2017), which typically do not support such direct and effective approach to private statistical inference.
dent replications, and ( is the estimator obtained from the b-th replication. The performance of the proposed methods is evaluated along with the above benchmarks through comparisons of estimation (mean squared error, MSE(B.) = B Σen |/3(0 131 privacy (c, d), and inference (coverage probability of 1-a confidence interval).
5.2. Results
Figures 2-5 present the empirical MSE of 3 and the cover- age probability of an approximate 95% confidence interval for B3 for Scenarios I-ll against different privacy parame- ters under E- and (s, dL-abeIDD constraints, respectively. Specifically, the top subfigures in each of these figures cor- respond to 6 O, while the bottom ones correspond to 8 = 10-5, In Figures 2 and 4, the y-axis shows the log- arithm of Mean Squared Error (MSE), and in Figures 3 and 5, it shows the empirical average coverage probability (CP) for an approximate 95% confidence interval of the true parameter B., with the z-axis representing the privacy pa- rameter in all cases. The chosen privacy budget values e are (0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.5. 0.7, 1}. which are not evenly spaced set according these values were deliberately selected in an irregular and somewhat random manner to extensively test the robustness of our method across a wide range of privacy constraints.
5. Simulation studies
In this section, we conduct extensive numerical experiments to evaluate the finite sample performance of the proposed mechanism with several competing methods when fitting RR binary response.
5.1. Settings
log scale. Instead,
We consider several benchmarks for comparison with our proposed methods.
NP. Non-DP RR binary response (Fox et al., 2018), which estimates B., without any privacy consideration.
All figures demonstrate that NP possesses the smallest MSE and the most accurate coverage probability as expected, since no privacy considered. Remarkably, our ORRbR
RRbR. Apply the traditional DP RR mechanism (Holohan et al., 2017) solely to the response variable, then fit binary
7

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
shows a higher coverage probability when is exceedingly small due the conservative nature of privacy-preserving estimators under stringent privacy constraints. Hence, under tight privacy constraints, the ORRbR method introduces significant noise to protect privacy. This could lead to overly conservative estimates, resulting in an increase coverage probability compared to the NP method. As the privacy budget increases, all comparing methods sacrifice accuracy for better privacy protection, consistent with theoretical analysis. Moreover, our proposed ORRbR outperforms the RRbR with a smaller MSE and a higher coverage probability due to the presence of covariate information in ORRbR.
Logistic
Probit
0.5
0.3
In addition, the performance of our proposed approach has improved as the privacy parameter & has increased for both values of 8. For example, in the top left plot of Figure 2, the log MSE is approximately 0 for it is approximately -15 for e = 0.7 and 5 O. Additionally, analogous results are depicted in the right panels of Figures 4 for Probit regression under Scenario II. Improvements were observed as increased for both E- and (s, LL-bbeIDP constraints. Furthermore, when comparing the panels in can observe
0.3
0.05 and 8 0, while
0.5
0.1
0.7
0.05
0.1
0.7
ORRER NP
RRbR
Figure 3. Empirical average coverage probability (CP) for three methods against different under Scenario I.
Figures and 4, as well as Figures 3 and significant differences between Scenarios I and II, particu- larly in terms of mean squared error (MSE) and coverage probability. Scenario II exhibits a larger MSE compared to Scenario I. Additionally, the coverage probability in Sce- nario II appears to be less accurate than in Scenario I. These disparities can be attributed to the underlying structural differences between the two scenarios.
6. Real-data application
In this section, we demonstrate the practical utility of our proposed method by applying it cused on measuring plagiarism among students, as detailed in (Jann et al., 2012). This dataset comprises responses from 474 German and Swiss students who were surveyed using either direct questioning or the crosswise (CW) technique to estimate the prevalence of plagiarism. For our analysis, the dataset was divided into two parts: 75% of the data was used to train the model, while the remaining 25% was re- served for testing. The covariates in our model include three key variables: pp (indicating whether the question pertains to partial or severe plagiarism), RR (denoting whether the crosswise method or direct questioning was used), and age (the student's 'ss age). These variables are essential to capture the nuances of the data and were included in the model to assess their impact responses.
dataset fo-
Logistic
Probit
-20
6=10
We implemented three methods NP, RRbR, and ORRbR, as used in our simulation studies to predict the probabili- ties of different responses. The performance of these models was assessed by calculating the MSE between the predicted probabilities and the actual responses in the test set. The results presented in Figure 6 highlight the trade-off between differential privacy and statistical accuracy. Compared to RRRR, the lower MSE values achieved by the proposed ORRbR method at various e levels demonstrate its superior performance. In particular, as E increases, the MSE for OR- RbR decreases even further, highlighting its effectiveness in balancing privacy and accuracy.
0.25
8.05
0.1
Figure Empirical log MSE three methods against different under Scenario I.
8

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
Logistic
Probit
Logistic
Probit
-20
0.3
6
0.3
A-8.6-
0.25
0.1
0.05
0.1
0.55
0.1
0.05
0.7
ORRER ARM
ORRER NP
RRbR
Figure Empirical log MSE three methods against different under Scenario II.
Figure 5. Empirical average coverage probability (CP) for three methods against different under Scenario IL.
7. Conclusion
32
This paper proposed two RR mechanisms with an opti- mal design for binary response models under é- and (s, 0)- LabelDP constraints. Theoretical and numerical results reveal that the proposed optimal LabelDP RR mechanism successfully outperforms the traditional optimal DP RR mechanism (Holohan et al., 2017) in terms of estimation, privacy, and statistical inference when applied to fit a binary response model with privatized responses. Starting with parametric models like GLMs is methodologically natural, they offer well-defined Fisher Information and their ana- lytical tractability allows for rigorous theoretical develop- ment. This mirrors the trajectory of many pioneering works in DP, which initially focused on basic statistical tasks such as mean and median estimation, before extending to more structured models like linear and logistic regression (e.g., Kulkarni al. (2021); Narayanan al. (2022); Asi al. (2022); Alparslan al. (2023); Kulesza et al. (2023); Brown al. (2024)). This balance between privacy protection and statistical efficiency marks a significant advancement in private data analysis. Our approach allows researchers to confidently release sensitive binary response data while im- proving statistical efficiency with strong privacy guarantees.
0.1
6.2
0.5
03
ORRER
HP
ARDR
Figure 6. Empirical MSE three methods against different e under the real-data application.
sign matrix in Definition 2.2 of our paper becomes k x stochastic matrix, and the optimality criterion would involve maximizing the Fisher Information's 's trace across multiple response probabilities under LabelDP constraints. This is an exciting direction that we are actively exploring. and pre- liminary theoretical analysis indicates promising scalability. We plan to provide more detailed theoretical developments and numerical evaluations in future work. Second, extend- ing our method to other concepts of privacy, such Rényi DP (Mironov, 2017) or Gaussian DP (Dong et al., 2022), remains challenging due to the unclear design of the feasible region in our equation (3). Lastly, investigating other opti- mal criteria, such as minimizing label classification errors, deserves further exploration to enhance the robustness and applicability of approach.
We conclude this paper by acknowledging several limita- tions and suggesting directions for future research. First, our method can be naturally extended to multiclass outcomes using the k-RR mechanism. The optimization strategy of maximizing the trace the Fisher Information Matrix re- mains valid for multiclass scenarios, as discussed Yao Wang (2019). Specifically, for k-class responses, the de-

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
Acknowledgements
differentially private mechanisms. arXiv:2210.133660 2022.
arXiv preprint
We would like to thank the anonymous reviewers and area chairs for great feedback on the paper. Xiaodong Yan was supported by the National Key R&D Program of China (No. 202 2023YFA1008701) 1008701) and the National Natural Science Foun- dation of China (No. 12371292) equally. Bei Jiang and Linglong Kong were partially supported by grants from the Canada CIFAR AI Chairs program, the Alberta Machine Intelligence Institute (AMII), and Natural Sciences and En- gineering Council of Canada (NSERC), and Linglong Kong was also partially supported by grants from the Canada Research Chair program from NSERC.
Awan, J. and Slavkovic, A. Differentially private infer- ence for binomial data. arXiv preprint arXiv:1904.0044 00449, 2019.
Barnes, L. P., Chen, W.-., and Özgür, A. Fisher infor- mation under local differential privacy. IEEE Journal on Selected Areas in Information Theory, 133):65--559 2020.
Biswas, S., Jung, K., and Palamidessi, C. Tight differ- ential privacy guarantees for the shuffle model with k-randomized response. In International Symposium on Foundations and Practice of Security, pp. 450-458. Springer, 2023.
Impact statement
The broader impact of this work extends across multiple domains where privacy preeeerire statistical inference is essential, including healthcare, social sciences, and finance. By providing optimal privacy protection while maintaining statistical accuracy, method enables more reliable in- ference in sensitive applications such medical diagnosis, online surveys, and secure machine learning. In scenarios where protecting individual responses is critical, our ap- proach ensures that privacy guarantees do not come at the cost of statistical efficiency, allowing researchers and prac- titioners draw meaningful conclusions from privatized data.
Blair, G., Imai, K., and Zhou, Y.-Y. Design and analysis of the randomized response technique. Journal of the American Statistical Association, 10:3411:100444119, 2015.
Boyd, S. P. and Vandenberghe, L. Convex optimization. Cambridge university press, 2004.
Brown, G., Dvijotham, K., Evans, G., Liu, D., Smith, A., and Thakurta, A. Private gradient descent for lincar re- gression: Tighter error bounds and instance-specific un- certainty estimation. arXiv preprint arXiv:2402.13531, 2024.
Additionally, our framework facilitates the construction of private confidence intervals, ensuring valid statistical in- ference under strict privacy constraints. This capability is particularly valuable for organizations and policymakers who require both privacy preservation and rigorous statis- tical validity in their analyses. Extensive empirical evalu- ations demonstrate that method outperforms existing approaches in estimation precision, coverage probability, and computational efficiency. By striking a balance between privacy and statistical accuracy, our approach lays strong foundation for advancing differentially private statistical modelling in real-world applications.
Busa-Fekete, R. I., Syed, U., Vassilvitskii, S., et al. Popu- lation level privacy leakage in binary classification wtih label noise. In NeurlPS 2021 Workshop Privacy Ma- chine Learning, 2021.
Busa-Fekett R. I., Medina, A. M., Syed, U., and Vassil- vitskii, S. Label differential privacy and private training data release. In International Conference on Machine Learning, pp. 3233-3253555 PMLR, 2023.
Chaudhuri, A. and Mukerjee, R. Randomized response: Theory and techniques. Routledge, 2020.
References
Chaudhuri, K. and Hsu, D. Sample complexity bounds for differennially private learning. In Proceedings of the 24th Annual Conference Learning Theory, pp. 155-186. JMLR Workshop and Conference Proceedings, 2011.
Alparslan, B., Yildırm, S., and Birbil, I. Differentially private distributed bayesian linear regression with memc. In International Conference on Machine Learning, pp- PMLL, 627-641. PMLR, 2023.
Coutts, E, and Jann, B. Sensitive questions online sur- veys: Experimental results for the randomized response technique (rrt) and the unmatched count technique (uct). Sociological Methods & Research, 40(1):169-1199 2011.
Asi, H., Feldman, V., and Talwar, K. Optimal algorithms for mean estimation under local differential privacy. In International Conference Machine Learning, pp. 1046- 1056. PMLR, 2022.
Dette, H., Melas, V., and Shpilev, P. T-optimal designs for discrimination between two polynomial models. The Annals of Statistics, 40(1):188-201 2012.
Asoodeh, S. and Zhang, H. Contraction of locally
10

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
Ding, J., Zhang, X., Li, X., Wang, J., Yu, R., and Pan, M. Differentially private and fair classification via calibrated functional mechanism. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 34, pp. 622-629, 2020.
Transactions on Information Forensics and Security, 12 (11):2726-2735, 2017.
Hout, A. v. d., Böckenholt, U., and Van Der Heijden, P. G. Estimating the prevalence of sensitive behaviour and cheating with a dual design for direct questioning and randomized response. Journal of the Royal Statistical Society Series C: Applied Statistics, 59(4):233-736, 2010.
Dong, J., Roth, A., and Su, W. J. Gaussian differential privacy. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(1):3-33 37, 2022.
Hsieh, S., Lee, S., and Shen, P. Logistic regression analysis of randomized response data with missing covariates. Journal of Statistical Planning and Inference, 140(4): 927-940, 2010.
Duan, R., Cao, M., Wu, Y., Huang. J., Denny, C., Xu, H., and Chen, Y. An empirical study for impacts of measurement errors ehr based association studies. In AMIA Annual Symposium Proceedings, volume 2016, pp. 1764. American Medical Informatics Association, 2016.
Jann, B., Jerke, J., and Krumpal, I. Asking sensitive ques- tions using the crosswise model: experimental survey measuring plagiarism. Public opinion quarterly, 76(1): 32-49, 2012.
Duarte, B. P., Atkinson, A. C., Granjo, J. F., and Oliveira, N. M. Optimal design of experiments for implicit models. Journal of the American Statistical Association, 1 (17(:))): 1424-142 1437, 2022.
Kairouz, P., Oh, S., and Viswanath, P. Extremal mechanisms for local differential privacy. The Journal of Machine Learning Research, 17(1):492-542, 2016.
Dwork, C., McSherry, F., Nissim, K., and Smith, A. Cal- ibrating noise to sensitivity in private data analysis. In Theory of Cryptogruphy: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3. pp. 265-284, Springer, 2006.
Kirchner, A. Validating sensitive questions: A comparison survey and register data. Journal of Official Statistics, 31(1):31-59, 2015.
Kulesza, A., Suresh, A. T., and Wang, Y. Mean estimation in the add-remove model of differential privacy. arXiv 2023.
Dwork, C., Roth, A., et al. The algorithmic foundations of differential privacy. Foundations and Trends@ in Theo- retical Computer Science, 9(3 (13-4):211-410) 2014.
preprint arXiv:2312.0666
Kulkarni, T., Jälkö, J., Koskela, A., Kaski, S., and Honkela. A. Differentially private bayesian inference for gener- alized linear models. In International Conference Machine Learning, pp. 5838-5849 PMLR, 2021.
Fox, J.PP., Veen, D., and Klotzke, K. Generalized linear mixed models for randomized responses. Methodology, 2018.
Garcelon, E., Perchet, V., Bike-Burke, C., and Pirotta, M. Local differential privacy for regret minimization in re- inforcement learning. Advances in Neural Information Processing Systems, 34:10561-10573, 2021.
McCullagh, P. Generalized linear models. Routledge, 2019.
Mironov, I. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263-275. IEEE, 2017.
Ghazi, B., Kumar, R., Manurangsi, P., and Pagh, R. Private counting from anonymous messages: Near-optimal accu- racy with vanishing communication overhead. In Interna- tional Conference on Machine Learning, pp. 3505-3514. PMLR, 2020.
Narayanan, S., Mirrokni, V., and Esfandiari, H. Tight and robust private mean estimation with few users. In Inter- national Conference on Machine Learning, pp. 16383- 16412. PMLR, 2022.
Nayak, T. K. and Adeshiyan, S. A. A unified framework for analysis and comparison of randomized response surveys of binary characteristics. Journal of Statistical Planning and Inference, 139(8): 2009.
Ghazi, B., Golowich, N., Kumar, R., Manurangsi, P., and Zhang, C. Deep learning with label differential privacy. Advances in Neural Information Processing Systems, 34: 27131-27145, 2021.
Oberski, D. L. and Kreuter, F. Differential privacy and
Ghazi, B., Kamath, P., Kumar, R., Leeman, E.,, Manurangsi, P., Varadarajan, A. V., and Zhang, C. Regression with la- bel differential privacy. arXiv preprint 27Xiv:2212.06074, 2022.
social science: An urgent puzzle. Harvard Data Science Review, 2(1):1-21, 2020.
Pastore, A. and Gastpar, M. Locally differentially-privale randomized response for discrete distribution learning. The Journal of Machine Learning Research, 22(1):5872 5927, 2021.
Holohan, N., Leith, J., and Mason, O. Optimal differen- tially private mechanisms for randomised response. IEEE

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
Pukelsheim, F. Optimal design of experiments. STAM, 2006.
Tutz, G. Regression for categorical data, volume 34, Cam-
bridge University Press, 2011.
Van den Hout, A., van der Heijden, P. G., and Gilchrist,
R. The logistic regression model with response variables subject to randomized response. Computational Statistics & Data Analysis, 51(12):6060 600000666666600 2007.
Waite, T. W. and Woods, D. C. Minimax efficient random experimental design strategies with application model- robust design for prediction. Journal of the American Statistical Association, 11 (7553)):1552-1 1465, 2022.
Wang, T., Blocki, J., Li, N., and Jha, S. Locally differen- tially private protocols for frequency estimation. In 26th USENIX Security Symposium (USENIX Security 17). PP. 729-745, 2017.
Wang, Y. Preserving differential privacy in complex data analysis. PhD thesis, The University of North Carolina at Charlotte, 2015.
Wang, Y., Wu, X., and Hu, D. Using randomized response for differential privacy preserving data collection. 2016.
Warner, S. L. Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60(309):63-69, 1965.
Waudby-Smith, I., Wu, S., and Ramdas, A. Nonparametric extensions of randomized response for private confidence sets. In International Conference on Machine Learning, pp. -77799967779 PMLR, 2023.
Xu, S., Wang, C., Sun, W. W., and Cheng, G. Binary clas- sification under local label differential privacy using ran- domized response mechanisms. Transactions on Machine Learning Research, 2023.
Yang, M., Biedermann, S., and Tang, E. On optimal designs for nonlinear models: a general and efficient algorithm. Journal of the American Statistical Association, 108(504); 1411-1420, 2013.
Yao, Y. and Wang, H. Optimal subsampling for softmax regression. Statistical Papers, 60:585-599, 2019.
12

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
Appendix
Proof of Lemma 4.3. Firstly, after some calculation on 2M Ope0
and 8 MMioopaa.PPal we have
8po0
am (00 poo. P11)
(Poo P11 1) [2p, (1 p.) + (poo + P11 1)(1 - 2p,) G (n.))
Opoo
n
[pi (1 p:)]
j=1
(G (mR)
2
X2.
On
8 (0: P1)
d
Pi (1 -p)) + (Poo + pn - 1)(1 - - G
Opho
+ (poo P11 1)3px (1 pa) (1- [ps (1 - (1-p:
aG (m))
X2
Onp
Note that p(pooopo-) Op
2 0, it is easy to verify that (.003:p0.P11). increases as Poo increases within the region R. or
Opeo
R'. We know that poo is greater than 1 P11 under Assumption 4.2. Since (M O, we can deduce that
Opes
0.M(9:poo.pu) when poo P11 It follows that M (B: Poo: Pı1) increases as poo increases.
Opoo
Secondly, after performing some calculations, have
0M (B: Poo. Pm1)
(Poo pı 1) [2p pı) (poo + P11 1)(1)))111)))) 2p,)G (n:)
0p11
[pi (1 p:)]
1=1
aG (m))
Om,
ijz
0² (B:
2
d
Pi (1 -p:) -(po + P11 1)(1 - (p (m)) )²
θpi1
1
{ (OG(m) }² x
8G (ni)
+ (Poo P11 - 11)3ps (1 ps) G2) G
}
(1- p3)
dm:
Similar to the reasoning above, we verify that M Poo, P11) is also an increasing function as P11 increases since B.M(B:po0PP11 when P11 > 1 Poo- Thus, the maximum of MM3; Poo: P11) for (Poo, Pı1) attained the boundaries 0pm1
of the feasible regions R or R'.
Proof of Theorem 4.4. Lemma 4.3 shows that the optimal mechanism occurs when the parameters (poo. P11) are on the boundaries of R. Accordingly, least one of the inequalities Pıi S e (1 Poo) and Poo I e -P1)) must be tight (i.e., either P11- = e (1 Poo) or Poo e (1 Pı1)), which can be listed as the following cases.
Case 1. Suppose P11 I e (1 Poo) holds. implies that P11 e (1 Poo) with the constraint that Pı1 is non-negative and Poo e (1 P1)). For & (0, 1J, we parameterize the line Pı1 = e (1 poo) as Poo r(t) and P1 s(t) with the
condition Poo + P11 >
r(t) = (1 (1-1) t) + 1t
t 1=
(A.1)
e
s(1) = 1 c
See the proof of Theorem 2 in (Holohan et al., 2017) for details the parameterization. After substituting Poo = r(1) and
13

---

Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
P11 s()) into Poo: have
t(e - 1)² [1+1 ( (m))
M(3:r(0),8(1)
(Ee
==
8G (m))
dm
OM (M)
0t
8G (m:)
Xtj.
Oni
Given e" > 1, we have 2M(3:)()) > O, which implies that the maximum value of maximizing M (B: s(t)) lies øt in the interval (0, and is equal to i... Poo Pí = 1/(e" 1). Therefore, we conclude that the privacy design matrix for the e-LabelDP RR mechanism while maximizing M (3: poo. Poo.P11) is
(
Pho
1 poo
)
Ps
1 -Pi
Case Let Poo S ef (1 (1-111p)) P11) tight that means poo = e (1 - P11); constrained by Poo 2 0 and pn1 S e (1 Poo). By symmetry of the equations Poo S ef (I)) and P11 S e (1 - poo). we define poo s(f) and Pı1 r(t). By the similar proof of Case 1, we obtain the same privacy design matrix shown in Case
Proof of Lemma 4.5. As the Proof of Theorem 4.4, parameterize the line P11 ef (1 Poo) 5 of the region R' as Poo = (t) and P11 ss(t):
(t) (1+e (1 1) 6 + 5 et + t
(A.2)
sa(t) e + d
(A.3)
for 1 E (0, 1]. By symmetry, we can parameterize the line Poo = = e1 P11) + 8 of the region R' as Poo = =(()) and
Pn1 = rt()). Hence, in the following, we only need to prove the result for the case that poo rt(() and pı = st()). By plugging (A.2) and (A.3) into M (B: Poo, P11), we have
M ((Brr((),sa0))) =
+ (1 (c-
OG (m) 2
Om: x/-
Denote
A(t) = (e - (1) +5+
Bi(t) = -1+)))))(((000(((000--------0111
14

---

+6 Differentially Private Analysis for Binary Response Models: Optimality, Estimation, and Inference
AP(() {OG(m)) ? 1-1 B,(0).100) Om
}
Hence, we rewrite M (t)
X2j. And thus we have
O. (s(t), ss(())
2A(1) -)) (1(1)(((1)
Ht.
(t) [e +1 -1,((), (1 + [1+ (e G)
(1(()((()
aG (ma)
On: X2.
0 M (B:rrt((t)..... st()))
d (1)+)) (1)))(((
0t2
(er - L)) - A(t) [et + - 2B.(t)]1++ (e= - G) (m:))
(1)))
()) (1+ (e - 1))
(t)
0G (m))
x?.
Omp
Note that 8 >>0 0 and e > 1, we have (M)(
2 O. Then,
812
arg mag.aaMMu (B; ss(()) S (1,u),
(E(1.)
which means that the maximal M (3) st)) on t [1. u| will occur at one of its extreme points or u.
Proof of Theorem 4.6. By Lemma 4.5, we can see that extreme point of Poo = e (1 Pm1) + 5 is located at Also, another extreme point can be found at to 8) 5(e*+1) that,
t that Poo Pi1 (1) (1)
5.6
e++1-
es
e.6 Poo (to) = 1 and pil st(o 5. Therefore, obtain
arg max. (tt, S {to, 1}.
1E(4,.1]
Next, we aim to determine the sign of M (B:rrsttu) st(t)) M (B:rs(1), If M (B:rs(to) M sx))) > 0, then the optimal privacy design matrix is
Pe.s = ( 1-8 (1-8 0) )
else the optimal privacy design matrix
Pe.s
Similarly, if we set pM (1 Poo) ro(t) M (1) 0, then the optimal privacy design matrix is
8 to be tight, and parameterize pn rs(() and poo so(1). Then if
P.. (o 8 1 )
else the optimal privacy design matrix is
Po. =
e++
1-5
and accordingly, we claim the assertion.
15


Reviews:
[{"review_id": "GtedVaUgsB", "reviewer": "Reviewer_7wrY", "summary": "This paper addresses statistical estimation and inference in binary response models while preserving the privacy of the labels via LabelDP. Covariate features $X$ are public, but binary response labels $Y$ are sensitive. The authors focus on using randomized response (RR) mechanisms – a classical privacy technique where each true binary label may be flipped with some probability​ to privatize the labels. They formulate it as an optimization problem: choose the RR flipping probabilities $(p_{00}, p_{11})$ (the probabilities of outputting the truthful label for $Y=0$ and $Y=1$, respectively) to maximize the trace of the Fisher Information Matrix of the model. This approach incorporates the influence of covariates into the privacy mechanism design, unlike prior methods that treated label randomization independently of $X$​", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question under Experimental Designs Or Analyses**\n\nWe acknowledge the concern of the reviewer about the large simulation sample size ($n=10^5$) for asymptotic approximations. This choice was intentional to validate our theoretical results (Theorem 4.7), where the guarantees of consistency and normality are held as $n \\rightarrow \\infty$. However, we emphasize that our method is equally applicable to smaller samples, as demonstrated in the real-data analysis (Section 6) with $n=474$ - a realistic sample size for sensitive surveys. Here, ORRbR achieved a 30% lower MSE than $\\operatorname{RRbR}$ at $\\varepsilon=0.1$ (Figure 7), proving its practical utility beyond asymptotic regimes.\n\nTo further address this point, we will add simulations for $n \\in \\\\{500,2000,5000\\\\}$ in revision, showing that ORRbR maintains superior coverage (>92% versus RRbR's <85% at $n=500, \\varepsilon=0.5$). Then we will clarify in Section 5 that the Fisher information optimization remains valid for small $n$, although variance estimates may require finite-sample adjustments (e.g., sandwich estimators).\n\n\n**Weakness 1 and Question 1.**\n\nWe appreciate the insightful suggestion of the reviewer on multiclass extensions. Indeed, our method can be naturally extended to multiclass outcomes using the $k$-RR mechanism. The optimization strategy of maximizing the trace of the Fisher Information Matrix remains valid for multiclass scenarios, as discussed in Yao and Wang (2019), Optimal subsampling for softmax regression. Specifically, for $k$-class responses, the design matrix in Definition 2.2 of our paper becomes a $k \\times k$ stochastic matrix, and the optimality criterion would involve maximizing the Fisher Information's trace across multiple response probabilities under LabelDP constraints. This is an exciting direction that we are actively exploring, and preliminary theoretical analysis indicates promising scalability. We plan to provide more detailed theoretical developments and numerical evaluations in future work.\n\nWe acknowledge the reviewer's observation regarding the large sample size of our simulation study ($n=10^5$). Our initial choice aimed to clearly demonstrate the theoretical properties under asymptotic conditions. However, we want to emphasize that our real data analysis employs a relatively small sample size of 474 German and Swiss students. This practical application highlights the robustness and applicability of our method even in small-sample contexts.\n\n**Weakness 2 and Question 2.**\n\nWe thank the reviewer for highlighting the potential for other optimality criteria. It is important to note that while both $T$-optimality and $D$-optimality aim at maximizing information, $D$-optimality involves maximizing the determinant of the Fisher Information Matrix, which is computationally challenging and more complex due to the involvement of determinant calculations, especially for high-dimensional parameter spaces. However, $T$-optimality is computationally simpler, making it significantly easier to implement in practical scenarios without sacrificing the quality of inference. Moreover, in many cases, the two criteria are equivalent or closely related, and maximizing the trace often provides a satisfactory approximation to maximizing the determinant.\n\nIn detail, our $T$-optimality criterion (maximizing the trace) is equivalent in asymptotic efficiency but far more practical:\n\n1. Computational tractability: The trace decomposes into a sum of variances (diagonal elements), reducing the problem to scalar optimization (Lemma 4.4). This allows closed-form solutions for $\\left(p_{00}, p_{11}\\right)$ on the boundary of $\\mathcal{R}$ (Theorems 4.5 and 4.7).\n\n2. Interpretability: The trace directly corresponds to minimizing the average variance of $\\widehat{\\boldsymbol{\\beta}}$, which aligns with our goal of precise estimation.\n\n3. Equivalence in large samples but different complexity: Both criteria yield consistent estimators, but $T$-optimality achieves this with $O(d)$ complexity versus $O\\left(d^3\\right)$ for $D$-optimality due to determinant calculations, where $d$ is the number of covariates.", "claims_and_evidence": "The paper’s main claims—that an optimal LabelDP randomized response mechanism exists for binary response models and that the proposed approach significantly outperforms baseline methods—are supported by proofs and simulation studies.", "methods_and_evaluation_criteria": "Yes, see above.", "theoretical_claims": "I did not check the correctness of the proofs in the appendix.", "experimental_designs_or_analyses": "The paper’s main experiment setup—comparing different randomized response strategies under various $\\varepsilon$-privacy budgets is sound. The sample size ($n=10^5$) is large, which supports asymptotic approximations, and they carefully document metrics (MSE, coverage probability) that align with the paper’s theoretical claims. One possible limitation is that only scenarios with a large $n$ and a single real dataset are shown, so small-sample performance is not really presented.", "relation_to_broader_scientific_literature": "The paper positions their contribution to the literature,  leveraging insights from local differential privacy to protect binary labels in the LabelDP setting.", "essential_references_not_discussed": "Not appliable.", "other_strengths_and_weaknesses": "Strengths:\n* The paper’s approach of integrating experimental design principles (maximizing Fisher Information) into differential privacy is an original angle.\n* The authors clearly derive the mechanism and proofs and show its impact in both simulations and a real-world plagiarism dataset.\n\nWeaknesses\n* The method is tightly scoped to binary label privacy with large-sample asymptotics; it does not address broader settings (e.g., multi-class outcomes, small-sample scenarios).\n* While they justify maximizing the trace of the Fisher Information, there is little exploration of other design criteria which might have yielded different insights.", "other_comments_or_suggestions": "No", "questions_for_authors": "* Could this method be extended to multi-class outcomes? (see the k-RR mechanism)\n* Did you consider other design criteria, such as $D$-optimality?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gU0KCeQ3sV", "reviewer": "Reviewer_v8Zr", "summary": "The authors propose an estimation method for a binary response model under LabelDP that is optimal in that it maximizes the trace of the Fisher information matrix. They leverage results regarding asymptotic normality of the MLE to derive confidence intervals for their estimator.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Experimental Designs or Analyses 1.**\n\nWe appreciate the reviewer pointing out this observation. The abrupt jumps around small $\\varepsilon$ values (particularly near $\\varepsilon=0.1$) observed in Figures 2 and 4 arise primarily from the specific data generation mechanism of our simulation study for the parameter $\\boldsymbol{\\beta}$. To clarify, after revising and slightly adjusting our data-generating procedure (especially for $\\boldsymbol{\\beta}$), we observed that these jumps significantly diminish and the curves become noticeably smoother. This smoother property aligns well with the results of our real data analysis (Figure 6) in our paper, where the curves demonstrate a much more stable and smooth pattern. This smoothness of the real data further supports that the initial observed fluctuations were due to the original simulation setup rather than an inherent instability of the proposed methodology.\n\nFollowing your suggestion, we will increase the simulation runs and provide smoother simulation results to better illustrate the effectiveness and robustness of our approach.\n\n**Experimental Designs or Analyses 2.**\n\nWe thank the reviewer for noting the ambiguity with respect to the scaling of the $x$-axis in Figures $2-6$. In fact, the chosen privacy budget values $(\\varepsilon)$-specifically $\\\\{0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.7, 1\\\\}$-were not evenly spaced or set according to a log-scale. Instead, these values were deliberately selected in an irregular and somewhat random manner to extensively test the robustness of our method across a wide range of privacy constraints.\n\nThis deliberate randomness and irregularity in the choice of $\\varepsilon$ values ensure our results are not sensitive to systematic or uniformly spaced intervals. Despite such randomness, our method consistently demonstrates strong performance, validating its robustness across varied privacy settings\n\n**Essential References Not Discussed**\n\nThe reviewer raised an insightful question about the connection between our optimal LabelDP setting with covariates and traditional DP scenarios (without covariates). Although both papers share the goal of optimal privacy-utility trade-offs for binary responses, our work makes two key advances beyond their framework.\n\n- Covariate Integration: Awan $\\\\&$ Slavković optimizes the RR mechanisms solely for the binary response $Y$ (minimizing $\\operatorname{Var}(\\widehat{\\sum{Y}})$, where $\\sum{Y}$ is the sample sum, which is a complete sufficient statistic for the binomial model in their paper) without considering covariate information $X$, while our Fisher information maximization (Eq. 5) explicitly incorporates covariate effects through the model $\\mathbb {E} \\left( {Y} _ i \\mid { {X} } _ i \\right) =1-p _ {00}+\\left( p _ {00} + p _ {11} - 1 \\right) G \\left( {\\beta} _ * ^ {\\top} X _ {i} \\right)$. This criterion, known as T-optimality, explicitly leverages the covariate structure, enhancing the estimation efficiency when covariates are present.\n\n- Inferential Guarantees: Beyond point estimation, our framework enables formal statistical inference with privacy guarantees. Specifically, Corollary 4.11 establishes the asymptotic normality of each coefficient $\\beta_ { * j}$ under $\\varepsilon$- and $(\\varepsilon,\\delta)$-LabelDP mechanisms, yielding valid confidence intervals that: \n  - Achieve nominal coverage (e.g., 95% intervals contain $\\beta_{* j}$ with probability 0.95 as $n \\rightarrow \\infty$). \n  - Preserve privacy through the optimized RR mechanism (Theorems 4.4 and 4.7). \n  - Account for covariate effects via the Fisher information matrix.\n\nWe will add a discussion in Section 2 comparing these approaches, citing Awan $\\\\&$ Slavković's marginal optimality as motivation for our novel optimality under the regression framework.\n\n**Other Comments or Suggestions.**\n\nWe agree with the reviewer on the readability issue. In our revised manuscript, we will significantly improve the readability of all figures, especially Figure 1, by:\n- Increasing font sizes to match the main text consistently.\n- Clarifying annotations and axis labels.\n- Enhancing visual clarity through improved formatting and spacing.", "claims_and_evidence": "The authors support their main theoretical claims in Section 4 with proofs in the appendix.", "methods_and_evaluation_criteria": "The methods and evaluation criteria make sense for the problem.", "theoretical_claims": "I did not read through the proofs in the appendix in too much detail.", "experimental_designs_or_analyses": "I did not notice any issues with the design of the experiments in Sections 5 and 6. However, I have a few points I want to raise about the accompanying Figures 2-6:\n\n1. The trends in the figures are strange at small $\\varepsilon$, especially in Figures 2 and 4. For example, what is causing the big jump in most of the curves at the point to the right of $\\varepsilon = 0.05$? Is this just Monte Carlo error? If so, the authors might consider running more simulations to reduce the error, if computationally feasible.\n\n2. Perhaps a minor point, but as I was trying to understand the trends in the figures, I was thrown by the scaling of the x-axis. The tick marks at 0.05, 0.1, 0.7, and 1 appear evenly spaced, but this does not correspond to a log-scale. How is the x-axis scaled?", "relation_to_broader_scientific_literature": "The authors improve over prior work in the LabelDP literature for this task, with the primary comparison being to Holohan et al. (2017). Notably, the proposed method achieves the nominal coverage rate in the simulation study, whereas the prior work did not.", "essential_references_not_discussed": "The key contribution is an optimal estimator for a setting with a binary response variable, which seems reminiscent of the optimality results of Awan & Slavkovic (JPC 2020). While the Awan & Slavkovic results apply to traditional DP with binary data (and no covariates), I am interested in whether there is a connection between these two notions of optimality in what seem to be related settings.", "other_strengths_and_weaknesses": "The work is well-written and the authors did a good job motivating their method.", "other_comments_or_suggestions": "1. The figures are very hard to read in their current form, especially Figure 1. I ask the authors to please update the text size to match that of the remainder of the document.", "questions_for_authors": "I have no additional questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MzVjEqVrA4", "reviewer": "Reviewer_rwTX", "summary": "This paper proposes a new method for ensuring label differential privacy in classification tasks through the randomized response mechanism with optimality guarantees. Furthermore,  the paper proposes differentially private confidence intervals based on the former method.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Validity of Definition 3.1**, **Theoretical Claims** and **Question 1.**\n\nTo satisfy $(\\epsilon, \\delta)$-LabelDP (and similarly for $\\epsilon$-DP), the conditional probabilities $p_{00}=P(Y^*=0 \\mid Y=0)$ and $p_{11}=P(Y^*=1 \\mid Y=1)$, which lie in $(0,1)$, must meet the following privacy constraints: $$\nP[Y^*=0 \\mid Y=0] \\leq e^{\\epsilon} P[Y^*=0 \\mid Y=1]+\\delta, \\\\; \\\\;\nP[Y^*=1 \\mid Y=1] \\leq e^{\\epsilon} P[Y^*=1 \\mid Y=0]+\\delta,\n$$\nand \n$$\nP[Y^*=0 \\mid Y=1] \\leq e^{\\epsilon} P[Y^*=0 \\mid Y=0]+\\delta, \\\\; \\\\;\nP[Y^*=1 \\mid Y=0] \\leq e^{\\epsilon} P[Y^*=1 \\mid Y=1]+\\delta.\n$$\nFollowing Holohan et al. (2017) and Wang (2015), we also assume $p_{00}>0.5$ and $p_{11}>0.5$, implying $p_{00}+p_{11}>1$, to reflect the idea that the RR mechanism is more likely to return a truthful label than a flipped one. It is then easy to verify that these inequalities lead to the feasible region of $(p_{00}, p_{11})$ pairs in Definition 3.1. Since this region is explicitly constructed from existing results, we present it as a definition rather than a theorem. However, we agree it would be clearer to write $p_{00}>0.5$ and $p_{11}>0.5$ directly in Definition 3.1, and we will update the paper accordingly.\n\n**Claims and Evidence Regarding the Optimality Criterion.**\n\nOur $T$-optimality refers to selecting the best RR mechanism by maximizing the trace of the Fisher Information Matrix (FIM) within the feasible region defined in Definition 3.1. The trace summarizes the total information across all parameters and reflects the average precision of estimates. Hence, our approach ensures the most statistically efficient estimation under the LabelDP constraint. This is not merely an empirical improvement over standard RR mechanisms, but a principled optimal solution within a clearly defined feasible set. In the revision, we will formally define the $T$-optimal criterion and clarify its interpretation.\n\n**Methods and Evaluation Criteria** and **Weakness 1.**\n\nWhile our main focus is on binary response models for theoretical clarity, our method can naturally extend to multiclass and high-dimensional settings. Initial results are promising, and we plan to explore these extensions in future work.\n\n**Experimental Designs or Analyses.**\n\nWe confirm that the R code was submitted as supplementary material. During the rebuttal, we also ran high-dimensional simulations, which showed strong performance, further supporting our method’s robustness. In simulation studies, scenario differences mainly come from varying correlation and variance. RRbR shows low coverage because it ignores covariates when privatizing labels, increasing variance. Our method incorporates covariates, leading to more accurate and stable coverage.\n\n**Weakness 2.**\n\nPlease refer to our responses to Weakness 1 and Question 1 in the rebuttal to Reviewer 7wrY.\n\n**Weakness 3.**\n\nAssumptions 4.8 and 4.9 are standard in asymptotic inference. Assumption 4.8 (on convexity and smoothness of the log-likelihood) holds in most binary regression settings, and Assumption 4.9 (positive definiteness of the FIM) ensures model identifiability, a common condition in GLMs. While asymptotic results assume large samples, these assumptions often hold well in practice, even with moderate sample sizes-as supported by our real data analysis.\n\n**Question 2.**\n\nAs explained above, we follow Wang (2015) to impose $p_{00} > 0.5$ and $p_{11} >0.5$ (our default assumption) to ensure that our $T$-optimal RR mechanism still tends to return truthful responses, more accurately than random guessing. \n\n**Question 3.** \n\nWe need Assumption 4.2 in our proof of Lemma 4.3 to guarantee that $\\frac{\\partial \\mathcal{M}(\\boldsymbol{\\beta} ; p_{00}, p_{11})}{\\partial p_{00}}>0$ (lines 571-572). Given that our feasibility region in Definition 3.1 explicitly enforces $p_{00} > 0.5$ and $p_{11} >0.5$, this assumption is naturally satisfied.\n\n**Question 4.**\n\nBy ''unsupervised aspects of the response'', we intended to highlight that traditional RR designs ignore covariate $X$ when privatizing the response. In contrast, our approach leverages covariate $X$ to inform the design of the RR mechanism, effectively making it a supervised design approach. We will revise the wording to make this clearer.\n\n**Question 5.** \n\nThe confidence intervals for RRbR are calculated by fitting the same binary response model used in our method, but using privatized responses generated by the traditional RR mechanism. Classical methods based on the asymptotic normality of maximum likelihood estimators are then applied to construct the intervals. However, this approach does not account for the additional bias and variability introduced by the RR mechanism, nor does it incorporate covariate information to mitigate such effects. As a result, it often leads to inaccurate and poorly calibrated confidence intervals, as shown in Figures 3 and 5.", "claims_and_evidence": "Most of the claims are supported by clear evidence. However, some concerns remain: (i) the Validity of Def. 3.1 (see below), and (ii) the meaning of \"optimality\". The paper states that the method borrows the T-optimality criterion. The criterion and the optimality guarantees, however, are never introduced. It remains unclear if the proposed method is indeed \"optimal\" (and the precise meaning thereof) or only an improvement over the standard RR mechanism.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense but are limited. Considering more high-dimensional and complex settings would be interesting to showcase and evaluate the optimality of the proposed LabelDP framework in practical settings.", "theoretical_claims": "Not all theoretical claims are stated as such and therefore are not proven. Especially for Def. 3.1. it is difficult to check the validity.", "experimental_designs_or_analyses": "The scope of the experiments is limited and very simplistic. Furthermore, the results are not reproducible, as no code is provided for the experiments.\n\nThe evaluation of the empirical results is not coherent—for example, lines 378-381 state that significant differences between both scenarios can be observed. However, this is not the case; rather, it raises a different question: The coverage of the CIs of RRbR is always far from the desired level. Why is this the case?", "relation_to_broader_scientific_literature": "The paper is closely related to literature on LabelDP using the RR mechanism. It distinguishes itself from the former in that it considers estimation performance when enforcing the privacy constraints.", "essential_references_not_discussed": "/", "other_strengths_and_weaknesses": "**Strengths:**\n- To the best of my knowledge, the idea of designing a LabelDP mechanism that considers specific optimality criteria for estimation is novel.\n\n**Weaknesses:**\n- The method is only compatible with GLMs. Non-parametric estimation is thus not possible. \n- The method only applies to binary labels. This limits the contribution of the work.\n- The confidence interval construction requires multiple assumptions that are unlikely to hold in practice.", "other_comments_or_suggestions": "/", "questions_for_authors": "1. Definition 3.1: This is not a definition. In my opinion, this is rather a theorem which has to be either references or proven.\n2. Lines 218/219: Why is this \"default assumption\" reasonable? How can it be tested?\n3. Assumption 4.2: Why is this assumption reasonable? Has this assumption been made in related work as well?\n4. Lines 163/164: What is meant by \"unsupervised aspects of the response\"?\n5. Experiments: How is the confidence interval for RRbR calculated? How can the extreme underoverage in Figures 3 & 5 be explained?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a statistically principled approach to label differential privacy (LabelDP) in binary response models. The key idea is to design a randomized response (RR) mechanism that is optimal with respect to Fisher Information. The method, termed Optimal Response under Randomized Binary Reporting (ORRbR), aims to minimize estimation variance while satisfying differential privacy constraints. The authors establish asymptotic consistency and normality for their estimator and demonstrate empirical performance gains in both simulations and a real-world application. Given that the method is technically sound, the presentation is clear, and the experimental results (while focused) convincingly validate the core claims, I recommend acceptance.
Decision: Accept (poster)