Hypothesis Testing for Differentially Private Linear Regression
Authors: Daniel Alabi
Salil Vadhan
Venue: NeurIPS 2022 Accept (2022)
Keywords: differential privacy
linear regression
robust statistics
small-area analysis

Abstract: In this work, we design differentially private hypothesis tests for the following problems in the general linear model: testing a linear relationship and testing for the presence of mixtures. The majority of our hypothesis tests are based on differentially private versions of the $F$-statistic for the general linear model framework, which are uniformly most powerful unbiased in the non-private setting. We also present another test for testing mixtures, based on the differentially private nonparametric tests of Couch, Kazan, Shi, Bray, and Groce (CCS 2019), which is especially suited for the small dataset regime. We show that the differentially private $F$-statistic converges to the asymptotic distribution of its non-private counterpart. As a corollary, the statistical power of the differentially private $F$-statistic converges to the statistical power of the non-private $F$-statistic. Through a suite of Monte Carlo based experiments, we show that our tests achieve desired \textit{significance levels} and have a high \textit{power} that approaches the power of the non-private tests as we increase sample sizes or the privacy-loss parameter. We also show when our tests outperform existing methods in the literature.

Hypothesis Testing for Differentially Private Linear Regression
Daniel Alabi *1 and Salil Vadhan f2
Departmeen of Computer Science and Data Science Institute, Columbia University Harvard School of Engineering and Applied Sciences
Abstract
In this work, we design differentially private hypothesis tests for the following problems in the general linear model: testing a linear relationship and testing for the presence of mixtures. The majority of our hypothesis tests are based on differentially private versions of the F-statistic for the general linear model framework, which are uniformly most powerful unbiased in the non-private setting. We also present another test for testing mixtures, based on the differentially private nonparametric tests of Couch, Kazan, Shi, Bray, and Groce (CCS 2019), which is especially suited for the small dataset regime. We show that the differentially private F-statistic converges to the asymptotic distribution of its non-private counterpart. As corollary, the statistical power of the differentially private F-statistic converges to the statistical power of the non-private F F-statistic. Through a suite of Monte Carlo based experiments, and have a high power that approaches the power of the non-private tests as we increase sample sizes the privacy-loss parameter. We also show when our tests outperform existing methods in the literature.
show that our tests achieve desired significance levels
Introduction
Linear regression is one of the most fundamental statistical techniques available to social scientists and economists (especially econometricians) [43, 20J. A graduate student successfully completes her classes: what is the effect on her future earnings? A country implements tough penalties for concealed carry permit violations: what is the effect on gun-related deaths? A school district cuts the size of the average class size for high school classes: what is the effect the standardized scores of the minority students?
All three questions about the (unknown) effect of changing (independent) variable X on another variable Y. Under reasonable assumptions, the relationship between X and Y can be modeled linear regression testing problem. One of the goals of performing regression analysis is for use in decision-making via point estimation (i.e., getting a single predicted value for the dependent variable). To increase the confidence of decision-makers and analysts in such estimates, it is often important also release accompanying uncertainty estimates for the point estimates [18, 19, 6].
In this work, we aim to provide differentially private linear regression uncertainty quantification via the use of hypothesis tests. Given the realistic possibility of reconstruction, membership, and inference attacks [46, 25], can rely on Differential Privacy (DP), rigorous approach to quantifying
'Email: alabidecs. col umbia. edu. D.A. was supported Fellowship from Meta Al and Cooperative Agreement CB20ADR0160001 with the Census Bureau. Work done while Ph.D. student Harvard University. 'Email: salil_ vadhan@harvard. edu. S.V. was supported by Cooperative Agreement CB20ADR0160001 and a Simons Investigator Award.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

privacy loss [24, 22]. The task of DP linear regression is to, given datapoints { 16)) point or uncertainty estimates for linear regression while satisfying differential privacy. The majority of our tests will rely on generalized likelihood ratio test F-statistics.
estimate
In previous works [41, 49, 13, 31, differentially private estimators for linear regression explored and key factors (such as sample size and variance of the independent variable) that affect the accuracy of these estimators are identified. The focus of these previous works for point estimate prediction. The predictive accuracy of such estimators can be measured in terms of a confidence bound or mean-squares error. We continue the study of the utility of such estimators for use in uncertainty quantification via hypothesis testing [41]. (See Section B for a more detailed background on hypothesis testing.)
Earlier work on uncertainty quantification for linear regression was done by Sheffet [41], who constructed confidence intervals and hypothesis tests based on the t-test statistic, and can be used to test a linear relationship. The random projection routine in [41], based on the Johnson-Lindenstrnn (JL) transform, only starts to correctly reject the null hypothesis when the sample size is very large (or the variables have a large spread). This observation is also supported by the work of [21]. Furthermore, the random projection routine requires extra parameters (e.g., for specifying the dimensions of the random matrix). In our work, we use the F -ttstiiittcc and our framework can be used to test mixture models, amongst other tests. We provide a general framework for DP tests based on the F-statistic. In addition, we will consider hypothesis testing for linear regression coefficients on both small and large datasets. For the mixture model tester, we additionally adapt and evaluate a nonparametric method, a DP analogue of the Kruskal-Wo -allis test due to Couch, Kazan, Shi, Bray, and Groce [21], which especially suited for the small dataset regime (See Section C.3.2 for full design, implementation, and proof details for the Kruskal- Wallis test.). To the best of our knowledge, our tests are the first to differentially privately detect mixtures in linear regression models, with accompanying experimental validation.
1.1 Our Contributions
In this work, we show that for the problem of differentially private linear regression, we can perform hypothesis testing for two problems in the general linear model: (I) Testing a Linear Relationship: is the slope of the linear model equal to some constant (e.g., slope is 0)? (2) Testing for Mixtures: does the population consist of one or more tob--ooulations with different regression coefficients?
We provide a differentially private analogue of the F-statistic which we, under the general lin- ear model, show converges in distribution to the asymptotic distribution of the F-statistic (Theo- rem 2.2). Furthermore, the DP regression coefficients converge, in probability, to the true coefficients (Lemma 1I1 In particular, in Lemma 1. we show a 1/vri statistical rate of convergence for the DP regression coefficients used for our hypothesis tests. This matches the optimal rate [38]. We then use our DP F-statistic and parametric estimates to obtain DP hypothesis tests using a Monte Carlo parametric bootstrap, following Gaboardi, Lim, Rogers, and Vadhan [30]. The Monte Carlo parametric bootstrap used to ensure that our tests achieve a target significance level of a (i.e., data generated under the null hypothesis is rejected with probability a). We experimentally compare these tests to their non non-private counterparts for univariate linear regression (i.e., one independent variable and one dependent variable). To the best of our knowledge, our tests are the first that use the F-statistic to perform tests on the problem of linear regression while ensuring privacy of the data subjects. In addition, our F-statistic based tests can be adapted to work on design matrices in any dimension i.c., the design matrix can be cast in the form X E 2 for any integer p Z 2 where represents the number of individuals in the dataset and represents the number of features per individual; we leave experimental evaluation of the multivariate case for future work.
We generate synthetic datasets with different distributions on the independent (or explanatory) variables. Specifically, we consider uniform, normal, and exponential distributions on the independent variables. We also vary the noise distribution of the dependent variable. (2) Opportunity Insights (OI) Data: We use a simulated version of the data used by the Opportunity Insights team (an economics research lab) to release the Opportunity Atlas tool, primarily used to predict social and economic mobility. We chose to use these datasets since they come from a real deployment of privacy-preserving statistics.
Experimental evaluation of our hypothesis tests is done on: (1) Synthetic Data:

---

3 The census tract-level datasets from these states can have a very small number of datapoints. (3) Bike Sharing Dataset: We use real-world dataset publicly available in the UCI machine learning repository. The dataset consists of daily and hourly counts (with other information such as seasonal and weather information) of bike rentals in the Capital bikeshare system years 2011 and 2012.
Our experimental findings are as follows:
1. Significance and Power: Across a variety of experimental settings, the significance is below the target significance level of 0.05. Thus, we have high confidence that we will not falsely reject the null hypothesis, should the null hypothesis be true. Our tests are designed to be conservative in the sense that they err on the side of failing reject the null (e.g.. when the DP estimate of a variance is negative). Consistently, the power of our tests increase as we increase sample sizes (from hundreds up to tens of thousands) parameter. The behavior of our DP tests tends toward that of the non-DP tests. The power of the DP OLS linear relationship tester increases as the slope of the model increases and as the noise in the dependent variable decreases. But when the DP estimate of the noise in the dependent variable is negative, the tests err on the side of failing to reject the null, leading to a lower power. The power of the mixture model tests also increases as the difference between the slopes in the two groups increases. And the more uneven the group sizes are, the lower the power.
as we relax the privacy
2. Alternative Methods:
(a) We compare our DP linear relationship tester, based on the F- statistic, to test that builds a DP parametric bootstrap method for confidence interval estimation. Ferrando, Wang. and Sheldon [29] prove that these intervals are consistent (in the asymptotic regime) and experimentally show that these intervals have good coverage. We can rely on such parametric bootstrap methods to decide to reject or fail to reject the null hypothesis. Such methods achieve the desired significance levels. However, we observe that the method is less powerful than the F-statistic approach. This hehavior could be attributed to the differences in the bootstrap process: whereas we use estimates of sufficient statistics under the null in the bootstrap procedure, [29] uses the entirety of the sufficient statistics estimated for the parametric model. (b) The DP nonparametrie method we provided, adapting the work of Couch, Kazan, Shi, Bray, and Groce [21]. has a higher power in the small dataset regime than the DP F-statistic method. As the dataset size increases or the difference in slopes between the groups increases, the gap closes. In addition, as the variance of the independent variable increases, the F-statistic method outperforms the nonparametric method. This observation echos findings in previous work [3] for the case of prediction rather than hypothesis testing. In that work, is shown that parametric methods for linear regression perform well in prediction as n, the variance of the independent variable, or the privacy loss parameter increase.
1.2 Overview of Techniques
We provide Algorithm 1. generic framework for DP Monte Carlo tests via a parametric bootstrap routine for estimating sufficient statisties. Algorithm 1 crucially relies on DPStats, a procedure that uses statistics of the independent and dependent variables to produce DP statistics. These statistics can be used to decide to reject or fail to reject the null hypothesis. DPStats must satisfy DP-in our case p-ZCDP ConeenCcneetr Differential Privacy) [12]. Algorithms 2 and 5 are SSP (Sufficient Statistic Perturbation) implementations of DPStats for the linear relationship tester and mixture model tester, respectively. (See Section C.2 for more details.) Algorithm 6 is a DP nonparametric test framework based on the Kruskal- test statistic for testing mixtures. We use the standard Gaussian mechanism to make these algorithms satisfy zCDP although the Laplace mechanism could be used instead (i.e., for pure DP). To make Algorithm 6 DP, we randomly pair datapoints so that the resulting transformation is I-Lipschitz (See Section C.3 for more details.). Theorem 2.2 shows that the DP F-statistic converges to the asymptotic distribution of its non-private counterpart, the chi- squared distribution. As corollary, the statistical power of the DP F F-statistic converges to the statistical power of the non- con-private I F-sttaisttcc. To prove this result, we first show the 1/ statistical rate of convergence for the DP coefficients. Then reformulate both the F-statistic and the DP F-statistic in terms of the 1/ /nn convergent quantities, including convergent functions of the Gram
3 See [19, 20J for more detailed description of the use of the Opportunity Atlas tool in predicting social and methods on Opportunity Atlas data.
economic mobility. [18, 3J evaluate
3

---

Matrix of the independent variable. Next, we construct random variables whose &y no-norm squared distributed as the (non-central) chi-squared distribution. Then we prove that DP analogues of the squared fz-norm of such random variables converge to the (non-central) chi-squared distribution. Finally, using such random variables, the reformulated DP F-statistic, reformulated F-statistic, and the continuous mapping theorem, we combine the convergent quantities to prove Theorem 2.2. See Section D for more details.
1.3 Other Related Work
Differentially Private Linear Regression Sheffet [41] considers hypothesis testing for ordinary least squares for specific test: testing for linear relationship under the assumption that the independent variable is drawn from a normal distribution. Wang [49] focuses on using adaptive algorithms for linear regression prediction. M-estimators [34], motivated by the field of robust statistics [35], are a simple class of statistical estimators that present a general approach to parametric inference. Dwork and Lei [23, 38] and Chaudhuri and Hsu [17] present differentially private M- estimators with near- optimal statistical rates of convergence.
[7] generalizes the M-estimator approach to differentially private statistical inference using an empirical notion of influence functions to calibrate the Gaussian mechanism. Alabi, McMillan, Sarathy, Smith, and Vadhan [3] proposed median-based based estimators for linear regression and evaluated their performance for prediction. All of these previous works show connections between robust statistics, M-estimators, and differential privacy. The ordinary least squares estimator is a classical M-estimator for prediction. Other examples include sample quantiles and the maximum likelihood estimation (MLE) objective. However, for differentially private hypothesis testing, as we show in this work, the optimal test statistic for DP linear regression depends on statistical properties of the dataset (such as variance and sample size). We also present novel differentially private test statistics that converge in distribution to the usymptotic distribution of the F-statistic. Bernstein and Sheldon [10] take a Bayesian approach to linear regression prediction and credible interval estimation. Through the Bayesian lens, there also work on how to approximately bias-correct some DP estimators while providing some uncertainty estimates in terms of (private) standard errors [27]. As a motivation for differentially private simple linear regression point and uncertainty estimation, Bleninger, Dreschsler, Ronning [11] show how an attacker could use background information to reveal sensitive attributes about data subjects used in a simple linear regression analysis.
General Differentially Private Hypothesis Testing Gaboardi, Lim, Rogers, and Vadhan [30] study hypothesis testing subject to differential privacy constraints. The tests they consider are: (1) goodness-of-fit tests on multinomial data to determine if data was drawn from multinomial distribution with a certain probability vector, and (2) independence tests for checking whether two categorical random variables are independent of each other. Both tests use the chi-squared test statistic. Rogers and Kifer [40] develop new test statisties for differentially private hypothesis testing categorical data while maintaining a given Type I error. Through the use of the subsample- and-aggregate framework, Barrientos et al. [9] compute univariate t-statistics by first partitioning the data into M disjoint subsets, estimating the statistic each subset, truncating the statistic at some threshold a, and then adding noise from a Laplace distribution to the average of the truncated t-statistics. Our framework requires a clipping parameter (similar to a) but does not require any others (e.g., the number of subsets). As noted in that work, the parameter M plays a significant role in the performance-ane tuning-of their tests while our tests require no such parameter tuning on partitioning of the data. A subset of previous work [47, 14, 45, 21] focus on differentially private independence tests between categorical and continuous variables. Some of these works produce nonparametric tests which require little or no distributional assumptions on the data generation process. Specifically, Couch, Kazan, Shi, Bray, and Groce [21] develop DP analogues of rank-based nonparametric tests such as Kruskal-Wei -allis and Mann-Whitney signed-rank tests. The Kruskal-Wallis test, for example, can be used to determine whether the medians of two or more groups are the same. We adapt their test to the setting of linear regression by using it to compare the distributions of slopes between the two groups. Wang, Lee, and Kifer [50] develop DP versions of likelihood ratio and chi-squared tests, showing a modified equivalence between chi-squared and likelihood ratio tests. In the space of differentially private hypothesis testing, previous work introduce methods for differentially private identity and equivalence testing of discrete distributions [1, 2, 4. 5). A differentially private version of the log-likelihood ratio test for the Jeyman-PParsor lemma has also been shown to exist [15]. Furthermore, Awan and Slavkovic [8] derive uniformly most powerful DP

---

tests for simple hypotheses for binomial data. Suresh [44] proposes a hypothesis test, which can be made to satisfy differential privacy, that is robust to distribution perturbations under Hellinger distance. Sheffet and Kairouz, Oh, Viswanath [42, 36] also consider hypothesis testing, although in the local setting of differential privacy.
1.4 Differential Privacy
For the definitions below, we say that two databases x and x' are neighboring, expressed notationally as d(x, x) for any x, E X", if differs from x' in exactly one row.
Definition 1.1 (Differential Privacy [24, 22]). Let M : X" + R be a (randomized) mechanism. For any 2 0. E [0, IJ. we say M is (e, didiffeerettlllly private if for all neighboring databases x,x' E n d(xxx x') and every S C R.
PIM(() E S (P(M(x') E ES] SS++1 +
If 8 = 0, then we say M is e-DP, sometimes referred to pure differential privacy. Typically, e small constant (e.g.. e [0.1, 1)) and 1/poly(n) cryptographically small.
Definition 1.2 (Zero-Concentrated Differential Privacy (zCDP) [12]). Let M x be (randomized) mechanism. For any neighboring databases x, E X², d(x,x') = 1, we say M satisfies p-zCDP if for all E (1,
Da(M(x)]IM(x)) a.
where (a(M()))]lM()) is the Rényi divergence of order between the distribution of MM()) and the distribution of Mxx).
In this paper, we will primarily use p-zCDP as our definition of differential privacy, adding noise from Gaussian distribution to ensure zCDP.
2 Differentially Private Linear Regression Testing
The goal of hypothesis testing is infer, based on data, which of hypothesis, Ho (the null hypothesis) or Hi (the alternative hypothesis), should be rejected. Let Pa be a family of probability distributions parameterized by Ω. For some unknown parameter 0 Ω. let Z Po be the observed data. Then the two competing hypothesis are: Ho E 24 vs. H1 0 E Ω1 where (Lo, Q2) form a partition of Ω. We will consider hypothesis testing in the linear model Y X3 + e, where X E x2* is matrix of known constants, 8 E Rº is the parameter vector that determines the linear relationship between X and the dependent variable Y. and e random vector such that for all i E [m]. E(e,] 0, var[e;] ] = o². Furthermore, for all of E [m], cov(e;, ej) = O. Note that the simple linear regression model, yo 33 + 81 I; + e; for scalars 2;; Yi and 6 Vi E [n], can be cast as linear model as follows: X E R where the first column is an all-ones vector and the second column (z)......2) ...) )..)".
We will consider the general linear model: Y ~ N T matrix. Let be an r-dimensional linear subspace of TRP and w/ be a q-dimensional linear subspace of w such that 0 < < r. We will consider hypothesis tests of the form: (i) Ho: 3 wai (ii) Hi: Wg.
Inxn), where Jax is the n x n identity
Let 3 and AN denote the least squares estimates under the alternative and null hypothesis respectively. In other words, JN argmingew.. || 2 β = argmingew || Yl
A related differential privacy notion, in terms of the Rényi divergence, is given in [39].
5

---

The test statistic (the F-statistic) we will use is equivalent to the generalized likelihood ratio test statistic
1
X 12 18 x1812
T
r
11 -x3112
|
118
n1
(2)
r
||Y
x
|
XBN 12
(3)
q
S
where S | x/811// r). The vectors Y -x3 and x3 X3N can be shown to be orthogonal, so that Y
||3 x X3N 1|² by the Pythagorean theorem [37]. 1, the test is most
When 1, this test is uniformly most powerful unbiased and for
powerful amongst all tests that satisfy certain symmetry restrictions [37].
To design Monte Carlo hypothesis test, we follow similar route to Gaboardi. Lim, Rogers, and Vadhan [30]. In Algorithm we provide a framework perform DP Monte Carlo tests using parametric bootstrap based on test statistic.
Let DPStats be procedure that uses can be used to reject or fail to reject the null hypothesis. In this paper, DPStats will satisfy p-zCDP (Zero-Concentrated Differential Privacy). T is the test statistic computation procedure. As done in [30], for example, we will assume the dataset sizes are public information.
statistics of X, Y produce DP statisties that
Let T = T(0;) be the non-private test statistic procedure given 0 = (X, The goal is to compute [()1) where 01 is an approximation of 01. DPStats returns Gu and 01. If 6 and 0; is not 1 (1 returned whenever the perturbed statistics cannot be used to simulate the null distributions), then we use 0 to compute the DP test statistic and Ag simulate the null. Pè, represents the distribution from which we will sample from to simulate the null distribution. When (X.y) Pg for 0 So and set 01 01 (X, Y) and sample (X', distribution as 0 ((X, v)). T will denote the test statistic applied to the DP sufficient statistics.
Pa. then 0 (XX','') has approximately the same
Algorithm 1 Monte Carlo DP Test Framework.
1: Data: X 22 R"
2: Input: (dataset size)) (privacy-loss parameter); a (target significance); T (test statistic)
3: (og. 0,) = DPStats(X: Y,
4: if 0g 01 = then
5: return Fail to Reject the null
6: end if
7: // non-DP test statistic applied to DP sufficient statistics
8: i T= T(Ø,)
9: Select K 1/a
10: for = =1.k-1... K do
Vi [n]. X,, Au Ps.
12: *Ø,. *0 = DPStats X, "v,, n.
13: Obtain to from ()))
14: end for
15: Sort t()) ... < t(k)
16: Compute threshold ((r) where r [[K + 1)(1 a)1
17: if 7> 1(r) then
18: return Reject the null
19: else
20: return Fail to Reject the null
end if

---

2.1 Differentially Private Algorithms
Notation For any constants A, E IR, we use 1 to mean that the (random) variable will be
truncated to have an upper bound of A and lower bound of B. -, 2, and 2 denotes convergence toward limit, convergence in probability, and convergence in distribution, respectively. See Section B the Appendix for more details.
Algorithm 2 p-zCDP procedure DPStataL
1: Data: X m R"
2: Input: integer n 2 2; r, 0, A 0
3: Set = p/5 and compute the following:
i= 1 n E
=i + (O,
24 pm
1 -= 1 E M1 + +N(0, 241).
- 2;10 + +0
2pn
Ty
= 2/4 E. 3. 1 + +N(0,,
24" pn?
=² 1 n
Ei= v²16²
A + +0
2pn
Ty
Ty
2
31
82
x22
12
x,2 J2
7
282nÿ n
86
ny 23 28 nzy + nBa
231 + Krry
$2
4: (0o 01) = (1,
5: if minn S2 (nx² nz //2)/(nn 1)) > then
6: 6 (B2, 2,,,,,7,5 2..0 n)
7: 0 = 2 n)
8: end if
9: return (0g, 01)
Lemma 2.1 (Lemma C.I). For any p, A > O, Algorithm 2 satisfies p-zCDP.
Instantiating Algurithm for the Linear Tester:
the procedure DPStats,, returns (1, then we fail to reject the null. Otherwise, we use the returned statistics 0 (3,, n) to
create the test statistic T(,)) -= and use 0 = (5, u x² Sh n) to simulate the null distributions (to decide to reject or fail to reject the null hypothesis). Po. is instantiated as normal
distribution and used to generate &z; distributed as N(, (nx² ) 1)) and ky as B2 + e. e (O,
( for all E [n].
Instantiating Algorithm I for the F-statistic Mixture Tester: If the procedure DPStats re- turns (I, then fail reject the null. Otherwise, we the returned statistics
0, (3) x2. x S2 ni, n22 n) to create the test statistic and use
0. = 2, St. n2, n2, n) to simulate the null distributions. Po, is instantiated as a normal
distribution and used to generate fr; distributed N (nz² nnn))(nn - 1)) and generate Ay: distributed as 3, xz + et. e, N S for either group I with size nı group 2 with size Ri-
7

---

2.2 Convergence
now state and interpret our main theoretical result. For more details on the proof and consequences of the theorem, see Section D.
Theorem 2.2 (Theorem D.I). Let - 2, 1 1, and B E IRP. For every E N with > r, let X, Raxp the design matrix where the first column an all-ones vector and the second column is (Z1....,Zn))
"n." Let A An > 0 be sequence of clipping bounds, p p= Pn. -., 11, Under the general linear model (GLM), Yn ~ (,,,, (x d..n))) Let 3 and BN be the DP least-squaree estimate of B, obtained in Algorithm 2, under the alternative and null hypotheses, respectively. Let T Tn be the DP F statisii computed from DP sufficient statistics via Algorithm 2 and Equation (3). Suppose the following conditions hold:
be sequence of privacy parameters, and Ti
1. 3cr Ca2 R such that & Cz. x2 Cg2, and C22 E2.
2. Br such that v/ + n??
3. 42 A - O,
4. PE E [n]. yi $ I- Anll and E [n], E I-An; An).
Then we obtain the following results: Under the null hypothesis: BN 3N L 3. Under the alternative hypothesis: B An E B. i T
Dg x ((0)
The condition that P| E [n)] Vs I- A.]] 0 (Condition 4 in Theorem 2.2), holds by Gaussian tail bound, if 36: > 0 such that for all E [n]. A, 18 + 321 + eev/ooo log 2
T, is the non-private F'-statistic while T, is the DP F-statistic constructed from DP sufficient statistics obtained via Algorithm 2. The main theorem in this section is Theorem 2.2, which shows the convergence, in distribution, of Tr to the asymptotic distribution of Tn, the chi-squared distribution. As a corollary, the statistical power of Tr, converges to the statistical power of T, While Theorem 2.2 specialized the simple linear regression setting (i.e., p 2), can easily be extended to multiple linear regression.
Without privacy, show that T, follows F-distribution and prove convergence for its components:
Theorem 2.3 (Theorem F.2). For every N with > let X Xx E R2xp be the design matrix. Under the general linear model = Y,, ~ NVX,, B, Inxn).
1x,3 3
11
T Tn F Pr-gm-r (n))
Nh
where Fr.m the F-distribution with parameters n, m, BN 1 q is the dimension of wo, and r is the dimension of w with 0 S q
Furthermore,
1.
||Y, - x2,112 X2 02
|| -
10,3-x,MM x2 X22 (1))o
2. If there exists 17 E R such that
1xn0-X, ag
+ 1/2 then
T T4 F. ----gn--ll (1)2 D2 x2 Xr-g (17)
3. We have
Y - x1
P, 02
8

---

(0.19/2
(0.1
(1)/22
11//2
Signiticacc
0.10
(37/2
Significance
0130
(3//2
0- (10/92
A- (10/92 non-private
non-private
0711
(a) Significance for testing linear relation- ship. ((0.5, 1), Ui 0. (0, 0.0013). 2
(b) Significance for testing linear relation-
ship. 2, (00.5, 1), ~ 0.0, +A(00, 12).
==2
Figure 1
The values 3 = 3 B[ are the expected values of our parameter estimates under the alternative and null hypotheses respectively.
Above, the noncentral F-distribution Fn,m (A), with parameters n, m and noncentrality parameter is the distribution of (2) the ratio of two scaled chi-squared random variables. X
3 Experimental Evaluation
In the appendix (Section E), we provide a more thorough suite of experiments to support our claims and fully discuss how we compute the empirical power.
General Parameter Setup for Synthetic Data:
For experimental evaluation synthetic datasets, we generated datasets with sizes between n = 100 and n 10, 000. For both the linear rela- tionship and mixture model tests synthetic data below, we consider the following values of p: (0.1²/2, 12/2, 52/2, 10 /2]. We draw the independent variables . Xi according to a few different distributions: Normal, Uniform, Exponential. For all tests below, the clipping parameter is either set to A 2 or A = 3. For the synthetic data, the dependent variable Y is generated using the linear or mixture model specification described in previous sections and by fixing or varying parameters (such as o.).
3.1 Testing Linear Relationship Synthetic Data
Evaluating the Significance for Normally Distributed Independent Variables: Generally, we see that the significance remains below the target significance level, on average, for all values of For the linear relationship tester, when the standard deviation of the dependent variable (o.,) is small (Figure la), we see that the true significance level is well below the target significance of 0.05, which fine (but conservative). We conjecture that this happens because when de is small: (i) we fail to reject when the noisy estimate of a, is < O; or (ii) the test statistic under the null distribution will be almost always 0 since under the null (even without privacy), the standard deviation of the test statistic is proportional to ae- In Figures la and 1b, see the significance of the linear tester attains the target (of 0.05) as we vary the noise the dependent variable o..
3.2 Testing Mixture Models on Synthetic Data
Evaluating the Power we Increase the Variance of the Independent Variable: In Figures 2a and 2b, we see that the F-statistic method outperforms the Koaskll-Wallin (KW) method when the variance of the independent variable is much larger (10x) than previously.

---

F-stet p= 10.5 5/12
F-stet p= 00.5//2
F-stat p- 131//2
F-stat p- (139/2
Power
F stat p- 137/12
Power
stat p- (5)
non-private
non non-private
KW p (0.55//
KW p (-0.5511
KW p (1.
KW p -1
KW (5)
KW =5)7/2
KW non-private
KW non-privvtee
(a) Power for Kruskal- Wallis versus the F. statistic. 2; ((0.5, 1), 24 -1 ((0, 1) for Group V 111.+++(000 for Group 2. A 3.
(b) Power for Kruskal-Wi Wallis versus the F- statistic. I2 N/0.5, 10), U -1 ((,, for Group y 11 for Group 2. A 3.
Figure 2
Pr(reject null]
(TX)
#trects 10 (NC)
#trects 10 1TX
Pr[reject
(TX)
non-private 4tracts (NC)
non-privace atracts - (TX)
#tracts (NC)
10
non-private #tracts (NC)
non-private #tracts 50 (TX)
32
(a) Probability of rejecting null for testing linear relationship NC. A 2.
(b) Probability of rejecting null for testing linear relationship TX.
Figure 3
3.3 Testing on Opportunity Insights Data
The Opportunity Insights (OI) team gave simulated data for census tracts from the following states in the United States: Idaho, Illinois, New York, North Carolina, Texas, and Tennessee. The dependent and independent variables the child and parent national income percentiles, respectively. Figures 3a, and 3b show the probability of rejecting the null as we increase the parameter when using the DP linear tester.
4 Conclusion
We have developed differentially private hypothesis tests for testing a linear relationship in data and for testing for mixtures in linear regression models. We also show that the DP F-statistic converges to the asymptotic distribution of the non-private F-statistic. Through experiments, we show that our Monte Carlo tests achieve significance that is less than the target significance level across wide variety of experiments. Furthermore, our tests generally have high power, getting higher as we increase the dataset size and/or relax the privacy parameter. Even on small datasets (in the hundreds) with small slopes, our tests retain the small significance while having good power. We have provided formal statements for the DP F-statistic in the asymptotic regime. We leave future work the task of theoretically analyzing the procedures in the non non- non-asymptotic regime. Experimental evaluation done on simulated data for the Opportunity Atlas tool, UCI datasets, and on synthetic datasets of varying distributions on the independent variable (normal, exponential, and uniform). We also identify regimes where the power of our tests are low (e.g., when the variance the dependent variable is very small).
10

---

References
[1] Acharya, C. L. Canonne, C. Freitag, and H. Tyagi. Test without trust: Optimal locally private distribution testing. In AISTATS, pages 2067-2076, 2019.
[2] J. Acharya, Z. Sun, and H. Zhang. Differentially private testing of identity and closeness of discrete distributions. In NewrIPS, pages 6879-6891, 2018.
[3] D. Alabi, A. McMillan, J. Sarathy, A. Smith, and S. P. Vadhan. Differentially private simple linear regression. Proc. Priv. Enhancing Technol. (022(2):184-204 2022.
[4] M. Aliakbarpour, Diakonikolas, D. Kane, and R. Rubinfeld. Private testing of distributions via sample permutations. In NewrIPS, pages 10877-10888, 2019.
[5] M. Aliakbarpour, Diakonikolas, and R. Rubinfeld. Differentially private identity and equiva- Ience testing of discrete distributions. In ICML, pages 169-178, 2018.
[6] Andrews, T. Kitagawa, and A. McCloskey. Inference on winners. Working Paper 25456, National Bureau of Economic Research, January 2019.
[7] M. Avella-Medina. Privacy-preserving parametric inference: case for robust statistics. Journal of the American Statistical Association, 000):-155 2020.
[8] A. Awan and A. Slavkovic. Differentially private inference for binomial data. Journal of Privacy and Confidentiality, 10(1), Jan. 2020.
[9] A. F. Barrientos, J. Reiter, A. Machanavajjhala, and Y. Chen. Differentially private significance tests for regression coefficients. Journal of Computational and Graphical Statistics, 28:440- 453, 2017.
[10] G. Bernstein and D. R. Sheldon. Differentially private bayesian linear regression. In NeurIPS, pages 523-533, 2019.
[11] P. Bleninger, J. Drechsler, and G. Ronning. Remote data access and the risk of disclosure from linear regression: An empirical study. PSD, volume 6344 of Lecture Notes in Computer Science, pages 220-233. Springer, 2010.
[12] M. Bun and T. Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In 7CC, pages 635-658, 2016.
[13] T. T. Cai, Y. Wang, and I.. Zhang. The cost of privacy in generalized linear models: Algorithms and minimax lower bounds. CoRR, abs/201 1. .03900, 2020.
[14] Z. Campbell, A. Bray, A. M. Ritz, and A. Groce Differentially private ANOVA testing. In ICDIS, pages 281-285, 2018.
[15] C. L. Canonne, G. Kamath, A. McMillan, A. D. Smith, and J. Ullman. The structure of optimal private tests for simple hypotheses. In STOC, pages 310-321, 2019.
[16] C. L. Canonne, G. Kamath, and T. Steinke. The discrete gaussian for differential privacy. In NeurlPS, 2020.
[17] K. Chaudhuri and D. J. Hsu. Convergence rates for differentially private statistical estimation. In Proceedings of the 29th International Conference on Machine Learning, ICML, 2012, Edinburgh, Scotland, UK, June 26 July 1, 2012, 2012.
[18) R. Chetty and J. N. Friedman. A practical method to reduce privacy loss when disclosing statistics based on small samples. American Economic Review Papers and Proceedings, 109:414- 420, 2019.
[19] R. Chetty, J. N. Friedman, N. Hendren, M. R. Jones, and S. R. Porter. The opportunity atlas: Mapping the childhood roots of social mobility. Working Paper 25147, National Bureau of Economic Research, October 2018.

---

[20] R. Chetty, N. Hendren, M. R. Jones, and S. R. Porter. Race and Economic Opportunity in the United States: an Intergenerational Perspective*. The Quarterly Journal of Economics, 135(2):711-7833 12 2019.
[21] S. Couch, Z. Kazan, K. Shi, A. Bray, and A. Groce. Differentially private nonparametric hypothesis testing. In CCS, pages 737-751 2019.
[22] C. Dwork, K. Kenthapadi, F. McSherry. I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, pages 486-4046 503, 2006.
[23] C. Dwork and J. Lei. Differential privacy and robust statistics. STOC, pages 371-380, 2009.
[24] C. Dwork, F. McSherry, K. Nissim, and A. D. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, pages 265-284, 2006.
[25] C. Dwork, Smith, Steinke, and Ullman. Exposed! survey of attacks on private data. Anmual Review of Statistics and Its Application, 4(1):61-84, 2017.
[26] E. S. Edgington. Randomization Tests, pages 1882-1883. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011.
[27] G. Evans, G. King, M. Schwenzfeier, and A. Thakurta. Statistically valid inferences from privacy protected data. 2019.
[28] H. T and J. Gama. Event labeling combining ensemble detectors and background knowledge. Prog. Artif. Intell., 2(2-3):113-12) 2014.
[29] C. Ferrando, S. Wang, and Sheldon. Parametric bootstrap for differentially private confidence intervals. CoRR, abs/2006.07749, 2021.
[30] M. Gaboardi, H. Lim, R. M. Rogers, and S. P. Vadhan. Differentially private chi-squared hypothesis testing: Goodness fit and independence testing. In ICMI,, pages 211 10222,,,, 2016.
[31] A. Gut. Probability: A Graduate Course. Springer Texts in Statistics. Springer New York, 2013.
[32] T. Hastie, R. Tibshirani, and H. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition. Springer Series Statistics. Springer, 2009.
[33] R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, 2 edition, 2012.
[34] P. Huber and E. Ronchetti. Robust Statistics. Wiley Series in Probability and Statistics. Wiley, 2011.
[35] P. J. Huber. Robust Estimation of Location Parameter. The Annals of Mathematical Statistics, 35(1):73
101, 1964.
[36] Kairouz, S. Oh, and P. Viswanath. Extremal mechanisms for local differential privacy. Mach. Learn, Res., 17:17:1-17:51, 2016.
[37] R. Keener. Theoretical Statistics: Topics for a Core Course. Springer Texts in Statistics. Springer New York, 2010.
[38] J. Lei. Differentially private m-estimators. In NeurlPS, pages 361-369, 2011.
[39] I. Mironov. Rényi differential privacy. In CSF, pages 263-275, 2017.
2017.
[40] R. Rogers and D. Kifer. A new class of private chi-square hypothesis tests. In AISTATS, pages 991-1000, 2017.
[41] O. Sheffet. Differentially private ordinary least squares. In ICML, pages 1114-3114,,,11111111111111111101 2017.
[42] O. Sheffet. Locally private hypothesis testing. In ICML, pages 4612-4621, 2018.
[43] Stock and M. Watson, Infroduction to Econometrics (3rd edition). Addison Wesley Longman,
2011.
12

---

[44] A. T. Suresh. Robust hypothesis testing and distribution estimation in hellinger distance. CoRR, abs/201 2020.
[45] M. Swanberg, I. Glrrus-Harris, I. Griffith, A. M. Ritz, A. Groce, and A. Bray. Improved differentially private analysis of variance. Proc. Priv. Enhancing Technol., 20193)) 2019.
[46] L. Sweeney. Weaving technology and policy together maintain confidentiality. The Journal of Law, Medicine & Ethics, 25(2-3):98-110, 1997.
[47] C. Task and C. Clifton. Differentially private significance testing on saired-sample data. In Proceedings of the 2016 SIAM International Conference on Data Mining, Miami, Florida, USA, May 5-7, 2016, pages 153-161. SIAM, 2016.
[48] A. v. d. Vaart. Asympiotic Statistics. Cambridge Series Statistical and Probabilistic Mathematics. Cambridge University Press, 1998.
[49] Y. Wang. Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain. In UAI, pages 93-103, 2018.
[50] Y. Wang, J. Lee, and Kifer. Differentially private hypothesis testing, revisited. CoRR, abs/1511.0333 2015.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See conclusion and experiments
section (our DP tests do not perform well for certain data distributions).
(c) Did you discuss any potential negative societal impacts of your work? [Yes] We discuss that in our work the size of the dataset is public whereas the record-level information remains private.
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? (Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See all thcorem and lemma statements.
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section E. We specify data splits for the mixture model tests and hyperparameters for both linear and mixture model testers.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] Our error bars correspond to the probability of rejecting the null hypothesis over thousands of trials. See Section E.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Section E.
4. If you using existing assets (c.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
13

---

(d) Did you discuss whether and how consent was obtained from people whose data you' re using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "sadWwWJaNg", "reviewer": "Reviewer_XTXZ", "summary": "This paper designs differentially private hypothesis tests in the generalized linear model (GLM) for the problems of testing a linear relationship (where the null hypothesis is that the slope of the linear model is (say) beta=0) and testing for the presence of mixtures. \nThe paper studies a DP version of the F-statistic and shows that converges asymptotically to the distribution of its non-private counterpart, as do the regression coefficients. Using this statistic, in combination with the Monte Carlo bootstrapping framework as used in Gaboardi et al (2016), the authors construct DP hypothesis tests both for testing a linear relationship and for the presence of mixtures.\nThree datasets are used to evaluate these testers experimentally. The first tester is being compared to its non-private counterpart and to a test based on DP parametric bootstrap method for confidence interval  and seems to perform better. For testing for the presence of mixtures, the authors provide additionally e DP nonparametric method based on Couch et al. which performs better than the one based on the F-statistic for smaller datasets.", "questions": "One thing that I missed is some discussion on the choice of the F-statistic for these problems. I think some references on how widely-used this statistic is or why it is a better choice than other approaches in the literature might be useful to a statistics non-expert.\n\nSmall typo:\nIn line 8 of algorithm 2, the assignment to theta_0,theta_1 should presumably not be there.\n", "limitations": "Both adequately discussed.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper clearly states and supports its claims and positions itself with respect to prior work.\n-  The convergence properties of the proposed DP version of the F-statistic are fully analysed. The authors show that the linear regression coefficients converge to their non-private counterparts with optimal rate and the power of the differentially private F-statistic converges to the statistical power of the non-private F-statistic.\n- The paper includes experimental evaluations that compare the performance of the tests to the non-private tests but also to other approaches (one different approach for each problem), which creates a more complete picture. \n\nWeaknesses:\n- I would say that there are not clear techniques that emerge from this work but analyzing the convergence of the statistic is not trivial and in general this is a minor point. It is a solid paper overall.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "oXCTQ7itS_1", "reviewer": "Reviewer_wjEQ", "summary": "The paper provides a framework for differentially private hypothesis tests for testing a linear relationship (when also uncertainty estimates are reported) and testing for the presence of mixtures. The proposed tests for detecting presence of mixtures are the first DP tests tailored for this particular task. There is both strong experimental and theoretical evidence (DP F-statistics converges to non-DP one).", "questions": "\n\nThe introduction says: \"For the mixture model tester, we additionally adapt and evaluate a nonparametric method, a DP analogue of the Kruskal-Wallis test due to Couch, Kazan, Shi, Bray, and Groce [21]..\". It remains very unclear without looking to the appendix what is meant by this.\n\nBeing non-expert in this topic, it sounds to me that there are absolutely no better fit among DP hypothesis testing methods you could reasonably compare against, am I correct?\n\nOne of first things coming to my mind: How can I trust that your adaptation of this median testing method is the best possible method to compare against, how can I trust this particular adaptation is reasonable? Why not have an adaption of some other DP hypothesis testing algorithm? In any case I think it would be good to have some more explicit motivation for these choices. \n\nAlso, you mention that you compare to the DP parametric bootstrap method for confidence intervals but the corresponding experiments are only in the appendix (without reference).\n\nFew remarks:\n\nPerhaps a typo in abstract: \"which are uniformly most powerful unbiased in the non-private setting\" ?\n\nFigures 3a and 3b: some dashed lines missing? (non-private baselines)\n\nThe notation for clipping used in Algorithm 2 is not explained anywhere.\n\nNotice: The abbreviation KW that you use in Experiments - section is not explained anywhere in the main text. ", "limitations": "Yes (the methods are widely applicable: hypothesis testing for linear regression)", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\n- Strong theory, showing that the DP F-statistics provided by these test converge in distribution to those of the non-DP ones.\n- Strong experimental evidence\n- An extensive literature review on this line of research (DP hypothesis testing)\n\nWeaknesses:\n\n- I think the paper would need some polishing (see details below)\n- Difficult to digest for a non-expert in this topic like me. Considering that problem is of quite general nature (DP hypothesis testing, linear models), I would imagine the presentation could be made more easily accessible to a wider audience. I am afraid that the results would be appreciated by only expert in this line of research.\n- On a related note, I think it is good that there is a short introduction to hypothesis testing in the appendix, but I think it needs also polishing: for example F-statistics is not explained anywhere.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "nN8FbXpKLEd", "reviewer": "Reviewer_f57B", "summary": "This paper studies the problem of hypothesis testing for two problems in the generalised linear model: testing a linear relationship and testing for the presence of mixtures. Most of the techniques involves are based on privatising the $F$-statistic for the general linear model. These techniques are used through a Monte Carlo parametric bootstrap. Other techniques for testing presence of mixtures in this model are based on the DP non-parametric tests of a prior work by Couch et al. 2019.\n\nThe main results are that the DP $F$-statistic converges to the asymptotic distribution of its non-private counterpart, which implies the convergence of statistical power of DP $F$-statistic to that of the non-private $F$-statistic. The Monte Carlo parametric bootstrap is used to ensure an $\\alpha$ target significance.\n\nAsymptotic results for these problems are obtained in this paper. Also, experiments are performed to compare these results with either the non-private counterparts of these tests or other available techniques for this problem.\n\nEdit: Updated score.", "questions": "I don't have a lot of questions, but just something for clarification.\n\n1. The results in this paper are valid for the case when the ambient dimension is $2$, with the claim that they could be extended to arbitrary dimensions. Is that extension straightforward? Or would you require techniques that are significantly different mathematically from the ones being currently used in this paper?\n\n2. Also, while generalising to higher dimensions, does the accuracy of the private tests still converge to that of the non-private ones?\n\n3. In Line 12 of Algorithm 1, why do we need to use DPStats? I thought those points are artificial points, so do we need privacy here?", "limitations": "I think the authors have been honest about wanting to have non-asymptotic results for this problem. Even though I have mentioned that as a weakness above, I don't think it's a big deal.\n\nI have mentioned my concerns about the writing above already. So, I don't have any other concerns. This results are quite self-contained and \"optimal\" already in the sense that the private convergence and the non-private convergence match.\n\nAlso, one more thing I would recommend is motivating the problem a bit more here. We know hypothesis testing and linear regression independently, but why is testing for this problem important? More like, why is privatising this is an important problem to solve? There are lot of problems that could need private solutions, but why is this an important problem to solve under DP constraints?", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Given that this problem has not been studied that extensively under DP constraints before (except in Sheffet 2017), I would say that this is a great starting point for this.\n2. The work is also fairly technical and rigorous. The analyses of the significance levels theoretically is very thorough even if it is dealing with just asymptotic guarantees.\n3. The convergence of statistical power of the DP $F$-statistic to that of its non-private counter part is a significant contribution. This means that the loss in accuracy due to privacy isn't much, which is usually the main target in most work on DP.\n4. This work also addresses one of the main issues in the prior work on this: the prior work only starts to reject the null hypothesis when the sample size is very large. In other words, their tests could give accurate results even on \"smaller\" datasets.\n5. One more thing I appreciate that the experiments have been performed both on synthetic and real-world datasets, as opposed to just on the former. It gives the impression that these tests are quite versatile in terms of the underlying data distributions.\n\nWeaknesses:\n1. I don't see this as a huge weakness (and is dependent on personal preferences), but getting non-asymptotic results would have made the paper a bit stronger. There are different schools of thoughts on this, of course. As I said though, this is not a big concern for me.\n2. I have some issue with the writing of the paper. It looks like it was a beautifully written paper, but chopping it down to nine pages made it a little hard to read in the first shot. I'll be more specific about this. (A) The fluff text for Algorithm 1 didn't do much for me. I agree that a text version of the algorithms is supposed to help, but an intuition behind why certain decisions and steps are taken within them is probably a more helpful discussion. (B) Saying which algorithms have been restated in the appendices would have been nice. It gets confusing when algorithms are stated in the main body, but the discussions refer to the ones in the appendices.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "YE4nRWmOe3K", "reviewer": "Reviewer_khGd", "summary": "Provides a GLRT & Monte-carlo based styled DP test for two kinds of hypothesis tests in the setting of linear regression: a) Testing a Linear Relationship, b) Testing for Mixtures. They take a holistic approach covering various aspects such as asymptotic  convergence in distribution results for the test-statistic. They provide empirical power analysis and significance analysis results. They compare against non-parametric tests such as the Kruskal-Wallis test.\n\nTypically, a classical way has been to understand the asymptotic distribution of a test-statistic conditioned on null hypothesis being true, followed by verifying its departure from that case based on observed test-statistic to have a guarantee on the performance of the test both in terms of its significance, power and worst-case characterizations. Note: power of a test depends on effect-size, sample size, significance level and so forth", "questions": "1.) line 269: is the notation a typo or intended in terms of a constant in using O(1) in log 2n^O(1)?\n\n2.) To this Q, the authors mention the conclusion section covers limitations-although it does not.: (Did I miss it?)\n     Did you describe the limitations of your work? [Yes] See conclusion and experiments \n     425 section (our DP tests do not perform well for certain data distributions). ", "limitations": "-some details in expt section on how the empirical power was computed would be useful.\n-the word appendix is referred to only once in the main paper. it instead refers to section A or C etc.\nPlease say 'Section A in appendix' and so forth.\n-The section on Overview of Techniques seems a bit too packed in a short space. It should ideally better help the navigation of the paper.\n\n-reference to paper on test statistic introduced in eqn 1 is needed or it needs to be clarified that it was introduced in this submission. it is unclear-which case it is in the beginning of the paper to the reader.\n\n-minor: The word general linear model (GLM), has been used multiple times in this submission to refer to linear regression with Gaussian assumptions. GLM is typically used for generalized linear models of which linear regression is just a special case. please change the terminology to avoid this overloading. \n-A table of main notations would be quite helpful.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:  The submission does quite a thorough job in covering various aspects of DP hyp-testing for linear regression. They provide DP-tests, study convergence of test-statistic, draw similarities to GLRT, study the power, provide parametric bootstrap based computational test and so forth.\n\nWeakness: It is bit unfair to expect this, given the page limit for the main paper: That said, the organization of the paper's approach & results in the appendix could have been better covered in the main paper. This is purely a comment in terms of providing clarity in the reading. \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept