SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1: Robust and Private Linear Regression
Input:  $S = \{(x_i,y_i)\}_{i = 1}^{3n}$  DP parameters  $(\varepsilon ,\delta),T$  , learning rate  $\eta$  , failure probability  $\zeta$  , target error  $\alpha$  distribution parameter  $K$    
1 Partition dataset  $S$  into three equal sized disjoint subsets  $S = S_{1}\cup S_{2}\cup S_{3}$    
2  $\delta_0\gets \frac{\delta}{2T},\varepsilon_0\gets \frac{\varepsilon}{4\sqrt{T\log(1 / \delta_0)}},\zeta_0\gets \frac{\zeta}{3},w_0\gets 0$    
3  $\Gamma \leftarrow$  PrivateNormEstimator(S1,εo,δo,ζo) // using Alg. 3, App. F   
4 Θ← K√2T loga(n/ζo)   
5 for t=0,1,2,...,T-1 do   
6  $\gamma_t\gets$  PrivateDistanceEstimator(S2,wt,εo,δo,α,ζo) // using Alg. 2, App. C   
7  $\theta_t\gets 2\sqrt{2\gamma_t}\cdot \sqrt{9C_2K^2\log(1 / (2\alpha))}.$    
8 Sample  $\nu_{t}\sim \mathcal{N}(0,\mathbf{I}_{d})$    
9  $\tilde{g}_i^{(t)}\gets \mathrm{clip}_{\Theta}(x_i)\mathrm{clip}_{\theta_t}(x_i^\top w_t - y_i)$    
10  $\phi_t = (\sqrt{2\log(1.25 / \delta_0)}\Theta \theta_t) / (\varepsilon_0n)$    
11  $w_{t + 1}\gets w_t - \eta \Big(\frac{1}{n}\sum_{i\in S_3}\tilde{g}_i^{(t)} + \phi_t\nu_t\Big)$    
12 Return  $w_{T}$
3.2 Analysis without adversarial corruption
We show that Alg. 1 achieves an improved sample complexity. We provide the proof for a more general class of distributions in App. H and a sketch of the proof in Sec. 5. We address the necessity of the assumptions in Sec. 3.4, along with some lower bounds.
Theorem 3.1. Alg. 1 is  $(\varepsilon, \delta)$ -DP. Under sub-Gaussian model of Asmp. 2.1, for any failure probability  $\zeta \in (0,1)$  and target error rate  $\alpha$ , if the sample size is large enough such that
$$
n = \tilde {O} \left(K ^ {2} d \log^ {2} \left(\frac {1}{\zeta}\right) + \frac {d + \log (1 / \zeta)}{\alpha^ {2}} + \frac {K ^ {2} d T ^ {1 / 2} \log (\frac {1}{\delta}) \sqrt {\log (\frac {1}{\zeta})}}{\varepsilon \alpha}\right), \tag {2}
$$
with a large enough constant, then the choices of a step size  $\eta = 1 / (C\lambda_{\mathrm{max}}(\Sigma))$  for any  $C\geq 1.1$  and the number of iterations,  $T = \tilde{\Theta}\left(\kappa \log \left(\| w^{*}\|\right)\right)$  for a condition number of the covariance  $\kappa \coloneqq \lambda_{\mathrm{max}}(\Sigma) / \lambda_{\mathrm{min}}(\Sigma)$ , ensures that, with probability  $1 - \zeta$ , Alg. 1 achieves
$$
\mathbb {E} _ {\nu_ {1}, \dots , \nu_ {T} \sim \mathcal {N} (0, \mathbf {I} _ {d})} \left[ \| w _ {T} - w ^ {*} \| _ {\Sigma} ^ {2} \right] = \tilde {O} \left(K ^ {4} \sigma^ {2} \alpha^ {2} \log^ {2} \left(\frac {1}{\alpha}\right)\right), \tag {3}
$$
where the expectation is taken over the noise added for  $DP$ , and  $\tilde{O}$  and  $\tilde{\Theta}(\cdot)$  hide logarithmic terms in  $K, \sigma, d, n, 1/\varepsilon, \log(1/\delta), 1/\alpha$ , and  $\kappa$ .
Remark 3.2. Omitting some constant and logarithmic terms, Alg. 1 requires
$$
n = \tilde {O} \left(\frac {d}{\alpha^ {2}} + \frac {\kappa^ {1 / 2} d}{\varepsilon \alpha}\right), \tag {4}
$$
samples to ensure an error rate of  $(1 / \sigma^2)\mathbb{E}[\| w_T - w^*\|_\Sigma^2 ] = \tilde{O} (\alpha^2)$ . From [16, Theorem 3.1], there exists an  $n = \Omega (d / \alpha^2 +d / (\varepsilon \alpha))$  lower bound, and our upper bound matches this lower bound up to a factor of  $\kappa^{1 / 2}$  in the second term and other logarithmic factors. Eq. (4) is the best known rate among all efficient private linear regression algorithms, strictly improving upon the state-of-the-art. The best existing efficient algorithm by [81] requires  $n = \tilde{O} (d / \alpha^2 +\kappa d / (\varepsilon \alpha) + \kappa^2 d / \varepsilon)$  to achieve the same error rate. Compared to Eq. (4), the second term is larger by a factor of  $\kappa^{1 / 2}$  compared to the second term in Eq. (4). Further, [81] requires  $\kappa^2 d / \varepsilon$ , which is not needed in Eq. (4).
Remark 3.3. Consider the standard settings of linear regression with  $x_{i} \sim \mathcal{N}(0, \mathbf{I}_{d})$  and  $z_{i} \sim \mathcal{N}(0, \sigma^{2})$  such that the condition number is one, our bound given by Eq. (4) nearly matches the lower bound ([16, Theorem 3.1]) up to logarithmic factors.
Remark 3.4. Note that the leading term in Eq. (4) is the first term  $d / \alpha^2$  when target error  $\alpha \leq \varepsilon /\kappa^{1 / 2}$ . Our first term is independent of  $\kappa$ , which matches the lower bound for non-private linear regression.
Remark 3.5. The third term  $\kappa^2 d / \varepsilon$  in [81] is independent of error rate  $\alpha$  but scales as  $\kappa^2$ . This term is required to ensure the privacy noise added in each iteration is small enough for their DP-SGD to make progress (Appendix, B.2.2 in [81]). Our algorithm is based on full-batch gradient descent, which uses all  $n$  samples and thus reduces the sensitivity of gradient average by a  $\kappa$  factor. As a result, we show in Eq. (59) that our algorithm only requires  $n = \tilde{O}((1 / \varepsilon)\sqrt{\kappa^{1/2}d / \alpha})$  to make progress for each iteration. This is strictly smaller than our dominant term  $\kappa^{1/2}d / (\varepsilon\alpha)$  and does not show up in our final guarantee. We provide a formal proof in App. H.
Remark 3.6. One of the key innovations in Alg. 1 is the adaptive distance estimator (Alg. 2 in App. C). The goal is to privately estimate the (shifted) distance of the current estimate, i.e.,  $\| w_t - w^*\|_{\Sigma} + \sigma^2$  without the knowledge of  $w^{*}$ . We show in Thm. C.1 that our novel distance estimator only requires an error-independent sample complexity  $n = \tilde{O} (\kappa^{1 / 2}d / \varepsilon)$  to achieve a constant multiplicative error. Note that the DP-STAT (Algorithm 3 in [81]) can also be used to estimate the distance. But it requires the knowledge of domain size  $\| w^{*}\|_{\Sigma} + \sigma$ . We completely remove this requirement, improve the dependence on  $K$  and  $\log (n)$ , and show it is also robust, as introduced in the next section. We provide the algorithms and analysis in App. C and the formal proof in App. D.
3.3 Robustness against label corruption
We assume there exists a good dataset  $S_{\mathrm{good}}$  that satisfies Asmp. 2.1. We only get access to a label-corrupted dataset under the standard definition of label corruption, e.g., [15]. There are variations in literature on the definition, which we survey in App. A.
Assumption 3.7 ( $\alpha_{\mathrm{corrupt}}$ -corruption). Given a dataset  $S_{\mathrm{good}} = \{(x_i, y_i)\}_{i=1}^n$ , an adversary inspects all the data points, selects  $\alpha_{\mathrm{corrupt}}n$  data points denoted as  $S_r$ , and replaces the labels with arbitrary labels while keeping the covariates unchanged. We let  $S_{\mathrm{bad}}$  denote this set of  $\alpha_{\mathrm{corrupt}}n$  newly labelled examples by the adversary. Let the resulting set be  $S_{\mathrm{corrupt}} := S_{\mathrm{good}} \cup S_{\mathrm{bad}} \setminus S_r$ .
Our goal is to estimate the unknown parameter  $w^{*}$ , given corrupted dataset  $S_{\mathrm{corrupt}}$ , distribution parameter  $K$ , and (an upper bound on) the corruption level  $\alpha_{\mathrm{corrupt}}$ .
Under the non-private scenario, i.e.,  $\varepsilon = \infty$ , recent advances led to optimal algorithms for linear regression that are robust to label corruptions [15, 21]; if the corruption level is smaller than the target error rate, i.e.,  $\alpha_{\mathrm{corrupt}} \leq \alpha$ , then  $n = \tilde{O}(d / \alpha^2)$  samples are sufficient to achieve an error rate of  $(1 / \sigma) \| \hat{w} - w^* \|_{\Sigma} = \alpha$ . The sample complexity of  $d / \alpha^2$  is optimal as it matches the information theoretic lower bound. The condition  $\alpha_{\mathrm{corrupt}} \leq \alpha$  is necessary since it is information theoretically impossible to achieve error  $\alpha$  less than  $\alpha_{\mathrm{corrupt}}$ , as we prove in Prop. 3.9. Setting the target error to the minimum possible value of  $\alpha = \alpha_{\mathrm{corrupt}}$ , we say that these algorithms achieve optimal robustness since the minimum robust error rate of  $(1 / \sigma) \| \hat{w} - w^* \|_{\Sigma} = O(\alpha_{\mathrm{corrupt}})$  can be achieved with minimal sample complexity of  $n = \tilde{O}(d / \alpha_{\mathrm{corrupt}}^2)$ . We aim to achieve such optimal robustness simultaneously with differential privacy in a computationally efficient manner.
Theorem 3.8. Under sub-Gaussian model of Asmp. 2.1 and  $\alpha_{\mathrm{corrupt}}$ -corruption of Asmp. 3.7, if the corruption level is below the target error rate,  $\alpha \geq \alpha_{\mathrm{corrupt}}$ , then  $n = \tilde{O}(d / \alpha^2 + \kappa^{1/2} d / (\varepsilon \alpha))$  samples are sufficient for Alg. 1 to achieve an error rate of  $(1 / \sigma^2) \mathbb{E}[\| \hat{w} - w^* \|_\Sigma^2] = \tilde{O}(\alpha^2)$ .
This is the first efficient approach to achieve robustness and  $(\varepsilon, \delta)$ -DP simultaneously. The existing such algorithms take exponential time [61, Corollary C.2] and [10], but achieve optimal sample complexity of  $n = O(d / \alpha^2 + d / (\varepsilon \alpha))$ . Notice that there is no dependence on  $\kappa$ . It remains an open question if computationally efficient private linear regression algorithms can achieve such an optimal  $\kappa$ -independent sample complexity. We make the first advance towards this ambitious goal with the above theorem. Our sample complexity is sub-optimal only by a factor of  $\sqrt{\kappa}$  in the second term. This is achieved by individually clipping the covariate,  $x_i$ , and the residual,  $(w_t^\top x_i - y_i)$ , in Alg. 1 and carefully tracking the bias of clipping with the use of resilience in the analysis in App. H.
3.4 Lower bounds
Necessity of our assumptions. A tail assumption on the covariate  $x_{i}$  such as Asmp. 2.1 is necessary to achieve  $n = O(d)$  sample complexity in Eq. (4). Even when the covariance  $\Sigma$  is close to identity, without further assumptions on the tail of covariate  $x$ , the result in [13] implies that for  $\delta < 1 / n$ , it is necessary for an  $(\varepsilon, \delta)$ -DP estimator to have  $n = \Omega(d^{3/2} / (\varepsilon \alpha))$  samples to achieve
Figure 1: Performance of various techniques on DP linear regression.  $d = 10$  in all the experiments.  $n = 10^7$ ,  $\kappa = 1$  in the  $2^{nd}$  experiment.  $n = 10^7$ ,  $\sigma = 1$  in the  $3^{rd}$  experiment, where  $\kappa$  is the condition number of  $\Sigma$  and  $\sigma^2$  is the variance of the label noise  $z_i$ .
$\| \hat{w} - w^{*}\|_{\Sigma} = \tilde{O}(\alpha)$  (see Eq. (3) in [83]). Note that this lower bound is a factor  $d^{1/2}$  larger than our upper bound that benefits from the additional tail assumption.
A tail assumption on the noise  $z_{i}$  such as Asmp. 2.1 is necessary to achieve  $n = O(d / (\varepsilon \alpha))$  dependence on the sample complexity in Eq. (4). For heavy-tailed noise, such as  $k$ -th moment bounded noise, the dependence can be significantly larger. [61, Proposition C.5] implies that for  $\delta = e^{-\Theta (d)}$  and 4-th moment bounded  $x_{i}$  and  $z_{i}$ , any  $(\varepsilon ,\delta)$ -DP estimator requires  $n = \Omega (d / (\varepsilon \alpha^2))$  which is a factor of  $1 / \alpha$  larger, to achieve  $(1 / \sigma^2)\| \hat{w} -w^{*}\|_{\Sigma} = \tilde{O} (\alpha)$ .
The assumption that only labels are corrupted is critical for Alg. 1. The average of the clipped gradients can be significantly more biased, if the adversary can place the covariates of the corrupted samples in the same direction. In particular, the bound on the bias of our gradient step in Eq. (44) in App. H would no longer hold. Against such strong attacks, one requires additional steps to estimate the mean of the gradients robustly and privately, similar to those used in robust private mean estimation [60, 56, 44, 8]. There is no known linear-time algorithm to achieve this, and this is outside the scope of this paper.
Lower bounds under label corruption. Under the  $\alpha_{\mathrm{corrupt}}$  label corruption setting (Asmp. 3.7), even with infinite data and without privacy constraints, no algorithm is able to learn  $w^{*}$  with  $\ell_2$  error better than  $\alpha_{\mathrm{corrupt}}$ . We provide a formal derivation for completeness.
Proposition 3.9. Let  $\mathcal{D}_{\Sigma, \sigma^2, w^*, K}$  be a class of distributions on  $(x_i, y_i)$  from sub-Gaussian model in Asmp. 2.1. Let  $S_{n,\alpha}$  be an  $\alpha$ -corrupted dataset of  $n$  i.i.d. samples from some distribution  $\mathcal{D} \in \mathcal{D}_{\Sigma, \sigma^2, w^*, K}$  under Asmp. 3.7. Let  $\mathcal{M}$  be a class of estimators that are functions over  $S_{n,\alpha}$ . Then there exists a constant  $c$  such that  $\min_{n, \hat{w} \in \mathcal{M}} \max_{S_{n,\alpha}, \mathcal{D} \in \mathcal{D}_{\Sigma, \sigma^2, w^*, K}, w^*, K} \mathbb{E}[\|\hat{w} - w^*\|_{\Sigma}^2] \geq c\alpha^2\sigma^2$ .
A proof is provided in App. I.1. A similar lower bound can be found in [11, Theorem 6.1].
4.1 DP Linear Regression
We present experimental results comparing our proposed technique (DP-ROBGD) with other baselines. We consider non-corrupted regression in this section and defer corrupted regression to the App. K. We begin by describing the problem setup and the baseline algorithms first.
Experiment Setup. We generate data for all the experiments using the following generative model. The parameter vector  $w^{*}$  is uniformly sampled from the surface of a unit sphere. The covariates  $\{x_{i}\}_{i = 1}^{n}$  are first sampled from  $\mathcal{N}(0,\Sigma)$  and then projected to unit sphere. We consider diagonal covariances  $\Sigma$  of the following form:  $\Sigma [0,0] = \kappa$ , and  $\Sigma [i,i] = 1$  for all  $i\geq 1$ . Here  $\kappa \geq 1$  is the condition number of  $\Sigma$ . We generate noise  $z_{i}$  from uniform distribution over  $[-\sigma ,\sigma ]$ . Finally, the response variables are generated as follows  $y_{i} = x_{i}^{\top}w^{*} + z_{i}$ . All the experiments presented below are repeated 5 times and the averaged results are presented. We set the DP parameters  $(\epsilon ,\delta)$  as  $\epsilon = 1, \delta = \min (10^{-6},n^{-2})$ . Experiments for  $\epsilon = 0.1$  can be found in Fig. 2 in the App. K.
Baseline Algorithms. We compare our estimator with the following baseline algorithms:
- Non private algorithms: ordinary least squares (OLS), one-pass stochastic gradient descent with tail-averaging (SGD). For SGD, step-size is  $1 / (2\lambda_{\mathrm{max}})$  and minibatch size is  $n / T$ , where  $T = 3\kappa \log n$ .
- Private algorithms: sufficient statistics perturbation (DP-SSP) [38, 83], differentially private stochastic gradient descent (DP-AMBSSGD) [81]. DP-SSP had the best empirical performance among numerous techniques studied by [83], and DP-AMBSSGD has the best known theoretical guarantees. The DP-SSP algorithm involves releasing  $X^T X$  and  $X^T \mathbf{y}$  differentially privately and computing  $(\widehat{X^T X})^{-1} \widehat{X^T \mathbf{y}}$ . DP-AMBSSGD is a private version of SGD where the DP noise is set adaptively according to the excess error in each iteration. For both algorithms, we use the hyper-parameters recommended in their respective papers. To improve the performance of DP-AMBSSGD, we reduce the theoretical clipping threshold by a constant factor.
DP-ROBGD. We implement Alg. 1 with the following key changes. Instead of relying on PrivateNormEstimator to estimate  $\Gamma$ , we set it to its true value  $\mathrm{Tr}(\Sigma)$ . This is done for a fair comparison with DP-AMBSSGD which assumes the knowledge of  $\mathrm{Tr}(\Sigma)$ . Next, we use  $20\%$  of the samples to compute  $\gamma_{t}$  in line 5 (instead of the  $50\%$  stated in Alg. 1). In our experiments we also present results for a variant of our algorithm called DP-ROBGD* which outputs the best iterate based on  $\gamma_{t}$ , instead of the last iterate. One could also perform tail-averaging instead of picking the best iterate. Both these modifications are primarily used to reduce the variance in the output of Alg. 1 and achieved similar performance in our experiments.
Results. Figure 1 presents the performance of various algorithms as we vary  $n, \kappa, \sigma$ . It can be seen that DP-ROBGD outperforms DP-AMBSSGD in almost all the settings (and DP-ROBGD* outperforms DP-ROBGD in all cases). DP-SSP has poor performance when the noise  $\sigma$  is low, but performs slightly better than DP-ROBGD in other settings. A major drawback of DP-SSP is its computational complexity which scales as  $O(nd^2 + d^\omega)$ . In contrast, the computational complexity of DP-ROBGD has smaller dependence on  $d$  and scales as  $\tilde{O}(nd\kappa)$ . Thus the latter is more computationally efficient for high-dimensional problems. More experimental results on both robust and private linear regression can be found in the App. K.
5 Sketch of the main ideas in the analysis
We provide the main ideas behind the proof of Thm. 3.1. The privacy proof is straightforward since no matter what clipping threshold we use the noise we add is always proportionally to the clipping threshold which guarantees privacy. In the remainder, we focus on the utility analysis.
The proof of the utility heavily relies on the resilience [73] (also known as stability [27]), which states that given a large enough sample set  $S$ , various statistics (for example, sample mean and sample variance) of any large enough subset of  $S$  will be close to each other. We define resilience as follows.
Definition 5.1 ([61, Definition 23]). For some  $\alpha \in (0,1)$ ,  $\rho_{1} \in \mathbb{R}_{+}$ ,  $\rho_{2} \in \mathbb{R}_{+}$ , and  $\rho_{3} \in \mathbb{R}_{+}$ ,  $\rho_{4} \in \mathbb{R}_{+}$ , we say dataset  $S_{\mathrm{good}} = \{(x_{i} \in \mathbb{R}^{d}, y_{i} \in \mathbb{R})\}_{i=1}^{n}$  is  $(\alpha, \rho_{1}, \rho_{2}, \rho_{3}, \rho_{4})$ -resilient with respect to  $(w^{*}, \Sigma, \sigma)$  for some  $w^{*} \in \mathbb{R}^{d}$ , positive definite  $\Sigma > 0 \in \mathbb{R}^{d \times d}$ , and  $\sigma > 0$  if for any  $T \subset S_{\mathrm{good}}$  of size  $|T| \geq (1 - \alpha)n$ , the following holds for all  $v \in \mathbb{R}^{d}$ :
$$
\left| \frac {1}{| T |} \sum_ {\left(x _ {i}, y _ {i}\right) \in T} \langle v, x _ {i} \rangle \left(y _ {i} - x _ {i} ^ {\top} w ^ {*}\right) \right| \leq \rho_ {1} \sqrt {v ^ {\top} \Sigma v} \sigma , \tag {5}
$$
$$
\left| \frac {1}{| T |} \sum_ {x _ {i} \in T} \langle v, x _ {i} \rangle^ {2} - v ^ {\top} \Sigma v \right| \leq \rho_ {2} v ^ {\top} \Sigma v, \tag {6}
$$
$$
\left| \frac {1}{| T |} \sum_ {\left(x _ {i}, y _ {i}\right) \in T} \left(y _ {i} - x _ {i} ^ {\top} w ^ {*}\right) ^ {2} - \sigma^ {2} \right| \leq \rho_ {3} \sigma^ {2}, \tag {7}
$$
$$
\left| \frac {1}{| T |} \sum_ {\left(x _ {i}, y _ {i}\right) \in T} \langle v, x _ {i} \rangle \right| \leq \rho_ {4} \sqrt {v ^ {\top} \Sigma v}. \tag {8}
$$
We give an overview of the proof for non-robust case as follows. First, we introduce some notations. Let  $g_{i}^{(t)} \coloneqq (x_{i}^{\top}w_{t} - y_{i})x_{i}$  be the raw gradient and  $\tilde{g}_i^{(t)}\coloneqq \mathrm{clip}_{\Theta}(x_i)\mathrm{clip}_{\theta_t}(x_i^\top w_t - y_i)$  be the clipped gradient. Note that when the data follows from our distributional assumption, with high probability, samples are not clipped by the norm:  $\mathrm{clip}_{\Theta}(x_i) = x_i$ . We can write down one step of gradient update
(see Alg. 1) as follows:
$$
w_{t + 1} - w^{*} = \underbrace{\left(\mathbf{I} - \frac{\eta}{n}\sum_{i\in S}x_{i}x_{i}^{\top}\right)(w_{t} - w^{*})}_{(i)} + \underbrace{\frac{\eta}{n}\sum_{i\in S}x_{i}z_{i}}_{(ii)} + \underbrace{\frac{\eta}{n}\sum_{i\in S}(g_{i}^{(t)} - \tilde{g}_{i}^{(t)})}_{(iii)} - \underbrace{\eta\phi_{t}\nu_{t}}_{(iv)}.
$$
In the above equation, the first term is a contraction, meaning  $w_{t}$  is moving toward  $w^{*}$ . The second term captures the noise from the randomness in the samples. The third term captures the bias introduced by the clipping operation, and the fourth term captures the added noise for privacy. The second term is standard and relatively easy to control, and our main focus is on the last two terms.
The third term  $(\eta / n) \sum_{i \in S} (g_i^{(t)} - \tilde{g}_i^{(t)})$  can be controlled using the resilience property. We prove that with our estimated threshold, the clipping will only affect a small amount of datapoints, whose contribution to the gradient is small collectively.
Now we have controlled the deterministic bias. Then, we upper bound the fourth term, which is the noise for the purpose of privacy, and show the expected prediction error decrease in every gradient step. The difficulty is that, since our clipping threshold is adaptive, the decrease of the estimation error depends on the estimation error of all the previous steps. This causes that in some iterations, the estimation error actually increases. In order to get around this, we split the iterations into length  $\kappa$  chunks, and argue that the maximum estimation error in a chunk must be a constant factor smaller than the previous chunk. This implies we will reach the desired error within  $\tilde{O} (\kappa)$  steps.
We provide a novel variant of DP-SGD algorithm for differentially private linear regression under label corruption. We show the first near-optimal rate that achieves privacy and robustness to label corruptions simultaneously. When there is no label corruption, our result also improves upon the state-of-the-art method [81] in terms of the condition number  $\kappa$ . Compared to [81], our algorithm has two innovations: 1) we introduce a novel adaptive clipping, which is critical in achieving robustness against label corruptions; and 2) we use full batch gradient descent and a novel convergence analysis to get the near-optimal sample complexity.
We thank Abhradeep Guha Thakurta for helpful discussions while working on this paper. This material is based upon work supported by the National Science Foundation under grants no. 2134012, 2019844, 2112471, and 2229876, and is supported in part by funds provided by the National Science Foundation, by the Department of Homeland Security, and by IBM. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or its federal agency and industry partners.