SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 3.3 (min-norm estimator with no look-alike clustering) Consider the linear response model (2.1), where the features are coming from the GMM (2.4). Under Assumption 1, the followings hold for the min-norm estimator  $\widehat{\pmb{\theta}}$  given by (3.1):
(a) (underparametrized setting) If  $\psi_d \leq 1$ , we have
$$
\operatorname {R i s k} (\widehat {\boldsymbol {\theta}}) \stackrel {\mathcal {P}} {\to} \frac {\sigma^ {2}}{1 - \psi_ {d}}.
$$
(b) (overparametrized setting) If  $\psi_d \geq 1$ , under Assumption 2, the prediction risk of  $\widehat{\pmb{\theta}}$  converges in probability
$$
\operatorname {R i s k} (\widehat {\boldsymbol {\theta}}) \stackrel {\mathcal {P}} {\rightarrow} \sigma^ {2} + \tilde {\gamma} _ {0} ^ {2} + \tilde {\alpha} ^ {\top} (I + \mu^ {2} \operatorname {d i a g} (\pi)) \tilde {\alpha}, \tag {3.4}
$$
where  $\tilde{\gamma}_0$  and  $\tilde{\alpha}$  are given by the following relations:
$$
\tilde {\boldsymbol {\alpha}} = \left(\boldsymbol {I} + \frac {\boldsymbol {I} + \mu^ {2} d i a g (\boldsymbol {\pi})}{\psi_ {d} - 1}\right) ^ {- 1} \boldsymbol {U} _ {\mathrm {s}} ^ {\top} \boldsymbol {\theta} _ {0, \mathrm {s}},
$$
$$
\tilde {\gamma} _ {0} ^ {2} = \frac {1}{\psi_ {d} - 1} \left(\sigma^ {2} + \tilde {\alpha} ^ {\top} (\boldsymbol {I} + \mu^ {2} d i a g (\boldsymbol {\pi})) \tilde {\alpha}\right) + \left(1 - \frac {1}{\psi_ {d}}\right) \left((1 - \rho) r _ {\mathrm {s}} ^ {2} + r _ {\mathrm {n s}} ^ {2}\right).
$$
Example 3.1 (Balanced clusters) In the case of equal cluster prior  $(\pi_1 = \ldots = \pi_k = 1 / k)$ , the risk characterization (3.3) depends on  $\alpha$  only through  $\| \alpha \|_{\ell_2}$  (and likewise, the risk (3.4) depends on  $\tilde{\alpha}$  only through its norm). This significantly simplifies these characterizations.
3.2 Extension to imperfect clustering estimation
In our previous results, we assumed that the underlying cluster memberships of users are known to the learner, so we could concentrate our analysis on the impact of training using anonymous cluster centers. However, in practice, clusters should be estimated from the features and thus includes an estimation error. In our next result, we combine our previous result with a perturbation analysis to bound the risk of the look-alike estimator based on estimated clusters.
Recall matrix  $M \in \mathbb{R}^{d \times k}$  from (2.3), whose columns are the cluster centers. Also, recall the matrix  $\Lambda \in \mathbb{R}^{k \times n}$  whose columns are the one-hot encoding of the cluster memberships. We let  $\widetilde{M}$  and  $\widetilde{\Lambda}$  indicate the estimated matrices, with the cluster estimation error rate  $\delta_{n} := \frac{1}{\sqrt{n}} \| M_{s}\Lambda - \widetilde{M}_{s}\widetilde{\Lambda} \|_{2}$ , where  $\| \cdot \|_{2}$  indicates spectral norm. Note that only the cluster estimation error with respect to the sensitive features matters because in the look-alike modeling only those features are anonymized (replaced by the cluster centers).
Proposition 3.4 Let  $\widetilde{\pmb{X}}^{\top} \coloneqq [(\widetilde{\pmb{M}}_s\widetilde{\pmb{\Lambda}})^{\top}, \pmb{X}_{\mathrm{ns}}^{\top}]$  be the feature matrix after replacing the sensitive features with the estimated cluster centers of users. We also let  $\widetilde{\pmb{\theta}}_L = (\widetilde{\pmb{X}}_L\widetilde{\pmb{X}}_L^{\top})^\dagger \widetilde{\pmb{X}}_L\pmb{y}$  be the look-alike estimator based on  $\widetilde{\pmb{X}}_L$ . Note that  $\widetilde{\pmb{\theta}}_L$  is the counterpart of  $\widehat{\pmb{\theta}}_L$  given by (3.2). Define the cluster estimation error rate  $\delta_n \coloneqq \frac{1}{\sqrt{n}} \| \pmb{M}_s\pmb{\Lambda} - \widetilde{\pmb{M}}_s\widetilde{\pmb{\Lambda}} \|_2$ , and suppose that either of the following conditions hold:
(a) Risk of  $\widehat{\pmb{\theta}}_L$  ( $\sigma = 1$ ,  $r_{\mathrm{ns}} = 2$ )
(d) Risk of  $\widehat{\pmb{\theta}} (\sigma = 1,r_{\mathrm{s}} = 1)$
(b) Risk of  $\widehat{\pmb{\theta}} (\sigma = 1,r_{\mathrm{ns}} = 2)$
(e) Risk of  $\widehat{\pmb{\theta}}_L$  ( $r_{\mathrm{s}} = 1$ ,  $r_{\mathrm{ns}} = 2$ )
Figure 2: Validation of theoretical characterizations of the risks. Curves correspond to (asymptotic) analytical predictions, and dots to numerical simulations (averaged over 20 realizations).
(c) Risk of  $\widehat{\theta}_L$  ( $\sigma = 1$ ,  $r_{\mathrm{s}} = 1$ )
(f) Risk of  $\widehat{\pmb{\theta}} (r_{\mathrm{s}} = 1,r_{\mathrm{ns}} = 2)$
-  $(i)$ $\psi_{d} - \psi_{p} < 0.5$  and  $\delta < \sqrt{1 - (\psi_d - \psi_p)} - \sqrt{\psi_d - \psi_p}$ .
- (ii)  $\psi_d - \psi_p > 2$  and  $\delta < \sqrt{\psi_d - \psi_p - 1} - 1$ .
$$
\operatorname {R i s k} \left(\widetilde {\boldsymbol {\theta}} _ {L}\right) \leq \operatorname {R i s k} \left(\widehat {\boldsymbol {\theta}} _ {L}\right) + C \delta ,
$$
for some constant  $C$  depending on the problem parameters.
In this section, we validate our theory with numerical experiments. We consider GMM with  $k$  clusters, where the centers of clusters are given by  $\mu \pmb{u}_{\ell}$ , for  $\ell \in [k]$ , where  $\pmb{u}_{\ell} \in \mathbb{R}^{d}$  are of unit  $\ell_{2}$ -norm. Also the vectors  $\pmb{u}_{\ell}$  are non-zero only on the first  $p$  entries, and their restriction to these entries form a random orthogonal constellation. Therefore, defining  $\pmb{U} = [\pmb{u}_1, \dots, \pmb{u}_k]$ , we have  $\pmb{U} = \begin{bmatrix} U_{\mathrm{s}} \\ 0 \end{bmatrix}$ , with  $\pmb{U}_{\mathrm{s}}^{\mathrm{T}}\pmb{U}_{\mathrm{s}} = \pmb{I}_{k}$ . In this setting there is no cluster structure on the non-sensitive features and the cluster centers on the sensitive features are orthogonal and of same norm.
Recall the decomposition of the model  $\pmb{\theta}_{0} = (\pmb{\theta}_{0,\mathrm{s}},\pmb{\theta}_{0,\mathrm{ns}})$ , with  $\pmb{\theta}_{0}$  the true underlying model (2.1) and  $\pmb{\theta}_{0,\mathrm{s}}$ ,  $\pmb{\theta}_{0,\mathrm{ns}}$  the components corresponding to sensitive and non-sensitive features. We generate  $\pmb{\theta}_{0,\mathrm{ns}}\in \mathbb{R}^{d - p}$  to have i.i.d standard normal entries and then normalize it to have  $\| \pmb{\theta}_{0,\mathrm{ns}}\|_{\ell_2} = r_{\mathrm{ns}}$ . For  $\pmb{\theta}_{0,\mathrm{s}}$ , we generate  $Z_{1},Z_{2}\sim \mathsf{N}(\mathbf{0}_{p},\mathbf{I}_{p})$ , independently and let
$$
\boldsymbol {\theta} _ {0, \mathrm {s}} = r _ {\mathrm {s}} \sqrt {\rho} \frac {\mathsf {P} _ {U _ {\mathrm {s}}} \boldsymbol {z} _ {1}}{\| \mathsf {P} _ {U _ {\mathrm {s}}} \boldsymbol {z} _ {1} \| _ {\ell_ {2}}} + r _ {\mathrm {s}} \sqrt {1 - \rho} \frac {\left(\boldsymbol {I} - \mathsf {P} _ {U _ {\mathrm {s}}}\right) \boldsymbol {z} _ {2}}{\| \left(\boldsymbol {I} - \mathsf {P} _ {U _ {\mathrm {s}}}\right) \boldsymbol {z} _ {2} \| _ {\ell_ {2}}},
$$
where  $\mathsf{P}_{\mathbf{U}_{\mathrm{s}}} := \mathbf{U}_{\mathrm{s}}\mathbf{U}_{\mathrm{s}}^{\top}$  is the projection onto column space of  $\mathbf{U}_{\mathrm{s}}$ . Therefore,  $\| \pmb{\theta}_{0,\mathrm{s}} \|_{\ell_2} = r_{\mathrm{s}}$  and  $\left\| \mathbf{U}_{\mathrm{s}}^{\top}\pmb{\theta}_{0,\mathrm{s}} \right\|_{\ell_2} = \sqrt{\rho} r_{\mathrm{s}}$ . Note that  $\rho$  quantifies the alignment of the model with the cluster centers, confined to the sensitive features.)
We will vary the values of  $r_{\mathrm{s}}$  and  $r_{\mathrm{ns}}$  in our experiments. We also consider the case of balanced clusters, so the cluster prior probabilities are all equal,  $\pi_{\ell} = 1 / k$ , for  $\ell \in [k]$ . We set the number of
cluster  $k = 3$ , dimension of sensitive features  $p = 200$  and the dimension of entire features vector  $d = 500$ . We also set  $\mu = 5$  and  $\rho = 0.3$ . In our experiments, we vary the sample size  $n$  and plot the risk of  $\widehat{\pmb{\theta}}_L$  and  $\widehat{\pmb{\theta}}$  versus  $\psi_d - \psi_p = (d - p) / n$ . We consider different settings, where we vary  $r_{\mathrm{s}}$ ,  $r_{\mathrm{ns}}$  and  $\sigma$  (noise variance in model (2.1)). In Figure 2, we report the results. Curves correspond to our asymptotic theory and dots to the numerical simulations. (Each dot is obtained by averaging over 20 realizations of that configuration.) As we observe, in all scenarios our theoretical predictions are a perfect match to the empirical performance.
5 When does look-alike clustering improve generalization?
In Section 3, we provided a precise characterization of the risk of look-alike estimator  $\widehat{\theta}_L$  and its counterpart, the min-norm estimator  $\widehat{\theta}$  which utilizes the full information on the sensitive features. By virtue of these characterizations, we would like to understand regimes where the look-alike clustering helps with the model generalization, and the role of different problem parameters in achieving this improvement. Notably, since the look-alike estimator offers  $m$ -anonymity on the sensitive features (with  $m$  the minimum size of clusters), our discussion here points out instances where data anonymization and model generalization are not in-conflict.
We define the gain of look-alike estimator as  $\Delta \coloneqq \mathrm{Risk}(\widehat{\pmb{\theta}}) / \mathrm{Risk}(\widehat{\pmb{\theta}}_L)$  to indicate the gain obtained in generalization via look-alike clustering. For ease in presentation, we focus on the case of balanced clusters (equal priors  $\pi_{1} = \ldots = \pi_{k} = 1 / k$ ), and consider three cases:
- Case 1 ( $\psi_d \leq 1$ ): In this case, both  $\widehat{\theta}_L$  and  $\widehat{\theta}$  are in the underparametrized regime and Theorems 3.1 and 3.3 (a) provide simple closed-form characterization of the risks of  $\widehat{\theta}_L$  and  $\widehat{\theta}$ , by which we obtain
$$
\Delta \stackrel {\mathcal {P}} {\rightarrow} \frac {(1 - \psi_ {d}) ^ {- 1}}{(1 + r _ {\mathrm {s}} ^ {2} / \sigma^ {2}) (1 - \psi_ {d} + \psi_ {p}) ^ {- 1} - \rho r _ {\mathrm {s}} ^ {2} / \sigma^ {2}}.
$$
Define the signal-to-noise ratio  $\mathrm{SNR} = r_{\mathrm{s}} / \sigma$ . Since  $\rho \leq 1$ , it is easy to see that  $\Delta$  is decreasing in the SNR. In particular, as  $\mathrm{SNR} \to 0$ , we have  $\Delta \to (1 - \psi_d + \psi_p) / (1 - \psi_d) > 1$ , which means the look-alike estimator  $\widehat{\theta}_L$  achieves lower risk compared to  $\widehat{\theta}$ . In Figure 3a we plot  $\log (\Delta)$  versus SNR, for several values of  $\psi_p$ . Here we set  $\psi_d = 0.9$  and  $\rho = 0.3$ . As we observe in low SNR, the look-alike estimator has lower risk. Specifically, for each curve there is a threshold for the SNR, below which  $\log (\Delta) > 0$ . Furthermore, this threshold increases with  $\psi_p$ , covering a larger range of SNR where  $\widehat{\theta}_L$  has better generalization.
(a)  $\rho = 0.3$
(b)  $\psi_p = 0.5$
Figure 3: Behavior of gain  $\Delta$  in the generalization of the look-alike estimator  $\widehat{\pmb{\theta}}_L$  over min-norm estimator  $\widehat{\pmb{\theta}}$  as we vary  $\mathrm{SNR} = r_{\mathrm{s}} / \sigma$ . Here,  $\psi_d = 0.9$ ,  $\sigma = 1$ , and we are in the underparametrized regime for both  $\widehat{\pmb{\theta}}_L$  and  $\widehat{\pmb{\theta}}$ .
In Figure 3b we report similar curves, where this time  $\psi_p = 0.5$  and we consider several values of  $\rho$ . As we observe, at fixed SNR the gain  $\Delta$  is increasing in  $\rho$ . This is expected since  $\rho$  measures the alignment of the underlying model  $\theta_0$  with the (left eigenvectors of) cluster centers and so higher  $\rho$  is to advantage of the look-alike estimator which uses the cluster centers instead of individuals' sensitive features.
- Case 2 ( $\psi_d \geq 1$ ,  $\psi_d - \psi_p \leq 1$ ): In this case, the look-alike estimator  $\widehat{\theta}_L$  is in the underparametrized regime, while the min-norm  $\widehat{\theta}$  is in the overparametrized regime. The following theorem uses the characterizations in Theorem 3.1 and and 3.3 (b), and shows that in the low  $\mathrm{SNR} = r_{\mathrm{s}} / \sigma$ , the look-alike estimator  $\widehat{\theta}_L$  has a positive gain. It further shows the monotonicity of the gain with respect to different problem parameters.
Theorem 5.1 Suppose that  $\psi_d \geq 1$  and  $\psi_d - \psi_p \leq 1$ , and consider the case of equal cluster priors. The gain  $\Delta$  is increasing in  $r_{\mathrm{ns}}$  and  $\rho$ , and is decreasing in  $\mu^2 / k$ . Furthermore, under the following condition
$$
\mathrm {S N R} ^ {2} := \left(\frac {r _ {\mathrm {s}}}{\sigma}\right) ^ {2} \leq \frac {1 + \left(\psi_ {d} - 1\right) ^ {- 1} - \left(1 - \psi_ {d} + \psi_ {p}\right) ^ {- 1}}{\left(1 - \psi_ {d} + \psi_ {p}\right) ^ {- 1} + \psi_ {d} ^ {- 1} - 1}, \tag {5.1}
$$
we have  $\Delta \geq 1$ , for all values of other parameters  $(\mu, k, \rho, r_{\mathrm{ns}})$ .
An interpretation based on regularization: We next provide an argument to build further insight on the result of Theorem 5.1. Recall the data model (2.1), where substituting from (2.2) and decomposing over sensitive and non-sensitive features we arrive at
$$
\begin{array}{l} y = \left\langle \boldsymbol {x} _ {\mathrm {s}}, \boldsymbol {\theta} _ {\mathrm {s}} \right\rangle + \left\langle \boldsymbol {x} _ {\mathrm {n s}}, \boldsymbol {\theta} _ {\mathrm {n s}} \right\rangle + \varepsilon \\ = \left\langle \pmb {\mu} _ {\mathrm {s}}, \pmb {\theta} _ {\mathrm {s}} \right\rangle + \left\langle \pmb {z} _ {\mathrm {s}}, \pmb {\theta} _ {\mathrm {s}} \right\rangle + \left\langle \pmb {x} _ {\mathrm {n s}}, \pmb {\theta} _ {\mathrm {n s}} \right\rangle + \varepsilon . \\ \end{array}
$$
Note that  $\langle z_{\mathrm{s}},\theta_{\mathrm{s}}\rangle \sim \mathsf{N}(0,\| \theta_{\mathrm{s}}\|^{2})$  . At low SNR, this term is of order of the noise term  $\varepsilon \sim \mathsf{N}(0,\sigma^2)$  Recall that the look-alike clustering approach replaces the sensitive feature  $x_{s}$  by the cluster center  $\mu_{\mathrm{s}}$  , and therefore drops the term  $\langle z_{\mathrm{s}},\theta_{\mathrm{s}}\rangle$  from the model during the training process. In other words, look-alike clustering acts as a form of regularization which prevents overfitting to the noisy component  $\langle z_{\mathrm{s}},\theta_{\mathrm{s}}\rangle$  , and this will help with the model generalization, together with anonymizing the sensitive features.
In Figure 4a we plot  $\log (\Delta)$  versus  $\mu$  for several values of  $r_{\mathrm{ns}}$ . Here,  $\psi_d = 2$ ,  $\psi_p = 1.7$ ,  $\sigma = 1$ ,  $r_{\mathrm{s}} = 0.5$  and so condition (5.1) holds. As we observe  $\log (\Delta)$  is positive, decreasing in  $\mu$  and also at any fixed  $\mu$ , it is increasing in  $r_{\mathrm{ns}}$ , all of which are consistent with the Theorem 5.1. In Figure 4b, we plot similar curves where this time  $r_{\mathrm{ns}} = 0.2$  and we try several values of  $\rho$ . As we see the look-alike estimator has larger gain  $\Delta$  at larger values of  $\rho$ .
(b)  $r_{\mathrm{ns}} = 0.2$
Figure 4: Behavior of gain  $\Delta$  in the generalization of the look-alike estimator  $\widehat{\theta}_L$  over min-norm estimator  $\widehat{\theta}$  as we vary  $\mu$  the energy of cluster centers.
- Case 3 ( $\psi_d - \psi_p \geq 1$ ): In this case, both  $\widehat{\pmb{\theta}}_L$  and  $\widehat{\pmb{\theta}}$  are in the overparametrized regime. Let us first focus on  $r_{\mathrm{ns}}$ , the energy of the model on the non-sensitive features. Invoking the equations (3.3) and (3.4) and hiding the terms that do not depend on  $r_{\mathrm{ns}}$  in constants  $C_1, C_2$  we arrive at
$$
\Delta \stackrel {(\mathcal {P})} {\rightarrow} \frac {C _ {1} + \left(1 - \frac {1}{\psi_ {d}}\right) r _ {\mathrm {n s}} ^ {2}}{C _ {2} + \left(1 - \frac {1}{\psi_ {d} - \psi_ {p}}\right) r _ {\mathrm {n s}} ^ {2}}.
$$
Therefore,  $\lim_{r_{\mathrm{ns}}\to \infty}\Delta = (1 - \psi_d^{-1}) / (1 - (\psi_d - \psi_p)^{-1}) > 1$ , indicating a gain for the look-alike estimator over  $\widehat{\theta}$ . In Figure 5a, we plot  $\log (\Delta)$  versus  $r_{\mathrm{ns}}$  for several values of  $\psi_p$ . As we observe, when  $r_{\mathrm{ns}}$  is large enough we always have a gain, which is increasing in  $\psi_p$ .
(a)  $r_{\mathrm{s}} = 0.5$
(b)  $r_{\mathrm{ns}} = 2$
Figure 5: Behavior of gain  $\Delta$  versus  $r_{\mathrm{ns}}$  and  $\mathrm{SNR} \coloneqq r_{\mathrm{s}} / \sigma$  for several values of  $\psi_p$ . Here,  $\psi_d = 4$ ,  $\sigma = 0.1$ ,  $\rho = 0.3$ ,  $\mu = 5$ ,  $k = 5$ . Here, we are in the overparametrized regime for both  $\widehat{\pmb{\theta}}_L$  and  $\widehat{\pmb{\theta}}$ .
Figure 6: Behavior of gain  $\Delta$  versus SNR for the nonlinear model described in Section 6. At small SNR, we observe a positive gain (lower risk of look-alike estimator  $\widehat{\theta}_L$  compared to  $\widehat{\theta}$ ).
We next consider the effect of  $\mathrm{SNR} = r_{\mathrm{s}} / \sigma$ . In Figure 5b we  $\log (\Delta)$  versus SNR, for several values of  $\psi_p$ . Similar to the underparametrized regime, we observe that in low SNR, the look-alike estimator has better generalization  $(\log (\Delta) > 0)$ .
6 Beyond linear models
In previous section, we used our theory for linear models to show that at low SNR, look-alike modeling improves model generalization. We also provided an insight for this phenomenon by arguing that look-alike modeling acts as a form of regularization and avoids over-fitting at low SNR regime. In this section we show empirically that this phenomenon also extends to non-linear models.
Consider the following data generative model:
$$
y \sim \operatorname {B i n o m i a l} (N, p _ {\boldsymbol {x}}), \quad p _ {\boldsymbol {x}} = \frac {1}{1 + \exp (- \langle \boldsymbol {x} , \boldsymbol {\theta} _ {0} \rangle + \varepsilon)},
$$
where  $\varepsilon \sim \mathsf{N}(0,\sigma^2)$ . We construct the model  $\theta_0 = (\theta_{0,\mathrm{s}},\theta_{0,\mathrm{ns}})$  similar to the setup in Section 4. We set  $n = 200$ ,  $d = 180$ ,  $k = 3$ ,  $\mu = 5$ ,  $\sigma = 1$ ,  $\rho = 0.3$ ,  $r_{\mathrm{ns}} = 2$  and  $N = 1000$  (number of trials in Binomial distribution). We vary SNR by changing  $r_{\mathrm{s}}$  in the set  $\{0.1,0.3,\dots,1.9\}$ . The estimators  $\widehat{\pmb{\theta}}$  and  $\widehat{\pmb{\theta}}_L$  are obtained by fitting a GLM with logit link function and binomial distribution. We compute the risks of  $\widehat{\pmb{\theta}}$  and  $\widehat{\pmb{\theta}}_L$  by averaging over a test set of size  $50K$ . In Figure 6, we plot the gain  $\log (\Delta)$  versus  $r_{\mathrm{s}}$  where each data point is by averaging over 50 different realizations of data. As we observe at low SNR,  $\log (\Delta) > 0$  indicating that the look-alike estimator  $\widehat{\pmb{\theta}}_L$  obtains a lower risk than the min-norm estimator.
This work is supported in part by the NSF CAREER Award DMS-1844481 and the NSF Award DMS-2311024.
[1] F. Abu-Khzam, C. Bazgan, K. Casel, and H. Fernau. Building clusters with lower-bounded sizes. In 27th International Symposium on Algorithms and Computation (ISAAC 2016). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2016.
[2] G. Aggarwal, T. Feder, K. Kenthapadi, R. Motwani, R. Panigrahy, D. Thomas, and A. Zhu. Approximation algorithms for k-anonymity. Journal of Privacy Technology, 2005112001:400, 2005.
[3] G. Aggarwal, R. Panigrahy, T. Feder, D. Thomas, K. Kenthapadi, S. Khuller, and A. Zhu. Achieving anonymity via clustering. ACM Transactions on Algorithms (TALG), 6(3):1-19, 2010.
[4] P. Bartlett, Y. Freund, W. S. Lee, and R. E. Schapire. Boosting the margin: A new explanation for the effectiveness of voting methods. The annals of statistics, 26(5):1651-1686, 1998.
[5] M. Belkin, S. Ma, and S. Mandal. To understand deep learning we need to understand kernel learning. In International Conference on Machine Learning, pages 541-549. PMLR, 2018.
[6] G. Brown, M. Bun, V. Feldman, A. Smith, and K. Talwar. When is memorization of irrelevant training data necessary for high-accuracy learning? In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, pages 123–132, 2021.
[7] J.-W. Byun, A. Kamra, E. Bertino, and N. Li. Efficient k-anonymization using clustering techniques. In Advances in Databases: Concepts, Systems and Applications: 12th International Conference on Database Systems for Advanced Applications, DASFAA 2007, Bangkok, Thailand, April 9-12, 2007. Proceedings 12, pages 188-200. Springer, 2007.
[8] Z. Deng, A. Kammoun, and C. Thrampoulidis. A model of double descent for high-dimensional binary linear classification. Information and Inference: A Journal of the IMA, 11(2):435-495, 2022.
[9] H. Ding, L. Hu, L. Huang, and J. Li. Capacitated center problems with two-sided bounds and outliers. In Algorithms and Data Structures: 15th International Symposium, WADS 2017, St. John's, NL, Canada, July 31-August 2, 2017, Proceedings 15, pages 325-336. Springer, 2017.
[10] A. Epasto, M. Mahdian, V. Mirrokni, and P. Zhong. Massively parallel and dynamic algorithms for minimum size clustering. In Proceedings of the 2022 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 1613-1660. SIAM, 2022.
[11] V. Feldman. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pages 954–959, 2020.
[12] V. Feldman and C. Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881-2891, 2020.
[13] Y. Gordon. On milman's inequality and random subspaces which escape through a mesh in  $r^n$ . In Geometric aspects of functional analysis, pages 84-106. Springer, 1988.
[14] H. Hassani and A. Javanmard. The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regression. arXiv preprint arXiv:2201.05149, 2022.
[15] A. Javanmard and M. Soltanolkotabi. Precise statistical analysis of classification accuracies for adversarial training. The Annals of Statistics, 50(4):2127-2156, 2022.
In this supplementary, we first provide an overview of our proof techniques in Appendix A and then in Appendix B provide the proofs of theorems and technical lemmas stated in the main paper.
A Overview of proof techniques
Our analysis of the generalization error is based on an extension of Gordon's Gaussian process inequality [13], called Convex-Gaussian Minimax Theorem (CGMT) [30]. Here, we outline the general steps of this framework and refer to the supplementary for complete details and derivations.
Consider the following two Gaussian processes:
$$
\boldsymbol {X} _ {\boldsymbol {u}, \boldsymbol {v}} := \boldsymbol {u} ^ {\top} \boldsymbol {G} \boldsymbol {v} + \psi (\boldsymbol {u}, \boldsymbol {v}),
$$
$$
\boldsymbol {Y} _ {\boldsymbol {u}, \boldsymbol {v}} := \| \boldsymbol {u} \| _ {\ell_ {2}} \boldsymbol {g} ^ {\mathsf {T}} \boldsymbol {v} + \| \boldsymbol {v} \| _ {\ell_ {2}} \boldsymbol {h} ^ {\mathsf {T}} \boldsymbol {u} + \psi (\boldsymbol {u}, \boldsymbol {v}),
$$
where  $\pmb{G} \in \mathbb{R}^{n \times d}$ ,  $\pmb{g} \in \mathbb{R}^n$  and  $\pmb{h} \in \mathbb{R}^d$ , all have i.i.d standard normal entries. Further,  $\psi: \mathbb{R}^d \times \mathbb{R}^n \to \mathbb{R}$  is a continuous function, which is convex in the first argument and concave in the second argument.
Given the above two processes, consider the following min-max optimization problems, which are respectively referred to as the Primary Optimization (PO) and the Auxiliary Optimization (AO) problems:
$$
\Phi_ {\mathrm {P O}} (\boldsymbol {G}) := \min  _ {\boldsymbol {u} \in \boldsymbol {S} _ {\boldsymbol {u}}} \max  _ {\boldsymbol {v} \in \boldsymbol {S} _ {\boldsymbol {v}}} \boldsymbol {X} _ {\boldsymbol {u}, \boldsymbol {v}}, \tag {A.1}
$$
$$
\Phi_ {\mathrm {A O}} (\boldsymbol {g}, \boldsymbol {h}) := \min  _ {\boldsymbol {u} \in \boldsymbol {S} _ {\boldsymbol {u}}} \max  _ {\boldsymbol {v} \in \boldsymbol {S} _ {\boldsymbol {v}}} \boldsymbol {Y} _ {\boldsymbol {u}, \boldsymbol {v}}. \tag {A.2}
$$
The main result of CGMT is to connect the above two random optimization problems. As shown in [30](Theorem 3), if  $S_{u}$  and  $S_{v}$  are compact and convex then, for any  $\lambda \in \mathbb{R}$  and  $t > 0$