Paper: Benign Overfitting in Two-Layer ReLU Convolutional Neural Networks for XOR Data (ICLR-2024)

Reviewer comment (Novelty):
It would be good to cite Hu \& Lu, 2022 and Xiao et al., 2022 alongside Misiakiewicz (2022); Xiao & Pennington (2022) for completeness when discussing multiple descents in kernel ridge regression in the related works.

Author response:
**A1**: Thank you for pointing out the missing references. We have cited them in the revision.

Reviewer comment (Theory):
In condition 3.1, is a max or min missing in the $\Omega$ notation?

Author response:
**A3**: Thanks for pointing it out.  We have corrected the typo and the correct version should be $d= \widetilde{\Omega}(\mathrm{max}\\{n^2,n||\boldsymbol{\mu}||_2^2\sigma_p^{-2}\\})\cdot \mathrm{polylog}(1/\varepsilon)\cdot \mathrm{polylog}(1/\delta)$.

Reviewer comment (Theory):
The paper studies two-layer CNNs, which is a valid and interesting model to study. However, to my understanding the analysis is for networks convolving over only 2 input patches, which seems very restrictive. In this case, it is not clear what is the purpose of studying CNNs instead of MLPs. The authors should explain why they made this choice of architecture? Do similar results hold for standard MLPs, and if so, why do they choose to focus on 2-patch CNNs? Can the results be extended to CNNs with larger number of patches?

Author response:
**A1**: The two-layer CNN model considered in this paper follows the same definition as in [1,2,3]. Considering a case with two patches is just for the simplicity of the presentation, and our analysis can be easily extended to the case where the data inputs consist of multiple patches. The proof will be almost identical to the current proof given in the paper.

The primary goal of this paper is to explore the capabilities of non-linear neural networks in learning complex and "non-linear" data. We believe our analysis can be extended to MLPs with some necessary modifications, and the condition for benign overfitting can still be derived as a function of signal strength, noise level, sample size and data dimension, similar to what we have presented in Theorem 3.2.

Reviewer comment (Theory):
Some details about the choice of input distribution are not clear. Specifically, in the choice of the randomly generated patch, why is the Gaussian covariance matrix chosen to be dependent on the "true" XOR vectors? Can you justify why such choice of random vectors is natural? Would a similar result hold for standard Normal distribution (or other more natural distributions)?

Author response:
**A2**: We define the covariance matrix of the Gaussian vector in this way so that the noise vectors are orthogonal to the signal vectors. This is to simplify the setting and give a clearer and more refined analysis. Similar definitions have been studied in [1,2,3]. If the noise is given by standard Gaussian, the general proof idea does not change but we will need to rely on the fact that in the high dimensional setting (when $d$ is sufficiently large), the noise and signal vectors will be almost orthogonal to each other (with high probability). Therefore, the proof details will be more complicated and stronger assumptions on $d$ may be required.

Reviewer comment (Theory):
In stating the conditions for the theoretical analysis (Condition 3.1), it is better to separate conditions on the learning problem (which are essentially assumptions about the "world"), from choices of learning parameters (learning rate, initialization) and sample complexity. The way the conditions are stated, it seems that e.g. the dimension should grow with the sample complexity. It makes more sense to first assume a problem is given with some fixed dimension, and then adapt the sample size to the problem (in which case, does this mean that the sample size needs to be upper bounded?).

Author response:
**A3**: We clarify that this paper indeed studies the case where the dimension $d$ grows with the sample size $n$. This is because the goal of this paper is to study the phenomena of benign and harmful overfitting. Achieving overfitting necessitates the growth of dimension $d$ alongside sample size $n$. And we indeed need to assume that the sample size $n$ is upper bounded by a quantity related to $d$ so that the problem setting allows overfitting. It is noteworthy that the setting where $d$ increases with $n$ is also a common scenario studied in learning theory (e.g., [4,5,6]).

Reviewer comment (Writing):
First sentence in the introduction: highly parameterized => over-parameterized.

Author response:
**A5**: Thank you for pointing out the typos. We have revised the paper and fixed the typos.

Reviewer comment (Theory):
Compared to the results in the Classic XOR regime (Section 3.1), the results in Section 3.2 are of greater interest. However, this part of the results relies on strong assumptions. Firstly, the first point of Condition 3.3 is a stronger high-dimensional assumption compared with other benign overfitting papers such as [1]. Then, considering Condition 3.3, we can derive the conditions in the second point of Theorem 3.4 as $\|\| \mu \|\|_2^2 \geq \frac{m^5 n^2 \sigma_p^2}{(1-\cos \theta)^2}$ which is much stronger than [1].

Author response:
**A1**: We believe that the classic XOR regime (Section 3.1) and the asymptotically challenging XOR regime (Section 3.2) are equally important. As commented in our paper, Section 3.1 studies the regime that covers the standard XOR problem, and therefore it is more appropriate to compare the results in Section 3.1 with previous works. In comparison, Section 3.2 discusses a less standard, but more challenging setting, which requires different theoretical analyses. Therefore, we believe that the two regimes are both important.

**A2**: We respectfully disagree with your comment. We strongly believe this comparison is unfair, and the weakness pointed out here is unreasonable. Our work investigates the XOR-type data, where the Bayes optimal classifier is nonlinear. For this XOR-type learning problem, the difficulty of learning (or how the noiseless samples are separated) depends on both the norms of the signal vectors and the angle between them. And the setting studied in Section 3.2 is a particularly challenging setting where the angle can be asymptotically small. In comparison, [1] only studied a much simpler learning problem where the Bayes-optimal classifier is linear, and in the setting of [1], there is no such concept of the angle between signals. Therefore, the settings in [1] and our work are not comparable, and the comparison between the rates in [1] and in Section 3.2 is unfair.

Reviewer comment (Theory):
I do not agree with the author's claim that they have discovered a sharp phase transition between benign overfitting and harmful overfitting, as in the case of Theorem 3.2, the transition between the second and third points may not necessarily be a phase transition but rather a continuous change.

Author response:
**A3**:  We would like to explain that the terminology "sharp phase transition" has been used in existing works [3,4] to describe the same type of transition between benign and harmful overfitting. Therefore, "sharp phase transition" is not invented by this paper, and we use this terminology mainly to highlight that the same type of transition between benign and harmful overfitting also happens in learning the XOR-type data.

We have also clarified in the revision (on Page 5) that by  "sharp phase transition", we mean that in "boundary cases", an increase of the sample size $n$ or the signal strength $|| \boldsymbol{\mu} ||\_2$ by a logarithmic factor is sufficient to change the test error from a constant level to the optimal rate $p+o(1)$.

**Rebuttal 2 (ID: rBqdvC1b5a)**

Dear reviewer:

Thanks for your further response.  We guess that you have concerns on the gap between $C_1$ and $C_3$. In fact, Theorem 3.2 can be modified to the version where the conditions for the second and third part are given using a same absolute constant $C_1$. This modification is simply done by adjusting the constants. We have revised the statement of Theorem 3.2 and the proof (Appendix F) accordingly in the latest version.

Hope this can address your concerns. We are happy to answer any further questions.

Authors

Reviewer comment (Novelty):
Could the author discuss the relationship and differences between this work and [2]? And is there any connection between this work and the Grokking?

Author response:
**A4**: We would like to first point out that our paper was submitted to ICLR before the appearance of [2] on arXiv, and it is generally unreasonable to make a comparison between our paper and [2]. To answer your questions, we list several key differences between our work and [2] as follows:

Differences in problem settings:

- Our data model covers varying angles between the signal vectors, while [2] assumes that the signal vectors in the XOR data are orthogonal (therefore, [2] may be more comparable with our results in Section 3.1). Moreover, we consider the setting where the signals and noises are on different patches, while [2] considers the setting where the signals and noises are added together.

- We analyze CNN models, while [2] studies fully connected neural networks.

Differences in results:

- [2] requires $||\boldsymbol{\mu} ||_2^2 = \Omega( n^{0.51} \sqrt{d} )$, and the signal strength needs to increase as the sample size increases. In comparison, our results indicate that if the sample size $n$ increases, then a smaller signal strength $|| \boldsymbol{\mu} ||_2$ may be sufficient to achieve benign overfitting.

- [2] does not give any direct guarantees of training loss minimization, while our results show that gradient descent can achieve an arbitrarily small training loss.

- The results in [2] are for neural networks trained up to $\sqrt{n}$ iterations. In comparison, our results are for CNNs trained until convergence.


- [2] establishes an upper bound of the test error of the trained network, while we establish upper and lower bounds of the test error under complimentary conditions, demonstrating that these upper and lower bounds are both tight.

References:

[1] Spencer Frei, Niladri S Chatterji and Peter Bartlett. Benign overfitting without linearity: Neural network classifiers trained by gradient descent for noisy linear data. COLT, 2022.

[2] Zhiwei Xu, Yutong Wang, Spencer Frei, Gal Vardi and Wei Hu. Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data. arxiv 2310.02541.

[3] Yiwen Kou, Zixiang Chen, Yuanzhou Chen and Quanquan Gu. Benign overfitting for two-layer relu networks. COLT, 2023.

[4] Yuan Cao, Zixiang Chen, Mikhail Belkin and Quanquan Gu. Benign overfitting in two-layer convolutional neural networks. NeurIPS, 2022.

Reviewer comment (Experiments):
It is unclear how the test error changes as the number of samples, and CNN width change.

Author response:
Our results in Theorem 3.2 suggest that as the sample size increases, the test error should decrease. Regarding the CNN width $m$, Theorem 3.2 indicates that the test error is not sensitive to $m$ as long as $m$ is large enough ($m = \Omega(\log(n/\delta))$ as is given in Condition 3.1). This is because in learning XOR problems,  the test error is mainly affected by how much "signal" (the $\boldsymbol{\mu}$ vectors in Definitions 2.1 and 2.2) the CNN filters can learn from the training data. As long as $m$ is large enough to ensure a diverse enough random initialization of the CNN filters, further increasing $m$ does not significantly change the training dynamics of the CNN model. We also added a new set of experiments in Appendix I to demonstrate the relation between $m$ and the test error. As shown in Figure 3, for $m \geq 3$, the test error does not significantly change with $m$. This backs up our claim that as long as $m$ is large enough, the test error is not sensitive to $m$.

It is worth noting that the conditions of benign and harmful overfitting in Theorem 3.4 explicitly depend on $m$. This is because our proof technique for the asymptotically challenging XOR regime can only cover a specific initialization scale $\sigma_0$ that depends on $m$. For general initialization scales that do not depend on $m$, we believe that the test error should not be sensitive to $m$.

Reviewer comment (Experiments):
As the paper studies the benign overfitting of neural networks, a figure of test accuracy versus the neural network width $m$ should be included.

Author response:
Our results in Theorem 3.2 suggest that as the sample size increases, the test error should decrease. Regarding the CNN width $m$, Theorem 3.2 indicates that the test error is not sensitive to $m$ as long as $m$ is large enough ($m = \Omega(\log(n/\delta))$ as is given in Condition 3.1). This is because in learning XOR problems,  the test error is mainly affected by how much "signal" (the $\boldsymbol{\mu}$ vectors in Definitions 2.1 and 2.2) the CNN filters can learn from the training data. As long as $m$ is large enough to ensure a diverse enough random initialization of the CNN filters, further increasing $m$ does not significantly change the training dynamics of the CNN model. We also added a new set of experiments in Appendix I to demonstrate the relation between $m$ and the test error. As shown in Figure 3, for $m \geq 3$, the test error does not significantly change with $m$. This backs up our claim that as long as $m$ is large enough, the test error is not sensitive to $m$.

It is worth noting that the conditions of benign and harmful overfitting in Theorem 3.4 explicitly depend on $m$. This is because our proof technique for the asymptotically challenging XOR regime can only cover a specific initialization scale $\sigma_0$ that depends on $m$. For general initialization scales that do not depend on $m$, we believe that the test error should not be sensitive to $m$.

Reviewer comment (Experiments):
Do we have any requirements on the number difference between two different data points?

Author response:
The balance among different types of data is implicitly given in Definitions 2.1 and 2.2. Following the definition of $\mathcal{D}\_{\mathrm{XOR}}(\mathbf{a},\mathbf{b})$ in Definition 2.1, it is easy to see that the different types of data are generated equally likely.

Our results in both the "classic XOR regime" and the "asymptotically challenging XOR regime" rely on the balance of the data. In the "classic XOR regime", we believe that Lemma 4.1 will still hold with unbalanced data. However, the subsequent analysis on the training dynamics will be significantly affected by unbalanced data. For instance, Lemma E.1 in Appendix E, which is a key lemma to show how much "signals" (the $\boldsymbol{\mu}$ vectors in Definitions 2.1 and 2.2) the CNN filters can learn from the training data, will no longer be valid. Similarly, in the "asymptotically challenging XOR regime", when the data are highly unbalanced, the results in Lemma G.17 will no longer hold.

Reviewer comment (Theory):
In p.5, the authors claim that  
> However, we note that our data model with $\|\mu\|_2 = \sigma_p = \Theta(1)$ would represent the same signal strength and noise level as in the 2-sparse parity problem, and in this case, our result implies that two-layer ReLU CNNs can achieve near Bayes-optimal test error with $n=\omega(d)$, which is better than the necessary condition $n=\omega(d^2)$ for kernel methods.  

However, as far as I understand, this explanation is wrong; by letting $\|\mu\|_2 = \sigma_p = \Theta(1)$, Condition 3.1 implies $d=\Omega(n^2)$ is required, while benign overfitting (in Theorem 3.2) requires $n =\Omega(d)$. To solve this issue, either $\|\mu\|_2$ or $\sigma_p$ should be larger than $O(1)$, in the asymptotic limit when $d$ and $n$ diverge. In other words, the signal-to-noise ratio should be larger than the standard analysis on the XOR data ([Ji & Telgarsky (2020)](https://arxiv.org/abs/1909.12292); [Barak et al. (2022)](https://arxiv.org/abs/2207.08799); [Telgarsky (2023)](https://arxiv.org/pdf/2208.02789.pdf)), although [Ba et al. (2023)](https://openreview.net/forum?id=HlIAoCHDWW) considers the anisotropic input data in the regression problem.

Author response:
**A1**: Thank you for pointing this out. We have revised the manuscript to emphasize that the comparison is not rigorous because Condition 3.1 requires $d = \tilde\Omega(n^2)$, which excludes the case $n=\omega(d)$. We have also discussed that the condition $d = \tilde\Omega(n^2)$ in Condition 3.1 is only a technical requirement to enable our current theoretical analysis (e.g. the proof of Lemma D.3). Therefore, Condition 3.1 can potentially be relaxed to cover the case $n=\omega(d)$ with different analysis tools, which can be an interesting future work direction. We have also cited the missing references in our discussion.

Reviewer comment (Theory):
Suppose that we take $d=\Theta(n^2)$ and $\sigma_p=\Theta(1)$ in Condition 3.1. Then, the initialization scale is $d^{-0.75}$, which is relatively small. This means that the output scale of the network at initialization is $o(1)$.

Author response:
**A2**: We assume a relatively small initialization to ensure sufficient "feature learning":
If the initialization is sufficiently small, the training updates (which consist of features from training data) of neural network parameters can quickly dominate their random initialization, thus during  training, the neural network can achieve sufficient feature learning. On the other hand, if the initialization is large, then it may dominate the training updates, which can lead to the "Neural Tangent Kernel (NTK) regime" where the model essentially uses kernels or random features [1] to fit the training data. Our analysis in this paper does not fall in the NTK regime, and we study the case where training updates dominate the random initialization. This is why we assume that the initialization is sufficiently small. Similar conditions of small initialization have been widely assumed such as [2,3,4,5].

Reviewer comment (Novelty):
I am basically satisfied with the first result (Theorem 3.2), but I do not see necessity to extend the problem into more general case, where two axes of the input XOR distribution is much closer.

Author response:
**A3**: Theorem 3.4 aims to study a more challenging case where the data are less separated. For simplicity, here we can consider a case without label flipping. In this case, the angle $\theta$ can be understood as the metric of the margin of the data, and a smaller angle means a smaller margin. For this more challenging case, our proof technique in Theorem 3.2 can no longer be applied, and therefore Theorem 3.4 and its proof may also have their unique technical value. This is why we also cover this setting in our paper.

Reviewer comment (Theory):
I want to know how the small initialization (as I pointed out) is required in the analysis.

Author response:
**A2**: We assume a relatively small initialization to ensure sufficient "feature learning":
If the initialization is sufficiently small, the training updates (which consist of features from training data) of neural network parameters can quickly dominate their random initialization, thus during  training, the neural network can achieve sufficient feature learning. On the other hand, if the initialization is large, then it may dominate the training updates, which can lead to the "Neural Tangent Kernel (NTK) regime" where the model essentially uses kernels or random features [1] to fit the training data. Our analysis in this paper does not fall in the NTK regime, and we study the case where training updates dominate the random initialization. This is why we assume that the initialization is sufficiently small. Similar conditions of small initialization have been widely assumed such as [2,3,4,5].

Reviewer comment (Theory):
In the proof sketch, the authors introduced "loss derivative comparison" technique. If the authors could provide more explanations about the motivation of such technique, I would appreciate it.

Author response:
**A4**: The loss derivative comparison results serve as the key to analyze the training of the CNN with equation 4.1. Specifically, the result of loss derivative comparison in the "classic XOR regime" is given in Lemma 4.1 and the result in the "asymptotically challenging XOR regime" is given by Lemma 4.4. 

The motivation of these results is that in equation 4.1, we hope to show that the terms without the $\cos\theta$ factor can dominate the terms with the $\cos\theta$ factor, which can significantly simplify our analysis. For example, with Lemma 4.1, we can see that in the "classic XOR regime" where $\cos\theta < 1/2$, for any $i,i'\in[n]$, a non-zero $\ell'^{(t)}\_i\cdot\boldsymbol{1}\\{\langle\mathbf{w}^{(t)}\_{+1,r},\boldsymbol{\mu}\_i\rangle>0\\}||\boldsymbol{\mu}||\_2^2$ will always dominate $\ell'^{(t)}\_{i'}\cdot \boldsymbol{1}\\{\langle\mathbf{w}^{(t)}\_{+1,r},\boldsymbol{\mu}\_{i'}\rangle>0\\}||\boldsymbol{\mu}||\_2^2\cos\theta$. For the "asymptotically challenging XOR regime", since a larger $\cos\theta$ that is close to $1$ is considered, we need the tighter bound on the ratios between loss derivatives given in Lemma 4.4. 

We have added more explanations of the motivation of "loss derivative comparison" in the revision.