Title: The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions
Venue: NeurIPS 2024
Accepted: yes
Keywords: Implicit Bias

Adversarial Robustness

Dynamic Analysis
Abstract: The implicit bias of gradient descent has long been considered the primary mechanism explaining the superior generalization of over-parameterized neural networks without overfitting, even when the training error is zero. However, the implicit bias toward adversarial robustness has rarely been considered in the research community, although it is crucial for the trustworthiness of machine learning models. To fill this gap, in this paper, we explore whether consecutive layers collaborate to strengthen adversarial robustness during gradient descent. By quantifying this collaboration between layers using our proposed concept, co-correlation, we demonstrate a monotonically increasing trend in co-correlation, which implies a decreasing trend in adversarial robustness during gradient descent. Additionally, we observe different behaviours between narrow and wide neural networks during gradient descent. We conducted extensive experiments that verified our proposed theorems.

The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions

Abstract

The implicit bias of gradient descent has long been considered the primary mechanism explaining the superior generalization of over-parameterized neural networks without overfitting, even when the training error is zero. However, the implicit bias toward adversarial robustness has rarely been considered in the research community, although it is crucial for the trustworthiness of machine learning models. To fill this gap, in this paper, we explore whether layers in neural networks collaborate to strengthen adversarial robustness during gradient descent. By quantifying this collaboration between layers using our proposed concept, co-correlation , we demonstrate a monotonically increasing trend in co-correlation, which implies a decreasing trend in adversarial robustness during gradient descent. Additionally, we observe different behaviours between narrow and wide neural networks during gradient descent. We conducted extensive experiments that verified our proposed theorems.

1 Introduction

As Artificial Intelligence (AI) has been widely applied in many industrial sectors, understanding the theoretical properties behind modern machine learning models is important, especially for neural networks due to their black-box nature. One such property is implicit bias , stemming from the phenomenon where over-parameterized neural networks, trained in a gradient descent manner, often exhibit great generalization without over-fitting. This implicit bias of gradient descent is often explained as steering neural networks towards solutions characterized by max-margin [3, 23, 12].

Another intriguing phenomenon is the existence of adversarial examples - imperceptible perturbations of inputs that alter classification results. Apart from existing work on attacks -algorithms for generating adversarial examples [5, 39, 37], defences against attacks, e.g., adversarial training [25, 36] and distillation [28], and verification [40, 34] to identify safe regions guaranteeing the absence of adversarial examples, recently some works are aiming at a theoretical understanding behind adversarial robustness. However, a principled way to comprehend the core contributors to the vulnerability of neural networks, especially a theoretical understanding of their relation to generalization capabilities, remains fragmented. This fragmentation is largely due to the intricate nature of neural networks, where robustness is interconnected with many factors spanning across input data distribution [33, 14], sampling complexity [1, 27], optimization techniques [25], weight initialization strategies [41], and model capacity and architectures [32, 2, 18, 35]. Not to mention that only very few works can address both generalization and adversarial robustness in a uniform framework. One such research by Frei et al. [13] investigates the implicit bias concerning both generalization and adversarial robustness ,

asserting that while implicit bias leads to solutions with improved generalization, it results in weaker adversarial robustness. However, this work ignores the architectural factor of neural networks and is hard to generalize to neural networks with more than two layers.

Inspired by this work and to investigate whether neural network layers collaborate against adversarial examples during training, this paper first adopts the novel concept of Dirichlet energy-originating from Partial Differential Equations (PDEs) to assess the variability of a function [11]-to evaluate the adversarial robustness of neural networks. We then theoretically demonstrate that Dirichlet energy serves as an effective measure of adversarial risk. By decomposing the Dirichlet energy across the entire neural network into its constituent layers, we can quantify the interactions between adjacent layers concerning adversarial robustness. We term this interaction collaboration correlation (co-correlation) and find that this metric reflects 'alignments' in feature selection between neighbouring layers. Furthermore, we conduct a dynamic analysis of co-correlation in two-layer MLPs, demonstrating with high probability a monotonic increase under gradient descent, which indicates diminishing adversarial robustness. Additionally, our experiments show that two-layer MLPs with small widths tend to enhance their performance through strengthened co-correlation, a pattern not observed in wide two-layer MLPs. Our key contributions in this paper can be summarized as follows:

To the best of our knowledge, this work is the first to study the implicit bias of interaction between layers. We have quantified the interactions between adjacent layers and theoretically demonstrated that co-correlation between layers strengthens during gradient descent in neural networks under mild assumptions, suggesting that it not only fails to collaborate against adversarial perturbations but may even hinder resistance to them during gradient descent.

We demonstrate how neural networks with a large width differ in behaviour from the neural network of a small width, showing that MLPs with larger widths exhibit more resistance to increased co-correlation and, therefore, are more adversarial robust, which is complementary for the work by Dohmatob and Bietti [8].

Extensive experiments have been conducted to validate our proposed framework. By controlling the weight initialization, a perspective also suggested by Zhu et al. [41], we challenge the argument, as proposed by Huang et al. [18], that a wide neural network does not necessarily lead to better adversarial robustness, through the lens of cross-layer collaboration.

2 Related Works

2.1 Implicit Bias of Gradient Descent

The mystery of over-parameterized neural network trained with gradient descent manner hardly over-fitting has long been studied. Chizat and Bach [3] study the two-layer neural network with infinite width and homogeneous activations, showing that gradient flow can be characterized as a max-margin classifier on exponentially tailed losses. Lyu et al. [24], Sarussi et al. [29] study the two-layer Leaky ReLU neural network on linearly separable data and claim that networks converge to a max-margin linear predictor by gradient descent manner. Frei et al. [12] confirms those claims on high-dimensional nearly orthogonal data.

Lyu and Li [23], Ji and Telgarsky [19] claims that the homogeneous neural networks with exponentially-tailed classification losses converge to a KKT point of a maximum-margin problem. Kunin et al. [21] extend these results to a more boarder family of quasi-homogeneous neural networks. A more recent research [13] considers both generalization and robustness for two-layer ReLU neural networks, arguing that gradient descent is biased towards solutions that generalise well but are more vulnerable against adversarial examples, even the neural network is highly over-parameterized.

2.2 Theoretical Investigation of Adversarial Robustness

Since the phenomenon of adversarial examples has been discovered [15], various works have been proposed to understand the theoretical fundamentals behind it, especially for neural networks. Some researchers argue that the source of adversarial vulnerability comes from the input data [33, 8, 31, 26, 14, 7, 30, 1, 27]. The more recent researches investigate the fragility of neural networks from an architectural perspective. Simon-Gabriel et al. [32] study the vulnerability of feed-forward neural

networks measured by L p norm of the loss function w.r.t. input data, suggesting that the vulnerability increases with input dimension independent of model structures. Daniely and Shacham [6] examined the ReLU neural network characterized by decreasing dimensions at each layer. They asserted that the manifestation of adversarial robustness is intrinsically tied to the network's architecture, which contrasts with the propositions put forth by Simon-Gabriel et al. [32]. Bubeck et al. [2] expanded the findings of Daniely and Shacham's work on two-layer neural networks from an "under-complete case" scenario to an "over-complete" one where the number of neurons surpasses the input dimension. They further broadened the conclusions drawn by Daniely and Shacham [6] and Bubeck et al. [2] to encompass Deep ReLU networks, hinting at a crucial role played by bottleneck layers in these networks. Zhu et al. [41], instead of merely considering random weights as the standard configuration, conducted a comprehensive analysis of the effects of weight initialization on adversarial robustness.

Unlike previous studies that focus solely on the overall assessment of neural networks while overlooking layer interactions, our research examines the synergistic involvement between layers within neural networks, taking into account both weight initialization and optimization.

3 Preliminary

3.1 General Setting

We follow the binary classification setting where the input data is X ⊆ R d , with the label Y ⊆ { 0 , 1 } . Given data set D = { ( x i , y i ) } n i =1 drawn from an unknown probability measure P on X × Y and the neural network f : X × Θ → R , where Θ denotes the set of parameters, our objective is to optimize f by updating the weights with gradient descent method such that it can predict the label accurately. The prediction result is shown in Equation (1).

where sig is the sigmoid function, i.e., sig ( x ) = 1 / (1 + e -x ) . We use Binary Cross-Entropy (BCE) loss in Equation (2) as our loss function. For simplicity, we denote u i = f ( x i , W ) , i ∈ [ n ] as the output of the neural network for input x i , where [ n ] = { k ∈ N + | k ≤ n } .

3.2 Neural Networks and Adversarial Risk

Our exploration starts from a basic linear model, then to Multilayer Perceptrons (MLP) . We provide the proof for both linear and 2-layer MLPs, which can be extended to MLPs with more layers. They are defined as

where x ∈ X is the input data, W ∈ R m × d denotes the linear transformation, m is the width of the networks. σ denotes the element-wise activation functions .

We follow the initialization setting in [9], where a is randomly initialized and fixed from a binary selection of {-1 √ m , 1 √ m } m . Additionally, we introduce a slightly different setting for the weights W , which are randomly initialized following the normal distribution N (0 , 1 m 1+2 q ) with q > 0 , instead of N (0 , 1 m ) . However, in our experiment, different settings of q are considered, including the scenario that q ≤ 0 .

Generalization ability is one of the most important concepts for machine learning models. Classifiers with better generalization power indicate lower natural risk for unseen data. Given data points ( x , y ) ∼ P and classifier f , the natural risk is defined as

When it comes to 0 -1 loss, the natural risk becomes the probability of misclassification for unseen data points.

Similar to natural risk, the adversarial risk is defined as the probability of misclassification under adversarial perturbations as is shown in Definition 3.1.

Definition 3.1 (Adversarial Risk) . Given data points ( x , y ) ∼ P , and a perturbation ε within a norm-ball, i.e.,

where r > 0 indicates the L 2 -norm budget for perturbations. The adversarial risk for neural network f on loss function L is defined as

Since adversarial perturbations are almost invisible to human eyes, we expect r to be quite small.

3.3 Dirichlet Energy

The concept of Dirichlet energy , originating from Partial Differential Equations (PDEs) , serves as a tool to assess the variability of a function [11]. However, as argued by Dohmatob and Bietti [8], it serves as a more effective measure of adversarial robustness than the Lipschitz constant . We extend this concept to mappings to make it more suitable for multi-dimensional problems, which is formally defined in Definition 3.2.

Definition 3.2 (Dirichlet Energy for Mappings) . Let convex set Z 1 ⊆ R m 1 and Z 2 ⊆ R m 2 . Given a differentiable mapping ϕ : Z 1 →Z 2 , the Dirichlet Energy w.r.t. x ∼ P x is defined as

where ∥ · ∥ 2 refers to the operator norm and J ϕ ( x ) denotes the Jacobian matrix of ϕ w.r.t. its input x .

To be clear, we use ∥·∥ 2 to denote the L 2 -norm for vectors and operator norm for matrices throughout our analysis. This concept can be extended to layers with ReLU activation by extending the derivative to ReLU ′ ( x ) = 1 { x ≥ 0 } , where 1 { x ≥ 0 } is the indicator function. With the concepts defined, we now establish the relationship between adversarial risk and Dirichlet energy.

4 Measure Adversarial Risk by Dirichlet Energy

In this section, we establish the relationship between the adversarial risks and Dirichlet energy , showing that Dirichlet Energy is a proper measurement for adversarial risk. Dohmatob and Bietti [8] only compares the Dirichlet energy with the Lipschitz constant. We, instead, illustrate that Dirichlet energy is a proper representation of the gap between natural risk and adversarial risk.

Theorem 4.1. Given data points ( x , y ) ∼ P and x ∼ P x , the relationship between adversarial risk and Dirichlet energy for classifier f with differentiable loss function L is shown as

where r > 0 is the largest perturbation budget and S ( L ( f )) = √ E x ∼ P x [ ∥∇ f L T · J f ( x ) ∥ 2 2 ] indicating the Dirichlet energy of the classifier on loss L .

As is shown, the Dirichlet energy is a proper representation of the gap between generalization and adversarial risk which indicates the adversarial robustness. The proof relies on the linear approximation of L ( f ) . The detailed proof is shown in Appendix A.

We also empirically show that the Dirichlet energy for classifier f , i.e., S ( f ) instead of S ( L ( f )) , is the part that influences the adversarial robustness, as is shown in Figure 1 where we compare the level of Dirichlet energy for f with the robust accuracy of the classifier attacked by Auto-attack [5].

Since the Dirichlet energy defined in Equation (7) can be used to measure the variability of mappings, it follows that this metric could be employed to evaluate the adversarial robustness of individual layers or modules within neural networks. Consequently, this allows for an assessment of whether there is collaboration between these components in terms of adversarial robustness.

4.1 Measure Correlation between Layers

To evaluate the collective impact of neighbouring layers on the adversarial robustness, we regard the neural network as compounded separate mappings, i.e., ϕ ◦ φ . By calculating the Dirichlet energy for each mapping ( S ( ϕ ) and S ( φ ) ) and comparing it with the overall Dirichlet energy for the whole classifier ( S ( ϕ ◦ φ )), we can quantify the collaboration between these two mappings, and we term this quantification co-correlation which is defined in 4.2.

Definition 4.2 (Co-correlation) . Let φ : Z 1 →Z 2 and ϕ : Z 2 →Z 3 be two successive mappings, where both Z 1 and Z 2 are convex. Given input x ∼ P x , the collaboration correlation (co-correlation) is defined as

To avoid any confusion, we use φ interchangeably to denote both the function φ : Z 1 →Z 2 and the output of the function φ = φ ( x ) for a given input x . Consequently, φ in J ϕ ( φ ) represents the outputs, while in J φ ( x ) , it represents the mapping. Since for each x ∈ P x , we have

We expect 0 ≤ ϱ ϕ , φ ≤ 1 . The concept of co-correlation can be explained by the feature alignment of layers in neural networks.

Interpretation of co-correlation. Intuitively, co-correlation can be viewed as feature 'alignment' between layers. We illustrate this intuition with a 2-layer neural network with linear activation functions, i.e.,

where W 1 and W 2 are weight matrices, and φ ( x ) = W 1 x represents the feature selection in layer-1, while ϕ ( φ ) = W 2 φ represents it in layer-2. By Definition 4.2, the co-correlation between ϕ and φ is

where ∥ · ∥ 2 denotes the operator norm. Let S φ = arg max ∥ x ∥ 2 =1 ∥ W 1 x ∥ 2 , representing the set of input features that maximize the L 2 -norm of their corresponding outputs. Similarly, let S ϕ = arg max ∥ z ∥ 2 =1 ∥ W 2 z ∥ 2 . Assume ϱ ϕ , φ = 1 , there exists x ∈ S φ such that φ ( x ) ∥ φ ( x ) ∥ 2 ∈ S ϕ . This implies that the maximal output in terms of the L 2 -norm at the first layer can lead the output at the second layer to reach its maximum. In other words, in terms of maximizing outputs, layer 1 aligns with layer 2. We can thus regard co-correlation as the degree of alignment in feature selection between adjacent layers within the context of output maximization. In the non-linear case, the weight matrices are replaced by the Jacobians of the respective layers, which can be equivalently viewed as weight matrices that vary based on their inputs.

Definition 4.3 (Other Related Statistics) . Given the same assumptions in Definition 4.2, we define the linear correlation between ∥ J ϕ ( φ ) ∥ 2 and ∥ J φ ( x ) ∥ 2 as

where φ ∼ φ ( x ) shows that φ follows the distribution of φ ( x ) , x ∼ P x . It is obvious that 0 ≤ ρ ϕ , φ ≤ 1 and when ρ ϕ , φ = 1 , ∥ J ϕ ( φ ) ∥ 2 and ∥ J φ ( x ) ∥ 2 are linear correlated.

The mean and variance for ∥ J ϕ ( φ ) ∥ 2 · ∥ J φ ( x ) ∥ 2 are defined as

and

(a) Robust Acc.

(b)

S

(

f

)

0

10

20

30

40

50

Epoch

0.4

0.6

0.8

1.0

ϱ

a

,W

width-16

width-32

width-64

width-128

width-256

width-512

width-1024

width-2048

width-4096

width-8192

0

10

20

30

40

50

Epoch

0.4

0.6

0.8

1.0

ϱ

a

,σ ○ W

width-16

width-32

width-64

width-128

width-256

width-512

width-1024

width-2048

width-4096

width-8192

(d) MLP

Remark 4.4 (Linear Correlation for L 2 -norm of the Jacobian) . We have ρ ϕ , φ = 1 iff there exist t ∈ R such that

implying that ρ ϕ , φ certainly can be used to assess the linear correlations. It reduces to Pearson correlation coefficient [4] when the mean of both random variables equals zero.

Now we give the theorem that binds them together.

Theorem 4.5 (Robustness Decomposition) . Given the same assumption in Definition 4.2, the measurement for overall adversarial robustness can be decomposed as

The proof is straightforward from the definition. Based on this theorem, we have conducted experiments with various linear models and 2-layer MLPs. These experiments demonstrate that, apart from the co-correlation, all other statistics are negligible, as shown in Figure 5 in the Appendix D. Consequently, our analysis primarily focuses on the co-correlation ϱ .

5 On Dynamics of Co-Correlation

5.1 Dynamics for Linear Model

Before delving into our analysis, we state our assumptions explicitly.

Assumption 5.1. We assume that each element w i,j in the weight matrix W (0) ∈ R m × d at initialization follows the Gaussian distribution N (0 , 1 m 1+2 q ) , with q > 0 . Additionally, each element a r , r ∈ [ m ] in a is randomly selected from the set {-1 √ m , 1 √ m } , and fixed during training.

Assumption 5.2. We assume that for each ( x i , y i ) ∈ D,i ∈ [ n ] , x i is L 2 norm bounded such that ∥ x i ∥ 2 = 1 for all i ∈ [ n ] .

Since we only assume bounded inputs and a specific weight initialization method, compared to existing works [23, 19, 21, 12], our approach can be easily extended to MLPs with more than two layers.

Now let us focus on the co-correlation defined in Equation (9) and show the dynamics of ϱ ϕ , φ for each step of gradient descent. We start from the linear model described in Equation (3a). Given the binary classification problem and the linear model described. Our first theorem demonstrates that co-correlation ϱ ϕ , φ gradually accumulates throughout gradient descent optimization. Despite the simplicity of the linear model, it effectively exhibits most of the core properties under consideration.

Since the weights are updated by the gradient descent, the update of the weights at step t ∈ N is

where η denotes the learning rate. Therefore, the dynamics of the weights can be expressed as

where ⊗ denotes the Kronecker product and ˜ x ( t ) is the error weighted input such that

Given the weight updates, we demonstrate that the dynamics of the co-correlation for the linear model, denoted as ˙ ϱ a ,W ( t ) , exhibit an increasing trend, particularly during the initial steps of training when most predictions are still essentially random.

Theorem 5.3 (Dynamics of the Co-correlation for Linear Model) . Given the linear model defined in Equation (3a) and training dataset D = { ( x i , y i ) } n i =1 . Assume that assumptions 5.1 and 5.2 hold for W and a . The gradient descent applied to the weights results in the dynamics of the co-correlation being expressed as:

and with high probability,

where the v ( t ) is the dominate eigenvector for W ( t ) W ( t ) T .

When m is sufficiently large, and during the initial steps of the optimization process, ˜ x ( τ ) , τ ∈ [ t ] are quite similar to each other in terms of cosine similarity, implying an acute angle to each other, which leads to ∑ t τ =1 ˜ x ( τ ) T ˜ x ( t ) ≥ 0 . As a result, we can conclude that C ( t ) ≥ 0 .

The detailed proof can be found in the Appendix B. This assertion is also corroborated by the results of our experiments as shown in Figure 1c. Even though Theorem 5.3 is based on a linear model, the essential properties are universally applicable and can be summarized as follows:

Property 1. The co-correlation ϱ a ,W develops during the initial stages of training and becomes saturated as training progresses to its later stages.

Property 2. The speed of the accumulation of co-correlation ϱ a ,W is inversely related to the operator norm of weights ∥ W ( t ) ∥ 2 .

Under the same weight initialization conditions specified in Assumption 5.1, an increase in network width leads to a decrease in the L 2 -norm of the weight, consequently causing a substantial rise in co-correlation.

5.2 Dynamics for MLP Model

For the non-linear case, we make certain assumptions regarding activation functions.

Assumption 5.4. The derivative of the activation function σ ′ ( x ) in non-linear neural networks is bounded by M . In other words, we have | σ ′ ( x ) | ≤ M .

With Assumption 5.4, Theorem 5.3 can be extended to MLPs, and the two properties still hold. Different from the linear model, the update of weights for non-linear MLP defined in (3b) is

where σ ′ ( w T r x i ) is the derivative of activation function w.r.t. its input. Hence,

(a) ResNets

0

10

20

30

40

50

Epoch

0.23

0.24

0.25

ϱ

ResNet50

WRN50

(b) A1-A1

0

10

20

30

40

50

Epoch

0.40

0.45

0.50

0.55

0.60

0.65

0.70

ϱ

ResNet50

WRN50

(c) B1-B2

where for each r ∈ [ m ] ,

With high probability,

where

As is shown in Equation (20), a is not the dominant eigenvector for ∆ W ∆ W T due to the difference of ˜ x r , r ∈ [ m ] from ˜ x . Thus to average out the difference, we define the weighted sum of inputs for the non-linear model as

where ( x i , y i ) ∈ D are realized r.v., and x ∼ P x . Given w i,j i.i.d. ∼ N (0 , 1 m 1+2 q ) for each element in W (0) , we have w (0) ∼ N (0 , 1 m 1+2 q I d ) . Furthermore, given that W ( t ) is calculated from W (0) , it is evident that each row in W ( t ) , denoted as w ( t ) , constitutes a random variable. As a result, both σ ′ ( w ( t ) T x ) and σ ′ ( w ( t ) T x i ) are bounded random variables that are contingent upon W (0) . Hence, we define α i ( t, x ) as

Since, x ∼ P x , α i ( t, x ) is still r.v. contingent to x and so does the ˜ x ∗ ( t ) . Now we show the dynamics of co-correlation for two-layer MLP defined in Equation (3b).

Theorem 5.5. (Dynamics of the Co-correlation for MLP) Given the MLP defined in Equation (3) with training dataset D = { ( x i , y i ) } n i =1 , x ∈ X such that x ∼ P x . Assume that Assumption 5.1 and 5.2 hold for W and a , and Assumption 5.4 holds for the activation function. we have

and v ( t ) denotes the dominant eigenvector for W ( t ) W ( t ) T , with ˜ x T ∗ is defined in Equation (21) . Similar to the Theorem 5.3, when m is sufficiently large, and during the initial steps of the optimization where the error-weighted inputs ˜ x T ∗ ( τ ) , τ ∈ [ t ] do not significantly fluctuate, we have that C ( t ) ≥ 0 .

The detailed proof is in Appendix C. In Theorem 5.5, ˜ x ∗ serves a similar purpose as ˜ x for the linear model. In addition, it considers the influence of the activation function. Property 1 still holds for the MLPs, and Property 2 extends to E x ∼ P x ∥ D ( t ) W ( t ) ∥ 2 2 = ∥ J σ ◦ W ( x ) ∥ 2 L ( P z ) .

0

10

20

30

40

50

Epoch

0.3

0.4

0.5

0.6

0.7

0.8

0.9

ϱ

a

,σ ○ W

q=-0.15

q=-0.2

q=-0.05

q=0

q=0.05

q=0.15

q=0.25

(a) width-32

0

10

20

30

40

50

Epoch

0.3

0.4

0.5

0.6

0.7

0.8

0.9

ϱ

a

,σ ○ W

q=-0.15

q=-0.1

q=-0.05

q=0

q=0.05

q=0.15

q=0.25

(b) width-512

0

10

20

30

40

50

Epoch

0.3

0.4

0.5

0.6

0.7

0.8

0.9

ϱ

a

,σ ○ W

q=-0.15

q=-0.1

q=-0.05

q=0

q=0.05

q=0.15

q=0.25

(c) width-2048

0

10

20

30

40

50

Epoch

0.3

0.4

0.5

0.6

0.7

0.8

0.9

ϱ

a

,σ ○ W

q=-0.15

q=-0.1

q=-0.05

q=0

q=0.05

q=0.15

q=0.25

(d) width-8192

6 Experiments

To estimate the co-correlation more efficiently and in parallel, we employ the Power Iteration algorithm [10] in conjunction with Functorch [17]. The corresponding pseudo-code is presented in the Appendix E.

We verify our proposed theorem on linear and MLP models on the MNIST dataset [22]. The width of hidden layers varied from 2 4 to 2 13 , with weights initialized via a Gaussian distribution N (0 , 1 m 1+2 q ) where q was set to values ranging from 0 . 25 to -0 . 15 . We train both the linear and MLP for 50 epochs with a batch size of 512 using the SGD optimizer with a learning rate of 0 . 003 . In addition, we also conduct the experiment on more complex ResNet50 [16] and WRN50 [38]. For both models, we opted for default random weight initialization and used the Adam optimizer with a learning rate of 0 . 0005 on CIFAR10 [20]. To calculate the co-correlation and other statistics, we cover the entire testset. We consider using L 2 Auto-attack [5] with ϵ = 0 . 5 for all MLPs we trained. The experiments were executed on a Nvidia RTX3090 GPU, using Python 3.9.7 and PyTorch 1.9.1. The code for the experiment is available at https://github.com/squarewang2077/co-correlation .

6.1 Empirical Evidence for Proposed Theorem

Figure 1 presents a comparison between the robust accuracy and the Dirichlet energy S ( f ) across all trained MLPs. As observed in Figure 1a and 1b, models with lower levels of Dirichlet energy S ( f ) tend to exhibit higher robust accuracy, suggesting that Dirichlet energy is an effective representation of adversarial robustness. Another noteworthy finding is that wider neural networks, with the same level of weight initialization, demonstrate improved adversarial robustness.

Figure 1 also depicts the dynamic behaviour of shallow neural networks with a weight initialization parameter of q = 0 . 25 . As is shown in Figure 1c and 1d, the co-correlation ϱ increases throughout training. Except for narrow widths like 2 4 and 2 5 , the majority of networks demonstrate an upward trend. This trend, however, flattens for non-linear models, suggesting potentially stronger adversarial robustness due to the non-linearity of the activation function introduced in MLPs.

Figure 2 shows the dynamics of the co-correlation on ResNet50 and Wide-ResNet50. Both networks are trained on CIFAR10 using the Adam optimizer. We divide them by the pattern of A1-A2 and B1B2, as shown in Figure 2a. The co-correlation outcomes for these divisions are displayed in Figure 2b and Figure 2c, it shows that even with the Adam optimizer, without specific weight initialization considerations, there is a noticeable rise in co-correlation.

6.2 The Impact of Width and Weight Initialization

Figure 3 illustrates the co-correlation dynamics under varying q for both linear and MLPs with widths of 32, 512, 2,048, and 8,192. The figure highlights that our proposed theorems' assumption of q > 0 is quite tight, as all trajectories with q < 0 remain flat throughout training. We can also observe that the speed of accumulation significantly increases with larger network widths.

Figure 4 displays the accuracy on testset and co-correlation for both linear and MLPs as heat-maps. Each cell in the heat-map represents a trained network. From Figure 4c and Figure 4d, we observe that the best performance and robustness are shown by the MLPs with the largest widths (width = 8192 ) and the smallest weight initializations ( q = -0 . 15 ). And when we alter the weight initialization to

(a) Linear Model/Acc.

(b) Linear Model/

ϱ

(c) MLP ReLU/Acc.

(d) MLP ReLU/

ϱ

Weight Initalization(q)

S0-10-500-0050051050

0.75

0.73

0.79

0.8

0.83

0.84

0.65

0.87

0.87

0.88

0.88

0.89

0.9

0.9

160160160

0.9

0.910.91

0.85

0.90

WeightInitalization(q)

0.25 0.15 0.05 0.0 -0.05 -0.1 -0.15

0.91

0.830.840.750.720.68

0.790.760.740.620.580.480.410.320.26

0.56

0.5

0.42

0.4

0.34

- 0.9

-0.8

Weight Initalization(q)

0.25 0.15 0.05 0.0 -0.05 -0.1 -0.15

0.11

0.21

0.73

0.730.77

0.8

0.840.870.890.910.920.920.93

260260260160680880980

0.9

0.8

0.7

Weight Initalization(q)

S10-10-500-00500510520

0.47

0.48

0.64

0.59

0.52

0.490.550.520.460.410.36

0.6

0.580.510.450.390.360.32

0.3

0.25

-0.8

-0.7

0.77

0.83

0.85

0.87

0.890.89

0.9

0.9

0.910.91

0.87

0.820.82

0.730.68

0.620.590.550.54

0.53

- 0.7

EO

0.67

680880980680

0.9

160160160

0.6

0.61

0.58

0.67

0.6

0.570.550.540.520.5

0.49

0.6

0.77

0.82

0.86

0.88

0.890.89

0.9

0.9

0.9

0.9

0.80

0.93

0.84

0.8

0.78

0.730.710.720.750.740.76

0.6

0.38

0.810.850.870.880.890.890.890.890.89

0.5

0.59

0.730.71

0.620.630.67

0.630.67 0.65 0.64

0.5

0.8

0.84

0.86

0.880.880.880.880.880880.88

0.850.860.83 0.83

0.79

0.86

0.82

0.890.9

- 0.5

0.34

0.830.860.880.880.880.880.870.87

0.4

0.53

0.730.72

0.7

0.7

0.68

0.71

0.730.71

0.75

0.8

28008019058028099098028090

0.75

0.9

0.86

0.94

0.83

0.91

0.9

0.95

6060

0.98

- 0.4

0.43

0.71

0.830.840.840.840.840.830.820.81

0.3

0.61

0.730.75

0.73

0.74

0.78

0.8

0.82 0.83 0.84

0.4

0.79

0.81

0.840.820.820.790.780.750.73

0.69

0.70

0.86

0.93

0.92

980

260

160

0.95

0.990.98

0.99

0.3

0.720.77

0.830.82

0.790.780.750.73

0.7

0.68

0.2

0.81

0.78

0.74

0.79

0.83

0.8

0.78

0.85

0.84

0.83

0.3

16

326

163264128256512

8192

16

32

6A

1632642825651

8192

control the co-correlation, making it increase from 0 . 25 to 0 . 83 , the accuracy declines accordingly from 0 . 93 to 0 . 68 . On the contrary, for another extreme case of models with a width of 16 , enhanced performance is accompanied by increased co-correlation. Consequently, an interesting conclusion can be drawn about the diverse behaviour of neural networks with small and larger width. Gradient descent tends to enhance the training of neural networks with smaller width by fostering co-correlation among layers, which is intrinsically brittle. However, wide networks are trained with less reliance on interlayer correlation, resulting in inherently more robust models.

7 Conclusion and Limitation

Our work investigates the implicit bias of gradient descent toward adversarial robustness from the perspective of collaboration between layers. By adapting Dirichlet energy to estimate the adversarial robustness of neural networks' individual components, we characterized the collaboration behaviour between consecutive layers and identified two fundamental properties for dynamics of the co-correlation. The first property shows that the co-correlation for MLPs will build up during gradient descent under mild assumptions for weight initialization. The second property shows that the speed of accumulation for co-correlation is inversely related to the operator norm of Jacobian for the corresponding sub-modules. In addition, we observed that networks with small widths tend to foster co-correlation among layers to improve performance, whereas wide networks' performance improvement does not heavily rely on establishing such co-correlation. Future research can expand upon this by examining the effects of increased network depth and more sophisticated structures on the observed phenomena.

Limitation Our work can be easily extended to multi-layer neural networks since we only assume that the inputs are bounded by the L 2 -norm. However, like many theoretical studies, extending our approach to more complex models is challenging. It remains unknown whether complex models exhibit the same behaviors.

References

R. Bhattacharjee, S. Jha, and K. Chaudhuri. Sample complexity of robust linear classification on separated data. In International Conference on Machine Learning , pages 884-893. PMLR, 2021.

S. Bubeck, Y. Cherapanamjeri, G. Gidel, and R. Tachet des Combes. A single gradient step finds adversarial examples on random two-layers neural networks. Advances in Neural Information Processing Systems , 34:10081-10091, 2021.

L. Chizat and F. Bach. Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss. In Conference on Learning Theory , pages 1305-1338. PMLR, 2020.

I. Cohen, Y. Huang, J. Chen, J. Benesty, J. Benesty, J. Chen, Y. Huang, and I. Cohen. Pearson correlation coefficient. Noise reduction in speech processing , pages 1-4, 2009.

F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning , pages 2206-2216. PMLR, 2020.

A. Daniely and H. Shacham. Most relu networks suffer from l 2 adversarial perturbations. Advances in Neural Information Processing Systems , 33:6629-6636, 2020.

E. Dobriban, H. Hassani, D. Hong, and A. Robey. Provable tradeoffs in adversarially robust classification. arXiv preprint arXiv:2006.05161 , 2020.

E. Dohmatob and A. Bietti. On the (non-) robustness of two-layer neural networks in different learning regimes. arXiv preprint arXiv:2203.11864 , 2022.

S. S. Du, X. Zhai, B. Poczos, and A. Singh. Gradient descent provably optimizes overparameterized neural networks. arXiv preprint arXiv:1810.02054 , 2018.

J. F. Epperson. An introduction to numerical methods and analysis . John Wiley & Sons, 2021.

L. C. Evans. Partial differential equations , volume 19. American Mathematical Society, 2022.

S. Frei, G. Vardi, P. L. Bartlett, N. Srebro, and W. Hu. Implicit bias in leaky relu networks trained on high-dimensional data. arXiv preprint arXiv:2210.07082 , 2022.

S. Frei, G. Vardi, P. L. Bartlett, and N. Srebro. The double-edged sword of implicit bias: Generalization vs. robustness in relu networks. arXiv preprint arXiv:2303.01456 , 2023.

J. Gilmer, L. Metz, F. Faghri, S. S. Schoenholz, M. Raghu, M. Wattenberg, and I. Goodfellow. Adversarial spheres. arXiv preprint arXiv:1801.02774 , 2018.

I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014.

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770778, 2016.

R. Z. Horace He. functorch: Jax-like composable function transforms for pytorch. https: //github.com/pytorch/functorch , 2021.

H. Huang, Y. Wang, S. Erfani, Q. Gu, J. Bailey, and X. Ma. Exploring architectural ingredients of adversarially robust deep neural networks. Advances in Neural Information Processing Systems , 34:5545-5559, 2021.

Z. Ji and M. Telgarsky. Directional convergence and alignment in deep learning. Advances in Neural Information Processing Systems , 33:17176-17186, 2020.

A. Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, University of Tront , 2009.

D. Kunin, A. Yamamura, C. Ma, and S. Ganguli. The asymmetric maximum margin bias of quasi-homogeneous neural networks. arXiv preprint arXiv:2210.03820 , 2022.

Y. LeCun, C. Cortes, and C. Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist , 2, 2010.

K. Lyu and J. Li. Gradient descent maximizes the margin of homogeneous neural networks. arXiv preprint arXiv:1906.05890 , 2019.

K. Lyu, Z. Li, R. Wang, and S. Arora. Gradient descent on two-layer nets: Margin maximization and simplicity bias. Advances in Neural Information Processing Systems , 34:12978-12991, 2021.

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

S. Mahloujifar, D. I. Diochnos, and M. Mahmoody. The curse of concentration in robust learning: Evasion and poisoning attacks from concentration of measure. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 4536-4543. Association for the Advancement of Artificial Intelligence (AAAI), 2019.

Y. Min, L. Chen, and A. Karbasi. The curious case of adversarially robust models: More data can help, double descend, or hurt generalization. In Uncertainty in Artificial Intelligence , pages 129-139. PMLR, 2021.

N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP) , pages 582-597. IEEE, 2016.

R. Sarussi, A. Brutzkus, and A. Globerson. Towards understanding learning in neural networks with linear teachers. In International Conference on Machine Learning , pages 9313-9322. PMLR, 2021.

L. Schmidt, S. Santurkar, D. Tsipras, K. Talwar, and A. Madry. Adversarially robust generalization requires more data. Advances in neural information processing systems , 31, 2018.

A. Shafahi, W. R. Huang, C. Studer, S. Feizi, and T. Goldstein. Are adversarial examples inevitable? arXiv preprint arXiv:1809.02104 , 2018.

C.-J. Simon-Gabriel, Y. Ollivier, L. Bottou, B. Schölkopf, and D. Lopez-Paz. First-order adversarial vulnerability of neural networks and input dimension. In International conference on machine learning , pages 5809-5817. PMLR, 2019.

D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry. Robustness may be at odds with accuracy. arXiv preprint arXiv:1805.12152 , 2018.

F. Wang, P. Xu, W. Ruan, and X. Huang. Towards verifying the geometric robustness of large-scale neural networks. arXiv preprint arXiv:2301.12456 , 2023.

Z. Wang and W. Ruan. Understanding adversarial robustness of vision transformers via cauchy problem. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2022, Grenoble, France, September 19-23, 2022, Proceedings, Part III , pages 562-577. Springer, 2023.

X. Yin and W. Ruan. Boosting adversarial training via fisher-rao norm-based regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 24544-24553, 2024.

X. Yin, W. Ruan, and J. Fieldsend. Dimba: discretely masked black-box attack in single object tracking. Machine Learning , 113(4):1705-1723, 2024.

S. Zagoruyko and N. Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146 , 2016.

H. Zhang, Y. Yu, J. Jiao, E. Xing, L. El Ghaoui, and M. Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning , pages 7472-7482. PMLR, 2019.

T. Zhang, W. Ruan, and J. E. Fieldsend. Proa: A probabilistic robustness assessment against functional perturbations. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 154-170. Springer, 2022.

Z. Zhu, F. Liu, G. Chrysos, and V. Cevher. Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization). Advances in Neural Information Processing Systems , 35:36094-36107, 2022.

A The Proof for Theorem 4.1

Suppose that the loss function L is differentiable w.r.t. f . Given data points ( x , y ) ∼ P and x ∼ P x , we have

where ∇ f L is the gradients of L w.r.t. f , ∇ f L T indicates that it is a row vector. J f ( x ) is the Jacobian matrix for classifier f .

B The Proof of Theorem 5.3

Before formally proving the theorem, we provide some lemmas which are useful for our proof.

Lemma B.1 (Dynamic of Weights for Linear Model) . Given gradient descent to optimize the weights, the dynamic of the weights at step t is

Proof. Given the step size η , the update for r th row of the weight matrix W is

where sig ( · ) denotes the sigmoid function. Hence, for weight matrix W , we have

After replacing with

we have

Since ∥ x i ∥ 2 = 1 , we also have ∥ ˜ x ∥ 2 ≤ 1 . Therefore, we can say that the dynamics of weights, i.e. ˙ W ( t ) , is

Lemma B.2 (The Concentration for L 2 Norm of Gaussian r.v.) . Let I m be identical matrix of size m × m . Given n Gaussian random vectors z 1 , · · · , z n such that z i i.i.d. ∼ N (0 , 1 m 1+2 q I m ) , q > 0 , ∀ i ∈ [ n ] . With probability at least 1 -δ , we have several conclusions:

1. Average of the norm

Average of the square norm

Square root of the average of the square norm

Proof. We first prove the concentration property for the average of the norm. Since √ m 1+2 q z i,r i.i.d. ∼ N (0 , 1) , i ∈ [ n ] , r ∈ [ m ] , we have that m 1+2 q z 2 i,r ∼ χ 2 (1) . Denote the sub-Exponential distribution as SE ( ν 2 , α ) , we have m 1+2 q z 2 i,r -1 ∈ SE (4 , 4) . Hence ∀ i ∈ [ n ] ,

Therefore, we have ∀ ϵ > 0

Equation (40) is because of the fact that given c > 0 fixed, ∀ x > 0 we have

Hence,

Equivalently, with probability at least 1 -δ ,

To prove the second convergence, we starts from the fact that

Since ( √ m 1+2 q z i,r ) 2 i.i.d. ∼ χ 2 (1) , ∀ i ∈ [ n ] , r ∈ [ m ] , similar to Equation (39), we have

Equivalently, with probability at least 1 -δ ,

Now we prove the third concentration, with similar trick of the first concentration, we have

Therefore,

with probability at least 1 -δ , we have

Now, we prove the theorem 5.3. Given the linear model and training dataset D = { ( x i , y i ) } n i =1 . Assume that each w i,j i.i.d. ∼ N (0 , 1 m 1+2 q ) , q > 0 and a is randomly initialized subject to the constraint ∥ a ∥ 2 = 1 and fixed during training. Hence, with high probability, we have

Proof. We first show that the derivative of co-correlation is

by the law of derivatives for inner product of matrices and the eigenvalue of matrix W ( t ) W ( t ) T , we have

where v ( t ) is the dominant eigenvector for W ( t ) W ( t ) T . Equation (60) is because J W ( x ) = W and ∥ a ∥ 2 = 1 by assumption. Since Equation (60) do not depend on x , we can safely drop the Expectation E x as is shown in Equation (61). Equation (63) is because ∥ a T W ∥ 2 2 ≤ ∥ a ∥ 2 2 ∥ W ∥ 2 2 = ∥ W ∥ 2 2 . By Lemma B.1, the weight matrix W ( t ) at training step t can be approximated as

then similarity,

Hence

1 is the main part of our theorem. And for 2 , it can be bounded as

Now since w i,j ∼ N (0 , 1 m 1+2 q ) , q > 0 and each w i,j is independent with each other, for given x i , i = 1 , ..., n , we have

where ∥ x i ∥ 2 2 = 1 by assumption. By the lemma B.2, with high probability,

with high probability, and it comes to our conclusion.

C The Proof of Theorem 5.5

To prove the Theorem 5.5, the following lemma is crucial.

Lemma C.1 (Dynamic of Weights for A Nonlinear Model for Initial Steps) . Given gradient descent to optimize the weights, the dynamic of the weights at the initial step t for the non-linear model is

Therefore, where

Proof. Similar to the linear case, the update for the r th row for the weight matrix is

Hence that the update of the weight matrix W ( t ) is

Hence our conclusion.

Lemma C.2 (Concentration for Weighted Sum and Square Root Scalar for Bounded Variables) . Suppose X r -µ ∈ [ l, u ] , ∀ i ∈ [ m ] is independent identically distributed r.v. with mean 0. Given a vector v ∈ R m , ∥ v ∥ 2 = 1 , and a such that a r ∈ {-1 , 1 } , r ∈ [ m ] we have concentration inequality

Proof. We first show the moment generation function of the r.v. we would like to estimate.

Now, we prove this inequality.

where t ≥ 0 . Hence by Chernoff's bound, we have that

Here, we prove the theorem 5.5. Given a shallow neural network, with high probability we have

Proof. Because of activation function depends on the input x , we cannot drop the expectation E x . Hence, we have

where D ( t ) = diag ( σ ′ ( w T 1 x ) , ..., σ ′ ( w T m x )) . And for ReLu activation function, σ ′ ( w T m x ) = 1 { w T m x ≥ 0 } . We assume that the activation does not change at an infinitely small change of t , implying ˙ D ( t ) = 0 . Hence, we have

And similarly,

1 can be

Since a r ∈ {-1 √ m , 1 √ m } and the derivative activation function σ ′ is bounded by M by assumption, we have that

where

Similarly,

Equivalently, with probability at least 1 -δ , we have

Let

Hence, with probability at least 1 -δ ,

Since

we have that for at least 1 -δ ,

Let a ∗ = √ m a ∈ {-1 , 1 } m and by the conclusion of lemma C.2,

Equivalently, with probability at least 1 -δ ,

Hence,

We have that for at least 1 -δ ,

For 2 , we have

where

Similar to the proof of the linear case, and accords to Lemma B.2, we have that with probability at least 1 -δ ,

Therefore, with high probability,

Follow the exact same procedure. With high probability, we have

We put all the information together, with high probability, we have

0

10

20

30

40

50

Epoch

0.005

0.010

0.015

0.020

0.025

0.030

std/µ

width-16

width-32

width-64

width-128

width-256

width-512

width-1024

width-2048

width-4096

width-8192

(a) Relative Std

0

10

20

30

40

50

Epoch

0.99965

0.99972

0.99979

0.99986

0.99993

1.00000

⍴

(b) Linear Correlation

4

5

6

7

8

9

10

11

12

13

width(2 n

)

0.0

0.3

0.6

tot Δϱ

a

,W

q=-0.15

q=-0.1

q=-0.05

q=0

q=0.05

q=0.15

q=0.25

4

5

6

7

8

9

10

11

12

13

width

0.0

0.3

0.6

tot Δϱ

a

,σ ○ W

D Extra Experiment Results

Figure 5 summarizes the statistics of linear correlation and relative variability, illustrating that co-correlation is the dominant factor in the decomposition outlined in Theorem 4.5.

Figure 6 shows the accumulated change in co-correlation during training across varying network widths. Each line, represented by different colours, corresponds to a unique weight initialization setting of q . A clear inference from the figure is the positive relationship between the accumulated co-correlation change and network width under the same q -the wider the network, the greater the accumulated co-correlation change. Moreover, as we shift the weight initialization setting from q = -0 . 15 to q = 0 . 25 , the accumulation of co-correlation also increases. These observations align with our Theorem 5.3 and 5.5, indicating that the increase in co-correlation is inversely proportional to ∥ W ( t ) ∥ 2 .

E Algorithm

We use the Power Iteration algorithm [10] along with Functorch [17] to compute co-correlation and other relevant statistics. This combination assists in approximating the L 2 -norm of the Jacobian for the layers under consideration. The algorithm to estimate the related statistics including cocorrelation are outlined in Algorithm 1. The power iteration used to obtain the L 2 -norm of the Jacobian is summarized in Algorithm 2.

Algorithm 1 Estimation of Related Statistics

Input : Input dataset and modules ϕ, φ for the given blocks in sequential { x i , i = 1 , .., N } Output : Approximation of co-correlations ̂ ϱ ϕ,φ , linear correlation ̂ ρ ϕ,φ , mean ̂ µ ϕ , ̂ µ φ and variance σ ϕ , σ φ of the Jacobian of the module w.r.t. their inputs

1: Computation of Jacobin of ∥ J ( x ) ∥ , ∥ J ( φ ( x )) ∥ for { x , i ∈ [ n ] } w.r.t. their inputs

φ i 2 ϕ i 2 i 2: ̂ ϱ ϕ,φ ← 1 N ∑ N i =1 ∥ J ϕ ( φ ( x i )) · J φ ( x i ) ∥ 2 1 N ∑ N i =1 ∥ J ϕ ( φ ( x i )) ∥ 2 ∥ J φ ( x i ) ∥ 2 3: ̂ ρ ϕ,φ ← 1 N ∑ N i =1 ∥ J ϕ ( φ ( x i )) ∥ 2 ∥ J φ ( x i ) ∥ 2 ( 1 N ∑ N i =1 ∥ J ϕ ( φ ( x i )) ∥ 2 2 ) 1 2 ( 1 N ∑ N i =1 ∥ J φ ( x i ) ∥ 2 2 ) 1 2 4: ̂ µ ϕ ← 1 N ∑ N i =1 ∥ J ϕ ( φ ( x i )) ∥ 2 ∥ J φ ( x i ) ∥ 2 5: ̂ σ ϕ ← [ 1 N -1 ∑ N i =1 ( ∥ J ϕ ( φ ( x i )) ∥ 2 ∥ J φ ( x i ) ∥ 2 -ˆ µ ϕ ) 2 ] 1 2

Algorithm 2 Power Iteration Based Computation

Input : Input dataset and module for the given sequential blocks { x i , i = 1 , .., N } , ϕ, φ Output : Approximation of co-correlations ϱ ϕ,φ , linear correlation ρ ϕ,φ , mean µ ϕ , µ φ and variance σ ϕ , σ φ of the Jacobian of the module w.r.t. their inputs

̸

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction have clearly stated the claims made, including the contributions made in the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: A separate section "Discussion and Limitations" has been designed and can be seen in the end of this paper.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The complete proofs are provided in the appendix. Theorems and Lemmas are properly referenced in this paper.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The algorithm and detailed experiment settings are provided.

Guidelines: The code is provided in the supplemental material.

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code is uploaded in github, including detailed README file.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All details can be checked in Experiment section.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The detailed methods can be found in Experiment section.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper introduces the configuration of the computer in Experiment section.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have read the NeurIPS Code of Ethics and confirm that the research conforms in every respect.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no societal impact of the work performed. This is a theoretical paper to understand existing ML methods.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have cited the original paper that produced the code package or dataset properly.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors study the difference between underparameterised and overparameterised networks in terms of the collaboration between consecutive layers. They find that under-parameterized networks tend to foster co-correlation among layers to improve performance, whereas performance of over-parameterized networks can improve without relying on this co-correlation. They relate the co-correlation to adversarial robustness.\n\nSTRENGTHS:\nThis paper adresses an important open question in deep learning: how should we relate the concepts of generalization, adversarial robustness and the network archictecture and size? The paper adresses this question from an original perspective. \n\nThe literature study is quite extensive and motivates the work (although it remains quite vague/confusing, see next). \n\nGreat care has been taken in a correct mathematical formulation off all definitions and proofs. \n\nThe authors have verified some of their theoretical claims with a series of experiments.\n\nWEAKNESSES:\n1.\tFrom your abstract, introduction and conclusion, it is not clear why you would study  co-correlation between layers in the first place, and/or how this relates to robustness and generalization. E.g., starting from line 40, the reader can infer: Dirichlet energy = measure of adversarial robustness -> Dirichlet energy per layer -> co-correlation -> gradient descent increases co-correlation. But how and why is this co-correlation exactly related to adversarial robustness? Etc. \n\n2.\tThis does not become really clear to me in the other parts of the paper either. I understand that the dirichlet can be used to measure the gap between generalization and adversial risk. But what this means for individual layers and why you would want to look into this remains unclear. E.g., Line 163: “Since the Dirichlet energy defined in Equation (7) can be used to measure the variability of mappings,it follows that this metric could be employed to evaluate the adversarial robustness of individual layers or modules within neural networks. Consequently, this allows for an assessment of whether there is collaboration between these components in terms of adversarial robustness.”: this seems an important point, but I could’t understand what you meant exactly, especially the second sentence.\n\n3.\tL117: why is the second layer random and fixed? If needed for a mathematical analysis, what is the influence of this on the generality of your findings?\n\n4.\tThe claim made in L160-161 cannot be supported by fig. 1 alone. It’s not because Dirichlet energy has a similar (inverse) profile over width and initialization as the adversarial robustness, that one influences the other directly. The same goes for the other figures.  More information/verification seems needed. \n\n5.\tL192-194: what about non-linear activation functions? \n\n6.\tThe experiments are only conducted on the simplified models used for the theoretical derivations, and as such do not show anything about how the theoretical findings relate to actual, practical neural networks (i.e., models with more layers, where no layers are held fixed). \n\nTypos/grammar/symbols: \n7.\tLine 64, 76, 77, 105, 111 (strange sentence), 127, caption figure 1, 275, …\n8.\tEq 1: f_W undefined\n9.\tFig 1: legend and labels too small, caption not clear\n\nQUESTIONS:\nSee points raised above. In general, I think this work could be a valuable contribution, but the quality of the text has to be increased to better motivate and explain the approach. In it’s current state, the work is hard to judge. \n\nIf you think a change to the manuscript is warranted, please specify how you would update it.\n\nLIMITATIONS:\nThe limitations are not addressed sufficiently. E.g., not clear what the influence of the strong assumption that the second layer remains fixed has on the generality of the findings. \n\nNo potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the implicit bias of gradient descent in over-parameterized neural networks, particularly focusing on the collaboration between consecutive layers. The study first introduces Dirichlet energy to evaluate the adversarial risk. Then it decomposes Dirichelt energy between layers and measures the alignment of feature selections between layers. The collaboration of Dirichlet energy between layers is called co-correlation. Through theoretical analysis and extensive experiments, the authors demonstrate that over-parameterized networks exhibit stronger resistance to increased co-correlation, thereby enhancing adversarial robustness.\n\nSTRENGTHS:\nThe paper introduces the novel concept of co-correlation to quantify collaboration between layers, offering a fresh perspective on implicit bias in neural networks. The notion of co-correlation helps with distinguishing the dynamics between over-parameterized and under-parameterized networks. Moreover, this paper provides robust theoretical analysis for the properties of the dynamics of co-correlation.\n\nWEAKNESSES:\n1. The theory part of the paper mainly analyzes the dynamics of co-correlation. However, the paper only explain the relation between Dirichlet energy and adversarial robustness. It does not directly connect adversarial robustness with co-correlation in analysis.Without a valid analysis, co-correlation dynamics cannot lead to the conclusions on adversarial robustness.\n2. In the analysis of co-correlation,  this paper proved $C(t)\\>0$ after training for some time. This result indicates the increeasing trend of co-correlation but not explaining the value and the rate of the convergence, which shows the strength of the implicit bias. Moreover, it could also be better if the authors discuss in the theory part about the effect of initialization shown in experiment.\n\nQUESTIONS:\n1. Why the final co-correlation of MLP is smaller than linear net with same width and initialization?\n2. How will the Dirichlet energy for each mapping $\\phi$ and $\\psi$ changes? Will this also affect the Dirichlet energy and adversarial robustness of the neural network?\n\nLIMITATIONS:\nThe authors have addressed the limitations of their work, acknowledging the challenges in extending the approach to more complex models and the need for broader evaluations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work focuses on studying the implicit bias of correlation between intermittent layers of neural nets and uses this metric to analyze the adversarial robustness of networks in under and over parameterized regimes. The authors further use these findings to suggest that in the under parametrized case, gradient descent enhances accuracy by forcing co-correlation between layers which in turns leads to worse adversarial robustness. However, in the overparameterized case, models depend less on the interlayer correlations and thus result in more robust models. \n\nThe authors approach the problem by using the Dirichlet energy to bound the adversarial risk. They then introduce the co-correlation metric, designed to describe the alignment of layers in a linear 2-layer neural net. Further, the authors manage to bound the Dirichlet energy using the co-correlation metric and additional terms but provide experimental evidence to indicate that main term changing throughout training in under over/under parameterized scenarios is the co-correlation metric. \n\nThe paper then focuses on studying the dynamics of co-correlation change during training on the linear model with emphasis on layer width and initialization, describing how co-correlation increase happens mainly during the early stages of training. Finally, the authors provide experimental results and extend their work to general MLPs.\n\nSTRENGTHS:\nOverall I found the work to be quite interesting, well written and described. The ideas were well explained, assumptions are mostly reasonable and the paper is easy to understand for me. Additionally I like seeing works done in the interlayer interactions of deepnets and believe such observations and results could have further experimental and theoretical implications. \n\nI am also quite interested in the results regarding implication of initialization and am somewhat surprised by the experimental results describing the importance of initialization.\n\nWEAKNESSES:\n1) While I understand that the focus of the work is on theory, I would like to see 1-2 more experimental results, potentially with CIFAR-10 or larger networks (maybe not MLP) just to confirm the results and give a better perspective. \n\n2) The authors often make claims regarding over-under parametrization vs width of layers. I find it a bit confusing whether some of the claims are considering the number of network parameters or the actual width of layers and would like to see some more distinction. For example in experimental cases, it's possible to try out narrower but deeper networks with the same number of parameters as wider ones to isolate the width effect. I understand that this could be hard in such controlled cases and small MLP’s of course.\n\nI will address some more of my concerns in the question section.\n\nQUESTIONS:\n1) To be clear the authors don’t provide any theoretical explanation for why the other terms used for the upper bound in Thm 4.5 are negligible ? And on that note, am I correct in assuming that the point of Fig 4 in appendix is to show how the variance/mean and linear correlation don’t change much with model width ? If so, shouldn't you consider different initializations as well ? \n\n2) On equation 19, the authors make the claim that the first term in the lower bound is large early on ? Wouldn’t this also be true during the late stage of training since the learning rate is smaller and $ \\tilde{x} $ stabilizes ? With enough training epochs, I expect that to be the case right ? \n\n3) Could authors provide further explanation on the potential impact of weight regularization and co-correlation given Property 2 ? \n\n4) The comment made on line 267, wouldn’t it be the opposite that $ \\tilde{x}_* $ fluctuates more during the early stages of training with higher learning rate, as mentioned previously for Thm 4.5 ?\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors study the tradeoff between generalization and adversarial robustness from the perspective of collaboration between the layers of a neural network (NN). They adapt the concept of Dirichlet energy to analyze the robustness of different layers in the network. Decomposing this across layers allows them to quantify the alignment between feature selection between consecutive layers as collaboration correlation (or co-correlation). They show that the co-correlation increases while training, which leads to lower adversarial robustness, MLPs with larger widths exhibit more resistance to increased co-correlation and present supporting experimental results.\n\nSTRENGTHS:\nThis work presents a novel *view* on the tradeoff between adversarial robustness and generalization of NNs trained with gradient descent.\n\nWEAKNESSES:\n- While the results are somewhat interesting, it is unclear what the implications of these results are, since many of the observations in this work have been noted in prior work, and are not surprising. It is also not clear if the proposed concept of co-correlation can be used can be used for other applications .\n\n- There are several typos/grammatical errors/formatting issues that should be corrected:\n    - In line 56, ‘adversarial’ should be ‘adversarially’.\n    - The last part of lines 64-65 should be rephrased.\n    - In line 77, ‘digger’ should be ‘dig’.\n    - The heading of section 3 should be ‘Preliminaries’.\n    - Some of the notations (e.g., $sig$) should be corrected (to sig).\n    - In line 127, ‘Slimier’ should be ‘Similar’.\n    - The formatting of Theorem 4.1 (lines 155-156) should be corrected.\n    - In line 179, $J\\mathbf{\\psi}$ should be $J_{\\mathbf{\\psi}}$.\n    - In Ass. 5.2, L-2 should be $L_2$.\n    - In Prop. 1, ‘flattened’ should be ‘saturated’.\n    - A reference is missing in line 275.\n    - Line 297 should be rephrased.\n    - Missing ‘.’ in line 311.\n\nQUESTIONS:\nPlease see the weaknesses section.\n\nLIMITATIONS:\nThere are no societal negative impacts of this work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}