Paper: Critical Learning Periods Emerge Even in Deep Linear Networks (ICLR-2024)

Reviewer comment (Theory):
The authors prove the dependence on depth and data distribution. But how can they claim independence from other factors, like architecture and non-linearities, especially when considering models other than deep linear networks?

Author response:
The reviewer is correct that the point is misphrased. Indeed, critical periods can depend, in part, on other factors (such as optimization algorithm and non-linearities), which has also been shown empirically in nonlinear networks (Achille et al., 2019). We rephrased the point in our introduction and discussion. We want to emphasize, however, that even ablating those details (nonlinearities and optimization algorithm), as done in our paper, we still observe critical periods that depend on architecture depth and data distribution. This means that the effect of other factors is not necessary for the emergence of critical periods in deep networks, and in general may have a less significant effect than architecture depth and data distribution. 

In our revised paper, we ran additional experiments in our multipathway setup where we included nonlinear activation functions (Relu and Tanh) and observed similar trends to in the deep multipathway linear case (Fig 11 and Fig 12 in revision).

Reviewer comment (Reproducibility):
Codebase not provided.

Author response:
We will provide our code along with the final version of our paper. 

Thank you for your constructive feedback. We are particularly appreciative of your responsiveness and consideration in revising your score after reading our response. We have anonymously uploaded our code for our multipathway experiments here: https://anonymous.4open.science/r/CriticalPeriodLinearMultiPath-AFBB and for the matrix completion experiments here: https://anonymous.4open.science/r/AnonCriticalPeriodMatrixCompletion-2AF7

Reviewer comment (Writing):
Typos:  
* Page 3: had minimum the → had the minimum  
* Page 6: training has only affects → training only affects

Author response:
We fixed the typos. Thanks for the careful reading and for catching those!

Reviewer comment (Experiments):
In the multi-pathway experiments, when a deficit was applied to a pathway (referred to as “blocking” or “gating” an input to a pathway in the paper), the desired target output was also shifted by a baseline amount corresponding to the deprived pathway’s output. This was to ensure that the normal pathway was only required to learn the unexplained component of the output, and not the entire output. The reviewer understood the use of PyTorch’s “.detch()” for blocking the flow of gradients in the deprived branch, but shifting the desired target output to ensure learning the unexplained component did not make sense to them and requires elaboration.

Author response:
Detaching is how it is implemented in our code. The alternative (and equivalent) interpretation as shifting the target was provided solely to relate it to a more biologically meaningful process. Specifically, consider a deficit where parameters along pathway A are not updated. Without the deficit the Loss is $L_{normal} = (y - W_A^{tot}x - W_B^{tot}x)^2$, and gradients are updated in both pathways. During the deficit (to pathway A) the loss is equivalent to $L_{def} = (\tilde{y} - W_B^{tot} x )^2$ where $\tilde{y} = y - W_A^{tot}x$, and gradients only occur in pathway B. We referred to $\tilde{y}$ as the shifted output.

Reviewer comment (Theory):
I think the assumption of the deep linear network is a bit strong. Since in real-world applications, most neural networks require non-linear activations. A deep linear network can be actually approximated by a single-layer linear network.

Author response:
While it is true that a deep linear network has the same expressive power as a single-layer linear network, we emphasize that the parameterization of a deep linear network leads to complex and depth-dependent non-linear learning dynamics that is sufficient to lead to the emergence of critical periods. By restricting our analyses to deep linear networks, we highlight that the deep parameterization alone (which is shared with non-linear networks) is enough to explain the emergence of critical periods and may indeed be the most important factor underlying them. Indeed, in our updated experiments in our revision paper that we posted, we show that even a non-linear network follows largely the exact same trends as the multipathway linear network in our experiments (Figure 11, Figure 12).

Reviewer comment (Theory):
Critical periods in artificial deep neural networks (DNNs) may be due to specificities of the optimization process, such as an annealing learning rate, or from defects in the artificial implementation and training, like ReLU units becoming frozen or gradients vanishing.

Author response:
We agree and acknowledge this directly in our introduction as one of the main motivations for studying deep linear networks in our paper. In particular we show that deep linear networks trained with constant learning rate and without nonlinearities also exhibit analogous phenomena, and also permit analytical analysis. As we replied to R1, we don't want to claim that in any model depth and structure of the data distribution are the only factors determining the existence of critical periods. In fact, other papers have shown that choice of optimization algorithm or regularization can significantly affect the critical period (Achille et al., 2019). What we show is that depth and structure of the data are a minimal set of requirements that alone can support the emergence of a critical period, and also enable differential equations that precisely characterize the learning dynamics. We clarified the text to better communicate this.

Reviewer comment (Experiments):
What's the model's generalization ability on more sparse data settings?

Author response:
We interpret this question to be referring to our matrix completion settings. In Fig. 5 right, we examine reconstruction error as a function of the number of entries. In general matrix completion becomes easier with a larger number of observed entries. When we have a very sparse number of entries (1000 entries) the network obtains a large reconstruction error even without pre-training on a different task (this corresponds to the point deficit removal at 0). In the sparse setting too, the network obtains worse reconstruction by initially training on the higher rank task (other points along the curves). 
We also found that pre-training on a subset of the observed entries adversely affects generalization (Fig 5, left). We added in an additional experiment to highlight that this effect also depends on the depth of the parameterization, with deeper architectures being more affected (Fig 13).

Reviewer comment (Theory):
The framing of the hypothesis in the beginning and conclusion is oversimplified or misframed. While the literal biochemical explanation may not hold in an artificial setting, analogous phenomena such as reduced plasticity could still occur. The reviewer does not see these as ‘competing’ hypotheses as they are framed; instead, plasticity, imperfect optimization, etc. are mechanisms by which critical periods could arise in both artificial and biological systems. This framing does not explain the underlying reason for critical periods (whether inherent to learning, due to substrate quirks, or something else). In the conclusion, “natural information processing constraints” are contrasted with biochemical processes like plasticity, but plasticity itself could be considered a natural information processing constraint. Demonstrating the presence of such phenomena in artificial systems does not establish them as general features of learning, except under a narrow definition encompassing only biological systems and ANNs. This leads to a somewhat circular argument about the meaning of “learning,” and the commonality is actually supported by the experiments in Fig. 3 recreating Guillery’s. The reviewer would prefer framing around mechanisms or instantiations of information processing constraints, with acknowledgement of the ambiguity of “learning systems in general,” while preserving clarity and narrative.

Author response:
We appreciate this thoughtful and insightful point. We updated the introduction and discussion to better highlight this nuance – indeed “natural information processing constraints” should not be contrasted with biochemical processes. In our introduction and discussion, we aim to emphasize that architecture and data distribution are sufficient for leading to critical periods in deep artificial networks, and that this offers an intriguing, and different lens for understanding critical periods in biological systems. We plan to further elaborate on this in subsequent revisions subject to space constraints and incorporating other reviewer feedback during the discussion period.

Reviewer comment (Writing):
Terms used in equations in Section 3.1 are not explained when first introduced. Although the equations are from a citation and space may be limited, failing to explain the terms could cause many readers to disengage, which is unfortunate given the clear prose used elsewhere in the paper.

Author response:
We updated our manuscript to better introduce notation (also pointed out by other reviewers).

Reviewer comment (Theory):
Around equation 9, the connection between matrix completion and the goals of the paper is unclear. The reviewer suggests making explicit how imputing missing values is equivalent to or allows specification of task relationships with flexibility.

Author response:
Thanks, we updated our manuscript to better introduce the matrix completion setting – the changes are in blue text of our revision.

Reviewer comment (Evaluation):
While the relationship between experiments and dependence on depth is clear and well explained, the insights and relationship between experiments and data distribution are much less clear.

Author response:
We appreciate the feedback. In general, pre-training on a modified data distribution for some duration can either be helpful or harmful, depending on the relationships between tasks. In the case of critical learning periods, we focus on the case when such pre-training is harmful. Empirically, Achille et al., (2019) found that pre-training on blurred images was harmful for subsequent generalization on normal images, but pre-training on pure noise for the same duration was much less harmful. This suggests that the learning dynamics depend on the complex relationship between the tasks.

In general it’s difficult to define the “complexity of a task” or the relationships between tasks. However, the matrix completion setting gives us the ability to completely specify the task, its complexity, and the relationship between tasks. We find that pre-training on a higher rank task (more complex) leads to worse generalization performance (Fig 5, left). We also found that pre-training on a subset of the observed entries (a partial view of the data, similar to the blurred image experiments) also leads to impaired generalization (Fig 6, left). We added in an experiment to highlight that this effect becomes more pronounced in deeper architectures (Fig 13 of revision). 

We updated the text to better motivate these experiments in text.

Reviewer comment (Experiments):
The reviewer would like to see connections made or discussed with nonlinear networks, including expectations for what should change versus remain the same according to cited sources (e.g., Saxe), or ideally through experiments with MLPs of increasing depth.

Author response:
We expect deeper networks to be more affected by a temporary deficit, and we now directly discuss this connection in the text. This finding that deeper architectures are more affected by an initially training on a related, partial dataset, is consistent with the empirical results of Achille et al., (2019, in particular see Fig 2, right panel of reference) and Kleinman et al., 2023 (in particular see Fig 5 center panel of reference) who empirically found that deeper convolutional architectures were increasingly affected by initially networks on blurred images before training on regular images. We also ran an additional experiment where we trained on a subset of the observed entries, and we found that while a single layer network is not affected, deeper architectures are increasingly affected by a sufficiently long initial deficit (Figure 13 of revision) We thank the reviewer for the helpful suggestion.

Reviewer comment (Evaluation):
A specific link could be made in Fig. 6 to scaling laws.

Author response:
We expect deeper networks to be more affected by a temporary deficit, and we now directly discuss this connection in the text. This finding that deeper architectures are more affected by an initially training on a related, partial dataset, is consistent with the empirical results of Achille et al., (2019, in particular see Fig 2, right panel of reference) and Kleinman et al., 2023 (in particular see Fig 5 center panel of reference) who empirically found that deeper convolutional architectures were increasingly affected by initially networks on blurred images before training on regular images. We also ran an additional experiment where we trained on a subset of the observed entries, and we found that while a single layer network is not affected, deeper architectures are increasingly affected by a sufficiently long initial deficit (Figure 13 of revision) We thank the reviewer for the helpful suggestion.

Reviewer comment (Writing):
There is inconsistent use of the terms “artificial neural network,” “deep net,” “neural net,” and others. While the reviewer finds this acceptable, it could be confusing for readers outside the field, so consistency is recommended.

Author response:
We thank the reviewer for the feedback and will better clarify the text. There were times where we wanted to emphasize the distinction between artificial and biological networks, as well as highlight the depths of the architectures, but we will strive for improved consistency and clarity.

Reviewer comment (Writing):
The reviewer did not understand the comment “and do not affect performance on the final task” in Fig. $. They ask whether this means that all configurations have equal performance.

Author response:
We clarified the sentence in our revision. Indeed, it means they lead to equivalent performance on the final task.

Reviewer comment (Theory):
The paper is less readable and requires a higher theoretical foundation. The description of the “linear multi-pathway framework” (Sec.3.1.) is not clear enough and lacks corresponding details.

Author response:
In our revised manuscript, we better introduce our notation in Sect 3.1. The updates are highlighted in blue text. We thank the reviewer for pointing this out, and we believe the changes lead to a more readable manuscript.

Reviewer comment (Experiments):
Only studying deep linear networks seems not enough to clearly understand why deep networks have critical periods, and network depth and data sources cannot guarantee sufficient persuasiveness.

Author response:
We don't want to claim that in any model depth and structure of the data distribution are the *only* factors determining the existence of critical periods. In fact, other papers have shown that choice of optimization algorithm or regularization can affect the critical period (Achille et al., 2019, Zilly et al., 2021). What we show instead is that depth and structure of the data are a minimal set of requirements that alone can support the emergence of a critical period, and also enable differential equations that precisely characterize the learning dynamics. This allowed us to better understand how depth, competition between sources, and data distribution affects critical learning periods. We have clarified the text in our revision.

Reviewer comment (Experiments):
There are fewer categories of experiments, and it would be better if the authors could provide more types of experiments to prove their claims.

Author response:
We ran additional experiments to further support our claims that we include in our revision. Related to the above question, for the multipathway simulations, we included results using Relu and Tanh nonlinearity and observed similar conclusions where deficits early on as in the deep linear case (Figure 11, and Figure 12 in revision)

For the matrix completion setup, we also ran an additional experiment where we trained on a subset of the observed entries, and we found that while a single layer network is not affected, deeper architectures are increasingly affected by a sufficiently long initial deficit (Figure 13). This finding that deeper architectures are more affected by an initially training on a related, partial dataset, is consistent with previous empirical results of Achille et al (2019; in particular see Fig 2, right panel of reference) and Kleinman et al (2023;  in particular see Fig 5, center panel of reference) who empirically found that deeper convolutional architectures (with nonlinearities) were increasingly affected by initially training on blurred images before training on regular images.

Reviewer comment (Theory):
I don’t find any support in the paper for what the abstract said: “Why critical learning periods emerge in biological and artificial networks”.

Author response:
Previous work emphasized several factors in the emergence of critical periods in deep networks such as the optimization algorithm and the non-linearities (Achille et al, 2019). However, these factors are absent from biological systems, thus making it difficult to establish a direct connection. We show that depth of the system is one of the main contributing factors to critical periods in DNNs, and is a more fundamental aspect which helps connecting with biological systems. This observation that deeper architectures are more prone to critical learning periods is consistent with neuroscience experiments beginning with Hubel and Wiesel that find that low-level sensory tasks (such as vision), which have deep circuits processing such information, are prone to critical periods.