C RITICAL L EARNING P ERIODS IN D EEP N ETWORKS
A BSTRACT
Similar to humans and animals, deep artiﬁcial neural networks exhibit critical periods dur- ing which a temporary stimulus deﬁcit can impair the development of a skill.
The extent of the impairment depends on the onset and length of the deﬁcit window, as in animal mod- els, and on the size of the neural network.
Deﬁcits that do not affect low-level statistics, such as vertical ﬂipping of the images, have no lasting effect on performance and can be overcome with further training.
To better understand this phenomenon, we use the Fisher Information of the weights to measure the effective connectivity between layers of a net- work during training.
Counterintuitively, information rises rapidly in the early phases of training, and then decreases, preventing redistribution of information resources in a phe- nomenon we refer to as a loss of “Information Plasticity”.
Our analysis suggests that the ﬁrst few epochs are critical for the creation of strong connections that are optimal relative to the input data distribution.
Once such strong connections are created, they do not appear to change during additional training.
These ﬁndings suggest that the initial learning tran- sient, under-scrutinized compared to asymptotic behavior, plays a key role in determining the outcome of the training process.
Our ﬁndings, combined with recent theoretical results in the literature, also suggest that forgetting (decrease of information in the weights) is critical to achieving invariance and disentanglement in representation learning.
Finally, critical periods are not restricted to biological systems, but can emerge naturally in learning systems, whether biological or artiﬁcial, due to fundamental constrains arising from learning dynamics and information processing
I NTRODUCTION
Critical periods are time windows of early post-natal development during which sensory deﬁcits can lead to permanent skill impairment (Kandel et al., 2013).
Researchers have documented critical periods affecting a range of species and systems, from visual acuity in kittens (Wiesel & Hubel, 1963b; Wiesel, 1982) to song learning in birds (Konishi, 1985).
Uncorrected eye defects ( e.g. , strabismus, cataracts) during the critical period for visual development lead to amblyopia in one in ﬁfty adults.
The cause of critical periods is ascribed to the biochemical modulation of windows of neuronal plasticity (Hensch, 2004).
In this paper, however, we show that deep neural networks (DNNs), while completely devoid of such regulations, respond to sensory deﬁcits in ways similar to those observed in humans and animal models.
This surprising result suggests that critical periods may arise from information processing, rather than biochemical, phenomena.
We propose using the information in the weights, measured by an efﬁcient approximation of the
Fisher Information, to study critical period phenomena in DNNs.
We show that, counterintuitively, the information in the weights does not increase monotonically during training.
Instead, a rapid growth in information (“memorization phase”) is followed by a reduction of information (“reorganization” or “forgetting” phase), even as classiﬁcation performance keeps increasing.
This behavior is consistent across different tasks and network architectures.
Critical periods are centered in the memorization phase.
artiﬁcial neural networks (ANNs) are only loosely inspired by biological systems (Hassabis et al., 2017).
Most studies to date have focused either on the behavior of networks at convergence (Representation
Learning) or on the asymptotic properties of the numerical scheme used to get there (Optimization).
The role of the initial transient, especially its effect in biasing the network towards “good” regions of the complex and high-dimensional optimization problem, is rarely addressed.
To study this initial learning phase of ANNs, we replicate experiments performed in animal models and ﬁnd that the responses to early deﬁcits are remarkably similar, despite the large underlying differences between the two systems.
In particular, we show that the quality of the solution depends only minimally on the ﬁnal, relatively well-understood, phase of the training process or on its very ﬁrst epochs; instead, it depends critically on the period prior to initial convergence.
In animals, sensory deﬁcits introduced during critical periods induce changes in the architecture of the corresponding areas (Daw, 2014; Wiesel & Hubel, 1963a;
Hendrickson et al., 1987).
To determine whether a similar phenomenon exists in ANNs, we compute the Fisher Information of the weights of the network as a proxy to measure its “effective connectivity”, that is, the density of connections that are effectively used by the network in order to solve the task.
Like others before us (Shwartz-Ziv & Tishby, 2017), we observe two distinct phases during the training, ﬁrst a “learning phase” in which the Fisher Information of the weights increases as the network learns from the data, followed by a “consolidation” or “compression” phase in which the Fisher Information decreases and stabilizes.
Sensitivity to critical-period-inducing deﬁcits is maximal exactly when the Fisher
Information peaks.
A layer-wise analysis of the network’s effective connectivity shows that, in the tasks and deﬁcits we consider, the hierarchy of low-level and high-level features in the training data is a key aspect behind the observed phenomena.
In particular, our experiments suggest that the existence of critical periods in deep neural networks depends on the inability of the network to change its effective connectivity pattern in order to process different information (in response to deﬁcit removal).
We call this phenomenon, which is not mediated by any external factors, a loss of the “Information
Plasticity” of the network.
R ELATED WORK D EEP A RTIFICIAL N EURAL N ETWORKS EXHIBIT CRITICAL PERIODS
A notable example of critical period-inducing deﬁcit, which also commonly affects humans, is amblyopia (reduced visual acuity in one eye) caused unilateral cataracts during infancy or childhood (Vaegan & Taylor, 1979; von Noorden, 1981): Even after surgical correction of the cataracts, the
Under review as a conference paper at ICLR in A and B respectively.
ability of the patients to regain normal acuity in the affected eye depends both on the duration of the deﬁcit and on its age of onset, with earlier and longer deﬁcits causing more severe effects.
In order to replicate this experimental setup in ANNs, we train a standard convolutional network (CNN) to classify objects in small⇥ 32 RGB images from the CIFAR-10 dataset (Krizhevsky & Hinton, 2009) in 10 classes.
To simulate the effect of cataracts, for the ﬁrst t 0 epochs the images in thedatasetaredownsampledto 8 ⇥ 8 andthenupsampledbackto 32 ⇥ 32 usingbilinearinterpolation, in practice blurring the image and destroying small-scale details.
1 After that, the training continues for 300 more epochs, giving the network enough time to converge and ensuring it is exposed to the same number of uncorrupted images as in the control ( t 0 = 0 ) experiment.
In Figure 1, we graph the ﬁnal performance of the network (described in Materials and Methods) as a function of the epoch at which the deﬁcit is corrected ( t 0 ).We
clearly observe the existence of a critical period for this deﬁcit in the ANN: if the blur is not removed within the ﬁrst 60 epochs, the ﬁnal performance is severely decreased when compared to the baseline (from a test error of ⇠ 6 . 4% , [In this plot it is 8%, it is 6.4% for the resnet later.
We can swap the plots or change the text] in the absence of a deﬁcit, to more than 18% when the blur is present over 140 epochs, a ⇠ 300% increase).
The proﬁle of the curve is also strikingly similar to the one obtained in kittens monocularly deprived from near birth and whose visual acuity upon eye-opening was tested and plotted against the length of the deﬁcit window (Mitchell, 1988).
Just like in humans and animal models (where critical periods are characteristic of early development), the critical period in the DNN also arises during the initial rapid learning phase.
At this stage, the network is quickly learning a solution before the test error plateaus and the longer asymptotic convergence phase begins.
Sensitivity to deﬁcit.
To quantify more accurately the sensitivity of the ANN to image blurring throughoutitsearlylearningphase, weintroducedthedeﬁcitinashortconstantwindow(40epochs), starting at different epochs, and then measured the decrease in the ANN’s ﬁnal performance com- pared to the baseline.
In Figure 2, we plot the ﬁnal testing error of the network against the epoch of onset of the deﬁcit.
We observe that the network’s sensitivity to blurring peaks in the central part of the early rapid learning phase (around 30 epochs), while later deﬁcits produce little or no effect.
A similar experiment was also performed on kittens by Olson and Freeman, using a window of 10-12 days during which the animals were monocularly deprived and using it to “scan” the ﬁrst 4 months after birth to obtain a sensitivity proﬁle (Olson & Freeman, 1980).
We subsequently evaluated the effect of other training data modiﬁcations: a more drastic deprivation where the input is substituted with random noise, simulating complete sensory deprivation, and two “high-level” modiﬁcations of the training data: vertical ﬂipping of the input image and permutation
We employed this method, instead of a simpler Gaussian blur, since it has a very similar effect and makes the quantiﬁcation of information loss clearer. a onset of short
The decrease in the ﬁnal performance can be used to measure the sensitivity to deﬁcits.
The most sensitive epochs corresponds to the early rapid learning phase, before the test error (dashed line) begins to plateau. Afterwards, the network is largely unaffected by the temporary deﬁcit. (D) This can be compared with changes in the degree of functional disconnection (normalized numbers
V1 monocular cells disconnected from the contralateral eye) as a function of the kittens’ age at the onset of a 10-12-day deﬁcit window (
& Freeman,
Our ﬁndings, described in Section 2, indicate that the early transient is critical in determining the ﬁnal solution of the optimization associated with training an artiﬁcial neural network.
In particular, the effects of sensory deﬁcits during a critical period cannot be overcome, no matter how much additional training is performed.
Yet most theoretical studies have focused on the network behavior after convergence (Representation Learning) or on the asymptotic properties of the optimization scheme used for training (SGD).
To study this early phase, in Section 3, we use the Fisher Information to quantify the effective connectivity of a network during training, and introduce the notion of Information Plasticity in learning.
Information Plasticity is maximal during the memorization phase, and decreases in the reorganization phase.
We show that deﬁcit sensitivity during critical periods correlates strongly with the effective connectivity.
In Section 4 we discuss our contribution in relation to previous work.
When considered in conjunction with recent results on representation learning (Achille & Soatto, 2018), our ﬁndings indicate that forgetting (reducing information in the weights) is critical to achieving invariance to nuisance variability as well as independence of the components of the representation, but comes at the price of reduced adaptability later in the training.
We also hypothesize that the loss of physical connectivity in biology (neural plasticity) could be a consequence, rather than a cause, of the loss of Information Plasticity, which depends on how the information is distributed throughout a network during the early stages of learning.
These results also shed light on the common practice of pre-training a model on a task and then ﬁne-tune it for another, one of the most rudimentary forms of transfer learning.
Our experiments show that, rather than helpful, pre-training can be detrimental, even if the tasks are similar ( e.g. , same labels, slightly blurred images).
E XPERIMENTS
A notable example of critical period-related deﬁcit, commonly affecting humans, is amblyopia (reduced visual acuity in one eye) caused by cataracts during infancy or childhood (Taylor et al., 1979; [Figure] von Noorden, 1981).
Even after surgical correction of cataracts, the ability of the patients to regain normal acuity in the affected eye depends both on the duration of the deﬁcit and on its age of onset, with earlier and longer deﬁcits causing more severe effects.
In this section, we aim to study the effects of similar deﬁcits in DNNs.
To do so, we train a standard All-CNN architecture based on
Springenberg et al.
(2014) (see Appendix A) to classify objects in small 32 × 32 images from the
CIFAR-10 dataset (Krizhevsky & Hinton, 2009).
We train with SGD using an exponential annealing schedule for the learning rate.
To simulate the effect of cataracts, for the ﬁrst t 0 epochs the images in the dataset are downsampled to 8 × 8 and then upsampled back to 32 × 32 using bilinear interpolation, in practice blurring the image and destroying small-scale details.
1 After that, the training continues for 160 more epochs, giving the network time to converge and ensuring it is exposed to the same number of uncorrupted images as in the control ( t 0 = 0 ) experiment.
DNNs exhibit critical periods: In Figure 1, we plot the ﬁnal performance of a network affected by the deﬁcit as a function of the epoch t 0 at which the deﬁcit is corrected.
We can readily observe the existence of a critical period: If the blur is not removed within the ﬁrst 40-60 epochs, the ﬁnal performance is severely decreased when compared to the baseline (up to a threefold increase in error).
The decrease in performance follows trends commonly observed in animals, and may be qualitatively compared, for example, to the loss of visual acuity observed in kittens monocularly deprived from birth as a function of the length of the deﬁcit (Mitchell, 1988).
We can measure more accurately the sensitivity to a blur deﬁcit during learning by introducing the deﬁcit in a short window of constant length (40 epochs), starting at different epochs, and then measure the decrease in the DNN’s ﬁnal performance compared to the baseline (Figure 1).
Doing this, we observe that the sensitivity to the deﬁcit peaks in the central part of the early rapid learning phase (at around 30 epochs), while introducing the deﬁcit later produces little or no effect.
A similar experiment performed on kittens, using a window of 10-12 days during which the animals are monocularly deprived, again shows a remarkable similarity between the proﬁles of the sensitivity curves (Olson & Freeman, 1980).
High-level deﬁcits are not associated with a critical period: A natural question is whether any change in the input data distribution will have a corresponding critical period for learning.
This is not the case for neuronal networks, which remain plastic enough to adapt to high-level changes in sensory processing (Daw, 2014).
For example, it is well-reported that even adult humans can rapidly adapt to certain drastic changes, such as the inversion of the visual ﬁeld (Stratton, 1896; Kohler, 1964).
In Figure 2, we observe that DNNs are also largely unaffected by high-level deﬁcits – such as vertical ﬂipping of the image, or random permutation of the output labels: After deﬁcit correction, the network quickly recovers its baseline performance.
This hints at a ﬁner interplay between the
[Figure]
structure of the data distribution and the optimization algorithm, resulting in the existence of a critical period.
Sensory deprivation: We now apply to the network a more drastic deﬁcit, where each image is replaced by white noise.
Figure 2 shows hows this extreme deﬁcit exhibits a remarkably less severe effect than the one obtained by only blurring images: Training the network with white noise does not provide any information on the natural images, and results in milder effects than those caused by a deﬁcit ( e.g. , image blur), which instead conveys some information, but leads the network to (incorrectly) learn that no ﬁne structure is present in the images.
A similar effect has been observed in animals, where a period of early sensory deprivation (dark-rearing) can lengthen the critical period and thus cause less severe effects than those documented in light-reared animals (Mower, 1991).
We refer the reader to Appendix C for a more detailed comparison between sensory deprivation and training on white noise.
Architecture, depth, and learning rate annealing: Figure 3 shows that a fully-connected network trained on the MNIST digit classiﬁcation dataset also shows a critical period for the image blur deﬁcit.
Therefore, the convolutional structure is not necessary, nor is the use of natural images.
Similarly, a ResNet-18 trained on CIFAR-10 also has a critical period, which is also remarkably sharper than the one found in a standard convolutional network (Figure 1).
This is especially interesting, since ResNets allow for easier backpropagation of gradients to the lower layers, thus suggesting that the critical period is not caused by vanishing gradients.
However, Figure 2 (Right) shows that the presence of a critical period does indeed depend critically on the depth of the network.
In Figure 3, we conﬁrm that a critical period exists even when the network is trained with a constant learning rate, and therefore cannot be explained by an annealed learning rate in later epochs.
Optimization method and weight decay: Figure 3 (Bottom Right) shows that when using Adam as the optimization scheme, which renormalizes the gradients using a running mean of their ﬁrst two moments, we still observe a critical period similar to that of standard SGD.
However, changing the
[Figure]
as it both slows the convergence of the network, and it limits the ability of higher layers to change to overcome the deﬁcit, thus encouraging lower layers to also learn new features.
F ISHER I NFORMATION A NALYSIS
We have established empirically that, in animals and DNNs alike, the initial phases of training are critical to the outcome of the training process.
In animals, this strongly relates to changes in the brain architecture of the areas associated with the deﬁcit (Daw, 2014).
This is inevitably different in artiﬁcial networks, since their connectivity is formally ﬁxed at all times during training.
However, not all the connections are equally useful to the network: Consider a network encoding the approximate posterior distribution p w ( y | x ) , parameterized by the weights w , of the task variable
y given an input image x .
The dependency of the ﬁnal output from a speciﬁc connection can be estimated by perturbing the corresponding weight and looking at the magnitude of the change in the ﬁnal distribution.
Speciﬁcally, given a perturbation w (cid:48)
= w + δw of the weights, the discrepancy between the p w ( y | x ) and the perturbed network output p w (cid:48)
( y | x ) can be measured by their KullbackLeibler divergence, which, to second-order approximation, is given by:
[Equation]
where the expectation over x is computed using the empirical data distribution ˆ Q ( x ) given by the dataset, and
[Equation]
is the Fisher Information Matrix (FIM).
The FIM can thus be considered a local metric measuring how much the perturbation of a single weight (or a combination of weights) affects the output of the network (Amari & Nagaoka, 2000).
In particular, weights with low Fisher Information can be changed or “pruned” with little effect on the network’s performance.
This suggests that the Fisher
Information can be used as a measure of the effective connectivity of a DNN, or, more generally, of the “synaptic strength” of a connection (Kirkpatrick et al., 2017).
Finally, the FIM is also a semideﬁnite approximation of the Hessian of the loss function (Martens, 2014) and hence of the curvature of the loss landscape at a particular point w during training, providing an elegant connection between the FIM and the optimization procedure (Amari & Nagaoka, 2000), which we will also employ later.
Unfortunately, the full FIM is too large to compute.
Rather, we use its trace to measure the global or layer-wise connection strength, which we can compute efﬁciently using (Appendix A):
[Equation]
In order to capture the behavior of the off-diagonal terms, we also tried computing the logdeterminant of the full matrix using the Kronecker-Factorized approximation of Martens & Grosse (2015), but we observed the same qualitative trend as the trace.
Since the FIM is a local measure, it is very sensitive to the irregularities of the loss landscape.
Therefore, in this section we mainly use
ResNets, which have a relatively smooth landscape (Li et al., 2018).
For other architectures we use instead a more robust estimator of the FIM based on the injection of noise in the weights (Achille &
Soatto, 2018), also described in Appendix A.
Two phases of learning: As its name suggests, the FIM can be thought as a measure of the quantity of information about the training data that is contained in the model (Fisher, 1925).
Based on this, one would expect the overall strength of the connections to increase monotonically as we acquire information from experience.
However, this is not the case: While during an initial phase the network acquires information about the data, which results in a large increase in the strength of the connections, once the performance in the task begins to plateau, the network starts decreasing the overall strength of its connections.
However, this does not correspond to a reduction in performance, rather, performance keeps slowly improving.
This can be seen as a “forgetting, or “compression” phase, during which redundant connections are eliminated and non-relevant variability in the data is discarded.
It is well-established how the elimination (“pruning”) of unnecessary synapses is a fundamental process during learning and brain development (Rakic et al., 1986) (Figure 4, Center); in Figure 4 (Left) an analogous phenomenon is clearly and quantitatively shown for DNNs.
Strikingly, these changes in the connection strength are closely related to the sensitivity to criticalperiod-inducing deﬁcits such as image blur, computed using the “sliding window” method as in
Figure 1.
In Figure 4 we see that the sensitivity closely follows the trend of the FIM.
This is remarkable since the FIM is a local quantity computed at a single point during the training of a network in the absence of deﬁcit, while sensitivity during a critical period is computed, using test data, at the end of the impaired network training.
Figure 4 (Right) further emphasizes the effect of deﬁcits on the FIM: in the presence of a deﬁcit, the FIM grows and remains substantially higher even after the deﬁcit is removed.
This may be attributed to the fact that, when the data are so corrupted that classiﬁcation is impossible, the network is forced to memorize the labels, therefore increasing the quantity of information needed to perform the same task.
Layer-wise effects of deﬁcits:
A layer-wise analysis of the FIM sheds further light on how the deﬁcit affects the network.
When the network (in this case All-CNN, which has a clearer division among layers than ResNet) is trained without deﬁcits, the most important connections are in the intermediate layers (Figure 5, Left), which can process the input CIFAR-10 image at the most informative intermediate scale.
However, if the network is initially trained on blurred data (Figure 5, top right), the strength of the connections is dominated by the top layer (Layer 6).
This is to be expected, since the low-level and mid-level structures of the images are destroyed, making the lower layers ineffective.
However, if the deﬁcit is removed early in the training (Figure 5, top center), the network manages to “reorganize”, reducing the information contained in the last layer, and, at the same time, increasing the information in the intermediate layers.
We refer to these phenomena as changes in “Information Plasticity”.
If, however, the data change occurs after the consolidation phase, the network is unable to change its effective connectivity: The connection strength of each layer remains substantially constant.
The network has lost its Information Plasticity and is past its critical period.
Critical periods as bottleneck crossings: The analysis of the FIM also sheds light on the geometry of the loss function and the learning dynamics.
Since the FIM can be interpreted as the local curvature of the residual landscape, Fig.
4 shows that learning entails crossing bottlenecks: In the initial phase the network enters regions of high curvature (high Fisher Information), and once consolidation begins, the curvature decreases, allowing it to cross the bottleneck and enter the valley below.
If the statistics change after crossing the bottleneck, the network is trapped.
In this interpretation, the early phases of convergence are critical in leading the network towards the “right” ﬁnal valley.
The end of critical periods comes after the network has crossed all bottlenecks (and thus learned the features) and entered a wide valley (region of the weight space with low curvature, or low Fisher Information).
[Figure]
D ISCUSSION AND R ELATED W ORK
Critical periods have thus far been considered an exclusively biological phenomenon.
At the same time, the analysis of DNNs has focused on asymptotic properties and neglected the initial transient behavior.
To the best of our knowledge, we are the ﬁrst to show that artiﬁcial neural networks exhibit critical period phenomena, and to highlight the critical role of the transient in determining the asymptotic performance of the network.
Inspired by the role of synaptic connectivity in modulating critical periods, we introduce the use of Fisher Information to study this initial phase.
We show that the initial sensitivity to deﬁcits closely follows changes in the FIM, both global, as the network ﬁrst rapidly increases and then decreases the amount of stored information, and layer-wise, as the network “reorganizes” its effective connectivity in order to optimally process information.
Our work naturally relates to the extensive literature on critical periods in biology.
Despite artiﬁcial networks being an extremely reductionist approximation of neuronal networks, they exhibit behaviors that are qualitatively similar to the critical periods observed in human and animal models.
Our information analysis shows that the initial rapid memorization phase is followed by a loss of Information Plasticity which, counterintuitively, further improves the performance.
On the other hand, when combined with the analysis of Achille & Soatto (2018) this suggests that a “forgetting” phase may be desirable, or even necessary, in order to learn robust, nuisance-invariant representations.
The existence of two distinct phases of training has been observed and discussed by Shwartz-Ziv &
Tishby (2017), although their analysis builds on the (Shannon) information of the activations , rather than the (Fisher) information in the weights .
On a multi-layer perceptron (MLP), Shwartz-Ziv &
Tishby (2017) empirically link the two phases to a sudden increase in the gradients’ covariance.
It may be tempting to compare these results with our Fisher Information analysis.
However, it must be noted that the FIM is computed using the gradients with respect to the model prediction, not to the ground truth label, leading to important qualitative differences.
In Figure 6, we show that the covariance and norm of the gradients exhibit no clear trends during training with and without deﬁcits, and, therefore, unlike the FIM, do not correlate with the sensitivity to critical periods.
However, a connection between our FIM analysis and the information in the activations can be established based on the work of Achille & Soatto (2018), which shows that the FIM of the weights can be used to bound the information in the activations.
In fact, we may intuitively expect that pruning of connections naturally leads to loss of information in the corresponding activations.
Thus, our analysis corroborates and expands on some of the claims of Shwartz-Ziv & Tishby (2017), while using an independent framework.
Aside from being more closely related to the deﬁcit sensitivity during critical periods, Fisher’s Information also has a number of technical advantages: Its diagonal is simple to estimate, even on modern state-of-the-art architectures and compelling datasets, and it is less sensitive to the choice estimator of mutual information, avoiding some of the common criticisms to the use of information quantities in the analysis of deep learning models.
Finally, the FIM allows us to probe ﬁne changes in the effective connectivity across the layers of the network (Figure 5), which are not visible in
Shwartz-Ziv & Tishby (2017).
A complete analysis of the activations should account not only for the amount of information (both task- and nuisance-related), but also for its accessibility, e.g. , how easily task-related information can be extracted by a linear classiﬁer.
Following a similar idea, Montavon et al.
(2011) aim to study the layer-wise, or “spatial” (but not temporal) evolution of the simplicity of the representation by performing a principal component analysis (PCA) of a radial basis function (RBF) kernel embedding of each layer representation.
They show that, on a multi-layer perceptron, task-relevant information increasingly concentrate on the ﬁrst principal components of the representation’s embedding, implying that they become more easily “accessible” layer after layer, while nuisance information (when it is codiﬁed at all) is encoded in the remaining components.
In our work we instead focus on the temporal evolution of the weights.
However, it’s important to notice that a network with simpler weights (as measured by the FIM) also requires a simpler smooth representation (as measured, e.g. , by the RBF embedding) in order to operate properly, since it needs to be resistant to perturbations of the weights.
Thus our analysis is wholly compatible with the intuitions of Montavon et al.
It would also be interesting to study the joint spatio-temporal evolution of the network using both frameworks at once.
One advantage of focusing on the information of the weights rather than on the activations, or behavior of the network, is to have a readout of the “effective connectivity” during critical periods, which can be compared to similar readouts in animals.
In fact, “behavioral” readouts upon deﬁcit removal, both in artiﬁcial and neuronal networks, can potentially be confounded by deﬁcit-coping changes at different levels of the visual pathways (Daw, 2014; Knudsen, 2004).
On the other hand, deﬁcits in deprived animals are mirrored by abnormalities in the circuitry of the visual pathways, which we characterize in DNNs using the FIM to study its “effective connectivity”, i.e. , the connections that are actually employed by the network to solve the task.
Sensitivity to critical periods and the trace of the Fisher Information peak at the same epochs, in accord with the evidence that skill development and critical periods in neuronal networks are modulated by changes (generally experience-dependent) in synaptic plasticity (Knudsen, 2004; Hensch, 2004).
Our layer-wise analysis of the Fisher Information (Figure 5) also shows that visual deﬁcits reinforce higher layers to the detriment of intermediate layers, leaving low-level layers virtually untouched.
If the deﬁcit is removed after the critical period ends, the network is not able to reverse these effects.
Although the two systems are radically different, a similar response can be found in the visual pathways of animal models: Lower levels ( e.g. , retina, lateral geniculate nucleus) and higher-level visual areas ( e.g. ,
V2 and post-V2) show little remodeling upon deprivation, while most changes happen in different layers of V1 (Wiesel & Hubel, 1963a;
Hendrickson et al., 1987).
An insightful interpretation of critical periods in animal models was proposed by Knudsen (2004):
The initial connections of neuronal networks are unstable and easily modiﬁed (highly plastic), but as more “samples” are observed, they change and reach a more stable conﬁguration which is difﬁcult to modify.
Learning can, however, still happen within the newly created connectivity pattern.
This is largely compatible with our ﬁndings: Sensitivity to critical-period-inducing deﬁcits peaks when connections are remodeled (Figure 4, Left), and different connectivity proﬁles are observed in networks trained with and without a deﬁcit (Figure 5).
Moreover, high-level deﬁcits such as imageﬂipping and label permutation, which do not require restructuring of the network’s connections in order to be corrected, do not exhibit a critical period.
Applying a deﬁcit at the beginning of the training may be compared to the common practice of pretraining, which is generally found to improve the performance of the network.
Erhan et al.
(2010) study the somewhat related, but now seldom used, practice of layer-wise unsupervised pre-training, and suggest that it may act as a regularizer by moving the weights of the network towards an area of the loss landscape closer to the attractors for good solutions, and that early examples have a stronger effect in steering the network towards particular solutions.
Here, we have shown that pre-training on blurred data can have the opposite effect; i.e. , it can severely decrease the ﬁnal performance of the network.
However, in our case, interpreting the deﬁcits effect as moving the network close to a bad attractor is difﬁcult to reconcile with the smooth transition observed in the critical periods, since the network would either converge to this attractor, and thus have low accuracy, or escape completely.
Instead, we reconcile our experiments with the geometry of the loss function by introducing a different explanation based on the interpretation of the FIM as an approximation of the local curvature.
Figure 4 suggests that SGD encounters two different phases during the network training: At ﬁrst, the network moves towards high-curvature regions of the loss landscape, while in the second phase the curvature decreases and the network eventually converges to a ﬂat minimum (as observed in
Keskar et al.
We can interpret these as the network crossing narrow bottlenecks during its training in order to learn useful features, before eventually entering a ﬂat region of the loss surface once learning is completed and ending up trapped there.
When combining this assumption with our deﬁcit sensitivity analysis, we can hypothesize that the critical period occurs precisely upon crossing of this bottleneck.
It is also worth noticing how there is evidence that convergence to ﬂat minima (minima with low curvature) in a DNN correlates with a good generalization performance (Hochreiter & Schmidhuber, 1997;
Li et al., 2018; Chaudhari et al., 2017; Keskar et al., 2017).
Indeed, using this interpretation, Figure 4 (Right) tells us that networks more affected by the deﬁcit converge to sharper minima.
However, we have also found that the performance of the network is already mostly determined during the early “sensitive” phase.
The ﬁnal sharpness at convergence may therefore be an epiphenomenon, rather than the cause of good generalization.
C ONCLUSION
Our goal in this paper is not so much to investigate the human (or animal) brain through artiﬁcial networks, as to understand fundamental information processing phenomena, both in their biological or artiﬁcial implementations.
It is also not our goal to suggest that, since they both exhibit critical periods, DNNs are necessarily a valid model of neurobiological information processing, although recent work has emphasized this aspect.
We engage in an “Artiﬁcial Neuroscience” exercise in part to address a technological need to develop “explainable” artiﬁcial intelligence systems whose behavior can be understood and predicted.
While traditionally well-understood mathematical models were used by neuroscientists to study biological phenomena, information processing in modern artiﬁcial networks is often just as poorly understood as in biology, so we chose to exploit well-known biological phenomena as probes to study information processing in artiﬁcial networks.
Conversely, it would also be interesting to explore ways to test whether biological networks prune connections as a consequences of a loss of Information Plasticity, rather than as a cause.
The mechanisms underlying network reconﬁguration during learning and development might be an evolutionary outcome obtained under the pressure of fundamental information processing phenomena.
A CKNOWLEDGEMENTS
We thank the anonymous reviewers for their thoughtful feedback, and for suggesting new experiments and relevant literature.
Supported by ONR N00014-17-1-2072, ARO W911NF-17-1-0304,
AFOSR FA9550-15-1-0229 and FA8650-11-1-7156.
A D ETAILS OF THE EXPERIMENTS A. A RCHITECTURES AND TRAINING
In all of the experiments, unless otherwise stated, we use the following All-CNN architecture, adapted from Springenberg et al.
[Equation]
where each conv block consists of a 3 × 3 convolution, batch normalization and ReLU activations.
conv denotes a 1 × 1 convolution.
The network is trained with SGD, with a batch size of 128, learning rate starting from 0.05 and decaying smoothly by a factor of .97 at each epoch.
We also use weight decay with coefﬁcient 0.001.
In the experiments with a ﬁxed learning rate, we ﬁx the learning rate to 0.001, which we ﬁnd to allow convergence without excessive overﬁtting.
For the
ResNet experiments, we use the ResNet-18 architecture from He et al.
(2016) with initial learning rate 0.1, learning rate decay .97 per epoch, and weight decay 0.0005.
When training with Adam, we use a learning rate of 0.001 and weight decay 0.0001.
When experimenting with varying network depths, we use the following architecture:
[Equation]
In order to avoid interferences between the annealing scheme and the architecture, in these experiments we ﬁx the learning rate to 0.001.
The Fully Connected network used for the MNIST experiments has hidden layers of size [2500 , 2000 , 1500 , 1000 , 500] .
All hidden layers use batch normalization followed by ReLU activations.
We ﬁx the learning rate to 0.005.
Weight decay is not used.
We use data augmentation with random translations up to 4 pixels and random horizontal ﬂipping.
For MNIST, we pad the images with zeros to bring them to size 32 × 32 .
A. A PPROXIMATIONS OF THE F ISHER I NFORMATION M ATRIX
To compute the trace of the Fisher Information Matrix, we use the following expression derived directly from the deﬁnition:
[Equation]
where the input image x is sampled from the dataset, while the label y is sampled from the output posterior.
Expectations are approximated by Monte-Carlo sampling.
Notice, however, that this expression depends only on the local gradients of the loss with respect to the weights at a point w = w 0 , so it can be noisy when the loss landscape is highly irregular.
This is not a problem for
ResNets Li et al.
(2018), but for other architectures we use instead a different technique, proposed in Achille & Soatto (2018).
More in detail, let L ( w ) be the standard cross-entropy loss.
Given the current weights w 0 of the network, we ﬁnd the diagonal matrix Σ that minimizes:
[Equation]
where β is a parameter that controls the smoothness of the approximation.
Notice that L (cid:48) can be minimized efﬁciently using the method in Kingma et al. (2015).
To see how this relates to the Fisher Information Matrix, assume that L ( w ) can be approximated locally in w 0 as L ( w ) =
L 0 + a · w + w · Hw .
We can then rewrite L (cid:48) as
[Equation]
Taking the derivative with respect to Σ , and setting it to zero, we obtain Σ ii = β/H ii .
We can then use Σ to estimate the trace of the Hessian, and hence of the Fisher information.
A. C URVE FITTING
Fitting of sensitivity curves and synaptic density proﬁles from the literature was performed using:
[Equation]
as the ﬁtting equation, where t is the age at the time of sampling and τ 1 , τ 2 , k and d are unconstrained parameters (Banks et al., 1975).
The exponential ﬁt of the sensitivity to the Fisher Information trace uses the expression
[Equation]
where a , b and c are unconstrained parameters, F ( t ) is the Fisher Information trace at epoch t of the training of a network without deﬁcits and S k is the sensitivity computed using a window of size k .
That is, S k ( t ) is the increase in the ﬁnal test error over a baseline when the network is trained in the presence of a deﬁcit between epochs t and t + k .
B A DDITIONAL PLOTS
[Figure]
N o r m .
I n f o i n
W e i g h t s to the lower-layers rather than higher-layers.
This may explain why it is easier for the network to reconﬁgure to solve the task after the deﬁcit is removed.
[Figure]
C E XPERIMENTAL DESIGN AND COMPARISON WITH ANIMAL MODELS
Critical periods are task- and deﬁcit-speciﬁc.
The speciﬁc task we address is visual acuity, but the performance is necessarily measured through different mechanisms in animals and Artiﬁcial Neural
Networks.
In animals, visual acuity is traditionally measured by testing the ability to discriminate between black-and-white contrast gratings (with varying spatial frequency) and a uniform gray ﬁeld.
The outcome of such tests generally correlates well with the ability of the animal to use the eye to solve other visual tasks relying on acuity.
Convolutional Neural Networks, on the other hand, have a very different sensory processing mechanism (based on heavily quantized data), which may trivialize such a test.
Rather, we directly measure the performance of the network on an high-level task, speciﬁcally image classiﬁcation, for which CNNs are optimized.
We chose to simulate cataracts in our DNN experiments, a deﬁcit which allows us to explore its complex interactions with the structure of the data and the architecture of the network.
Unfortunately, while the overall trends of cataract-induced critical periods have been studied and understood in animal models, there is not enough data to conﬁdently regress sensibility curves comparable to those obtained in DNNs.
For this reason, in Figure 1 we compare the performance loss in a DNN trained in the presence of a cataract-like deﬁcit with the results obtained from monocularly deprived kittens, which exhibit similar trends and are one of the most common experimental paradigms in the visual neurosciences.
Simulating complete visual deprivation in a neural network is not as simple as feeding a constant stimulus: a network presented with a constant blank input will rapidly become trivial and thus unable to train on new data.
This is to be expected, since a blank input is a perfectly predictable stimulus and thus the network can quickly learn the (trivial) solution to the task.
We instead wanted to model an uninformative stimulus, akin to noise.
Moreover, even when the eyes are sutured or maintained in the darkness, there will be background excitation of photoreceptors that is best modeled as noise.
To account for this, we simulate sensory deprivation by replacing the input images with a dataset composed of (uninformative) random Gaussian noise.
This way the network is trained on solving the highly non-trivial task of memorizing the association between the ﬁnitely-many noise patterns and their corresponding labels.