Original: Brain-inspired predictive coding dynamics improve the robustness of deep neural networksAuthor(s
Revised: Brain-inspired predictive coding dynamics improve the robustness of deep neural networks

Original: AfﬁliationAddress

Original: Abstract
Revised: Abstract

Original: Deep neural networks excel at image classiﬁcation, but their performance is farless robust to input perturbations than human perception.
Revised: Deep neural networks excel at image classiﬁcation, but their performance is far less robust to input perturbations than human perception.

Original: In this work we addressthis shortcoming by incorporating brain-inspired recurrent dynamics in deep convolutional networks.
Revised: In this work we address this shortcoming by incorporating brain-inspired recurrent dynamics in deep convolutional networks.

Original: We augment a pretrained feedforward classiﬁcation model(VGG16 trained on ImageNet) with a “predictive coding” strategy: a frameworkpopular in neuroscience for characterizing cortical function.
Revised: We augment a pretrained feedforward classiﬁcation model (VGG16 trained on ImageNet) with a “predictive coding” strategy: a framework popular in neuroscience for characterizing cortical function.

Original: At each layer of thehierarchical model, generative feedback “predicts” (i.e., reconstructs) the patternof activity in the previous layer.
Revised: At each layer of the hierarchical model, generative feedback “predicts” (i.e., reconstructs) the pattern of activity in the previous layer.

Original: The reconstruction errors are used to iterativelyupdate the network’s representations across timesteps, and to optimize the network’s feedback weights over the natural image dataset–a form of unsupervisedtraining.
Revised: The reconstruction errors are used to iteratively update the network’s representations across timesteps, and to optimize the network’s feedback weights over the natural image dataset–a form of unsupervised training.

Original: We demonstrate that this results in a network with improved robustnesscompared to the corresponding feedforward baseline, not only against varioustypes of noise but also against a suite of adversarial attacks.
Revised: We demonstrate that this results in a network with improved robustness compared to the corresponding feedforward baseline, not only against various types of noise but also against a suite of adversarial attacks.

Original: We propose thatmost feedforward models could be equipped with these brain-inspired feedbackdynamics, thus improving their robustness to input perturbations.
Revised: We propose that most feedforward models could be equipped with these brain-inspired feedback dynamics, thus improving their robustness to input perturbations.

Original:  Introduction
Revised:  Introduction

Original: Recent studies have stressed the importance of feedback connections in the brain [1, 2] and haveshown how artiﬁcial neural networks can take advantage of such feedback for various tasks such asobject recognition with occlusion [3] or panoptic segmentation [4].
Revised: Recent studies have stressed the importance of feedback connections in the brain [1, 2] and have shown how artiﬁcial neural networks can take advantage of such feedback for various tasks such as object recognition with occlusion [3], or panoptic segmentation [4].

Original: Feedback connections conveycontextual information about the state of the higher layers down to the lower layers of the hierarchy;this way, they can constrain lower layers to represent inputs in meaningful ways.
Revised: Feedback connections convey contextual information about the state of the higher layers down to the lower layers of the hierarchy; this way, they can constrain lower layers to represent inputs in meaningful ways.

Original: In theory, this couldmake neural representations more robust to image degradation [5].
Revised: In theory, this could make neural representations more robust to image degradation [5].

Original: Merely including feedback in thepattern of connections, however, may not always be sufﬁcient; rather, it should be combined withproper mechanistic principles.
Revised: Merely including feedback in the pattern of connections, however, may not always be sufﬁcient; rather, it should be combined with proper mechanistic principles.

Original: To that end, we employ a predictive coding framework (supported byample neuroscience evidence [6, 7]) to build a large-scale hierarchical network with both feedforwardand feedback connections that can be trained using backpropagation.
Revised: To that end, we employ a predictive coding framework (supported by ample neuroscience evidence [6, 7, 8, 9, 10]) to build a large-scale hierarchical network with both feedforward and feedback connections that can be trained using backpropagation.

Original: Several prior studies haveexplored this interesting avenue of research [8, 9, 10, 11], but with important differences with ourapproach (see Appendix A for a more extensive discussion).
Revised: Several prior studies have explored this interesting avenue of research [11, 12, 13, 14], but with important differences with our approach (see Section 2).

Original: We demonstrate that our proposednetwork has interesting properties, especially when viewed from the perspective of robustness.
Revised: We demonstrate that our proposed network has interesting properties, especially when viewed from the perspective of robustness.

Revised: In a separate companion paper [15], we also demonstrate that the proposed dynamics help the networks perceive illusory contours akin to humans.

Original: Ourcontributions can be summarized as follows:
Revised: Our contributions can be summarized as follows:

Original: • We propose a novel strategy for effectively incorporating recurrent feedback connections based on the neuroscientiﬁc principle of predictive coding.
Revised: • We propose a novel strategy for effectively incorporating recurrent feedback connections based on the neuroscientiﬁc principle of predictive coding.

Original: • We implement this strategy in a feedforward architecture, and show that this improves its robustness against different types of natural and adversarial noise.
Revised: • We implement this strategy in a feedforward architecture, and show that this improves its robustness against different types of natural and adversarial noise.

Original: • We suggest and verify that the network iteratively shifts noisy representations towards the corresponding clean representations—a form of ‘projection towards the learned manifold’as implemented in other strong adversarial defense methods.  
Revised: • We suggest and verify that the network iteratively shifts noisy representations towards the corresponding clean representations—a form of ‘projection towards the learned manifold’ as implemented in other strong adversarial defense methods.  Prior work

Revised: Other studies have also tried implementing predictive coding mechanisms in machine learning models [11, 12, 13, 14].

Revised: Out of these, our implementation is most similar to the Predictive Coding

Revised: Networks (PCNs) of [13].

Revised: These hierarchical networks were designed with a similar goal in mind: improving object recognition with predictive coding updates.

Original: Our approach
Revised: Their proposed dynamics are roughly comparable to ours without the feedforward term, i.e. β n = 0 . However the most signiﬁcant difference between their work and ours is that their network (including the feedback connection weights) is solely optimized with a classiﬁcation objective. As a result, their network does not learn to uniformly reduce reconstruction errors over timesteps (see Figure 4a in the Appendix), as the predictive coding theory would mandate. We also found that their network performs relatively poorly until the ﬁnal timesteps (see Figure 4b in the Appendix), which does not seem biologically plausible: biological systems typically cannot afford to wait until the last iteration before detecting a prey or predator.

Revised: We discuss these PCNs [13] further in the Appendix, together with our own detailed exploration of their network’s behavior.

Revised: Other approaches to predictive coding for object recognition include Boutin et al.

Revised: [14], who used a PCN with an additional sparsity constraint.

Revised: The authors showed that their framework can give rise to receptive ﬁelds which resemble those of neurons in areas V1 and V2 of the primate brain.

Revised: They also demonstrated the robustness of the system to noisy inputs, but only in the context of reconstruction—they did not show that their network can perform (robust) classiﬁcation, and they did not extend their approach to deep neural networks.

Revised: Spratling [16] also described PCNs designed for object recognition, and demonstrated that their network could effectively recognise digits and faces, and locate cars within images.

Revised: Their update equations differed from ours in a number of ways: they used divisive/multiplicative error correction (rather than additive), and a form of biased competition to make the neurons ‘compete’ in their explanatory power.

Revised: The weights of the network were not trained by error backpropagation, making it difﬁcult to scale it to address modern machine learning problems.

Revised: Indeed, the tasks on which they tested their network are simpler than ours, and the datasets much smaller.

Revised: Huang et al.

Revised: [17] also tried to extend the principle of predictive coding by incorporating feedback connections such that the network maximizes ‘self consistency’ between the input image features, latent variables and label distribution.

Revised: The iterative dynamics they proposed, though different from ours, improved the robustness of neural networks against gradient-based adversarial attacks on datasets such as Fashion-MNIST and CIFAR10.

Original: Predictive coding, as introduced by [12], is a neurocomputational theory positing that the brainmaintains an internal model of the world, which it uses to actively predict the observed stimulus.
Revised: Predictive coding, as introduced by [18], is a neurocomputational theory positing that the brain maintains an internal model of the world, which it uses to actively predict the observed stimulus.

Original: Within a hierarchical architecture, each higher layer attempts to predict the activity of the layerimmediately below, and the errors made in this prediction are then utilized to correct the higher-layeractivity.
Revised: Within a hierarchical architecture, each higher layer attempts to predict the activity of the layer immediately below, and the errors made in this prediction are then utilized to correct the higher-layer activity.

Original: To establish our notation, let us consider a hierarchical feedforward network equipped with generativefeedback connections, as represented in Figure 1.
Revised: To establish our notation, let us consider a hierarchical feedforward network equipped with generative feedback connections, as represented in Figure 1.

Original: The network counts N encoding layers e n and Ncorresponding decoding layers d n − 1 (see Figure 1).
Revised: The network counts N encoding layers e n and N corresponding decoding layers d n − 1 (see Figure 1).

Original: The feedforward weights connecting layer n − 1to layer n are denoted by W fn − 1 ,n , and the feedback weights from layer n + 1 to n by W bn +1 ,n .
Revised: The feedforward weights connecting layer n −to layer n are denoted by W fn − 1 ,n , and the feedback weights from layer n + 1 to n by W bn +1 ,n .

Original: For agiven input image, we initiate the activations of all encoding layers with a feedforward pass.
Revised: For a given input image, we initiate the activations of all encoding layers with a feedforward pass.

Original: Then,over successive recurrent iterations (referred to as timesteps t ), both the decoding and encoding layerrepresentations are updated using the following equations :
Revised: Then, over successive recurrent iterations (referred to as timesteps t ), both the decoding and encoding layer representations are updated using the following equations :

Original: [Equation]
Revised: [Equation]

Original:  where (cid:2) · (cid:3) + is the ReLU function and β n , λ n , and α n ( 0 ≤ β n + λ n ≤ 1 ) act as layer-dependentbalancing coefﬁcients for the feedforward, feedback, and error-correction terms, respectively.
Revised: (1) where (cid:2) · (cid:3) + is the ReLU function and β n , λ n , and α n ( 0 ≤ β n + λ n ≤ 1 ) act as layer-dependent balancing coefﬁcients for the feedforward, feedback, and error-correction terms, respectively.

Original: (cid:15) n − 1 ( t )denotes the reconstruction error at layer n − 1 and is deﬁned as the mean squared error (MSE) betweenthe representation e n − 1 ( t ) and the predicted reconstruction d n − 1 ( t ) at that particular timestep.
Revised: (cid:15) n − 1 ( t ) denotes the reconstruction error at layer n − 1 and is deﬁned as the mean squared error (MSE) between the representation e n − 1 ( t ) and the predicted reconstruction d n − 1 ( t ) at that particular timestep.

Original: Layere 0 is deﬁned as the input image and remains constant over timesteps.
Revised: Layer e 0 is deﬁned as the input image and remains constant over timesteps.

Original: Finally, all the weights W fn − 1 ,nand W bn +1 ,n are ﬁxed during these iterations.
Revised: Finally, all the weights W fn − 1 ,n and W bn +1 ,n are ﬁxed during these iterations.

Original: Each of the four terms in Equation 1 contributes different signals: (i) the feedforward term (controlledby parameter β ) provides information about the (constant) input and changing representations in the lower layers, (ii) the feedback term (parameter λ ) conveys information about higher-levelrepresentations, as proposed in [13], (iii) the memory term acts as a time constant to retain the currentrepresentation over successive timesteps, and (iv) the error correction term (controlled by parameter α )fulﬁlls the objective of predictive coding, i.e., it corrects representations in each layer such that theirgenerative feedback can better match the preceding layer.
Revised: Each of the four terms in Equation 1 contributes different signals: (i) the feedforward term (controlled by parameter β ) provides information about the (constant) input and changing representations in the lower layers, (ii) the feedback term (parameter λ ) conveys information about higher-level representations, as proposed in [19], (iii) the memory term acts as a time constant to retain the current representation over successive timesteps, and (iv) the error correction term (controlled by parameter α ) fulﬁlls the objective of predictive coding, i.e., it corrects representations in each layer such that their generative feedback can better match the preceding layer.

Original: For this error correction term, we directlyuse the error gradient ∂(cid:15) n − 1 /∂e n to take full advantage of modern machine learning capabilities.
Revised: For this error correction term, we directly use the error gradient ∂(cid:15) n − 1 /∂e n to take full advantage of modern machine learning capabilities.

Original: While the direct computation of this error gradient is biologically implausible, it has been notedbefore that it is mathematically equivalent to propagating error residuals up through the feedbackconnection weights ( W b ) T , as often done in other predictive coding implementations [12, 10].
Revised: While the direct computation of this error gradient is biologically implausible, it has been noted before that it is mathematically equivalent to propagating error residuals up through the feedback connection weights ( W b ) T , as often done in other predictive coding implementations [18, 13].

Original: While it is certainly possible to train such an architecture in an end-to-end fashion, by combininga classiﬁcation objective for the feedforward weights W f with an unsupervised predictive codingobjective (see Section 3) for the feedback weights W b , we believe that the beneﬁts of our proposedscheme are best demonstrated by focusing on the added value of the feedback pathway onto apre-existing state-of-the-art feedforward network. 
Revised: While it is certainly possible to train such an architecture in an end-to-end fashion, by combining a classiﬁcation objective for the feedforward weights W f with an unsupervised predictive coding objective (see Section 4) for the feedback weights W b , we believe that the beneﬁts of our proposed scheme are best demonstrated by focusing on the added value of the feedback pathway onto a preexisting state-of-the-art feedforward network. Consequently, we implement the proposed strategy with an existing feedforward deep convolutional neural network (DCNN) architecture as the backbone: VGG16 trained on ImageNet. We show that predictive coding confers higher robustness to this network.

Original:  Model architecture and training
Revised:  Model architecture and training

Original: We start from a VGG16 pretrained on ImageNet and implement its predictive version, PVGG.
Revised: We start from a VGG16 pretrained on ImageNet and implement its predictive version, PVGG.

Original: Thenetwork ‘body’ (without the classiﬁcation head) is split into a cascade of ﬁve sub-modules.
Revised: The network ‘body’ (without the classiﬁcation head) is split into a cascade of ﬁve sub-modules.

Original: Eachconvolution block is considered as a sub-module which plays the role of an e n in equation (1).
Revised: Each convolution block is considered as a sub-module which plays the role of an e n in equation (1).

Original: We then add deconvolution as feedback layers d n − 1 connecting each e n to e n − 1 , with kernel sizesaccounting for the increased receptive ﬁelds of the neurons in e n (see Appendix A.1).
Revised: We then add deconvolution as feedback layers d n − 1 connecting each e n to e n − 1 , with kernel sizes accounting for the increased receptive ﬁelds of the neurons in e n (see Appendix A.1).

Original: We then trainthe parameters of the feedback deconvolution layers with an unsupervised reconstruction objective(with all other parameters frozen).
Revised: We then train the parameters of the feedback deconvolution layers with an unsupervised reconstruction objective (with all other parameters frozen).

Original: We minimize the reconstruction errors just after the ﬁrst forwardpass (i.e. no error correction or predictive coding recurrent dynamics are involved at this stage):
Revised: We minimize the reconstruction errors just after the ﬁrst forward pass (i.e. no error correction or predictive coding recurrent dynamics are involved at this stage):

Original: [Equation]
Revised: [Equation]

Original: where N is the total number of encoders, e n is the output of the n th encoder after the ﬁrst forwardpass, and d n is the estimated reconstruction of e n via feedback/deconvolution (from e n +1 ).
Revised: where N is the total number of encoders, e n is the output of the n th encoder after the ﬁrst forward pass, and d n is the estimated reconstruction of e n via feedback/deconvolution (from e n +1 ).

Original: After training the feedback deconvolution layers, we freeze all of the parameters, and choose the valuesof hyperparameters β n , α n and λ n for the encoders/decoders update equations (1).
Revised: After training the feedback deconvolution layers, we freeze all of the parameters, and choose the values of hyperparameters β n , α n and λ n for the encoders/decoders update equations (1).

Original: These values arechosen manually to decrease the prediction errors over successive iterations.
Revised: These values are chosen manually to decrease the prediction errors over successive iterations.

Original: See Appendix A for thechosen hyperparameter values.
Revised: See Appendix A for the chosen hyperparameter values.

Original: The resulting model, along with pretrained weights, will be madeavailable on Github .
Revised: The resulting model, along with pretrained weights, will be made

Original:  Results and Discussion
Revised:  Results and Discussion

Original: When considered at time step 0 (i.e., after a single feedforward and feedback pass through the network), the model and its accuracy are—by construction—exactly identical to the standard pretrained
Revised: When considered at time step 0 (i.e., after a single feedforward and feedback pass through the network), the model and its accuracy are—by construction—exactly identical to the standard pretrained

Original: VGG16.
Revised: VGG16.

Original: Over successive time steps, however, the inﬂuence of feedback and predictive codingiterations becomes visible.
Revised: Over successive time steps, however, the inﬂuence of feedback and predictive coding iterations becomes visible.

Original: On the standard ImageNet test dataset (clean images), accuracy remainsroughly constant over time (Figure 2b brown plot-line); that is, feedback connections do not improve(nor impair) performance.
Revised: On the standard ImageNet test dataset (clean images), accuracy remains roughly constant over time (Figure 2b brown plot-line); that is, feedback connections do not improve (nor impair) performance.

Original: Our hypothesis, on the other hand, is that feedback can confer robustnessto image perturbations.
Revised: Our hypothesis, on the other hand, is that feedback can confer robustness to image perturbations.

Original: Gaussian noise robustness: To evaluate the network robustness, we ﬁrst inject additive Gaussiannoise to the ImageNet test dataset, and quantify the model performance.
Revised: Gaussian noise robustness: To evaluate the network robustness, we ﬁrst inject additive Gaussian noise to the ImageNet test dataset, and quantify the model performance.

Original: In terms of accuracy, oversuccessive iterations the PVGG model progressively discards some of the noise and improves itsperformance relative to the feedforward VGG baseline (see Figure 2b).
Revised: In terms of accuracy, over successive iterations the PVGG model progressively discards some of the noise and improves its performance relative to the feedforward VGG baseline (see Figure 2b).

Original: To understand this robustnessto noisy images, we evaluate the quality of image reconstructions generated by the network with
Revised: To understand this robustness to noisy images, we evaluate the quality of image reconstructions generated by the network with

Original: MSE (see Figure 2).
Revised: MSE (see Figure 2).

Original: The reconstructions become progressively cleaner over timesteps.
Revised: The reconstructions become progressively cleaner over timesteps.

Original: It shouldbe noted that the feedback connections were trained only to reconstruct clean images; therefore,this denoising property is an emerging feature of the model dynamics.
Revised: It should be noted that the feedback connections were trained only to reconstruct clean images; therefore, this denoising property is an emerging feature of the model dynamics.

Original: Next, we test whether thisdenoising behavior is also observed in higher layers of the model.
Revised: Next, we test whether this denoising behavior is also observed in higher layers of the model.

Original: To quantify similarity at the levelof neural representations rather than pixels, we use the correlation distance.
Revised: To quantify similarity at the level of neural representations rather than pixels, we use the correlation distance.

Original: We pass clean and noisy versions of the same image through the network, and measure the correlation distance betweenthe clean and noisy representations at each layer and for each timestep.
Revised: We pass clean and noisy versions of the same image through the network, and measure the correlation distance between the clean and noisy representations at each layer and for each timestep.

Revised: These correlation distances were normalized with the distance measured at t=0 (i.e., relative to a standard feedforward

Original:  The correlation distancesdecrease consistently over timesteps across all layers of the network (Figure 2d).
Revised: VGG). The correlation distances decrease consistently over timesteps across all layers of the network (Figure 2d).

Original: This implies thatfeedback predictive coding iterations help the network steer the noisy representations closer to therepresentations elicited by the corresponding (unseen) clean image.
Revised: This implies that feedback predictive coding iterations help the network steer the noisy representations closer to the representations elicited by the corresponding (unseen) clean image.

Original: This is an important property for robustness.
Revised: This is an important property for robustness.

Original: When compared to clean images, noisy images canresult in different representations at higher layers [14].
Revised: When compared to clean images, noisy images can result in different representations at higher layers [20].

Original: Various defenses have aimed to protectneural networks from adversarial attacks by constraining the images to the ‘original data manifold’.
Revised: Various defenses have aimed to protect [Figure] l [Figure]

Revised: i

Revised: [Figure]

Revised: n

Revised: [Figure]

Revised: i s t a n

Revised: [Figure]

Revised: ) clean version.

Revised: neural networks from adversarial attacks by constraining the images to the ‘original data manifold’.

Original: Accordingly, studies have used generative models such as GANs [15, 16, 17, 18] or PixelCNNs [19]to constrain the input to the data manifold.
Revised: Accordingly, studies have used generative models such as GANs [21, 22, 23, 24] or PixelCNNs [25] to constrain the input to the data manifold.

Original: Similarly, multiple efforts have been made to clean therepresentations in higher layers and keep them closer to the learned latent space [20, 21, 22, 14].
Revised: Similarly, multiple efforts have been made to clean the representations in higher layers and keep them closer to the learned latent space [26, 27, 28, 20].

Original: Here,we demonstrate that feedback predictive coding iterations can achieve a similar goal by iterativelyprojecting noisy representations towards the manifolds learned during training, both in pixel spaceand representation spaces.
Revised: Here, we demonstrate that feedback predictive coding iterations can achieve a similar goal by iteratively projecting noisy representations towards the manifolds learned during training, both in pixel space and representation spaces.

Original:  robustness: Finally, we quantify the PVGG model’s accuracy on different typesof perturbations.
Revised: Benchmarking robustness: Finally, we quantify the PVGG model’s accuracy on different types of perturbations.

Original: We ﬁrst use ImageNet-C, a test dataset provided by [23] using 19 types of imagecorruption across 5 severity levels each.
Revised: We ﬁrst use ImageNet-C, a test dataset provided by [29] using 19 types of image corruption across 5 severity levels each.

Original: Using the recommended mean Corruption Error (mCE)score, we observe a steady improvement of accuracy over timesteps (see Figure 3a and 5), relative tothe standard VGG16 feedforward model.
Revised: Using the recommended mean Corruption Error (mCE) score, we observe a steady improvement of accuracy over timesteps (see Figure 3a and 5), relative to the standard VGG16 feedforward model.

Original: We also test the model’s robustness with a suite of targetedadversarial attacks.
Revised: We also test the model’s robustness with a suite of targeted adversarial attacks.

Original: The attack suite consists of gradient-based attacks: BasicIterativeMethod [24],
Revised: The attack suite consists of gradient-based attacks: BasicIterativeMethod [30],

Original: RandomProjectedGradientDescent [25], CarliniWagner attacks [26], and non-gradient based Boundary attacks [27].
Revised: RandomProjectedGradientDescent [31], CarliniWagner attacks [32], and non-gradient based Boundary attacks [33].

Original: The use of a non-gradient based attack ensures that any robustness we observe is notdue to gradient-masking , a well-known inferior form of defense [28, 29].
Revised: The use of a non-gradient based attack ensures that any robustness we observe is not due to gradient-masking , a well-known inferior form of defense [34, 35].

Original: Each attack is used against the model, on images that were initially correctly classiﬁed, to ﬁnd aminimal perturbation such that the resulting "adversarial image" is misclassiﬁed as another predeﬁnedcategory.
Revised: Each attack is used against the model, on images that were initially correctly classiﬁed, to ﬁnd a minimal perturbation such that the resulting "adversarial image" is misclassiﬁed as another predeﬁned category.

Original: Figure 3b-f shows the median perturbation required by each attack to fool the network (see
Revised: Figure 3b-f shows the median perturbation required by each attack to fool the network (see

Original: Appendix A.3 for further details).
Revised: Appendix A.3 for further details).

Original: Over successive iterations, increasing perturbations are required,suggesting that predictive coding can help the model to withstand adversarial perturbations.
Revised: Over successive iterations, increasing perturbations are required, suggesting that predictive coding can help the model to withstand adversarial perturbations.

Original: So far, the most promising strategy for achieving robustness has been adversarial training, wherebyadversarial datapoints are added to the training dataset.
Revised: So far, the most promising strategy for achieving robustness has been adversarial training, whereby adversarial datapoints are added to the training dataset.

Original: While efﬁcient, this strategy was also shownto be strongly limited [30, 31].
Revised: While efﬁcient, this strategy was also shown to be strongly limited [36, 37].

Original: Most importantly, it shares very little, if any, resemblance to theway the brain achieves robustness.
Revised: Most importantly, it shares very little, if any, resemblance to the way the brain achieves robustness.

Original: Instead, here we start from biological principles and show thatthey can lead to improved adversarial robustness.
Revised: Instead, here we start from biological principles and show that they can lead to improved adversarial robustness.

Original: It is worth mentioning that PVGG’s robustness isachieved here totally via unsupervised training.
Revised: It is worth mentioning that PVGG’s robustness is achieved here totally via unsupervised training of the feedback connections (while of course, the backbone VGG feed-forward network that we used was pretrained in a supervised manner).

Original: We avoid using costly adversarial training, or tuningour hyperparameters speciﬁcally for classiﬁcation under each attack or noise condition.
Revised: We avoid using costly adversarial training, or tuning our hyperparameters speciﬁcally for classiﬁcation under each attack or noise condition.

Original: This likelyexplains why the model, while improving in robustness compared to its feedforward version, remainsfar from state-of-the-art adversarial defenses.
Revised: This likely explains why the model, while improving in robustness compared to its feedforward version, remains far from state-of-the-art adversarial defenses.

Original: On the other hand, we believe that addition of thesemethods (adversarial training, hyperparameter tuning) to the training paradigm, in future work, couldfurther improve the network’s adversarial robustness.
Revised: On the other hand, we believe that addition of these methods (adversarial training, hyperparameter tuning) to the training paradigm, in future work, could further improve the network’s adversarial robustness.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper we demonstrated how a state-of-the-art feedforward neural network can be augmentedwith unsupervised brain-inspired dynamics.
Revised: In this paper we demonstrated how a state-of-the-art feedforward neural network can be augmented with unsupervised brain-inspired dynamics.

Original: Speciﬁcally, we showed that predictive coding updatescan improve the robustness of the network to a wide variety of random noises, as well as targetedadversarial attacks.
Revised: Speciﬁcally, we showed that predictive coding updates can improve the robustness of the network to a wide variety of random noises, as well as targeted adversarial attacks.

Original: We demonstrated that the network achieves this robustness by learning a formof manifold projection.
Revised: We demonstrated that the network achieves this robustness by learning a form of manifold projection.

Original: In future work, we intend to generalize these results to other feedforwardarchitectures (e.g. ResNet [32]).
Revised: In future work, we intend to generalize these results to other feedforward architectures (e.g. ResNet [38]).

Original: We believe that this work contributes to the general case forcontinuing to draw inspiration from biological visual systems in computer vision, both at the level ofmodel architecture and dynamics.
Revised: We believe that this work contributes to the general case for continuing to draw inspiration from biological visual systems in computer vision, both at the level of model architecture and dynamics.

Original: Broader Impacts
Revised: Broader Impacts

Original: The research discussed above proposes novel ways of using brain-inspired dynamics in currentmachine learning models.
Revised: The research discussed above proposes novel ways of using brain-inspired dynamics in current machine learning models.

Original: Speciﬁcally, it demonstrates a neuro-inspired method for improving therobustness of machine learning models.
Revised: Speciﬁcally, it demonstrates a neuro-inspired method for improving the robustness of machine learning models.

Original: It further highlights one of the many important propertiesnecessary for obtaining this robustness.
Revised: It further highlights one of the many important properties necessary for obtaining this robustness.

Original: Given that such models are employed by the general public,and are simultaneously shown to be heavily vulnerable, research efforts to increase (even marginally)or to understand their robustness against mal-intentioned adversaries has high societal relevance.
Revised: Given that such models are employed by the general public, and are simultaneously shown to be heavily vulnerable, research efforts to increase (even marginally) or to understand their robustness against mal-intentioned adversaries has high societal relevance.

Original: Importantly, the research also aims to bridge techniques between two different ﬁelds–neuroscienceand machine learning, which can potentially open new avenues for studying the human brain.
Revised: Importantly, the research also aims to bridge techniques between two different ﬁelds–neuroscience and machine learning, which can potentially open new avenues for studying the human brain.

Original: Forexample, it could help better understand the unexplained neural activities in patients, to improve theirliving conditions, and in the best case, in the treatment of their conditions.
Revised: For example, it could help better understand the unexplained neural activities in patients, to improve their living conditions and in the best case, in the treatment of their conditions.

Original: While this may also beassociated with inherent risks (related to privacy or otherwise), there are clear potential beneﬁts tosociety.
Revised: While this may also be associated with inherent risks (related to privacy or otherwise), there are clear potential beneﬁts to society.

Original: A AppendixA.1 Network Architectures
Revised: A Appendix A.1 Network Architectures

Original: Here we provide the details on how VGG16 is converted to a predictive coding network–PVGG.
Revised: Here we provide the details on how VGG16 is converted to a predictive coding network–PVGG.

Original: VGG16 consists of ﬁve convolution blocks and a classiﬁcation head.
Revised: VGG16 consists of ﬁve convolution blocks and a classiﬁcation head.

Original: Each convolution block containstwo or three convolution+ReLU layers with a max-pooling layer on top.
Revised: Each convolution block contains two or three convolution+ReLU layers with a max-pooling layer on top.

Original: For each e n in PVGG, weselected the max-pooling layer in block n − 1 and all the convolution layers in block n of VGG16(for n ∈ { 1 , 2 , 3 , 4 , 5 } ) as the sub-module that provides the feedforward drive.
Revised: For each e n in PVGG, we selected the max-pooling layer in block n − 1 and all the convolution layers in block n of VGG (for n ∈ { 1 , 2 , 3 , 4 , 5 } ) as the sub-module that provides the feedforward drive.

Original: Afterwards, for eache n a deconvolution layer d n − 1 is added which takes the e n as the input and reconstructs (or predicts)the output of e n − 1 .
Revised: Afterwards, for each e n a deconvolution layer d n − 1 is added which takes the e n as the input and reconstructs (or predicts) the output of e n − 1 .

Original: We call a bundle of ( e n , d n − 1 ) as a Predictive Coder module ( PC n ).
Revised: We call a bundle of ( e n , d n − 1 ) as a Predictive Coder module ( PC n ).

Original: Please notethat we consider e 0 as the input image.
Revised: Please note that we consider e 0 as the input image.

Original: Table 1 summarizes PVGG’s architecture.
Revised: Table 1 summarizes PVGG’s architecture.

Original: The hyperparameter values that we used are provided in Table2.
Revised: The hyperparameter values that we used are provided in Table

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: A.2 Prior work
Revised: A.2 Prior work: PCNs

Original: Other studies have also tried implementing predictive coding mechanisms in machine learningmodels [8, 9, 10, 11]. Out of these, our implementation shares some resemblance to the Predictive Coding Networks (PCNs) of [10] - we discuss this work [Table]
Revised: To better understand the model proposed by Wen et al.

Original:  and the differences between this and ourapproach, shortly.
Revised: [13] and its differences to ours

Original: Other approaches to predictive coding for object recognition include Boutin etal.

Original: [Table]
Revised: [Table]

Original: who used a PCN with an additional sparsity constraint.

Original: The authors showed that theirframework can give rise to receptive ﬁelds which resemble those of neurons in areas V1 and V2 of theprimate brain.

Original: They also demonstrated the robustness of the system to noisy inputs, but only in thecontext of reconstruction—they did not show that their network can perform (robust) classiﬁcation,and they did not extend their approach to deep neural networks. Spratling [33] also described PCNsdesigned for object recognition, and demonstrated thattheir network could effectively recognisedigits and faces, and locate cars within images. Their update equations differed from ours in a numberof ways: they used divisive/multiplicative error correction (rather than additive), and a form of biasedcompetition to make the neurons ‘compete’ in their explanatory power. The weights of the networkwere not trained by error backpropagation, making it difﬁcult to scale it to address modern machine
Revised: we conducted several experiments using the code that they provided

Original: [Figure]
Revised: [Table]

Original: learning problems.

Original: Indeed, the tasks on which they tested their network are simpler than ours, andthe datasets much smaller.

Original: Wen et al.

Original: [10] designed a hierarchical network with a similar goal in mind: improving objectrecognition with predictive coding updates. After the initial feedforward pass, the representationsare transformed iteratively by predictive coding, so their proposed dynamics would be roughlycomparable to ours without the feedforward and feedback terms, i.e. β n = 0 and λ n = 0 . In addition,their predictive coding update term only approximates error correction, under the assumption ofsymmetric feedforward and feedback connections, i.e. W b = ( W f ) T . But perhaps the mostsigniﬁcant difference between their work and ours is that their network (including the feedbackconnection weights) is solely optimized with a classiﬁcation objective.
Revised: and report here our most compelling observations.

Original: As a result, their networkcannot learn to uniformly reduce reconstruction errors over timesteps (see Figure 4a), as the predictivecoding theory would mandate. To better understand the model proposed by Wen et al. [10] and itsdifferences to ours, we conducted several experiments using the  codethat they provided, and reporthere our most compelling observations.
Revised: A ﬁrst striking shortcoming was that the accuracy of their feedforward baseline was far from optimal. Using their code

Original: The ﬁrst striking shortcoming of the model proposed by [10] is that the accuracy reported for theirfeedforward baseline was far from optimal: in our own explorations of their model, even minortweaking of the learning rate during training could bring the accuracy up from 60% to 70% –just a few percentage points below their recurrent network.
Revised: [Table] with relatively minor tweaks to the learning rate schedule [Table] we were able to bring it up from 60% to 70% – just a few percentage points below their recurrent network.

Original: We expect that this could be furtherimproved with a more extensive and systematic hyperparameter search.
Revised: We expect that this could be further improved with a more extensive and systematic hyperparameter search.

Original: In other words, their traininghyperparameters appeared to have been optimised for their predictive coding network, but not (or notas much) for their feedforward baseline.
Revised: In other words, their training hyperparameters appeared to have been optimised for their predictive coding network, but not – or not as much – for their feedforward baseline.

Original: Further, we found that a minor change to the architectureusing group normalisation layers after each ReLU – leads to a feedforward network which performson par with the recurrent network, with a mean over 6 runs of 72% and best of 73%.
Revised: We further found that a minor change to the architecture - using group normalisation layers after each ReLU – leads to a feedforward network which performs on par with the recurrent network, with a mean over 6 runs of 72% and best of 73%.

Original: Adding thesame layers to the recurrent network did not lead to a corresponding improvement in accuracy.
Revised: Adding the same layers to the recurrent network did not lead to a corresponding improvement in accuracy.

Original: Wealso found that the network had poor accuracy (underperforming the optimized feedforward baseline)until the ﬁnal timestep, as can been seen in Figure 4b.
Revised: We also found that the network had poor accuracy (underperforming the optimized feedforward baseline) until the ﬁnal timestep, as can been seen in Figure 4b.

Original: This can be clariﬁed by a closer reading of
Revised: This can be clariﬁed by a closer [Figure] u a r [Figure]

Revised: d d i s t

Original: Figure 3 of their paper: the reported improvements over cycles from 60% at time step 0 to morethan 70% at time step 6 are for seven distinct networks, each evaluated only at the timestep theywere trained for.
Revised: [Figure] n c [Figure] u r

Revised: [Figure]

Revised: c y

Revised: [Figure]

Revised: reading of Figure 3 of their paper: the reported improvements over cycles from 60% at time step 0 to more than 70% at time step 6 are for seven distinct networks, each evaluated only at the timestep they were trained for.

Original: So in fact, in their model the predictive coding updates do not gradually improveon an already reasonably guess.
Revised: So in fact, in their model the predictive coding updates do not gradually improve on an already reasonably guess.

Original: This is clearly not biologically plausible: visual processing wouldbe virtually useless if the correct interpretation of a scene only crystallised after a number of ‘timesteps’.
Revised: This is clearly not biologically plausible: visual processing would be virtually useless if the correct interpretation of a scene only crystallised after a number of ‘time steps’.

Original: By the time a person has identiﬁed an object that object is likely to have disappeared or, in aworst case scenario, eaten them.
Revised: By the time a person has identiﬁed an object that object is likely to have disappeared or, in a worst case scenario, eaten them.

Original: We also experimented with feeding the classiﬁcation error at eachtimestep into an aggregate loss function, but this lead to a network which, while performing well,essentially did not improve over timesteps.
Revised: We also experimented with feeding the classiﬁcation error at each timestep into an aggregate loss function, but this lead to a network which, while performing well, essentially did not improve over timesteps.

Original: Figure 4a shows that the network does not uniformly minimise reconstruction errors over time for alllayers, and thus is not performing correct predictive coding updates.
Revised: Figure 4a shows that the network does not uniformly minimise reconstruction errors over time for all layers, and thus is not performing correct predictive coding updates.

Original: In fact the total reconstructionerror (across all layers) increases exponentially over timesteps.
Revised: In fact the total reconstruction error (across all layers) increases exponentially over timesteps.

Original: There are a number of possible explanations for this.
Revised: There are a number of possible explanations for this.

Original: Firstly, in the case of the network with untied weights, the authors chooseto make a strong assumption in the update equations (seen as the equivalence of their Equations 5and 6): that the feedback weights can be assumed to be the transpose of the feedforward weights,i.e. W b =
Revised: Firstly, in the case of the network with untied weights, the authors choose to make a strong assumption in the update equations (seen as the equivalence of their Equationsand 6): that the feedback weights can be assumed to be the transpose of the feedforward weights, i.e. W b =

Original: ( W f ) T .
Revised: ( W f ) T .

Original: They thus propagate the feedforward error through the feedforward weights.
Revised: They thus propagate the feedforward error through the feedforward weights.

Original: However, it might be that the network learns feedback weights which essentially invert the feedforwardtransformation as assumed, but this is not guaranteed, and nor is it explicitly motivated through theclassiﬁcation loss function.
Revised: However, it might be that the network learns feedback weights which essentially invert the feedforward transformation as assumed, but this is not guaranteed, and nor is it explicitly motivated through the classiﬁcation loss function.

Original: Indeed, because the network is not motivated to learn a representation atearlier timesteps which produces a good prediction, it does not necessarily need to learn the inversetransformation: it can instead learn some other transformation which, when applied with the updateequations, leads the network to end up in the right place.
Revised: Indeed, because the network is not motivated to learn a representation at earlier timesteps which produces a good prediction, it does not necessarily need to learn the inverse transformation: it can instead learn some other transformation which, when applied with the update equations, leads the network to end up in the right place.

Original: That being said, this assumption is valid forthe network with tied weights, and this network also does not uniformly reduce reconstruction errorover timesteps.
Revised: That being said, this assumption is valid for the network with tied weights, and this network also does not uniformly reduce reconstruction error over timesteps.

Original: Possibly, the presence of ReLU non-linearities means that the forward convolutionmay still not be perfectly invertible by a transposed convolution.
Revised: Possibly, the presence of ReLU non-linearities means that the forward convolution may still not be perfectly invertible by a transposed convolution.

Original: Finally, in line with this unexpectedincrease of reconstruction errors over time, we have also failed to extract good image reconstructionsfrom the network as seen in Figure 5 of their paper, although in private communication the authorsindicated that this was possible with some other form of normalisation.
Revised: Finally, in line with this unexpected increase of reconstruction errors over time, we have also failed to extract good image reconstructions from the network as seen in Figure 5 of their paper, although in private communication the authors indicated that this was possible with some other form of normalisation.

Original: In short, while the ideas put forward in [10] share similarities with our own, their exact implementationdid not support the claims of the authors, and the question of whether predictive coding can beneﬁtdeep neural networks remained an open one.
Revised: In short, while the ideas put forward in [13] share similarities with our own, their exact implementation did not support the claims of the authors, and the question of whether predictive coding can beneﬁt deep neural networks remained an open one.

Original: We hope that our approach detailed in the present studycan help resolve this question.
Revised: We hope that our approach detailed in the present study can help resolve this question.

Original: A.3 Details on adversarial attacks
Revised: A.3 Details on adversarial attacks

Original: Adversarial attacks are hard-mined perturbations that aim to change the classiﬁcation label of anetwork with little perturbation.
Revised: Adversarial attacks are hard-mined perturbations that aim to change the classiﬁcation label of a network with little perturbation.

Original: We here use a suite of targeted attacks, which aim to changethe classiﬁcation label to a strictly predeﬁned category.
Revised: Here, we use a suite of targeted attacks, which aim to change the classiﬁcation label to a strictly predeﬁned category.

Original: The suite included iterative attacks thatrelied on gradient information (BasicIterativeMethod [24], RandomProjectedGradientDescent [25],
Revised: The suite included iterative attacks that relied on gradient information (BasicIterativeMethod [30], RandomProjectedGradientDescent [31],

Original: CarliniWagner [26]) and one attack that did not rely on gradient information (BoundaryAttack) [27].
Revised: CarliniWagner [32]) and one attack that did not rely on gradient information (BoundaryAttack) [33].

Original: Attacks like boundary attacks are used to ensure that the defense does not utilize a technique knownas gradient-masking , wherein a defense knowingly or unknowingly relies on hiding the gradientsfrom the attacker.
Revised: Attacks like boundary attacks are used to ensure that the defense does not utilize a technique known as gradient-masking , wherein a defense knowingly or unknowingly relies on hiding the gradients from the attacker.

Original: Stronger attacks that can estimate the gradients in the target network can easilybreak such defenses [28].
Revised: Stronger attacks that can estimate the gradients in the target network can easily break such defenses [34].

Original: To allow for a meaningful comparison across timesteps, we add an additional criteria that any imageused has to be correctly classiﬁed by the model on all the timesteps chosen for attack.
Revised: To allow for a meaningful comparison across timesteps, we add an additional criteria that any image used has to be correctly classiﬁed by the model on all the timesteps chosen for attack.

Original: For boundaryattacks, since we need to use a target image (instead of a target-class), we also enforce this criteria fortarget images and use the same target image across all timesteps.
Revised: For boundary attacks, since we need to use a target image (instead of a target-class), we also enforce this criteria for target images and use the same target image across all timesteps.

Original: The numbers for attacks BIM L ∞ ,
Revised: The numbers for attacks BIM L ∞ ,

Original: RPGD L ∞ ,BIM L 2 , CarliniWagner L 2 are 500, 127, 165, and 112 respectively for all timesteps.
Revised: RPGD L ∞ , BIM L 2 , CarliniWagner L 2 are 500, 127, 165, and 112 respectively for all timesteps.

Original: These numbers were restricted mainly by computational requirements.
Revised: These numbers were restricted mainly by computational requirements.

Original: For boundary attacks, thenumbers are 209, 72, 98 and 95 for timesteps 0, 2, 4, and 6 respectively.
Revised: For boundary attacks, the numbers are 209, 72, 98 and 95 for timesteps 0, 2, 4, and 6 respectively.

Original: All the attacks wereconducted using Foolbox 2.3.0 API.
Revised: All the attacks were conducted using Foolbox 2.3.0 API.

Original: The adversarial perturbations are then used as a base populationto bootstrap and estimate the median perturbation.
Revised: The adversarial perturbations are then used as a base population to bootstrap and estimate the median perturbation.

Original: The standard deviation of the medians is denotedas error bars in Figure 3.
Revised: The standard deviation of the medians is denoted as error bars in Figure 3.

Revised: A.4 CE Scores

Original: [Figure]
Revised: [Figure]