Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
Authors: Bhavin Choksi
Milad Mozafari
Callum Biggs O'May
B. ADOR
Andrea Alamia
Rufin VanRullen
Venue: NeurIPS 2021 Poster (2021)
Keywords: predictive coding
neuroscience
deep learning
robustness
noisy image classification
python package

Abstract: Deep neural networks excel at image classification, but their performance is far less robust to input perturbations than human perception. In this work we explore whether this shortcoming may be partly addressed by incorporating brain-inspired recurrent dynamics in deep convolutional networks. We take inspiration from a popular framework in neuroscience: "predictive coding". At each layer of the hierarchical model, generative feedback "predicts" (i.e., reconstructs) the pattern of activity in the previous layer. The reconstruction errors are used to iteratively update the network’s representations across timesteps, and to optimize the network's feedback weights over the natural image dataset--a form of unsupervised training. We show that implementing this strategy into two popular networks, VGG16 and EfficientNetB0, improves their robustness against various corruptions and adversarial attacks. We hypothesize that other feedforward networks could similarly benefit from the proposed framework. To promote research in this direction, we provide an open-sourced PyTorch-based package called \textit{Predify}, which can be used to implement and investigate the impacts of the predictive coding dynamics in any convolutional neural network. 

Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
Bhavin Choksi" CerCo CNRS, UMR 5549 & Université de Toulouse bhavin. www..cccccccooooo finksifrenrsfrr
Milad Mozafari" CerCo CNRS. UMR 5549 & IRIT CNRS, UMR 5505 milad, mwwamariicnnr
Callum Biggs O'May CerCo CNRS UMR 5549
Benjamin Ador CerCo CNRS
Andrea Alamia CerCo CNRS
Rufin VanRullen
CerCo CNRS, UMR 5549 & ANITI, Université de Toulouse rufin. vanrull ennrulee@eenrs.cm fr
UMR 5549
UMR 5549
Abstract
Deep neural networks excel at image classification, but their performance is far less robust to input perturbations than human perception. In this work explore whether this shortcoming may be partly addressed by incorporating brain-inspired recurrent dynamics in deep convolutional networks. We take inspiration from a popular framework in neuroscience: "predictive coding". At each layer the hierarchical model, generative feedback "predicts" (i.e., reconstructs) the pattern of activity in the previous layer. The reconstruction errors are used to iteratively update the network's representations feedback weights over the natural image dataset-a form unsupervised training. We show that implementing this strategy into two popular networks, VGG16 and Efficien(NetBO, improves their robustness against various corruptions and adversarial attacks. We hypothesize that other feedforward networks could similarly benefit from the proposed framework. To promote research in this direction, we provide an open- open-sourced PyTorch-baat based package called Predify, which can be used to implement and investigate the impacts of the predictive coding dynamics in any convolutional neural network.
timesteps, and optimize the network's
Introduction
Deep convolutional neural networks (DCNNs), initially inspired by the primate visual cortex archi- tecture, have taken big strides solving computer vision tasks in the last decade. State-of-the-ar networks can learn to classify images with high accuracy from huge labeled datasets (1-]]. This rapid progress and the resulting interest in these techniques have also highlighted their various shortcom- ings. Most widely studied is the sensitivity of neural networks, not only to perturbations specifically designed fool them (so-called "adversarial examples") but also regular noises typically observed in natural scenes 7 These shortcomings indicate that there is still room for improvement in current techniques.
One possible way to improve the robustness of artificial neural networks could be to take further inspiration from the brain. In particular, one major aspect of the cerebral cortex that missing from standard feedforward DCNNs is the presence of feedback connections. Recent studies have stressed the importance of feedback connections in the brain [10] IT), and have shown how artificial neural
Equal Contribution
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

LAYER I
LAYER II
.00
LAYER N
DENSE
UNSUPERVISED SUPERVISED FEEDBACK FEEDFORWARD
eN-1
eN
W
W
W NUN-1
do 1
22
NN-1
Figure 1: General overview of our predictive coding strategy as implemented in feedforward hierarchical network with generative feedback connections. The architecture (roughly similar to stacked auto-encoders) consists of N encoding layers en and N decoding layers d,. W Wm.. denotes the connection weights from layer m to layer n, with W and W for feedforward and feedback connections, respectively. The reconstruction errors at each layer are denoted by 6n. The feedforward connections (green arrows) are trained for image classification (in supervised fashion), while feedback weights (red arrows) are optimized for a prediction (i.e. reconstruction) objective (unsupervised). Predictive coding minimizes the reconstruction errors in each layer by updating activations in the next layer accordingly (black arrows). Self-connections (memory) are represented by blue arrows.
networks can take advantage of such feedback for various tasks such as object recognition with occlusion [2], or panoptic segmentation [13. Feedback connections convey contextual information about the state of the higher layers down to the lower layers of the hierarchy; in this way, they can constrain lower layers to represent inputs in meaningful ways. In theory, this could make neural representations more robust to image degradation [14]. Merely including feedback in the pattern of connections, however, may not always be sufficient; rather, it should be combined with proper mechanistic principles.
To that end, we explore the potential of recurrent dynamics for augmenting deep neural networks with brain-inspired predictive coding (supported by ample neuroscience evidence 19). We build large-scale hierarchical networks with both feedforward and feedback connections that can be trained using error backpropagation. Several prior studies have explored this interesting avenue of research 23) but with important differences with our approach (see Section 3). We demonstrate that our proposed method adds desirable properties to feedforward DCNNs, especially when viewed from perspective of robustness.
Our contributions can be summarized as follows:
We propose a novel strategy for effectively incorporating recurrent feedback connections based on the neuroscientific principle of predictive coding.
We implement this strategy in two pre-trained feedforward architectures with unsupervised training of the feedback weights, and show that this improves their robustness against different types of natural and adversarial noise.
We suggest and verify that an emergent property of the network is to iteratively shift noisy representations towards the corresponding clean representations form of "projection towards the learned manifold" as implemented in certain adversarial defense methods.
To facilitate research aimed at using such neuroscientific principles in machine learning, we provide Python package called Predify that can easily implement the proposed predictive coding dynamics in any convolutional neural network with a few lines code.

---

2 Our Approach
2.1 The proposed predictive coding dynamics
Predictive coding, as introduced by [24], is neurocomputational theory positing that the brain maintains an internal model of the world, which it uses to actively predict the observed stimulus. Within a hierarchical architecture, each higher layer attempts to predict the activity of the layer immediately below, and the errors made in this prediction are then utilized to correct the higher-layer activity.
To establish our notation, us consider hierarchical feedforward network equipped with generative feedback connections, as represented in Figure ! The network contains N encoding layers (n N) and N corresponding decoding layers dn-1- The feedforward weights connecting layer
n n- 1 to layer n are denoted by W a=1.n and the feedback weights from layer n + to n by W a+11n For given input image, we first initiate the activations of all encoding layers with a feedforward pass. Then, over successive recurrent iterations (referred to as timesteps f), both the decoding and encoding layer representations updated using the following equations (also refer to Pseudocode I
(t) W Cn+1nt+
(1)
(t =
Sn- (++ an
- - Ven-i(t)
(2)
where Bn, Xn (0 3 A < 1), and an act as layer-dependent balancing coefficients for the feedforward, feedback, and correction terms, respectively. Gn-1(t) denotes the reconstruction 1 and is defined the mean squared enror (MSE) between the representation error at layer (1) and the predicted reconstruction (1) at that particular timestep. Layer en defined as
the input image and remains constant over timesteps. All the weights w n-1nn and W 1l1. are fixed during these iterations.
Each of the four terms Equation 2 contributes different signals, reflected by different arrow colors in Figure [ (i) the feedforward term (green arrows; controlled by parameter B) provides information about the (constant) input and changing representations in the lower layers, (ii) the feedback correction term (red arrows; parameter λ), proposed in [24 25). guides activations towards their representations from the higher levels, thereby reducing the reconstruction errors over time, (iii) the memory term (blue arrows) acts time constant retain the current representation over successive timesteps, and (iv) the feedforward error correction term (black arrows; controlled by parameter a) corrects representations in each layer such that their generative feedback can better match the preceding layer. For this error correction term, we directly use the error gradient to take full advantage of modern machine learning capabilities (where k the number of elements in e,,). While the direct computation of this error gradient is biologically implausible, it has been noted before that mathematically equivalent to propagating error residuals
Ven-1 0en-1 den-1 8el ...., de
Pseudocude Predictive Coding lierations
Input image: 60
2: for N de 3: en +- Conv(ea-1)
d=-1 deConu(en)
1ld,, en 11.
6:
end for
to do for N do
8:
9:
I1 + Bn Conv(en. 1)
10:
fb 0
11:
in < N then
12:
A,, d.,
13:
and
en
fb (1- An) enn an Ven-1
15:
d, f- deConv(e..)
16:
+ |1 11
17:
18:
end for
3

---

up through the (transposed) feedback connection weights (W")*T, as often done in other predictive coding implementations [22 24] Together, the feedforward and feedback error correction terms fulfill the objective of predictive coding as laid out by Rao and Ballard [24]. We discuss the similarities and
differences between our equations and those proposed in the original Rao and Ballard implementation
in the Appendix A.6
While it is certainly possible to train such an architecture in an end-to-end fashion, by combining a classification objective for the feedforward weights W/ with an unsupervised predictive coding objective (see Section 2.2) for the feedback weights W we believe that the benefits of our proposed scheme are best demonstrated by focusing on the added value of the feedback pathway onto pre-existing state-of-the the-hhhrrrr feedforward network. Consequently, we implement the proposed strategy with two existing feedforward DCNN architectures as backbones; VGG16 and EfficientNetBO, both trained on ImageNet. We show that predictive coding confers higher robustness to these networks.
2.2 Model architectures and training
We select VGGI6 and EfficientNetBO, two different pre-trained feedforward networks on ImageNet, and augment them with the proposed predictive coding dynamics. The resulting models are called PVGG16 and PEfficientNetBO, respectively. The networks' "bodies" (without the classification head) split into cascade of N sub-modules, where each plays the role of en in equation (2). We then add deconvolutions as feedback layers d connecting each en to Gn-1, with kernel sizes accounting for the increased receptive fields of the neurons in Cn or upsampling layers to match the size of the predictions and their targets (see Appendix [..2). We then train the parameters of the feedback deconvolution layers with unsupervised reconstruction objective (with all feedforward parameters frozen). We minimize the reconstruction errors just after the first forward pass, and after single deconvolution step (i.e. no error correction predictive coding recurrent dynamics involved at this stage):
N Σ ==
C
en dn 112.
where en is the output of the th encoder after the first forward pass and d, is the estimated reconstruction of e, via feedhack/deconvllttion (from ex+1)
For both the networks, after training the feedback deconvolution layers, we freeze all of the weights, 8n, 0.8, A,, 0.1, and a, 0.01 for all the encoder-
and set the values of hyperparameters s/decoders in Equation (2). We also explore various strategies for further tuning hyperparameters to improve the results (see Appendix A.7 for the chosen hyperparameter values).
2.3 Predify
To facilitate and automate the process of adding the proposed predictive coding dynamics to existing deep neural networks, we have developed open-source Python package called Predify. The package developed based on PyTorch (26) and provides flexible object oriented framework to convert any Py TyTorhh-copatible network into predictive network. While an advanced user may find it easy to integrate Predify in their project mamually, a simple text-based user interface (in TOMI format) is also provided to automate the steps. For the sake of improved performance and flexibility, Predify generates the code of the predictive network rather than the Python object. Given the original network and configuration file (e.g. 'config, toml ') that indicates the intended source and target layers for the predictive feedback, three lines of code are enough to construct the corresponding predictive network:
from predify import predify
net load PyTorch network predify (net, ./config aeconfig.toml toml') 1) config file indicates the layers that # will act as outputs of encoders.
ftttps://tomm.1//00

---

The Appendix prrvvids further details on the package, along with a sample config file and certain default behaviours. Predify an ongoing project available on GitHut under GNU General Public License v3.0. Scripts for creating PVGG16 and PRfficientNetB00 from their feedforward instances and reproducing the results presented this paper, as well as the pre-trained weights are also available on another GitHub repository
3 Related work
There is long tradition of drawing inspiration from neuroscience knowledge to improve machine learning performance, Some studies suggest using sparse coding, a concept closely related to pre- dictive coding [16 27430), for image denoising [3]] and robust deep learning [32 33 while other studies focus on implementing feedback and horizontal recurrent pathways to tackle challenges
beyond the core object recognition [13 [4 25 B9].
Here, we focus specifically on those studies that tried implementing predictive coding mechanisms in machine learning models [20)23]. Out of these, our implementation is most similar to the Predictive Coding Networks (PCNs) of [22] These hierarchical networks were designed with a similar goal in mind: improving object recognition with predictive coding dynamics. However, their network (including the feedback connection weights) is solely optimized with a classification objective. As a result, their network does not learn to uniformly reduce reconstruction errors over timesteps, as the predictive coding theory would mandate. We also found that their network performs relatively poorly until the final timestep (see corresponding Figure iS in the Appendix A.5). which does not seem biologically plausible: biological systems typically cannot afford to wait until the last iteration before detecting prey predator. In the proposed method, we incorporate the feedforward drive into similar PC dynamics and train the feedback weights in an unsupervised way using a reconstruction loss. We then show that these modifications help resolve PCNs issues. We discuss these PCNs [22] further in the Appendix A.55 together with our own detailed exploration of their network's 's behavior.
Other approaches to predictive coding for object recognition include Boutin et al. [2 who used a PCN with an additional sparsity constraint. The authors showed that their framework can give rise to receptive fields which resemble those of neurons in areas V1 and V2 of the primate brain. They also demonstrated the robustness of the system to noisy inputs, but only in the context of reconstruction. Unlike ours, they did not show that their network can perform (robust) classification, and they did not extend their approach to deep neural networks.
Spratling [40 also described PCNs designed for object recognition, and demonstrated that their network could effectively recognise digits and faces, and locate cars within images. Their update equations differed from ours in a number of ways: they used divisive/multiplicatiie error correction (rather than additive), and a form of biased competition to make the neurons "compete" in their explanatory power. The weights of the network were not trained by error backpropagation, making it difficult to scale to address modern machine learning problems. Conversely, our proposed network architecture and PC dynamics are fully compatible with error backpropagation, making them suitable option for large-scale problems. Indeed, the tasks on which they tested their network are simpler than ours, and the datasets much smaller.
Huang et al. [I also aimed to extend the principle of predictive coding by incorporating feedback connections such that the network maximizes "self consistency" between the input image features, latent variables and label distribution. The iterative dynamics they proposed, though different from ours, improved the robustness of neural networks against gradient-based adversarial attacks on datasets such as Fashion-MNIS, and CIFAR10.
4 Results
Here we contrast the behavior of feedforward networks with their predictive coding augmentations. When considered at timestep 0 (i.e., after a single feedforward and feedback pass through the model), the deep predictive coding networks (DPCNs) and their accuracy are- are-by constrution-exxcee identical to their standard pretrained feedforward versions. Over successive timesteps, however,
5

---

(a)
(b)
Improvement Accuracy
PVGG16
PEfficientNetB0
Normalized MSE distance @ clean images
PVGG16
PEfficient
1.0
1.0
(%6
20.94
0.94
Timesteps
15
Timesteps
Timesteps 15
Timesteps
15
- 0.0
d- 0.5
a- 0.0
a- 0.5
a- 1.0
d=
a= 0.25
a 0.75
2.0
(c)
Original Input
Reconstruction
PVGG16
= 10
PEfficientNet50
Noisy Input
0 1
Reconstruction
PVGG16
10
PEffcientNetB00
(d)
PVGG16
PEfficientNetB0
Normalized Correlation
Noisa 0-0.55
Naise D-025
Noisa d-0.5
Noised-1.0
1.0
1.0
1.0
1.0
1.01
1.0
Distance
0.82
0.82
0.62
0.82
0.82
0.82
Timesteos
Timesteps
Timesteps
Timesteps
Timesteps
Timesteps
Encoder
Encoder
Encoder
Encoder
Encoder
Encoder
Encoder
Figure 2: Performance under Gaussian noise and projection towards the learned manifold. (a) Improvement recognition accuracy with reference to the feedforward baseline under various levels of Gaussian noise. Both networks demonstrate significant accuracy improvement across timesteps under noisy conditions. while maintaining performance close the feedforward level for clean images. (b) Normalized MSE distance between the image reconstruction (dg) and the clean image (eg). Irrespective of the noise level, image reconstruction consistently gets closer to the clean image across timesteps in both models. (c) Examples of clean and noisy input images together with their final reconstruction by the model (the row order from top to bottom is: original image, PVGG16 reconstruction, PEfficientNeiB00 reconstruction; noisy image, PVGG16 reconstruction. PEfficientNe(BO reconstruction). For best viewing, we recommend zooming in on electronic version. (d) Normalized correlation distance between representation of clean and noisy images for each encoder (e,) across timesteps. The values are normalized with respect to the feedforward baseline (timestep 0). In both models and all encoders, the noisy representations tend to toward the clean copies.

---

the influence of feedback and predictive coding iterations becomes visible. Here, we investigate for both DPCNs (PVGG16 and PEfficientNetBO): (i) how the PC dynamics update the networks' representations across timesteps, and in which direction relative to the learned manifold; (ii) how the networks benefit from PC under noisy conditions, or against adversarial attacks.
4.1 Performance under Gaussian noise
To understand the evolution of representations and the behavior of the proposed DPCNs, we first investigate their performance under the influence of different levels of Gaussian noise. To this end, we inject additive Gaussian noise to the ImageNet validation set, and monitor the models' performance across timesteps.
In Figure 2h we provide the classification accuracy on these noisy images and absolute values in the Table S4 We observed that both models progressively improve their recognition accuracy relative to their Teedforward baseline (timestep 0) over successive iterations while imposing only a minor performance reduction on clean images. other words, the networks are able discard some of the noise by leveraging the predictive coding dynamics over timesteps.
4.2 Projection towards the learned manifold
In order quantify DPCNs' denoising ability, we evaluate the quality of image reconstructions generated by each network using the mean squared error (MSE) between the clean image and its reconstruction generated by the first decoder. For each DPCN, we normalize these distances, by dividing them by the value obtained for the corresponding feedforward network (at t=0), We provide the absolute values in the Table [3] As Figures p2-c illustrate, the reconstructions become progressively cleaner over timesteps. It should be noted that the feedback connections were trained only to reconstruct clean images; therefore, this denoising property is an emerging feature of the PC dynamics.
Next, we test whether the higher layers of the proposed DPCNs also manifest this denoising property. Hence, we pass clean and noisy versions of all images from the ImageNet validation set through the networks, and measure the average correlation distance between the clean and noisy representations of each encoder at each timestep. As done above, these correlation distances are then normalized with the distance measured timestep (i.e., relative to the standard feedforward network), For both the networks, the correlation distances decrease consistently over timesteps across all layers (see Figure d). This implies that predictive coding iterations help the networks to steer the noisy representations closer to the representations elicited by the corresponding (unseen) clean image.
This is an important property for robustness. When compared clean images, noisy images result in different representations at higher layers (42) and consequently, produce significant classification errors. Various defenses have aimed protect neural networks from perturbations and adversarial attacks by constraining the images to the "original data manifold". Accordingly, studies have used generative models such as GANs 46 or PixeICNNs 47 constrain the input to the data manifold. Similarly, multiple efforts have been made to clean the representations in higher layers and keep them closer the learned latent space 48 50). Here, we demonstrate that feedback predictive coding iterations can achieve similar goal by iteratively projecting noisy representations towards the manifolds learned during training, both pixel (Figure and representation spaces (Figure lit).
4.3 Benchmarking robustness ImageNet-C
Given the promising results with additive Gaussian noise (Figure 2). we extend the noise variety and quantify the classification accuracy of the networks under different types of perturbations. We use ImageNet-C, a benchmarking dataset for noise robustness provided by [ including 19 types of image corruptions across 5 severity levels each. To begin with, we evaluate DPCNs with pre-defined hyperparameter values (as provided in subsection 2.2). We observe that they improve the Corruption Error (CE) scores over timesteps for several of the additive-noise corruptions: Gaussian noise, shot noise, impulse noise or speckle noise (see Figure 33), but fail to improve the overall mean Corruption Error, or mCE score (the recommended score for this benchmark (8).
7

---

(a)
(b)
PUSG16
PEffcicntNett80
(eprovated)
(splimned)
1.0
1.0
1.0
normallzed CE scorss
normalizee MCE sxures
0.94
0.98
0.98
Timestoos
Timesteps
Timesteos
Timesteps
Figure 3: Benchmarking robustness to ImageNet-C. (a) Normalized corruption enrors (CE) of PVGG16 and PEfficientNetBO under four types of additive noise corruptions. The values are normalized with respect to the feedforward bascline. Both networks show consistent reductions in the errors across timesteps. (b) Normalized mean Corruption Error (mCE) scores for PVGG16 and PEfficientNetBO hyperparameters are used (as described in the Appendix A.7). The values are normalized with respect to the feedforward baseline. In both the panels, error bars represent the standard deviation of the bootstrapped estimate of the mean value.
all the 19 corruptions available in the ImageNet-C dataset, when optimized
Thus, instead of using pre-defined hyperparameter values, we fine-tune them using two different methods (see Appendix [A.7). and repeat the above experiment. As shown in Figure 3p, when the hyperparameters are more appropriately tuned for the task, the PC dynamics can increase noise robustness more generally across noise types, resulting in improvements of the mcan Corruption Error (mCE) score. The CE plots for individual perturbations along with other recommended metrics (values normalized with AlexNet scores, Relative mCE scores) are provided the Appendix [A.8]
Furthermore, in the Appendix A.9 we demonstrate that we can replicate these observations with version of PEfficientNetBO provided by [51] that is robust to corruptions in the ImageNet-C dataset. We show that the recurrent dynamics we propose still help in further improving the mCE score of this already robust network.
4.4 Benchmarking robustness ad adversarial attacks
Finally, we evaluate the robustness of the networks across timesteps against adversarial attacks. The proposed DPCNs recurrent models, meaning that their layer representations change on every timestep, and consequently, so do the classification boundaries in the last layer, leading to different accuracy and generalization errors across time (as seen above). To mitigate this effect and properly assess the changes in robustness due to the PC dynamics, for each network we start by selecting 1000 images from the ImageNet validation dataset such that they are correctly classified across all timesteps. Also, we only perform targeted attacks so that for each image, the same attack target is given for all timesteps. Using the Foolbox library [52]. we conduct targeted Basic_ Iterative Method attacks (BIM, with Loc norm) [53] for both networks; although it would prove computationally prohibitive to systematically explore all standard types of adversarial attacks, we also evaluated random Projected Gradient Descent attacks (RPGD, with L2 norm) [54) and non non-gradient- based HopSkipJump attacks [55] on a subset of 100 images, specifically for PEfficientNerBO. Across various levels of allowed image perturbations (denoted as es), the predictive coding iterations tend to decrease the success rate of the attacks across timesteps, for both networks and attacks (see Figure 4 That is, DPCNs are more robust against these adversarial attacks than their feedforward counterparts.
5 Discussion and Conclusion
In this work, we explore the use of unsupervised recurrent predictive coding (PC) dynamics, based on neuroscientific principles, to augment modern deep neural networks. The resulting models have an initial feedforward sweep, compatible with visual processing human and macaque brains [I]] 566-
58). Following this feedforward sweep, consecutive layers iteratively exchange information regarding

---

Lo
BIM
L2 RPGD
Lop HopSkipJump
PVGG16
PEfficienNNet00
PEfficientNett0
PEfficientNeet0
Success Rate Attack
(baseline -carrecddeee
1.0
1.0
1.0
1.0
0.99
0.9
0.9
0.93
4
0
8
0
0
Timestops
Timestops
Timestops
Timesteps
2+0.001
C-0.003
8-0.005
C-0.7
2-0,8
0-0.05
2-0.3
C-0.004
C-0.01
5-0.0
C-0.3
Figure 4: Benchmarking robustness adversarial attacks. Plots show the success rate of targeted adversarial attacks against DPCNs across timesteps. The values are baseline-correcten relative to the success rate at timestep (feedforward baseline). Though we see some signs of reversals for few perturbations, on average. both networks demonstrate improving robustness across timesteps to different types and/or levels of perturbations.
predictions and prediction errors, aiming to converge towards a stable explanation of the input. This dynamic system is inspired by, and reminiscent of, the "canonical microcircuit" (a central component of cortical structure (15)) that relies on feedback signaling between hierarchically adjacent layers to update its activity. Overall, the augmented networks are closer to the architecture of biological visual systems, while gaining some desirable functional properties. For example, in (59]. we also demonstrated that the proposed dynamics help the networks perceive illusory contours in a similar way to humans.
Here, we implemented these PC dynamics in state-of-the-art DCNs, VGG16 and EfficientNetBO, and showed that they helped to improve the robustness of the networks against various corruptions (e.g- ImageNet-C). We demonstrated that this behavior, at least partly, stems from PC's ability to project both the corrupted image reconstructions and neural representations towards their clean counterparts.
We also tested the impact of our network augmentations against adversarial attacks; here again, we showed that PC helps to improve the robustness of the networks. So far, the most promising strategy for achieving robustness has been adversarial training, whereby adversarial datapoints are added to the training dataset. While efficient, this strategy was also shown to be strongly limited [60] 61). Apart from factors like the choice of the norms used for training, or the high computation requirements, it is ultimately performed with a supervised loss function that can alter the decision boundaries in undesirable ways 661 62) Most importantly, adversarial training shares very little, if any, resemblance to the way the brain achieves robustness. Instead, here we start from biological principles and show that they can lead to improved adversarial robustness. It is worth mentioning that both our networks achieved robustness totally via unsupervised training of the feedback connections (while of course, the backbone feed-forward networks that we used were pretrained in a supervised manner). We avoided using costly adversarial training, or tuning our hyperparameters specifically for classification under each attack. This likely explains why the models, while improving in robustness compared to their feedforward versions, remain far from faute-o-thh--ae adversarial defenses. On the other hand, we believe that addition of these methods (adversarial training, hyperparameter tuning) to the training paradigm, future work, could further improve the networks' adversarial robustness.
For the present experiments, we made a choice of using different objectives for training the feedfor- ward and feedback weights: pre-trained feedforward weights optimized for classification, feedback weights trained with a reconstruction objective (computed after single time-step). On the one hand, we note that is perfectly feasible train a similar predictive coding architecture with single objec- tive (classification, reconstruction, or otherwise) for both feedforward and feedback weights [59 63 On the other hand, our choice has several advantages. First, using a feedforward backbone pretrained for classification allowed us to demonstrate the effect of our dynamics pre-existing state-of-the-as neural networks. Some authors have tried training both feedforward and feedback connections to- gether for classification 222 at the final timestep for relatively smaller networks, but we discussed in our explorations in the Appendix[A.3 we found that the resulting network ended up classifying correctly at the last timestep, with very poor performance during carly timesteps. This problem could

---

be addressed by training over time-averaged metrics, such as the average cross-entropy loss for N timesteps. Nonetheless, training the feedback weights for reconstruction instead of classification has the additional advantage that it can be done entirely without supervision. We chose to train the feedback weights for a single time-step, because training with recurrence over multiple timesteps would have required unrolling the network over time. Hence, training a large network like PVGGI6 for say 5 or 10 timesteps would incur significant computational challenges. Furthermore, our use of a one-step reconstruction objective allowed us to train the feedback weights independently of the various hyperparameters of our predictive coding dynamics (8, A, and a),, which only influence the model behavior after the second timestep. Training these weights using recurrence would have required to (i) either fix the values of these hyperparameters beforehand, leading to constraints of expensive hyperparameter explorations: (ii) directly train these hyperparameters parameters of the model, probably with additional constraints to prevent the network from reaching trivial values (e.g., if all hyperparameters but the feedforward term B converge to zero, the network performs identically to a feedforward one). Finally, from a neuroscience perspective, whether and how the brain combines discriminative and generative representations has been an open question addressed by many researchers, e.g. [5] 64, (65). Our approach of a discriminative (classification-tii -rriined)- feedforward coupled with generative (reconstruction-traine) feedback could be considered another attempt in this direction.
We speculate that the proposed PC dynamics could help improve robustness in most feedforward neural architectures. To facilitate further explorations in this direction, we provided a Python package, called Predify, which allows users to implement recurrent PC dynamics in any feedforward DCN, with only a few lines of code. Predify automates the network building, and thus simplifies experiments. On the other hand, there is yet no established method criteria to automate the process of identifying the appropriate number of encoding layers, their source and target layers in the DCN hierarchy, and the corresponding hyperparameter values. This remains an open research question, and requirement for manual explorations and tuning from Predify users. For instance, our own explorations with augmenting ResNets through Predify proved difficult, and failed in some situations but succeeded in others. More specifically, as developed in [63] using Predify, ResNet augmentations always achieved noise robustness when the hyperparameter values (controlling the feedforward, feedback, and memory terms) could be tuned separately for each noise type; but we found it challenging to identify single set of hyperparameters that could generalize all noise types. Nonetheless. we are hopeful that the package will prove useful to the community. The code is structured such that users can readily adapt it to test their hypotheses. In particular, it should allow both proponents and opponents of the predictive coding theory to investigate its effects on any DCN.
Overall, this work contributes to the general case for continuing draw inspiration from biological visual systems in computer vision, both at the level of model architecture and dynamics. We believe that our user-friente fre-ffiindly Python package Predify can open new opportunities, even for neuroscience researchers with little background in machine learning, to investigate bio- dio-inspired hypotheses in deep computational models. and thus bridge the gap between the two communities.
Broader Impacts
The research discussed above proposes novel ways of using brain-inspired dynamics in current machine learning models. Specifically, it demonstrates a neuro-inspired method for improving the robustness of machine learning models. Given that such models are employed by the general public, and are simultaneously shown to be heavily vulnerable, research efforts to increase (even marginally) or to understand their robustness against mal-intentioned adversaries has high societal relevance.
Importantly, the research also aims to bridge techniques between two different fields neuroscience and machine learning, which can potentially open new avenues for studying the human brain. For example, could help better understand the unexplained neural activities in patients, to improve their living conditions, and in the best case, in the treatment of their conditions. While this may also be associated with inherent risks (related to privacy or otherwise), there are clear potential benefits to society.
The likelihood of sentient AI arising from this line of research is estimated be rather low.
10

---

Acknowledgments and Disclosure of Funding
This work was funded by an ANITI (Artificial and Natural Intelligence Toulouse Institute) Research Chair to RV (ANR grant ANR-19-PI3A -000044, as well as ANR grants AI-REPS (ANR-18-CE37. 0007-01) and OSCI-DEEP NNR-I19-EUC--0004)
References
[1] Alex Krizhevsky, Ila Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097-1105, 2012.
[2] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1rXXiv:409.1555 2014.
[3] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. corr abs/1512.03385 (2015), 2015.
[4] Gao Huang. Zhuang Liu, and Kilian Q Weinberger. Densely connected convolutional networks. corr abs/1 ftbs//608.nog 0009306993 (2016). arXiv preprint arXiv: 06993, 2016.
[5] Haider Al-Tahan and Yalda Mohsenzadeh. Reconstructing feedback representations in the ventral visual pathway with a generative adversarial autoencoder. PLoS Computational Biology, 1008775, 2021.
[6] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, pages 6105-61 01556614,, 14. PMLR, 2019.
[7] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks, arXiv preprint 1312v 2013.
[8] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint 1rXiv:1903.1223 12261, 2019.
[9] Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confi- dence predictions for unrecognizable images. arxiv e-prints, art. arXiv preprint arXiv::442 1897, 2014.
[10] Tim C Kietzmann, Courtney J Spoerer, Lynn KA Sörensen, Radoslaw M Cichy, Olaf Hauk, and Nikolaus Kriegeskorte. Recurrence is required to capture the representational dynamics of the human visual system. Proceedings the National Academy Sciences, 116(43)21854--1166. 2019.
[11] Kohitij Kar, Jonas Kubilius, Kailyn Schmidt, Elias B Issa, and James DiCarlo. Evidence that recurrent circuits critical to the ventral stream's 's execution of core object recognition behavior. Nature neuroscience, 22(6))974-983, 2019.
[12] Markus Roland Ernst, Jochen Triesch, and Thomas Burwick. Recurrent connections aid occluded object recognition by discounting occluders. In International Conference Artificial Neural Networks, pages 295- Springer, 2019.
[13] Drew Linsley, Alekh Karkada Ashok, Lakshmi Narasimhan Govindarajan, Rex Liu, and Thomas Serre. Stable and expressive recurrent vision models. arXiv preprint arXiv:2005. 2020.
[14] Dean Wyatte, Tim Curran, and Randall O' O'Reilly. The limits of feedforward vision: Recurrent processing promotes robust object recognition when objects are degraded. Journal of Cognitive Neuroscience, 24(11):2248 222222212 2012.
[15] Andre M Bastos, W Martin Usrey, Rick A Adams, George R Mangun, Pascal Fries, and Karl J Friston. Canonical microcircuits for predictive coding. Neuron, 76(4)6995-71, 2012.
[16] Yanping Huang and Rajesh PN Rao. Predictive coding. Wiley Interdisciplinary Reviews: Cognitive Science, 255) 2011.

---

[17] Micha Heilbron and Maria Chait. Great expectations: is there evidence for predictive coding in auditory cortex? Neuroscience, 389:54-73, 2018.
[18] Laurence Aitchison and Máté Lengyel. With or without you: predictive coding and bayesian inference the brain. Current opinion neurobiology, 46:219-227, 2017.
[19] Kevin S Walsh, David P McGovem, Andy Clark, and Redmond G O'Connell. Evaluating the neurophysiological evidence for predictive processing as a model of perception. Annals of the New York Academy of Sciences, 1464( 2020.
[20] Rakesh Chalasani and Jose C Principe. Deep predictive coding networks. arXiv preprint urXiv:1301.3553 2013.
[21] William Lotter, Gabriel Kreiman, and David Cox. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv: arXiv:/605,0000 1605.081044 2016.
[22] Haiguang Wen, Kuan Han, Junxing Shi, Yizhen Zhang, Eugenio Culurciello, and Zhongming Liu. Deep predictive coding network for object recognition. arXiv preprint arXiv:1802.04760 2018.
[23] Victor Boutin, Angelo Franciosini, Frederic Chavane, Franck Ruffier, and Laurent Perrinet. Sparse deep predictive coding captures contour integration capabilities of the early visual system. arXiv preprint
2019.
[24] Rajesh PN Rao and Dana Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-ficld effects. Nature neuroscience, 2(1):79-87,
1999.
[25] David Heeger. Theory of cortical function. Proceedings of the National Academy of Sciences,
1 114(8):173--1782 2017.
[26] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024 8035. Curran Associates, Inc., 2019.
[27] Bruno A Olshausen and David Field. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, (61833)3) 1996.
[28] Erkki Oja, Aapo Hyvärinen, and Patrik Hoyer. Image feature extraction and denoising by sparse coding. Pattern Analysis & Applications, 2(2):104-110, 1999.
[29] Matthew Chalk, Olivier Marre, and Gašper Tkačik. Toward unified theory of efficient, predictive, and sparse coding. Proceedings of the National Academy of Sciences, 115(1):186- 191. 2018.
[30] Dylan M Paiton, Charles G Frye, Sheng Y Lundquist, Joel Bowen, Ryan Zarcone, and Bruno A Olshausen. Selectivity and robustness of sparse coding networks. Journal of Vision, (0(12):10-10 2020.
[31] Xiaoqiang Lu, Yuan Yuan, and Pingkun Yan. Sparse coding for image denoising using spike and slab prior. Neurocomputing 106:12-20, 2013.
[32] Jeremias Sulam, Ramchandran Muthukumar, and Raman Arora. Adversarial robustness of supervised sparse coding. Advances in neural information processing systems, 2020.
[33] Edward Kim, Jocelyn Rego, Yijing Watkins, and Garrett T Kenyon. Modeling biological immunity to adversarial examples. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattrnn Recognition, pages 4666-4675, 2020.
12

---

[34] Courtney J Spoerer, Patrick McClure, and Nikolaus Kriegeskorte. Recurrent convolutional neural networks: a better model of biological object recognition. Frontiers in psychology. 8:1551, 2017.
[35] Drew Linsley, Junkyung Kim, Vijay Veerabadran, Charles Windolf, and Thomas Serre. Learning long-range spatial dependencies with horizontal gated recurrent units. In Advances in neural information processing systems, pages 152-164, 2018.
[36] Nicholas Frosst, Sara Sabour, and Geoffrey E. Hinton. Darcce: Detecting adversaries by reconstruction from class conditional capsules. ArXiv, abs/1811 100/1881,06699, 2018.
[37] Dmitry Krotov and John Hopfield. Dense associative memory is robust to adversarial inputs. Neural computation, 30(12):3151-316) 2018
[38] Jonas Kubilius, Martin Schrimpf, Kohitij Kar, Rishi Rajalingham, Ha Hong. Najib Majaj. Elias Issa, Pouya Bashivan, Jonathan Prescott-Roy Kailyn Schmidt, Aran Nayebi, Daniel Bear, Daniel I. Yamins, and James J DiCarlo. Brain-like object recognition with pigh-performing shallow recurrent anns. H. Wallach, H. Larochelle, Beygelzimer, F. d' d'Alché- Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[39] Karim Rajaei, Yalda Mohsenzadeh, Reza Ebrahimpour, and Seyed-Mahdi Shaligh-Razavi. Beyond core object recognition: Recurrent processes account for object recognition under occlusion. PLoS computational biology, 15(5)ee107000 2019.
[40] Michael Spratling. A hierarchical predictive coding model of object recognition in natural images. Cognitive computation, 9(): 2017.
[41] Yujia Huang. James Gornet, Sihui Dai, Zhiding Yu, Tan Nguyen, Doris Tsao, and Anima Anandkumar. Neural networks with recurrent generative feedback. Advances in Neural Information Processing Systems, 33, 2020.
[42] Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 501-509, 2019.
[43] Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan: Protecting classifiers against adversarial attacks using generative models. International Conference Learning Representations, 2018
[44] Guoqing Jin, Shiwei Shen, Dongming Zhang, Feng Dai, and Yongdong Zhang. Ape-gan: Adversarial perturbation elimination with gan. In ICASSP 2019 2019 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 3842-3846, 2019.
[45] Dongyu Meng and Hao Chen. Magnet: a two-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 135-147, 2017.
[46] Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, and Alexandros G Dimakis. The robust man- ifold defense: Adversarial training using generative models. arXiv preprint arXiv: 1712.09196, 2017.
[47] Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In International Conference on Learning Representations, 2018
[48] Guanhong Tao, Shiqing Ma, Yingqi Liu, and Xiangyu Zhang. Attacks meet interpretabil- ity: Atribute-steered detection of adversarial samples. In Advances Neural Information Processing Systems, pages 7717-7728, 2018.
[49] Emin Orhan and Brenden M Lake. Improving the robustness of imagenet classifiers using elements of human visual cognition. arXiv preprint arXiv: 2019.
13

---

[50] Kevin Roth, Yannic Kilcher, and Thomas Hofmann. The odds are odd: A statistical test for detecting adversarial examples. In International Conference on Machine Learning, pages 5498 5507. PMLR, 2019.
[51] Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoe V Le. Adver- sarial examples improve image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages $19-828, 2020.
[52] Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox benchmark the robustness of machine learning models. In Reliable Machine Learning in the Wild Workshop, 34th International Conference on Machine Learning, 2017.
[53] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
[54] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
[55] Jianbo Chen, Michael I Jordan, and Martin J Wainwright. Hopskipjumpattakk: A query- efficient decision- based attack. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1277-1294. IEEE Computer Society, 2020.
[56] Simon Thorpe, Denis Fize, and Catherine Marlot. Speed of processing in the human visual system. nature, 381 (6522)5500-52, 1996.
[57] Chou P Hung, Gabriel Kreiman, Tomaso Poggio, and James DiCarlo. Fast readout of object identity from macaque inferior temporal cortex. Science, 310(5749):863 866, 2005.
[58] Rufin VanRullen. The power of the feed-forward sweep. Advances in Cognitive Psychology. 3(1-2):167, 2007.
[59] Zhaoyang Pang, Callum Biggs O'May, Bhavin Choksi, and Rufin VanRullen. Predictive coding feedback results in perceived illusory contours in recurrent neural network. Neural Networks, 144:164 175, 2021.
[60] Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data. In Advances in Neural Information Processing Systems, pages 5014-5026, 2018.
[61] Haichao Zhang and Jianyu Wang. Defense against adversarial attacks using feature scattering- based adversarial training. In Advances in Neural Information Processing Systems, pages 1831-1841, 2019.
[62] Thomas Tanay and Lewis Griffin. A boundary tilting persepective on the phenomenon of adversarial examples. arXiv preprint arXiv:1608.07690 2016.
[63] Andrea Alamia, Milad Mozafari, Bhavin Choksi, and Rufin VanRullen. On the role of feedback in visual processing: a predictive coding perspective. arXiv preprint arXiv:2106.042: 04225,, 2021.
[64] James J DiCario, Ralf Haefner, Leyla Isik, Talia Konkle, Nikolaus Kriegeskorte, Benjamin Peters, Nicole Rust, Kim Stachenfeld, Joshua Tenenbaum, Doris Tsao, et al. How does the brain combine generative models and direct discriminative computations in high-level vision? CCN 2021 Workshop GAC, 2021.
[65] Derek Huffman and Craig EL Stark. Multivariate pattern analysis of the human medial temporal lobe revealed representationaally categorical cortex and representationally agnostic hippocampus. Hippocampus, 24(11):1994-1403 2014.
[66] Michael W Spratling. Predictive coding as a model of biased competition in visual attention. Vision research, 48(1 18(12):139111408 2008
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Please refer Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A] We draw comparisons with Rao and Ballard's implementation, and provide step-by-step derivations in the Appendix.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or URL)? [Yes] The scripts and data of the main experiments will be available online on GitHub. At the review stage, included the source code in the supplementary materials for the purpose of anonymization.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] All the important details are provided in Section 2 and the Appendix. (c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] The corresponding details
given in Figure captions. (d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] See subsection 2.3
(c) Did you include any new assets either the supplemental material or as URL? [Yes]
We include the proposed Predify package in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A] All assets used were open-source with appropriate licenses.
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
15


Reviews:
[{"review_id": "qckgDDA7DoV", "reviewer": "Reviewer_4h3w", "summary": "Taking inspiration from the theory of predictive coding in neuroscience, the authors implement CNNs with layerwise predictive activity, as a form of unsupervised training. Experiments show that this approach is helpful in case of image corruption.", "questions": "", "limitations": "", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is generally well written and easy to follow.  The illustrations are effective. The section dedicated to the description with code snapshots of the developed library is unnecessary in the paper and could be relegated to supplementary material or link documentation. I would use the space available to provide more references to the state of the art and describe the proposed approach in more detail.\n\nThe idea of predictive coding applied to neural network layers is not new. The authors should expand the related works section to be more inclusive of more recent work. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The novelty of the work is very limited. Since the model is not new, they only show that the application brings advantages in the case of corrupted images with perturbations. \n\nThe experimental part is also limited. The authors only evaluate one type of image noise. It would have been interesting to see if this theory could contribute to improving robustness against adversarial attacks. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "R-dJge_7KG", "reviewer": "Reviewer_WgTo", "summary": "The authors proposed a new algorithm to implement a popular neuroscience theory, i.e. predictive coding, in deep neural networks (DNN). The feedforward connection weights are pretrained for object recognition and the feedback connection weights are trained for image reconstruction, and then predictive coding introduces recurrent computations with the trained DNNs. The authors use two image recognition DNNs (VGG16 and EfficientNetB0) in experiments. They found that the recurrent computation brings improvement in object recognition and image/feature reconstruction over time given noisy inputs. This property of predictive coding improves DNN robustness to noisy examples and adversarial attacks. The authors also provide a python package to allow community to explore the predictive coding algorithm.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Predictive coding is an important theory in neuroscience and recently some researchers have explored several implementations with deep neural network to help understand the theory. How neurons recurrently process information is still unknown in predictive coding theory. This paper proposed a new implementation (though have some similarity to previous works) and did extensive experiments and found desirable properties of recurrent computation over time. This is very interesting and valuable to both neuroscience and deep learning communities. The writing is clear and easy to understand.\n\nThe feedforward connection weights in DNN are trained with forward propagation for object recognition (pretrained) and the feedback connect weights are trained with reconstruction of lower layer features. No recurrent computation is involved in training the connection weights, which may not be the case in the brain. It would be good if the author can discuss this in the perspectives of both neuroscience and machine learning.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "KXeC_gkN0W", "reviewer": "Reviewer_NhQe", "summary": "The authors draw inspiration from neuroscience and propose augmenting feedforward neural networks with recurrent dynamics to improve their adversarial and corruption robustness. They provide a PyTorch package that can augment any convolutional neural network with recurrence. Their empirical tests show that recurrence mostly helps with robustness.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Review summary:*\n\nThe proposed method is well explained and the provided recurrence software package could be a valuable contribution to the community. The paper could be improved by reporting more interpretable metrics and addressing inconsistencies in the results. Additionally, the literature review needs to be improved and the novelty made more explicit. I do not believe the paper is good enough for publication in its current form, but am ready to revise my score if my concerns are adequately addressed.\n\n*Strengths:*\n\nThe paper is well written in terms of grammar and fluency. The introductory paragraphs provide good motivation for exploring recurrence in DNNs, and the model description is clear and concise. The concluding and broader impact statements are also well argued. The breadth of experiments (denoising, corruption robustness, and adversarial robustness) is helpful for providing a holistic understanding for how recurrence influences resilience to perturbations. While I didn’t look at the code itself, I believe that the provided codebase could be helpful for important future work in this space. I also appreciate the detailed comparisons to [22] and [24] in the Appendix.\n\n*Weaknesses:*\n\nThis is a fairly long list that is not in order of importance (sorry). However, I hope many of these issues can be easily resolved by the authors.\n\na) I recommend additional background discussion on related prior work. There is a wealth of research attempting to use predictive coding ideas to improve DNN robustness. If we only focus on recurrence then the scope broadens further {2, 3, 8}. Although note that {8} heavily draws on the concept of analysis-by-synthesis, which is also a core idea in predictive coding. A more related example would be sparse coding, which is a predictive coding model that has been used for denoising for decades {6, 7}. Recently, several papers have also demonstrated adversarial robustness with sparse coding {1, 4, 5}. I have provided some examples here, but I would encourage the authors to look at citations of and within them to inform a proper discussion on the context of their work such that readers can properly assess the novelty of their ideas. An additional (minor) issue is that the existing references should also be updated. Many papers cited as arxiv preprints are now published in peer-reviewed venues.\n\nb) It would be easier to compare the networks (VGG vs EfficientNet) if the authors used shared vertical axes in adjacent plots. This recommendation applies to almost every subfigure. The study’s goal is clearly not to compare VGG against EfficientNet, but understanding the relative differences between baselines can give better context for the change in performance when recurrence is added.\n\nc) It is not clear to me what motivated the stopping criteria for the recurrence in any given experiment. The authors appear to arbitrarily stop the model at 4, 8, 10, 15, and 20 time steps. Why is this? I would like the authors to extend the time steps as long as necessary (given computational constraints) to give a complete picture of how recurrence influences the representations. Presumably all of the curves will eventually converge to a steady state -- when is that? If not, why? In a few instances (e.g. figure 4 for eps=0.003 and figure 3(b) for PEfficentNet), the performance does a reversal, which should at least be discussed.\n\nd) When interpreting the gaussian noise experiment results (figure 2) the authors state that “Both networks demonstrate significant accuracy improvement across timesteps under noisy conditions, while maintaining a performance close to the feedforward level for clean images.” I do not believe this statement is supported by the data. For one, the use of the word “significant” implies that they did some sort of significance test, which they did not as far as I can tell. Perhaps more importantly is that for certain settings (e.g. the blue and orange curves in figure 2(a)) the recurrence appears to hurt performance. Due to the axis rescaling (see point b above), it is hard to see whether these differences matter. Finally, they should note what data that accuracy was computed on -- I assume one of the ImageNet test sets?\n\ne) I do not understand the metric used in figure 2(b). I believe a normalized MSE distance of 1.0 indicates that the reconstruction quality with recurrence matches that without. The authors should be explicit about this. This metric is not as helpful to understand as reporting the raw MSE values (this is a theme across my suggestions). I find it interesting that there is not a consistent correlation or trend when comparing MSE against noise level. In fact, the PVGG net and PEfficientNet appear to have opposite trends, where the noise free case has the highest score for PVGG and almost lowest for PEfficientNet. There is a similar inconsistency (and similar metric confusion) in figure 2(d), where there is not a monotonic relationship between layer and distance, as I would expect. I would appreciate it if the authors would comment on this. \n\nf) I think it would be helpful if the authors used the same images for the two networks in figure2(C) so that readers can understand the relative difference in the baselines used.\n\ng) In terms of the corruption experiments, given that the clean accuracy has changed I think it makes more sense to use the relative CE score from [8] than the regular CE. Also, the authors do not say if they are using the same AlexNet baseline as was used in [8]. At timestep 0 they always report CE of 1.0, which leads me to guess that they are using the standard VGG or EfficientNet score as the denominator. At a minimum the authors should be explicit about how they are computing the metric instead of deferring to the citation (unless they are indeed computing the score exactly as was done in [8]). Ideally they should use AlexNet as their denominator so that the reader can properly compare their performance with other works in the field. I appreciate that the authors intended to focus on the added value of feedback, instead of achieving SOTA on some benchmark. However, it is still helpful to understand the added value if we have a good relative context.\n\nh) The authors alternate between showing individual CE scores without hyperparameter tuning in figure 3.(a) and mCE with tuning in figure 3.(b). But in the appendix figure 6 we can see that there are tuned CE scores that perform worse with recurrence (e.g. zoom & fog for VGG). To clarify the results I recommend that the authors provide more information in the main paper. Specifically, they should acknowledge negative CE results from the tuned/untuned models in the main paper. And if the goal is to compare the results “with tuning” against “without tuning”, then they should use the same metric in both scenarios. \n\ni) The robustness metrics used are also not standard in that field. The appendix figures 8, 9, and 10 are much more in line with what is typically reported. I don’t see the added benefit of the presentation in figure 4 when we can easily see the “baseline corrected” score by comparing time step 0 with those that are > 0 (this argument applies to many of the above points as well). Again, even though the authors wish to stress a relative comparison it is important that they report their results in such a way that it can be viewed in the context of other work. As a point of comparison, the results reported in {9} also used FoolBox and are more aligned to typical setup and formatting.\n\nj) I’m not sure if the concerns raised in {4, 8} about backpropagating through recurrence are relevant here, but it would be helpful if the authors would explicitly say that the white-box attacks did not suffer from gradient obfuscation issues (see {9}). Even better would be if they additionally used gradient-free attacks as in {8}.\n\n{1} Sulam et al., Adversarial Robustness of Supervised Sparse Coding, https://proceedings.neurips.cc/paper/2020/hash/170f6aa36530c364b77ddf83a84e7351-Abstract.html\n\n{2} Krotov et al., Dense associative memory is robust to adversarial inputs, https://direct.mit.edu/neco/article-abstract/30/12/3151/8426/Dense-Associative-Memory-Is-Robust-to-Adversarial?redirectedFrom=fulltext\n\n{3} Frosst et al., DARCC: Detecting Adversaries by Reconstruction from Class Conditional Capsules, ​​https://arxiv.org/abs/1811.06969\n\n{4} Paiton et al., Selectivity and robustness of sparse coding networks, https://jov.arvojournals.org/article.aspx?articleid=2772000\n\n{5} Kim et al., Modeling Biological Immunity to Adversarial Examples, https://openaccess.thecvf.com/content_CVPR_2020/html/Kim_Modeling_Biological_Immunity_to_Adversarial_Examples_CVPR_2020_paper.html\n\n{6} Lu et al., Sparse coding for image denoising using spike and slab prior, https://www.sciencedirect.com/science/article/abs/pii/S0925231212007643?via%3Dihub\n\n{7} Oja et al., Image Feature Extraction and Denoising by Sparse Coding, https://link.springer.com/article/10.1007/s100440050021\n\n{8} Schott et al., Towards the first adversarially robust neural network model on MNIST, https://openreview.net/forum?id=S1EHOsC9tX\n\n{9} Athalye et al. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples http://proceedings.mlr.press/v80/athalye18a.html", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "1Rr28Xf8yV", "reviewer": "Reviewer_WCib", "summary": "This paper implemented the predictive coding dynamics into deep neural nets, and observed the increase in robustness against various corruptions. The authors also provided a pytorch-based package for easy implementation of the proposed dynamics into deep neural nets. ", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The predictive coding principle has gained increasing attention in the machine learning community. Despite not being the first to incorporate the predictive coding dynamics into neural networks, this paper did a good job in formulating the predictive coding principle in a clean way and they were able to test out the ideas on large scale datasets like Imagenet. The paper was well written and provided enough details to reproduce their results. In general, I enjoyed reading this paper. \nHowever, I do have some comments and questions regarding the proposed mechanism as well:\n1) In Eq. 2, the feedforward representations were updated to reduce the reconstruction error in the lower level (the alpha term). However, if we believe that the feedforward representations for noisy images are far away from that of the clean images and will cause wrong classification, why reducing the distance between the feedback representation and this \"wrong\" forward representation helps robustness? I understand that experimentally we see some positive results, but it would be good to comment on this point. \nAlso, it may be good to take a pretrained robust feedfoward network (for example, network trained with data augmentation techniques or adversarial training), and see whether the proposed mechanism can still improve the robustness.\n2) In the paper, the authors only reported relative improvements, but I am curious to see the absolute value of robustness accuracy. Since the authors take pretrained networks, I am wondering if similar robustness improvements can also be achieved by training the forward model differently (for example, using data augmentation or adversarial training). This is related to the last point I mentioned in 1). It would be good to show whether feedback can further improve robustly trained feedforward models.\n3) Since the authors introduced more weights in the model (the feedback path), could the robustness improvement be a result of using larger model? It would be good to take a deeper or wider feedforward model to match the number of parameters in the feedback model and see if the improvement is still there. \n4) In the pseudo code, it looks like the authors loop through all PCoder modules first, and then move on to the next time step. How would this compared with performing several update steps first until one PCoder is convergent and then move on to the PCoder in the next level?\n5) For the adversarial robustness experiment, I was wondering how the authors perform the attack? Are they performing end-to-end attack considering the predictive coding dynamics? If not, this should be evaluated. In addition, when performing end-to-end attack, adversarial attack may suffer from gradient obfuscation problem, so it would also be good to add black-box attack evaluation.\n6) How sensitive is the method's performance with respective to the coefficient \\alpha in Eq. 2? In Table 2 and Table 3 in appendix, \\alpha is fixed to be 0.01. But this term is an very important term for the predictive coding principle. Also the authors scaled the gradient based on feature size, so it seems that this is a sensitive hyper-parameter. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have provided sufficient discussion on this.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a framework for incorporating recurrent feedback connections based on predictive coding principles into feed-forward networks. Overall, the reviewers expressed excitement for the integration of neuroscience with ML and thought that the evaluations were convincing. However, there was initially high spread in reviewer scores due to concern over the novelty of the model and approach. After the detailed author responses and a thorough discussion, the consensus moved towards acceptance and two reviewers increased their scores.  
Decision: Accept (Poster)