Brain-like Flexible Visual Inference by Harnessing Feedback Feedforward Alignment
Authors: Tahereh Toosi
Elias Issa
Venue: NeurIPS 2023 poster (2023)
Keywords: Visual inference
Bio-plausible learning algorithm
Feedback connections
Visual imagery
Occlusions
Noise
TL;DR: A learning algorithm that leverages feedback and feedforward pathways as mutual credit assignment computational graphs, enabling flexible visual inference such as resolving occlusions and generating visual imagery.

Abstract: In natural vision, feedback connections support versatile visual inference capabilities such as making sense of the occluded or noisy bottom-up sensory information or mediating pure top-down processes such as imagination. However, the mechanisms by which the feedback pathway learns to give rise to these capabilities flexibly are not clear. We propose that top-down effects emerge through alignment between feedforward and feedback pathways, each optimizing its own objectives. To achieve this co-optimization, we introduce Feedback-Feedforward Alignment (FFA), a learning algorithm that leverages feedback and feedforward pathways as mutual credit assignment computational graphs, enabling alignment. In our study, we demonstrate the effectiveness of FFA in co-optimizing classification and reconstruction tasks on widely used MNIST and CIFAR10 datasets. Notably, the alignment mechanism in FFA endows feedback connections with emergent visual inference functions, including denoising, resolving occlusions, hallucination, and imagination. Moreover, FFA offers bio-plausibility compared to traditional backpropagation (BP) methods in implementation. By repurposing the computational graph of credit assignment into a goal-driven feedback pathway, FFA alleviates weight transport problems encountered in BP, enhancing the bio-plausibility of the learning algorithm. Our study presents FFA as a promising proof-of-concept for the mechanisms underlying how feedback connections in the visual cortex support flexible visual functions. This work also contributes to the broader field of visual inference underlying perceptual phenomena and has implications for developing more biologically inspired learning algorithms.

Brain-like Flexible Visual Inference by Harnessing Feedback-Feefforward Alignment
Tahereh Toosi *
Elias B. Issa Department of Neuroscience
Center for Theoretical Neuroscience
Zuckerman Mind Brain Behavior Institute Columbia University New York, NY
Zuckerman Mind Brain Behavior Institute
Columbia University
New York, NY elias. issa@coluubies edu
tahereh.
edu
Abstract
In natural vision, feedback connections support versatile visual inference capabili- ties such as making sense of the occluded or noisy bottom-up sensory information mediating pure top-down processes such as imagination. However, the mech- anisms by which the feedback pathway learns to give rise to these capabilities flexibly not clear. We propose that top-down effects emerge through alignment between feedforward and feedback pathways, each optimizing its own objectives. introduce Feedback Feedback-Feedforwann Feedforward Alignment
To achieve this co-optimization, (FFA), learning algorithm that leverages feedback and feedforward pathways mutual credit assignment computational graphs, enabling alignment. In our study, we demonstrate the effectiveness of FFA in co-optimizing classification and reconstruction tasks on widely used MNIST and CIFARI datasets. Notably, the alignment mechanism in FFA endows feedback connections with emergent visual inference functions, including denoising, resolving occlusions, hallucination, and imagination. Moreover, FFA offers bio- bio-plausibility compared to traditional back- propagation (BP) methods implementation. By repurposing the computational graph of credit assignment into a goal-driven feedback pathway. FFA alleviates weight transport problems encountered BP, enhaneing bio-plausibility of the learning algorithm. Our study presents FFA promising proof-of-concep for the mechanisms underlying how feedback connections in the visual cortex support flexible visual functions. This work also contributes to the broader field of visual inference underlying perceptual phenomena and has implications for developing more biologically inspired learning algorithms.
Introduction
Humans possess remarkable abilities infer the properties of objects even in the presence of occlusion or noise. They can mentally imagine objects and reconstruct their complete forms, even when only partial information available, regardless of whether they have ever seen the complete form before. The process of visual inference noisy uncertain stimuli requires additional time, implying cognitive processes that go beyond simple feedforward pass on visual input and suggest the involvement of additional mechanisms such as feedback and recurrence (Kar et al., 2019; Kietzmann et al., 2019; Gilbert and Sigman, 2007: Debes and Dragoi, 2023; Kreiman and Serre, 2020). Despite the abundant evidence on the involvement of feedback connections in various cognitive processes, understanding the precise mechanisms through which they flexibly give rise to the ability to infer or generate perceptual experiences is not clear.
"Corresoondence to: Taherch Toosi itahere.toosic @ cmai
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

While hierarchical feedforward models of the ventral visual cortex based on deep learning of dis- criminative losses have achieved remarkable success in computer vision tasks (Yamins al., 2014; Khaligh-Razavi and Kriegeskorte, 2014; Lindsay, 2021), alternative frameworks, such as predictive processing models, offer a distinct perspective on visual processing. Predictive processing models propose that the brain generates fine- fine-grained predictions about incoming sensory inputs and compare them with actual sensory signals to minimize prediction errors (Rao and Ballard, 1999; Friston, 2009; Clark, 2013). These models emphasize the role of feedback connections in the visual cortex, with higher-level areas sending top-down predictions to lower-level areas to guide perception (Wen et al., 2018; Choksi et al., 2021; Whittington and Bogacz, 2017: Millidge al., 2020). Unlike deep learning models that learn from large-scale datasets, predictive processing models prioritize the role of prior knowledge and expectations in shaping perception originally emphasized far back as Helmholtz (Helmholtz et al., 1909). While predictive processing models emphasize the active role of top-down predictions and prior knowledge in shaping visual perception, they lag behind feedforward models in mechanistic specificity and thus direct neurophysiological evidence (Walsh et al., 2020; Clark, 2013; Keller and Mrsic-Flogett 2018). Moreover, most large-scat scale implementation of predictive coding or, in general, recurrence relies on feedback connections (Choksi al., 2021; Tscshantz et al., 2023; Spoerer et al., 2017), which suffers from many issues regarding bio-plausible implementation (Lillicrap and Santoro, 2019a).
(BPTT) to train the recurrence and
Another main constraint on the space of models is the learning algorithm to train the model. Classical error backpropagation (BP) has been a workhorse algorithm for training discriminative, feedforward deep neural networks, particularly for visual object recognition (Rumelhart et al., 1986; Krizhevsky et al., 2012). Despite its immense success in training the state-of-the-art, BP has been criticized on number of implementational issues, some of which also call into question its bio-plausibility for the brain: weight symmetry requires weight transport to the feedback network (Grossberg, 1987), the feedback network is not used during runtime inference, and feedforward discrimination performance is not robust to noise (Goodfellow et al., 2015; Akrout, 2019). Beyond these issues, BP is infinites- imally local estimate of the gradient, and other higher-order methods for computing the gradient could accelerate learning. As pointed out by (Bengio, 2014), the inverse of the weight matrix, rather than the transpose, may provide a valid path for credit assignment, learning a linear extrapolation of the underlying landscape (Bengio, 2014). However, attempts to match BP by learning the inverse weights instead of the transpose in a stage-wise fashion, also called target propagation (TP), have yielded limited practical success for reasons that are not entirely clear (Lee et al., 2014; Bartunov et al., 2018)- potentially related to the difficulty of learning an inverse function using noisy gradients as opposed to the relative ease of taking transpose, noiseless procedure (Kunin et al., 2020).
Here, we simultaneously learn feedforward and feedback functions that are mutual global inverses of each other such that each path can perform credit assignment for the other during the training pass. We term this Feedback- FeedbackkFFeddorvarrd Alignment (FFA) since the discriminator (encoder) contributes the gradients for the reconstructor (decoder) and vice versa. We show that rather than trading against each other as in typical, single-objective settings, co-optimizing discrimination and reconstruction objectives can lead to mutualistic symbiotic interaction.
Next, we explore the potential of the gradient path as model of feedback connections. We are inspired by the structural similarity of the credit assignment computational graph and the feedforward pass, which parallels the anatomically reciprocated feedforward and feedback connections in the primate visual cortex (Markov et al., 2013, 2014). Importantly we hypothesized an objective function for feedback connections motivated by the Higghresolution buffer hypothesis by (Lee and Mumford, 2003) regarding the primary visual cortex (V1), arguing that VI is uniquely situated to act high-resolution buffer to synthesize images through generative processes. We hypothesized that by co-tuning feedforward and feedback connections to optimize two different but dependent objective functions, we could cxplain the properties of flexible visual inference of visual detail under occlusion, denoising, dreaming, and mental imagery.
The contributions of this study are as follows:
Based on the role of feedback connections in the brain, we propose novel strategy to train neural networks to co-optimize for two objective functions.
We leverage the credit assignment computational graph as feedback connections during learning and inference.
2

---

Backpropagation
Feedback Alignement
Feedback Feedforward Alignment
y
Lossy: (lffo) y
Wr
Wi
Wry
R
Wf
Wb,
1100 yy
Losss desoo
Wr
W
Wf,
R
Wf,
Wb,
4000
Wf
WF
Lossy
Connections
.
Wf
Wb.
forward inference
credit assignment Interence
Learning
C
EC
Ct
Bt-1
while 4, S A do It = =+ + A + noise end while
Inference
noise
A = 3t
St
Figure 1: Feedback-Feedforwarrt Alignment. Learning: backpropagation and feedback alignment train a discriminator with symmetric W7 or fixed random R₃ weights, respectively. FFA maps input x latents y as a discriminator but also reconstructs the input from the latent. The forward and backward pathways also pass gradients back for their counterpart performing inference in the opposite direction. Inference: We run forward and feedback connections trained under FFA a loop update the activations (æ) for each of the inference tasks e.g. mental imagery. A shows the difference between the imput signal and the reconstructed (output). * shows the imposed upper bound. See algorithm 2 in Section 8.4.
We suggest and verify that training feedforward and feedback connections for discrimination and reconstruction respectively, induces noise robustness.
We show that FFA can flexibly support an array of versatile visual inferences such as resolving occlusion, hallucination, and visual imagery.
Related Work
2.1 Models of visual perception and inference in the brain
In going beyond purely feedforward models, there is a large hypothesis space of recurrent neural network models (RNNs), and training inference into RNN via backpropagation through time raises severe questions about bio-plausibility of the learning algorithm as well as architecture (Lilllirapp and Santoro, 2019b). Prior work RNNs for visual classification whether through complex architecture search (Nayebi et al., 2022) or through imposing theoretically motivated lateral recurrent connections (Tang al., 2014, 2018) has shown benefits for the classification loss but was not geared improve our understanding of how feedback recurrence supports visual inference. On the other hand, there increasing evidence supporting distinct phases of processing pertaining to perception and inference which parallels the notion of bottom-up versus top-down processing. Recent studies suggest that feedforward and feedback signaling operate through distinct "channels," enabling feedback signals to influence the forward processing without directly affecting the forward-propagated activity, including feedback interaction being more dominant during spontaneous, putatively internally generated activity periods (Semedo et al., 2022; Kreiman and Serre, 2020). Thus, implementing recursion

---

through feedforward and physiological basis.
phases, as we suggest in FFA, has an anatomical and
Algorithm 1 Training simple three-layer network by FFA
Learning
parameters: Wh. W Why Wb, for epoch < n_ epochs do y= W, h = Wn
ef -
Lossj 2e7 ef
AW -Eg hif A. -Wo, eg X";
// forward updates
X =W hb: hb Wo,- -Y
En = -8
Loss; z66
AWb, -og : hig AWb, W cb YT;
// backward updates
end
Internally generated perceptual experiences, such as hallucinations, dreams, and mental imagery evoke vivid experiences that mimic the perception of real-world stimuli. Neuroimaging studies demonstrate an overlap in neural activation between internally generated experiences and perception suggesting shared neural substrate for generating and processing sensory information (Ganis et al., 2004; Pearson, 2019; Pearson et al., 2008; Abid et al., 2016; Dijkstra et al., 2017). While studies have provided insights into the brain regions involved in these phenomena, the neural mechanisms and computations underlying hallucination and imagery remain a topic of ongoing research and debate. One challenge is that hallucinations and imagery are subjective experiences that difficult to objectively measure and study (Pearson et al., 2008). Additionally, the neural correlates of these experiences can vary across individuals and different types of hallucinations (Suzuki al., 2017, 2023). Thus, computational modeling of how comparable phenomena can emerge in neural networks, without explicitly training for complex -ooppu-lllluible generative objective functions (Goodfellow et al., 2014), would help elucidate the neural mechanisms that may underpin these internally- internally-generated perceptions.
2.2 Bio-plausible training
Our work also falls within the class of bio-plausible extensions of the original BP algorithm that try to avoid the weight transport problem (Grossberg, 1987). In an effort to mitigate the weight transport problem, (Lillicrap et al., 2016) developed algorithm known Feedback Alignment (FA). This algorithm employs random and fixed feedback connections to propagate errors, initiating line of research that explored different variations of random, yet fixed, feedback connections for credit assignment (Nøkland, 2016; Crafton et al., 2019; Liao et al., 2015). Another line of work uses strategy that still aims for BP-like symmetric weights while circumventing weight transport by designing a training objective for the feedback path that encourages symmetry (Akrout et al., 2019). For example, augmenting reconstruction loss with weight decay will constrain solutions to the transpose in the linear setting (Kunin et al., 2019, 2020). However, our method differs in two key ways. First, those methods required invoking separate gradient pass to train the feedback weights whereas we accomplish the training of feedback with the same feedforward network, thus adding no other hidden paths. Second, those methods explicitly seek symmetry whereas we do not constrain the stage-wiss wise feedback weights, only their end-to-ent goal. Our algorithm resembles the stage-wise reconstruction target propagation (TP) which could also result in end-to-end propagation of latent representations back to inputs if noise at each local propagation step is sufficiently small (Bengio, 2014; Lee et al., 2014). Unlike the layerwise implementation of TP, our training approach end-to-end, and we do not use any BP training the penultimate layer of the discriminator.

---

MNIST
CIFAR10
Reconstruction (correltioo)
Reconstruction (correlatio)
0.8
0.6
FFA
0.4
BP
CIFAR10
FA
0.2
Liscrimination
0.0
resesto
-0.2
custemand
vesnett
100
cuatommezd
ruaedt22
90
Discrimination (accuracy
Discrimination (accuracy)
100
80-
Tost accuracyy
95
70
MNIST
90
60
100
Impuut nols
85
50-
Toat accuracy
40
CIFAR10
0
100
150
0
50
100
150
200
0.0
0.5
epochs
Inpur noise
Figure 2: Co-optimization in FFA. A) Accuracy and reconstruction performance for FFA and control algorithms as a function of epochs. B) Dual-task performance for a variety of feedforward discriminative and autoencoder architectures trained under BP FA compared to FFA training (details for architecture in Suppl. 8.1). The shaded area represents the desired corner. C) Robustness input Gaussian noise (μu = 0 and varying o² between 0 and 1) measured by test accuracy the noisy input.
3 Feedback-Feedforvaan Alignment
During the training, BP uses a computational graph to backpropagate the error to the hidden layers (Figure 1). This computational graph is linear neural network that is the transpose of the forward neural network and is constantly updated every time the forward weights are updated. FA and in general the family of the random feedback gradient path such DFA, random values and do not update the backward weights during the training. FFA in essence runs two FA algorithms to train the forward pass and backward pass alternatively. The FFA diagram in Figure 1 highlights its two distinguishing features: feedback (decoder) has an end-to-end goal and co-opting of the forward discriminator path (encoder) train this decoder. A pseudo-code for simple three-layer neural network to clarify the parameter updates in FFA is in algorithm 1.
Below, we compare how FFA operates on MNIST across two architectures (fully connected and convolutional) and on CIFARIO using a ResNet architecture by directly reconstructing from the onndimensional discriminator output. For details on the architecture please refer to Supplementary material 8.1. For each architecture, we compare FFA to BP and feedback alignment (FA) (Lillicrap al., 2016) training of single objective (feedforward discrimination or an autoencoder loss) resulting in 5 control models: FFA, BP, FA, BP-AE, and FA-AE. The purpose of these controls was to verify that the properties of gradient descent on single loss at the output does not trivially invoke reconstruction of input for example in BP-trained discriminative networks.
3.1 FFA achieves the co-optimization of discrimination and reconstruction
We highlight performance results on a convolutional architecture but also report results on fully connected architecture. Convolutional architectures are potentially of greater interest because they are used for scaling up algorithms to larger datasets. Furthermore, convolutional architectures tend to expose greater performance gaps between BP and FA (Bartunov et al., 2018). FFA-trained networks achieved digit discrimination performance on par with FA but slightly below BP (Figure 2). However, on MNIST, reconstruction performance is exceedingly high. Critically, were also interested in
5

---

A
noise robustness o=m0.0
B
100
FFA
FA-Autoencoder
BP-Autoencoder
Test Test accuracy
2
4
7
FFA
60-
BP
itr 0 inputs
40-
FA
7
2
4
reconstructed images
0
noise robustness d#=0.44
2
80-
7
Test accuracy
60-
itr itr
7
2
4
60-
20-
2
2
0 iteration
Figure 3: Denoising in FFA. Closed-loop inference on noisy inputs (a² = 0.4) performed by FFA and control algorithms assuming static read-out for discrimination by iteration O. Shown right, the sample reconstructions recovered by FFA and control autoencoders over 4 iterations (no clipping or other processing performed these images).
seeing if FFA could co-train, using only the discriminator weights for credit assignment, digit reconstruction path. We found that FFA produced reconstruction on par with a BP-trained autoencoder for convolutional architectures while slightly lagging the autoencoder standard for reconstruction on fully connected architectures. Thus, within the same network. FFA co-optimizes two objectives at levels approaching the high individual standards set by BP-trained discriminator and a BP- trained reconstructor (see Figure 2 and Figure Suppl. 7). In FFA, like FA, the feedforward and feedback weights aligned over training (Lillicrap et al., 2016), but only in FFA, alignment is useful for reconstruction, presumably because both paths are free to align to each other which breaks the random feedback constraint of FA. In examining discrimination versus reconstruction performance, these can be mutually exclusive. For example, single objective networks tend to improve along one axis the other. In contrast, FFA-trained networks moved toward the top-right corner of the plot indicating co-optimization along both axes (Figure 2, scatter plot). As shown in Figure 2B for CIFARIO, FFA and FA both struggle keep up with BP, so for the rest of the paper regarding inference, we focus on MNIST.
3.2
FFA induces robustness to image noise and adversarial attacks
Although in FFA training, we did not use any noise augmentation, as we show in this section, the network trained under FFA developed robustness noise and adversarial attacks relative to the BP control. Previous works showed that BP networks are vulnerable to noise and highlighted that FA-trained networks are surprisingly robust (Goodfellow al., 2015: Akrout, 2019). When pixel noise was used to degrade input characters, we found that FFA was more robust than BP conferring some of the same robustness seen FA (Figure 2C). This advantage of FFA and FA over BP was also true for gradient-based white-box adversarial attacks (Figure Suppl. 9).
Flexible visual inference through recursion
While FFA is not explicitly recurrent network, by coupling the feedforward and feedback pathways through mutual learning of dual, complementary losses, it may indirectly encourage compatibility in their inference processes. That is, we can run the network in closed loop. passing å from the decoder back in as input to the encoder (replacing x)) (see Figure 1). In this section, we explore the capabilities of FFA dealing with missing information (noise or occlusion) and in generation (visual imagery, hallucinations, or dreams). It is worth noting that FFA was not trained to perform any of these tasks and was only trained for discrimination and reconstruction of clean images, conditioned on this discrimination. The inference algorithm in this section relies on two main components:

---

Original
Occluded
High Noise
Low Noise
llerations
lterations
5
8
9
Figure Resolving occlusion. A 15x15 black square occludes the digits in the first columns as shown in the second column. Each row shows sample occluded digit (5, 8, and 9) and the corresponding resolved images under high noise and low noise conditions. For high noise and low noise visual inference, the resolved digit is depicted in 5th and the last columns, respectively. For details regarding the sample intermediate iterations refer to Figure Suppl. 10.
recursion and noisiness of inference in each recursion. The algorithm was developed in (Kadkhodaie and Simoncelli, 2021) for denoising autoencoders based on Empirical Bayes Theorem (Miyasawa, 1961). Although FFA is not trained as a denoiser autoencoder (no noisy input was used during training), we hypothesized that since exhibits robustness to noise properties, then the theory applics here and the algorithm can be adapted to draw effective inferences from the representation learned by FFA. We especially focused on the effect of the noisiness of inference to inform the computational role of noisy responses in actual biological neurons, as the utility of noisy responses remains largely unknown despite extensive active research (Echeveste and Lengyel. 2018; Findling and Wyart, 2021; McDonnell and Ward, 201 1).
0
High Noise
E
6
D
Low Noise
X
Figure 5: Hallucination. Without external input, let the inference algorithm run on the FFA-trained network until convergence (the last column) for high noise (upper) and low noise (lower) inference. The sample iterations linearly spaced and for high noise, there typically twice many iterations needed. Refer to Section 8.8 for iteration values.
7

---

4.1 Denoising
As a first step toward future recurrent processing within FFA, we simply ran the network in a closed loop, passing E from the decoder back in as input to the encoder (replacing x) (see Figure 1) and found that both discrimination and reconstruction performance is sustained over iterations similar to autoencoder whereas BP and FA discriminators change over multiple closed loop iterations when their feedback path from training was used for generative inference in runtime and thus would require dynamic decoder recover any performance (Figure 3).
4.2 Resolving occlusions
We occlude parts of input image by blank square and run the network inference. The assumption here is that the occluded image was briefly presented and during the inference, the original image is not accessible throughout inference. Figure shows examples of completion of the pattern using FFA. Even though for high noise inference more iterations were needed, the generated samples do not reflect any superiority compared to low noise inference which took fewer iterations to converge.
4.3 Hallucination
Visual hallucinations refer to the experience of perceiving objects events even when there is no corresponding sensory stimulation that would typically give rise to such perceptions. As mentioned above, the spontaneous activity in V1 is linked to the vividness of hallucinated patterns. Here, let the FFA-trained network run through the inference algorithm starting from Gaussian noise at the input and adding noise in each iteration. As shown in Figure 5, when in the high noise regime (3 = 0.2), the quality of hallucinated digits is better compared to the low noise regime (3 = 0.99, for the definition of 33 see Section 8.5). Given that the noise in the inference algorithm controls the convergence rate (Kadkhodaie and Simoncelli, 2021), 1), these results suggest that the computational role of spontaneous activity in generating stronger hallucinated percepts may be the refinement of the hallucinated patterns.
4.4 Mental imagery
Visual mental imagery refers to the ability to create mental representations or pictures of visual information in the absence of actual sensory input (Pearson al., 2015; Colombo, 2012). A key distinction between mental imagery and hallucinations is that mental imagery involves voluntarily creating mental images through imagination, while hallucinations To implement the voluntary, top-down activation of percept (c.g. '9"), we add the average activation pattern of the category the latent layer to each recursion the inference algorithm. Presumably, the brain has recollection of the category which can be read out from memory during mental imagery.
involuntary sensory perceptions.
Figure shows that as noise the inference goes higher, so does the quality of the imagined digits.
5 Limitations
We acknowledge several limitations of the in its current form. One key limitation is the difficulty of scaling FFA larger datasets such as ImageNet. While we observed gaps in performance compared to classical backpropagation (BP) on CIFAR10, found little difference compared the Feedback Alignment (FA) baseline in discrimination performance on smaller datasets such as CIFARIO. However, it is yet possible that FFA could be more suitable large scale for specific architectures, such transformers, where layer sizes do not decrease towards the output layer. Scaling up FFA requires further theoretical and empirical exploration. Another limitation is related to the assessment of the generated inferences. Currently, the evaluation relies primarily on visual inspection. Although we included classifier accuracy reports for denoising, assumes that perception arises solely from top activations and that bottom hierarchy activation (such as VI) does not directly contribute to perception. Enhancing the evaluation methodology to incorporate more objective measures and quantitative assessments of generated inferences would strengthen the framework. Furthermore, while FFA demonstrates balance between discrimination performance, efficient learning, and robust recurrent inference, important to acknowledge that FFA may not fully capture all aspects of the biological brain. The
Aligument (FFA) framework
8

---

Imagine '5'
Imagine '3'
samples
samples
5
5
5
6
8
6
3
3
Higher noise
5
5
5
5
3
3
3
3
3
5
5
5
5
5
3
3
3
3
3
5
5
5
5
5
3
3
3
3
3
iterations
iterations
5
5
5
5
5
8
8
8
a
3
Higher noise
5
3
3
3
3
3
9
5
3
3
5
8
3
Figure 6: Visual imagery. Generated samples (upper panels) using the inference algorithm on the FFA- trained network when top-down signal '5' (left) and '3' (right) was activated. The sample iterations (equally spaced) for sample generations were shown the lower panel. Each row corresponds to an inference noise level. Refer to Section 8.7 for iteration and 3 values
framework represents a step towards understanding the brain's 's mechanisms but may still fall short in faithfully replicating the intricacies of neural processing such as the precise dynamics of neural responses or differences in tuning among feedforward and feedback neural subtypes, though it takes a step toward those directions. Overall, these limitations highlight the need for further research and development to address the scalability of FFA, refine evaluation methodologies, and gain deeper insights into the biological plausibility of the framework. Overcoming these limitations will pave the way for more effective and robust alternatives to BP, advancing the understanding and application of neural network training algorithms to neurobiology.
Conclusions
In moving beyond classical error backpropagation training of single-objective, feedforward network, we have presented sedfoorwadd-eedbbal algorithm that trains neural networks to achieve mutualistic optimization of dual objectives. Co-optimization provides attendant advantages: avoids weight transport, increases robustness to noise and adversarial attack, and gives feedback its own runtime function that allows closed-loop inference. Through our experiments, we demonstrated that the network trained using the FFA approach supports various visual inference tasks.
7 Broader Impacts
This work has broader impacts that include advancing our understanding of human perception, enhancing the robustness and performance of neural networks, helping to identify the emergence of closed-loop inference in larger networks for real-time applications, and potential implications for clinical research of mental disorders. By studying the neural mechanisms underlying visual perception, this research contributes to our understanding of natural and artificial vision.

---

Acknowledgments
T.T. supported by NIH IK99EY035357-0L. This work was supported by NIH RFIDA056397, NSF 1707398, Gatsby Charitable Foundation GAT3708, NIH R34 (NS116739), Klingenstein-Sine -imeons fellowship. Sloan Foundation fellowship and Grossman Kavli Scholar Award, as well as NVIDIA GPU grant, and was performed using the Columbia Zuckerman Axon GPU cluster.
References
Abid, H., Ahmad, F., Lee, S. Y., Park, H. W., Im, D., Ahmad, I., and Chaudhary, S. U. (2016). A functional magnetic resonance imaging investigation of visual hallucinations in the human striate cortex. Behav. Brain Funct., 12(1):31.
Akrout, M. (2019), On the Adversarial Robustness of Neural Networks without Weight Transport. arXiv:I908. 03560 [cs, stat]. arXiv: 1908.03560.
Akrout, M., Wilson, C., Humphreys. P. C.. Lillicrap. T., and Tweed, D. (2019). Deep Learning without Weight Transport. arXiv: 1904.0339 [cs, star]. arXiv: 1904.05391.
Bartunov, S., Santoro, A., Richards, B. A., Marris, L., Hinton, G. E., and Lillicrap, T. (2018). Assessing the Scalability of Biologically-Motivan. Deep Learning Algorithms and Architectures. arXiv: 1807.0/807 [cs, stat]. arXiv: 0057004577.
Bengio, Y. (2014). How Auto-Encoders Could Provide Credit Assignment in Deep Networks via [cs]. arXiv: 1407.7906.
Target Propagation.
Choksi, B., Mozafari, M., O'May, C. B., Ador, B., Alamia, A., and VanRullen, R. (2021). Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics.
Clark, A. (2013). Whatever next? predictive brains, situated agents, and the future of cognitive science. Behav. Brain Sei., 36(3):181-204.
the Colombo, B. (2012). Mental imagery. In Encyclopedia of the Sciences of Learning, pages 2187-2191. Springer US Boston, MA.
Crafton, B., Parihar, A., Gebhardt, E., and Raychowdhury, A. (2019). Direct feedback alignment with sparse connections for local learning. Front. Neurosci., 13:525.
Debes, S. R. and Dragoi, V. (2023). Suppressing feedback signals to visual cortex abolishes attentional modulation. Science, 379(6631) 473..
Dijkstra, N., Bosch, S. E., and van Gerven, M. A. J. (2017). Vividness of visual imagery depends on the neural overlap with perception in visual areas. J. Neurosci., 375):
Echeveste, R. and Lengyel, M. (2018). The redemption of noise: Inference with neural populations. Trends Neurosci., 41611):767-770
Findling, C. and Wyart, V. (2021). Computation noise in human learning and making: origin, impact, function. Curr. Opin. Behav. Sci., 38:124-132.
Friston, K. (2009). The free-energy principle: rough guide the brain? Trends Cogn. Sci., 13(7):293-301.
Ganis, G., Thompson, W. L., and Kosslyn, S. M. (2004). Brain areas underlying visual mental imagery and visual perception: an fMRI study. Cognitive Brain Research, 2062):26-2441
Gilbert, C. D. and Sigman, M. (2007). Brain states: top-down influences in sensory processing. Neuron, 54(5):677-666.
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial networks.
Goodfellow, I. J., Shlens, J., and Szegedy, C. (2015). Explaining and Harnessing Adversarial Examples. arXiv:1412.655 [cs, stat]. arXiv: 1412.6572.
10

---

Grossberg, S. (1987). Competitive learning: From interactive activation to adaptive resonance. Cognitive Science, 11(1):23-63.
He, K., Zhang. X., Ren, S., and Sun, (2015). Deep Residual Learning for Image Recognition. arXiv:1512.03335 [cs]. arXiv: 1512.03385.
Helmholtz, H. v., Gullstrand, A., Kries, J. v., Nagel, W. A., and University College, I. L. S. (1909). Handbuch der Physiologischen Optik [electronic resource]. Hamburg ; Leipzig verlag von Leopold Voss.
Kadkhodaie, Z. and Simoncelli, E. (2021). Stochastic solutions for linear inverse problems using the prior implicit in denoiser. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P. and Vaughan, J. W., editors, Advances in Neural Information Processing Systems, volume 34, pages 13242-13254. Curran Associates, Inc.
Kar, K., Kubilius, J., Schmidt, K., Issa, E. B., and DiCarlo, J. J. (2019). Evidence that recurrent circuits critical to the ventral stream's execution of core object recognition behavior, Nat. Neurosci,, 22(6))974-983
Keller, G. B. and Mrsic-Flogel. T. D. (2018). Predictive Processing: A Canonical Cortical Computa- tion. Neuron, 100(2)4224435.
Khaligh-Razavi, S.-M. and Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Comput. Biol., 10(1 1):e
Kietzmann, T. C., Spoerer, C. J., Sörensen, L. K. A., Cichy, R. M., Hauk, O., and Kriegeskorte, N. (2019). Recurrence is required to capture the representational dynamics of the human visual system. Proc. Natl. Acad. Sci. U. S. A., (1644)):2154-22186
Kreiman, G. and Serre, T. (2020). Beyond the feedforward sweep: feedback computations in the visual cortex. Ann. N. Acad. Sci., 1464(1) 2222-241.
Krizhevsky, A., Sutskever, I., and Hinton, E. (2012). ImageNet Classification with Deep Convolu- tional Neural Networks. In Pereira, F., Burges, C. J. C., Bottou, L., and Weinberger, K. Q.. editors, Advances Neural Information Processing Systems 25, pages 1097-1105. Curran Associates, Inc.
Kunin, D., Bloom, M., Goeva, A., and Seed, C. (2019). Loss Landscapes of Regularized Linear Autoencoders. 190.iv:900.0011101 [cs, stat]. arXiv: 1901.08166
Kunin, D., Nayebi, A., Sagastuy- J., Ganguli, S., Bloom, J., and Yamins, D. L. K. (2020). Two Routes to Scalable Credit Assignment without Weight Symmetry. arXivv2003.01513 [cs, q-bio, stat]. arXiv: 2003.01513.
Lee, D.-H., Zhang, S., Fischer, A., and Bengio, Y. (2014). Difference Target Propagation. 141i [cs]. arXiv: 1412.7525.
Lee, T. S. and Mumford, D. (2003). Hierarchical bayesian inference in the visual cortex. JOSA A, 14444.434-1448.
Liao, Q., Leibo, Z., and Poggio, T. (2015). How important weight symmetry in backpropagation?
Lillicrap, T. P.. Cownden, D., Tweed, B., and Akerman, C. J. (2016). Random synaptic feedback
weights support error backpropagation for deep learning. Nature Communications, 7:13276
Lillicrap. T. P. and Santoro, A. (2019a). Backpropagation through time and the brain. Curr. Opin. Neurobiol, 55:82-89.
Lillicrap, T. and Santoro, A. (2019b). Backpropagation through time and the brain. Current Opinion in Neurobiology, 55:82-80 89.
Lindsay, G. W. (2021). Convolutional neural networks as model of the visual system: Past, present, and future. Cogn. Neurosci, 33(10):2017-20::
Markov, T., Fresey-Ravasz, M., Essen, D. C. V., Knoblauch, K., Toroezkai, 7.,, and Kennedy, H. (2013). Cortical Hiihh-Density Counterstream Architectures. Science,

---

Markov, N. T., Vezoli, J., Chameau. P., Amaad Falchier, Quilodran, R., Huissoud, C., Lamy, C., Misery, P., Giroud, P., Ullman, S., Barone, P., Dehay, C., Knoblauch, K., and Kennedy, H. (2014). Anatomy of hierarchy: Feedforward and feedback pathways in macaque visual cortex. Journal of Comparative Neurology, 522(1):255-259
McDonnell, M. D. and Ward, L. M. (2011). The benefits of noise in neural systems: bridging theory and experiment. Nat. Rev. Neurosci., 12(7):415-426.
Millidge, B., Tschantz, A., and Buckley, C. L. (2020). Predictive coding approximates backprop along arbitrary computation graphs.
Miyasawa, K. (1961). An empirical Bayes estimator of the mean of a normal population. Bull. Inst. Internat. Statist., 38:181-188
Nayebi, A., Eogastuy-Brenal J., Bear, M., Kar, K., Kubilius, J., Ganguli, S., Sussillo, D., DiCarlo, J. J., and Yamins, D. L. K. (2022). Recurrent connections in the primate ventral visual stream mediate a trade-off between task performance and network size during core object recognition. Neural Comput., 34(8):1652-16733
Nøkland, (2016). Direct feedback alignment provides learning in deep neural networks.
Pearson, J. (2019). The human imagination: the cognitive neuroscience of visual mental imagery. Nat. Rev. Neurosci., 20(10):624-634
Pearson, J., Clifford, C. W. G., and Tong, F. (2008). The functional impact mental imagery on conscious perception. Curr. Biol., 18(13):982-988
Pearson, J., Naselaris, T., Holmes, E. A., and Kosslyn, S. M. (2015), Mental imagery: Functional mechanisms and clinical applications. Trends Cogn. Sci., 1910))590-600.
Rao, N. and Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1):79-88 87.
Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning representations by back- propagating errors. Nature, 323(6088):533-536
Semedo, J. D., Jasper, A. I., Zandvakili, A., Krishna, A., Aschner, A., Machens, C. K., Kohn, A., and Yu, B. M. (2022). Feedforward and feedback interactions between visual cortical areas use different population activity patterns. Nat. Commun. 13(1):1099.
Spoerer, C. J., McClure, P., and Kriegeskorte, N. (2017). Recurrent convolutional neural networks: A better model of biological object recognition. Front. Psychol., 8.
Suzuki, K., Roseboom, W., Schwartzman, D. J., and Seth, A. K. (2017). deep-dream virtual reality platform for studying altered perceptual phenomenology. Sci. Rep.. 751):19982.
Suzuki, K.. Seth, A. K., and Schwartzman, D. (2023). Modelling phenomenological differences in aetiologically distinct visual hallucinations using deep neural networks.
Tang, H., Buia, C., Madhavan, R., Crone, N. E., Madsen, J. R., Anderson, W. S., and Kreiman, G. (2014). Spatiotemporal dynamics underlying object completion in human ventral visual cortex. Neuron, 83(3):736-748.
Tang, H., Schrimpf, M., Lotter, W., Moerman, C., Paredes, A., Ortega Caro, J., Hardesty, W., Cox, D., and Kreiman, G. (2018). Recurrent computations for visual pattern completion. Proc. Natl. Acad. Sci. U. S. A., 115((5):8833-
Tseshantz, A.. Millidge, B., Seth, A. K., and Buckley, C. L. (2023). Hybrid predictive coding: Inferring, fast and slow. PLoS Comput. Biol., 190))e1011220.
Walsh, K. S., McGovem, D. P., Clark, A., and O'Connell, R. G. (2020). Evaluating the neuro- physiological evidence for predictive processing as model of perception. Ann. N. Y. Acad. Sci., 1464(1))242-268.
12

---

Wen, H., Han, K.. Shi, J.. Zhang, Y., Culurciello, E., and Liu, Z. (2018). Deep predictive coding network for object recognition. In Dy, and Krause, A., editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5266-5275. PMIR.
Whittington, J. C. R. and Bogacz, R. (2017). An approximation of the error backpropagation algorithm in predictive coding network with local hebbian synaptic plasticity. Neural Comput., 2969):1229-126..
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., and DiCarlo, J. (2014). Performance-optinized hierarchical models predict neural responses in higher visual cortex. Proc. Nail. Acad. Sci. U. S. A., 1 1116884861--662
13

---

8 Supplementary
8.1 Model architectures
For the experiment on a fully connected architecture, we used 4-layer network with [1024, 256,256,10] neurons in each layer and ReLU non-linearity between layers. For the experiment on convolutional architecture, we used modified version of ResNet (He et al., 2015), where the last convolutional layer has the same number of channels as classes, and an adaptive average pooling operator is used to read out of each channel (see below). Since the last layer doesn't it have any learnable parameters, the penultimate layer can be as large as desired which works fine for FFA. The convolutional architecture consists of 11 convolutional layers with 658,900 trainable parameters in total. For autoencoder controls (trained under BP or FA), we additionally trained a linear decoder on the activations of the penultimate layer to assess the linear separability of the representation learned by autoencoders.
Code is available at: http:://ghhtt.commooo////cccdced Freedioowarr_Aliggmentt
8.2 More control networks: autoencoders
MNIST
CIFAR10
Reconstruction (correlation)
FFA
(comelation!
1.0
1.0
0.75
0.75
0.5
0.25
0-
50
100
150
100 epochs
200
epochs
100
Discrimination (accuracy)
Discrimination (accuracy)
100
80
80
40
70
20
100
150
100 epochs
200
epochs
Reconstruction (MSE)
102
Reconsttruttion (MSE)
10a
10-1
10%
10
50
100
150
100
200
epochs
epochs
Figure 7: Co-optimization in FFA compared to single objective (either discrimination or reconstruc- tion) control networks for MNIST and CIFAR1O (extensive version Figure 2).
14

---

8.3 Alignment between feedback and feedforward
A) MNIST
Alignment
Discrimination
Reconstruction
FFA
FA
perforancc (%)
LIFF. FB)
correlation
B) CIFAR10
Alignment
Discrimination
Reconstruction
80'
100
0.8
FFA
FA
perfomance (55)
correlation
500
600
epoch
Figure 8: Alignment of feedback (FB) and feedforward (FF) weights for A) MNIST B) CIFARIO datasets over the course of training shown in the first column. The smaller the angle between the FF and FB weights, the higher the alignment. The second and third columns show discrimination and reconstruction performances test sets.
8.4 Robustness assessment
We added Gaussian noise with zero mean and varied the variance o² 0.2.0. 1.0] to assess the robustness of models to imput noise. We also performed a widely used white box adversarial attack Fast Gradient Sign Method (FGSM) (Goodfellow al., 2015). FGSM can summarized by
+
where a is the magnitude of the perturbation, J is the loss function and ys the label of x. While in BP this perturbation is computed through transposed forward parameters, for FFA and FA, used their gradient pass parameters which are learned feedback and random feedback, respectively. We used range of cover the interval between 0 I.
8.5 Visual inference algorithm
adapt the sampling algorithm developed (Kadkhodaie and Simoncelli, 2021) to implement the
visual inference in FFA-trained networks. | parameter which varies between 0 and 1 controls the proportion of injected noise (3 indicates no noise). The injected noise together with the gradient step size controls the number of iterations for convergence.
15

---

100
FFA
BP
80
Test accuracy
accuracy
60
40
40
20
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
input noise o²
Attack
Figure 9: Robustness to Gaussian noise and adversarial attacks for MNIST: Robustness to noise and adversarial attacks in input (image) space for FFA and control algorithms. FA and FFA both exhibit more robustness than BP-trained discriminators.
Algorithm 2
The core visual inference algorithm (adapted with minimal modification from Kadkhodaie and Simoncelli (2021)))
parameters: oo. JL, hps
initialization: / = 1, draw un (0..5, 1)
trained feedforward
G: trained feedback
while ;- S OL do
h [thg(t-1)
L, 11- = (F
d, Ft-1 54-1
of 1|d.11
V1 ((1 (ha)² - (1 h2))) (1)7))² o?
Draw ~ 10,
Ft +- x--1 + hy + 2
t ++1
end
16

---

8.6 Visual occlusion
original
occluded
itro
itrl0
itr21
itr31
itr42
5
itro
itr10
itr20
itr30
itr41
High Noise
8
itrO
itrl0
itr21
itr32
itr43
9
original
occluded
itrO
itr7
itr15
itr22
itr30
5
Low Noise
itrO
itr7
itr14
itr21
itr28
8
itr0
itr7
itr15
itr22
itr30
9
Figure 10: Visual occlusion related to Figure in the main text.
17

---

8.7
Visual imagery
itr
20
itr 28
36
44
= 1.00
5
5
5
5
5
itr
itr
itr
62
itr 77
B 0.80
i0
itr 44
itr 69
9tr
itr 118
B = 0.40
20
60
1rr 100
140
181
B 0.20
5
B 1.00
a
a
20
33
Itr
|tr
75
g = 0.80
3
20
itr 45
itr 70
95
itr 121
B 0.40
3
20
61
144
B 0.20
Figure 11: Sample visual imagery related to Figure 6 in the main text.
18

---

8.8 Hallucinations
itr
itr18
Ir24
Ir77
Ird3
hr44
1tr0
itr8
It:17
1r25
Itr34
Itr42
[t55]
It599
Trs8
RV77
inra
itr
itrs
Itr10
itr15
Itr20
itr25
itr30
itr35
itr40
itré6
7
Itr
It:12
1r:25
IL:32
IL/38
Itr45
Rr51
It588
0.99
Figure 12: Sample hallucinations related to Figure 5 in the main text.
19


Reviews:
[{"review_id": "1JBs9I4PEg", "reviewer": "Reviewer_KkB4", "summary": "The authors propose a mechanism for biologically plausible learning in visual circuits that can endow the system with additional epiphenomenal capabilities that resemble phenomena humans are known to exhibit. The mechanism resembles feedback alignment, where the weight transport problem is resolved by substituting different feedback weights in the backward pass instead of the transpose of the forward weights. In contrast to previous approaches that dynamically train the backward weights to achieve symmetry through local learning, the authors suggest concurrently training the feedback weights to reconstruct the system's visual inputs based exclusively on activations in the final latent layer. They then show that these feedback weights can be used for a wide variety of additional tasks (similar to Kadkhodaie & Simoncelli 2021), including image generation, conditional image generation based on a weakly clamped latent layer, and denoising. ", "questions": "Do multilayer perceptron networks trained on these tasks show qualitatively similar inpainting and generation? The authors say that the MLP results are present, but I've been unable to find them in the text or supplemental. Networks that do not have within-layer weight sharing are of more interest from the perspective of biological plausibility, and while I believe that an MLP should produce similar results, it would be good to have visual confirmation.\n\nCould you discuss the biological plausibility of BatchNorm, which appears to be used in all tested network architectures? Have similar results been obtained in the absence of BatchNorm?", "limitations": "Yes, the authors do adequately discuss the limitations of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The concept for the paper is a very strong one: it is a major failing of the 'feedforward paradigm' of visual cortex modeling that it cannot account for capabilities and phenomena the visual system is known to exhibit, including mental imagery, hallucination, and to some extent de-occlusion (e.g. filling in the visual blind spot caused by the optic nerve). That a network can be shown to accomplish this through exclusively through a reasonably local learning algorithm (a feedback alignment variant) is very impressive. Furthermore, the notion that the feedback pathway could have additional functional utility beyond exclusively providing credit assignment signals for learning is a much-needed perspective for the biologically plausible plasticity field, especially the subfield that is primarily focused on variants of the feedback alignment algorithm. ", "weaknesses": "On the theory of learning side of things, the paper falls short in several ways.\n\nFirst and foremost, the authors do not actually explain their learning algorithm in the detail that would be required to reproduce it. The authors should provide detailed math explaining the actual steps required to implement the proposed rule at each layer of their system, as well as potentially pseudocode. This is required for me to increase my confidence and maintain my score--what I imagine the authors to be doing based on my own understanding of the feedback alignment algorithm is a perfectly viable approach, but it's not possible to confirm this based on the text itself. For reproducibility purposes, the authors should also commit to providing a github link to their code implementation of the model.\n\nSecond, (as with the original FA paper) it is not obvious why the authors' approach works at all. It is possible to draw some loose similarities between the proposed algorithm and target propagation, but the authors provide no mathematics that explain why the proposed algorithm should actually produce learning. Given that the empirical performance results begin to drop off steeply for CIFAR10 (and presumably fail entirely for imagenet), the convergence properties of this algorithm deserve serious attention.\n\nFurthermore, the authors do not perform any form of direct comparison to alternative biologically plausible algorithms that would be expected to show similar results. In particular, both the predictive coding (PC) approach and the Wake-Sleep algorithm should show similar effects, and energy-based models like the Boltzmann machine, PC, and even Hopfield networks have a long history of demonstrating pattern completion effects and image generation on simple datasets like binarized MNIST. The paper would benefit immensely from a discussion of the experimentally testable predictions that are specific to this model and not to other approaches, as well as a discussion of the factors that make this method conform to known experimental data more than these methods.\n\nLastly, the inpainting effects produced in this paper are qualitatively worse than those observed in Kadkhodaie & Simoncelli 2021. The paper would benefit from an empirical comparison to the model actually used in this paper, or at least a discussion of the similarities & differences, as well as why we should expect worse results here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> \"\nStrengths:\nThe concept for the paper is a very strong one: it is a major failing of the 'feedforward paradigm' of visual cortex modeling that it cannot account for capabilities and phenomena the visual system is known to exhibit ...\"\n\nWe appreciate that the reviewer recognizes the significance of the concept of this work and we thank the reviewers for their inputs on our work.\n> \"\nWeaknesses:\nFirst and foremost, the authors do not actually explain their learning algorithm in the detail that would be required to reproduce it...\"\n\nWe apologize for the lack of clarification regarding the learning algorithm. We now added the learning algorithm as pseudo codes to the paper (and in the general response above). As we stated, we will provide the github link to our code including training models by FFA and control algorithms.\n> \"\nSecond, (as with the original FA paper) it is not obvious why the authors' approach works at all. It is possible to draw some loose similarities between the proposed algorithm and target propagation...\"\n\nPlease see the theory section and scalability section in the general response where we addressed this concern. We add here that TP invokes targets at each layer whereas we uniquely only set targets at the discriminator and reconstructor outputs allowing for end-to-end training of the dual loss. End-to-end training tends to perform well relative to local losses with the drawback of requiring backprop, but we benefit from end-to-end goals in a bio-plausible way by learning the backpropagation path implicitly through alignment and not having to invoke a separate, symmetric pathway.\n\n> \"Furthermore, the authors do not perform any form of direct comparison ...\"\n\nRegarding bio-plausibility of algorithms, the premise of our work was to discover a bio-plausible learning algorithm for mainly feedforward deep neural networks which share many neuronal and behavioral similarities to biological vision (Schrimpf et al., 2018, 2020). The generative properties of the implementation were not a direct aim of our set-up but illustrate downstream consequences of this type of feedback, and we tried to stay away from any claims regarding the superiority of FFA in terms of pattern completion or generation. Rather, the fact that feedforward and feedback pathways used for training each other can give rise to generative properties serves as a proof-of-concept illustrating the utility of bio-plausible implementation. Though theoretically intriguing, Hopfield networks, Boltzmann machine and Predictive coding approaches in general, lag behind the DNN class of models in classification performance ((Belyaev & Velichko, 2020) reports a Hopfield network with 56.2% on MNIST) and are have poor predictivity of neural responses and human behavior (Schrimpf et al., 2018) (brain-score.org). \n\n> \"Lastly, the inpainting effects produced in this paper are qualitatively worse than those observed in Kadkhodaie & Simoncelli 2021. The paper would benefit from an empirical comparison to the model actually used in this paper, or at least a discussion of the similarities & differences, as well as why we should expect worse results here.\"\n\n\nKadkhodaie and Simoncelli 2021 trained autoencoders using backpropagation to be robust to input noise by including noisy images in the training set. As we pointed out in the paper (section 3.2), networks trained using FFA were never shown noisy images during training, and we only used noise for evaluation of robustness and inference algorithm. The point we make in the paper is that FFA induces the properties of a denoising autoencoder (presumably by a form of structural regularization, please see the theory section in general response) without receiving any noise training, even if weaker than the denoising autoencoder trained with BP.\n\n> \" Questions:\nDo multilayer perceptron networks trained on these tasks show qualitatively similar inpainting and generation? The authors say that the MLP results are present, but I've been unable to find them in the text or supplemental. Networks that do not have within-layer weight sharing are of more interest from the perspective of biological plausibility, and while I believe that an MLP should produce similar results, it would be good to have visual confirmation.\"\n\nWe acknowledge the ambiguity present in the text. We tested both fully connected (labeled as customized architecture) and resnet-like convolutional architectures for the co-optimization verification (section 3). However, the results in the visual inference were on the convolutional architecture. We agree that weight-sharing in a convolution module has been raised as a concern for bio-plausibility although MLP models which allow connection of any neuron in a layer to any other neuron is also problematic from the biological perspective (neurons tend to send projections to nearby neurons and connectivity strength falls quickly over distance). Also, the theoretical foundation of sampling implicit prior does not assume any architectural constraints (Kadkhodaie & Simoncelli, 2021) confirming the reviewer’s intuition regarding MLP should produce similar results. However, we plan to add this to the supplementary should the paper get accepted.\n\n> \"Could you discuss the biological plausibility of BatchNorm, which appears to be used in all tested network architectures? ...?\"\n\n\nWe agree that BatchNorm in the current form can hardly be considered a bio-Plausible implementation and in future work we plan to replace that with closely related but bio-inspired modules such as (Shaw et al., 2020). BatchNorm has been used in other research related to bio-plausibility of learning algorithms (Kunin et al., 2020; Liao et al., 2016), presumably with the same logic that it is rather feasible to replace this module with a more bio-plausible implementation as the community discovers more about BatchNorm and the biological mechanisms underlying the regularization in the brain. \n"}, {"review_id": "BIXcsg45wi", "reviewer": "Reviewer_6SSt", "summary": "The paper proposes a bio-inspired computer vision algorithm for classification that includes feedback and aims to use alignment rather than backprop for training the models of feedforward and feedback.", "questions": "Are the codes going to be released?", "limitations": "NA", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 1, "contribution": 4, "strengths": "The paper provides a novel idea for including feedback (or recurrence) in visual classification that would be interesting for neuroscientists to investigate further.\nThe emerging noise robustness is a big strength for this model and confirms a lot of neuro theories about feedback.", "weaknesses": "The explanation of the methods is severely inadequate. The training phase is not well explained. I usually prefer to have a visual explanation over equations but in this case I believe that the learning phase, the equations for the learning process should be included as well especially given that there are variables and paramters that are included without explanation of how they contribute to the process in figure 1. The supplements also shows two separate networks for feedforward and feedback and the alignment process should be included.\n\nThe literature review is also not proper and misses many important publications in the field, for example:\n- Spoerer et al 2017: Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition\n- Abdelhack and Kamitani 2018 Sharpening of hierarchical visual feature representations of blurred images\n- Huang et al. 2019 Brain-inspired Robust Vision using Convolutional Neural Networks with Feedback\n\nThe first analysis shows performance on both CIFAR and MNIST while all the subsequent analyses are only including MNIST. This makes me wonder if the performance on CIFAR is not as impressive as it was not included even in supplementary materials. I think the reader would appreciate full transparency on this.\n\nIt was unclear why training on Imagenet would not be feasible. Is it a computing problem or other?\n\nMinor: p 5 line 156 typo: discrimination -> reconstruction ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> \" Strengths:\nThe paper provides a novel idea for including feedback (or recurrence) in visual classification that would be interesting for neuroscientists to investigate further. The emerging noise robustness is a big strength for this model and confirms a lot of neuro theories about feedback.”\n\nWe are extremely grateful for recognizing the novelty of the idea and for providing constructive feedback to improve the work.\n\n> \" Weaknesses:\nThe explanation of the methods is severely inadequate. The training phase is not well explained. I usually prefer to have a visual explanation over equations but in this case I believe that the learning phase, the equations for the learning process should be included as well especially given that there are variables and paramters that are included without explanation of how they contribute to the process in figure 1. The supplements also shows two separate networks for feedforward and feedback and the alignment process should be included.”\n\n\nWe apologize for the lack of clarity in the training phase. We now included a pseudo code to the paper (section 3) to explain the alignment process. \n\n> \" The literature review is also not proper and misses many important publications in the field, for example:\n\n* Spoerer et al 2017: Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition\n\n* Abdelhack and Kamitani 2018 Sharpening of hierarchical visual feature representations of blurred images\n\n* Huang et al. 2019 Brain-inspired Robust Vision using Convolutional Neural Networks with Feedback”\n\n\nThank you for pointing out the above list. We now added citations to the above mentioned works in literature review. However, we highlight the fact that both (Huang et al., 2019; Spoerer et al., 2017) used classical error backpropagation which as we discussed falls out of the bio-plausible implementation of learning. While (Abdelhack & Kamitani, 2018)’s finding regarding the deblurred representation in the brain is interesting, however, they don’t propose a model based on their analysis. Thus, the relevance of this paper is not immediately clear to us. \n\n> \"The first analysis shows performance on both CIFAR and MNIST while all the subsequent analyses are only including MNIST. This makes me wonder if the performance on CIFAR is not as impressive as it was not included even in supplementary materials. I think the reader would appreciate full transparency on this.”\n\n\n\nAs we stated in the paper (section 3.1), we only verified generative properties on the MNIST dataset  (as do virtually all similar studies on generative properties of a bio-plausible implementation) due to lack of backgrounds and less visual ambiguity in the MNIST images compared to CIFAR which makes the results of the generative processes more accessible to visual inspection . We included CIFAR results on the performance and robustness in our attempt to scale up FFA to natural images.\n\n> \"It was unclear why training on Imagenet would not be feasible. Is it a computing problem or other?”\n\n\nAt core, FFA is essentially two feedback alignment algorithms and although there are some theoretical attempts to explain why FA does not scale up to imagenet, there is still no single convincing theory to explain why (please refer to the general response-theory section where we explained the challenges of learning symmetric weights in the feedback). We suspect that with newer architectures maybe a variant of FA such as FFA has a chance and in future work we aim to explore this idea by implementing transformer-based architectures. A good analogy here could be the autoencoders, as for a long time it was assumed that autoencoders are unable to be scaled up to large image sets like imagenet for representational learning purposes, but masked autoencoders proved otherwise.\n\n> \"Minor: p 5 line 156 typo: discrimination -> reconstruction”\n\n\n\n\nThank you for pointing us to this typo. We now fixed this.\n\n> \"Questions:\nAre the codes going to be released?”\n\n\nAbsolutely, as we stated in the checklist. \n\n"}, {"review_id": "pCL35TYmNt", "reviewer": "Reviewer_gcpd", "summary": "This paper explores the intriguing subject of integrating feedback connections into neural networks during both learning and inference stages, drawing inspiration from the brain. This study has the potential to significantly advance artificial intelligence (AI). Feedback connections are a topic of interest in both theoretical and experimental neuroscience, and establishing a link between neuroscience and AI research could mark a major advancement in both fields. Therefore, the motivation behind this study is commendable.\n\nHowever, despite its compelling premise, the paper has issues with the presentation of the algorithm, its novelty, and the explanation for its superior performance over standard auto-encoders. Specifically, Figure 1 and the corresponding explanation in the main body of the text are brief and do not provide clear insight into how their system operates. Figure 1C suggests that the feedforward weights are trained using Feedback Alignment (FA) and not Backpropagation (BP) at any stage or layer of the network, yet this point is not explicitly clarified.\n\nThe authors also neglected to specify whether noise was introduced into the hidden layers to replicate the inherent noise in neurons, despite stating that no noise was added to augment the data during training. Additionally, during the inference stage, the reconstruction is updated and passed as an input in the forward pass, going through multiple iterations. However, the paper does not provide guidelines on how and when this iterative process should cease to deliver a final inference result.\n\nIn Figure 3A, there is a noticeable degradation in performance and reconstruction as a function of iterations. This degradation is less pronounced than with the FA-autoencoder or BP-autoencoder, but its presence is still problematic.\n\nA significant point of confusion is the authors' claim that training their architecture with FA versus BP in both forward and feedback processes would lead to better generalization. This claim is not sufficiently explained in the paper. The scalability of the proposed method is another major concern. Like other FA training methods, the presented approach appears incapable of scaling to more complex problems, such as ImageNET, which is a significant limitation. Considering that there is no explanation provided for why their algorithm is more robust to noise in simpler problems like MNIST, the lack of scalability is a particular concern.\n\nFurthermore, the claim of adversarial robustness is not adequately substantiated, as it should include a more comprehensive set of white box attacks. Since 2015, numerous studies have shown that networks improving robustness to the Fast Gradient Sign Method (FGSM) are not necessarily robust against stronger attacks, such as boundary-based optimization methods.\n\nIn summary, while the paper delves into an intriguing research area and is well-motivated, it lacks clarity regarding the proposed algorithm and its novelty compared to standard autoencoders. Additional concerns include the paper's omission of details about potential noise introduced in hidden layers, and the lack of a clear process for terminating the iterative inference stage. Additionally, the authors' assertion of better generalization through FA versus BP is not clearly explained. Issues of scalability and incomplete testing for adversarial robustness also cast doubt on the practicality and comprehensive effectiveness of the proposed method. If these issues are clearly addressed in a revision, this paper could contribute significantly to AI research.\"\n\n", "questions": "Explain the algorithm much better and compare to previous work like auto-encoders. Provide theoretical or intuitive explanation why training this particular architecture with FA is better than BP. Was not clear if noise is introduced in the hidden layers. ", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Important problem \nInteresting idea to train both forward and feedback weights with FA ", "weaknesses": "Need more clarity \nMore explanation why training with FA works better for generalization than BP\nLack scalability to more complex problems \nNeeds stronger white box attacks to substantiate claims about robustness to adversarial attacks ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> \"Summary:\nThis paper explores the intriguing subject of integrating feedback connections into neural networks....\n.”\n\n\nWe would like to thank the reviewer for recognizing the significance of the aim of this work.\nWe agree that the algorithm was not clarified in detail and we apologize for this oversight. We now include pseudo code (section 3) to supplement Figure 1’s visual presentation of the algorithm.\n\n> \"The authors also neglected to specify whether noise was introduced ...”\n\n\nWe apologize for lack of clarity on noise and the inference algorithm. We should emphasize that there was no added noise during the training. We did not model the inherent noisiness of the biological neurons. Noisiness of neurons being present in both training and evaluation presumably would require a probabilistic read-out such as what is offered in random smoothing models (Cohen et al., 2019). Thus to avoid complexity and ease of comparison to the control algorithms we avoid modeling the internet noisiness of biological neurons. \nRegarding the inference algorithm, the reviewer is absolutely correct. Although we had included the inference algorithm pseudocode to the supplementary and a schematic of inference in Figure 1, we failed to provide details on the criteria to stop the algorithm explicitly. Essentially, the algorithm stops, when the difference between the reconstructed image $\\hat{x_{t-1}}$ and the input image $x_{t-1}$ falls below an imposed upper bound (sec 8.4: $ \\sigma_{t-1} \\leq \\sigma_L $). We added details on the inference algorithm to the main paper.\n\n> \"In Figure 3A, there is a noticeable degradation in performance and reconstruction as a function of iterations. This degradation is less pronounced than with the FA-autoencoder or BP-autoencoder, but its presence is still problematic.”\n\n\nThe inclusion of iterations greater than 2 were for the purpose of full transparency. Since 1) in FFA the training for the reconstruction when there is no iteration (iteration 0 in the plot) and 2) FFA does not use any input noise in the training, the main purpose of the experiment was to examine the functional benefits of FFA in terms of an implicit regularization effect (denoising). The reviewer is correct that the degradation of performance over iterations happens even for FFA, but please consider the fact that this analysis is simply highlighting an emergent property of having this type of feedback to even allow for iteration – no optimization was done to handle the iterations and thus any maintenance of the performance over the iterations. It’s likely implausible to only perform end-to-end recurrence in this fashion. Future work could examine more plausible recurrent connections between each layer of the network; we simply ran the most basic one first. \n\n\n> \"A significant point of confusion is the authors' claim that training their architecture with FA versus BP in both forward and feedback processes would lead to better generalization...”\n\n\nWe refer to the  generalization of FFA to noise-degraded images which was shown in Figure 3. We agree that the scalability remains a limitation in this work as we addressed in the limitation section, but please refer to the general response above in the theory section for an explanation of why robustness to noise can be expected in FFA training.\n\n> \" Furthermore, the claim of adversarial robustness is not adequately substantiated, as it should include a more comprehensive set of white box attacks...”\n\n\nThe reviewer is absolutely correct that robustness to FGSM does not prove robustness to stronger attacks. Essentially, there are a lot of attacks and counter-attacks that have been proposed over the years in the literature to examine adversarial robustness. Here, we used FGSM as a simple yet fundamental measure of robustness that was used in previous studies of bio-plausible learning algorithms (Akrout, 2019).\n\n> \"In summary, while the paper delves into an intriguing research area and is well-motivated,...\"”\n\n\nWe appreciate the summary of the constructive comments from the reviewer. To clarify regarding the proposed algorithm, we added a pseudocode to the paper. Regarding the novelty compared to the standard autoencoders, we should clarify that standard autoencoders use backpropagation for training which as we discussed in the paper cannot be implemented using biological neurons because of the weight transport problem. Standard denoiser autoencoders are trained using added noise to the input while the loss function includes the original noiseless image to induce robustness to noise. However, training using FFA 1) does not require weight transport hence a bio-plausible implementation and 2) does not require input noise training to induce robustness.\nWe believe the reviewer is referring to noise robustness when asking about model generalization. However, in our general rebuttal above, we acknowledge that we are not entirely certain about the existence of an encompassing theory and welcome any follow-up comments if our explanation is not suitably directed to the intended generalization phenomenon.\nAs we acknowledged in the limitation section of the paper, scalability is a limitation at the current stage of the work; however, as in other original, proof-of-concept work (as indicated directly in the abstract), we agree that future work relying on novel architectures and more relevant datasets and increasingly more compatible loss functions can improve the scalability to more interesting datasets. We appreciate that the reviewer could conceptualize the significance of this type of interdisciplinary research (neuroscience and AI) to the broader community of AI research.\n\n>\" Weaknesses:\nNeed more clarity ...\"\n\nPlease see the theory section in general response.\n\n>\"Questions:\nExplain the algorithm ...\"\n\nWe now include pseudocode and intuitive theoretical accounts behind FFA training (please see general response). As stated above, no noise was introduced in the hidden layers.  \n"}, {"review_id": "B2U3Vo43rG", "reviewer": "Reviewer_jn5o", "summary": "The paper presents a new neural network architecture called Feedback Alignment (FFA) that demonstrates the effectiveness of co-optimizing classification and reconstruction tasks on MNIST and CIFAR10 datasets. FFA uses a goal-driven feedback pathway to alleviate weight transport problems encountered in traditional backpropagation (BP) methods, enhancing the bio-plausibility of the learning algorithm. The alignment mechanism in FFA endows feedback connections with emergent visual inference functions, including denoising, resolving occlusions, hallucination, and imagination. The study presents FFA as a promising proof-of-concept for the mechanisms underlying how feedback connections in the visual cortex support flexible visual functions and contributes to the broader field of visual inference underlying perceptual phenomena. ", "questions": "See the weaknesses stated above and also the following questions.\nQ1. Please explain why FFA is bio-plausible more clearly.\nQ2. Please broaden the datasets to validate the superior performance of FFA.\nQ3. Some details should be added, such as how the noises are added, why choose discrimination and reconstruction as the training tasks.\n", "limitations": "While the idea is interesting, more explanations of the choices and experimental details should be added.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. This paper proposes a novel strategy to train neural networks to co-optimize for two objective functions based on the role of feedback connections in the brain, and leverages the credit assignment computational graph as feedback connections during learning and inference\nS2. FFA for discrimination and reconstruction respectively induces noise robustness, and can flexibly support an array of versatile visual inferences such as resolving occlusion\nS3. This paper is well written overall.\n", "weaknesses": "W1. Although the paper claims that FFA is bio-plausible, the discrimination and reconstruction in biological system are not processed through feedforward and feedback pathways.\nW2. The datasets for test are way too simple and small.\nW3. They reason for choosing discrimination and reconstruction as the tasks is not clear, as the visual system has multiple abilities.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> \"Strengths:\nS1. This paper proposes a novel strategy to train neural networks to co-optimize for two objective functions based on the role of feedback connections in the brain, and leverages the credit assignment computational graph as feedback connections during learning and inference S2. FFA for discrimination and reconstruction respectively induces noise robustness, and can flexibly support an array of versatile visual inferences such as resolving occlusion S3. This paper is well written overall.”\n\n\nWe thank the reviewer for their evaluation of our work and their helpful comments to improve the manuscript.\n\n\n> \"Weaknesses:\nW1. Although the paper claims that FFA is bio-plausible, the discrimination and reconstruction in biological system are not processed through feedforward and feedback pathways.”\n\nThough interesting to consider, the above point is not readily clear to us. For clarification, is the reviewer’s point regarding the insignificance of contribution of anatomical feedforward or feedback to discrimination and reconstruction in biology (what other pathways remain? Perhaps lateral recurrence?) or is the point  about a specific lack of evidence for discrimination or reconstruction optimization in the brain? \nWhile we acknowledge more neural experiments are needed to shed light on the exact objectives of neural circuits and the significance of the hierarchical connections between layers to compute such objectives, we believe that modeling efforts such as the one presented here could help produce testable hypotheses for future neural experiments.\n \n\n> \"W2. The datasets for test are way too simple and small.”\n \nWe agree that MNIST and CIFAR10 are small datasets when compared to other AI research but in the field of models for the brain, especially under the constraints of no use of backpropagation of error, these datasets are typical for creating proof-of-concept NeuroAI research to make the foundation for future large-scale models (Lillicrap et al., 2016; Meulemans et al., 2020; Nøkland, 2016). Please see the limitation section in the paper and the scalability section in the general response.\n\n> \"W3. They reason for choosing discrimination and reconstruction as the tasks is not clear, as the visual system has multiple abilities.”\n\nWe agree that this is a valid question in general for studies modeling vision in the brain as they tend to mostly focus on discrimination and sometimes reconstruction (or prediction of the next stimulus). The brain exhibits many functions. Discovering these functions is an important area of research, but object discrimination is at least one of these functions and there is broad consensus that this is performed in the ventral visual stream from neural recordings (DiCarlo et al., 2012) . So we think solving a discrimination task is a good first test of any alternative to backprop. Combined with a dual task of reconstruction, this led our network to produce many other behaviors such as stability with iteration, de-noising, and imagery.\n\n> \"Questions:\nSee the weaknesses stated above and also the following questions. Q1. Please explain why FFA is bio-plausible more clearly. “\n\n\nWe apologize for the lack of clarity on the bio-plausibility of FFA. We now added the pseudocode for FFA to make the distinction to BP. One of the key concerns for a bio-plausible implementation is that backpropagation necessitates the transmission of error signals from downstream neurons back to upstream neurons through an identical and symmetrical replica of the synaptic weight matrix that exists in the downstream path (Figure 1, BP: $W$). To elaborate further, in the process of backpropagation, the error signals (denoted as $e_f$) are multiplied by the transpose of the forward synaptic connections weight matrix ($W^T$), whereas in FFA the error signals are transported via learned existing connections $W_b$, so no other weights need to be invoked for learning besides those used in inference\n\n> \"Q2. Please broaden the datasets to validate the superior performance of FFA. “\n\nThis is certainly our plan to scale up to larger datasets; however, as discussed in the general response (scalability section), there are challenges shared between our algorithm and many of the bio-plausible implementations of learning that we hope will ultimately be solved with the advancement of our theoretical understanding of learning algorithms and with new architectures that could overcome the scalability issue.\n\n> \"Q3. Some details should be added, such as how the noises are added, why choose discrimination and reconstruction as the training tasks.”\n\nThere was no noise added during the training as indicated in the first sentence of section 3.2, and noise was used only for evaluation of noise robustness. e added the following to the caption of Figure 2:\nC) Robustness to input Gaussian noise ($\\mu =0$ and varying $\\sigma^2$ between 0 and 1) as measured by test accuracy on the noisy input.\n\n\n \n> \"Limitations:\nWhile the idea is interesting, more explanations of the choices and experimental details should be added.”\n\n\n\nWe thank the reviewer how found the idea interesting, we now added more explanations in the form of pseudo code on the algorithms (section 3) and on the experimental details (caption of Figure 3 and section 8.4)\n\n"}, {"review_id": "7sPeU2rWIW", "reviewer": "Reviewer_3rRy", "summary": "This article proposes and exploits a bio-plausible mechanism inspired from the Feedback Alignement : the Feedfback-Feedforward Alignement (FFA). Through a series of experiments (classification, robustness, denoising, generation…) the authors demonstrate that the FFA could be used for various applications. ", "questions": "1) Could you detail a bit more the implementation of the FA and FFA algorithms. You are showing the losses in Figure 1, but it does not tell us how the FFA algorithm is trained. It would suggest to strongly clarify the paper to include a ‘pseudo code’ to explain the training procedure ? \n2) The legend of the Figure 4 is not clear enough ! What are the rows 3 and 4 ? Is that the type of noise you apply ? The reconstruction ? \n3) In Figure 5 : What is the difference between the low noise and the high noise setting ? Perceptually speaking, the noise seems to be so high in both cases that it is very hard to make the difference between both. \n4) Have you controlled for the number of parameters of all compared networks ? \n\nMinor Typo : \nL 53 : critiqued —> criticized ", "limitations": "The authors have correctly discussed the limitation part. I considered that small black and white dataset is a limitation ONLY if you compare yourself with machine learning models (which is what the authors are doing). On my opinion, this is not a limitation anymore if you leverage those ’simple’ datasets to model and probe the human cognition.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "I have found this paper original, as it proposes a solution that solves the weight transport problem in biology without imposing additional constraints such as weight symmetry. I have also found interesting the versatility of the FFA as well as recursion-like inference.", "weaknesses": "Despite the very interesting topic and idea, I was not convinced by the experimental parts (see more details below). The main point of this article is the  bio-plausibility of the method: this is a good motivation, but not a result in itself. I would have expected more results in line with biology/psychophysics (see Major Concern 2 ), to show that the proposed algorithms is a good model of the human cognition. Overall the article presents a series of experiments that lacks comparison with 1) similar model in machine learning (see Major Concern 1) and 2) biological data.\n\nMajor Concerns:\n1) The literature review is weak, especially in the field of machine learning. Plenty of algorithms have already been proposed to model human cognition using feedforward and feedback connections. Some of these works should have served as a benchmark in the proposed experiments. Here are more details:\n[1] proposed a deep-learning friendly implementation of the predictive coding algorithms (using backdrop). They are demonstrating that the prediction accuracy gets refined through time. It seems that the model you proposed has lowered accuracy through time (see Fig 3 A : the red curve tends to decrease whatever the noise). How do you explain that ? what’s the point of degrading performance through time ?\n[2] proposes a similar network to [1] but more in line with bio-plausible training. It shows refined denoising abilities thought time. How do you compare the FFA with those models ?\n[3], [4] have developed VAE-models based on predictive coding. Those are showing nice ‘hallucination’ and denoisiing abilities as well as good classification accuracy in challenging environment (noisy image…). Such models might serve as a good baseline to compare with.\n[5], [6] have published numerous articles to show that backprogation could be approximated (in a biology-friendly way) using the predictive coding model. The authors don’t even mention those works !\n[7] : have proposed a predictive coding model doing discrimination and classification in the same time. Such a network would have served as a good comparison for the FFA\n\n2) The main interest of the FFA seems to be its bio-plausibility, but the authors keep evaluating it as a machine learning model rather than a biological model. Most of the experiments the authors are showcasing actually show the weaknesses of the model rather than its advantage. Here are some exemples : \n* Fig 1 shows that FFA is not as good as a classical Autoencoder to reconstruct, and not as good as a BP feedforward model to classify… The only argument in favor of a FFA is that it could do both in the same time… but with lower performance… this is not very appealing (even more when we look at the CIFAR10 curve).\n* Fig 1c shows that FFA is not as good as FA for classification of noisy MNIST and not as good as the BP model for classifying noisy images..\n* Fig 3 shows that the more you let the iterative process going, the more the performance degrade… Then what’s the point oh having an iterative inference process ?\n\n\nI would suggest the authors to modify the way they are evaluating the model. For example why not testing if the FFA is sensitive to perceptual illusion (as in [8]), are why not comparing the number of iterative inference steps with the human reaction time ? Those comparison would tend to show that the FFA is a good model of human cognition.\n\n\n[1] : Wen, Haiguang, et al. \"Deep predictive coding network for object recognition.\" International conference on machine learning. PMLR, 2018.\\\n[2] : Choksi, Bhavin, et al. \"Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics.\" Advances in Neural Information Processing Systems 34 (2021): 14069-14083.\\\n[3] Marino, Joseph. \"Predictive coding, variational autoencoders, and biological connections.\" Neural Computation 34.1 (2022): 1-44.\\\n[4] Boutin, Victor, et al. \"Iterative VAE as a predictive brain model for out-of-distribution generalization.\" arXiv preprint arXiv:2012.00557 (2020).\\\n[5] Whittington, James CR, and Rafal Bogacz. \"An approximation of the error backpropagation algorithm in a predictive coding network with local hebbian synaptic plasticity.\" Neural computation 29.5 (2017): 1229-1262.\\\n[6] Millidge, Beren, Alexander Tschantz, and Christopher L. Buckley. \"Predictive coding approximates backprop along arbitrary computation graphs.\" Neural Computation 34.6 (2022): 1329-1368.\\\n[7] Tschantz, Alexander, et al. \"Hybrid predictive coding: Inferring, fast and slow.\" arXiv preprint arXiv:2204.02169 (2022).\\\n[8] : Pang, Zhaoyang, et al. \"Predictive coding feedback results in perceived illusory contours in a recurrent neural network.\" Neural Networks 144 (2021): 164-175.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their constructive comments and suggestions to improve the manuscript. \n\n> \"Weaknesses ...”\n\nWe acknowledge that a model of the brain ideally should encompass the structural constraints of the neural circuits (bio-plausible) and exhibit hallmarks of human cognition (brain-like behavior) as addressed in general response. \n\n> \"Major ..1.”\n\nWe thank the reviewer for pointing out the important literature, as we also acknowledged the importance of this class of models in the Introduction. However, since (Wen et al., 2018) used back-propagation through time (BPTT) to update the predictive coding network (PCN) parameters and any BPTT implementation suffers from weight transport as well as bio-implausibility specific to temporal credit assignment (Lillicrap & Santoro, 2019), we refrained from including PCN + BPTT as a benchmark. \nRegarding the decrease in accuracy over iterations as reported in Fig. 3A, we didn’t train FFA for more iterations (as opposed to PCN + BPTT). Here, we only wanted to show that even though FFA doesn’t utilize credit assignment through time, the accuracy holds up over iterations, as opposed to FA and BP which degrade performance. \n\nPredify mentioned by the reviewer also used BPTT to train feedback connections. Hence the learning algorithm severely suffers from bio-implausibility.\n\nThe predictive coding formulated in (Whittington & Bogacz, 2017) and (Millidge et al., 2020) required the strict symmetry of feedforward and feedback connections at least at the initialization stage (so weight transport is still a problem in this work, see (Song et al., 2020) and the reviews). At its core, (Tscshantz et al., 2023) also uses the (Whittington & Bogacz, 2017) formulation to implement PC using Hebbian plasticity (page #8), which requires the symmetry of FF and FB connections at least at the initialization to ensure the convergence, limiting the bio-plausibility of the algorithm (In pybrid git repo, self.weight in \"layers\" is used in both forward and backward). \nWe included these discussions to introduction thanks to the reviewer’s recommendation. \n\n\n> \"2... “\n\n\nWe tested the effectiveness of training for both cost functions as it is standard practice in all learning algorithm studies including the PCN studies referenced above. In sec. 4, we show the effectiveness of the model in exhibiting brain-like inferential abilities such as imagery.\n\n> \"Most ...”\n\n\nWe invite the reviewer to consider the heavy constraint asymmetrical weights put on FFA. FFA can optimize two cost functions AND avoid the weight transport problem. The biological brain might favor producing such a diversity of functions over raw performance on any one task. The experiments we included show the superiority of FFA over FA in performance and over BP in robustness.\n\n> \"Fig 3 ...”\n\n\nWe apologize for the lack of clarity. Motivated by the fact that training algorithms over time (e.g. BPTT) are complex and not easily implementable by bio-plausible circuitry, we think that the demonstration of stable recurrence without BPTT is an advance. We wanted to test to what extent each of these algorithms can maintain the visual information over iteration as a test for evaluating the learned synergy between feedforward and feedback connections in FFA.\n\n> \"I would ...”\n\nWe acknowledge that sensitivity to perceptual illusions and comparing to human reaction times would have been a great addition to the battery of inference tests in the paper. However, there is a tension between bio-plausibility of the learning algorithm and scalability to natural image sets. For instance,  (Pang et al., 2021) used BPTT and a pre-trained FF on ImageNet to test the ability of recurrence in producing illusory contours. Scalability is also relevant when comparison to human reaction time such as in (Tang et al., 2018). The fact that their model’s base was AlexNet (trained with BP) gave the model the ability to process natural images which were shown to human subjects when collecting reaction times. Overall, the tension between scalability and bio-plausibility has hindered filling the gap between modeling high-level cognitive abilities while being faithful to the biological constraints on neural circuitry. In our study we attempted to take the challenge of filling the gap, at least partially, by constraining the learning algorithm to biological constraints and evaluating the model in exhibiting high-level cognitive abilities such as imagery, hallucinations, and de-occlusion which has been previously studied almost exclusively in BP-trained networks (Kadkhodaie & Simoncelli, 2021; Tang et al., 2018). If anything, the dual challenge faced by the field of bio-plausibility and scalability highlights the need for further entertaining novel algorithms.\n\n\n> \"​​Questions\"\n\nWe apologize for the oversight on clarifying the algorithm. We now included a pseudo-code in general response. Feedback alignment (FA) is an alternative learning algorithm to the backpropagation that utilizes fixed random feedback weights for error propagation removing the requirement for symmetric weight updates. FFA in essence uses FA to train both feedback and feedforward weights.\n\n> \"The legend ...”\n\nWe apologize for the lack of clarity of the legend of Fig.4. Please see the pdf in general response. As we stated in the first sentence of section 3.2, there is no noise used during the training. \n> \"In Fig...”\n\nThe noise in the legend refers to the noise in the inference algorithm that controls the convergence (Fig.1 & sec. 8.4). This experiment shows, qualitatively, that the higher noise prevents premature convergence.\n> \"Have you ..”\n\nThe number of trainable parameters in the FFA matches exactly to that of control BP and FA autoencoders. The feedforward control BP and FA algorithms have exactly half the number of trainable parameters as FFA by definition, since their feedback connections are not trainable. \n\n\n> \"Typo..”\n\nThank you for pointing this out.\n"}, {"review_id": "7f5RwVcCOz", "reviewer": "Reviewer_WNmV", "summary": "The authors present a novel artificial neural network architecture which they relate to visual processing in the brain. In particular, the authors avoid the weight transport problem by proposing that feedback weights can be learned by optimising them to reconstruct the original input (autoencoder). The authors test the model on image datasets and show that the network can both discriminate and reconstruct the original image, as well as replicating biological visual phenomena with respect to resolving occlusions, hallucinations and imagination. ", "questions": "- are there no other computational models of top-down architectures to compare to? The authors cite works in experimental neuroscience which support top-down feedback during visual processing, do other explicit implementations not exist? e.g. are bidirectional variational autoencoders not related? (e.g. Kingma et al. Neurips 2016)\n- in the fig 1 inference box, should it not be delta_{t-1} >= delta*? Same for Algorithm 1. I'm not sure the figure caption text '* shows the desired value' is the right language, isn't delta* rather an imposed upper bound which we want to fall below?\n- Fig 1 caption last sentence: add full stop\n- line 99: \"Prior work on RNNs..was not geared to improve our understanding of how feedback or recurrence supports inference of visual details\": Could the authors clarify what is meant by 'visual details'?\n- For discrimination performance figures percentage signs should be added\n- Fig 2B: 'customised' is unclear. Are these fully connected feedforward nets?\n- line 132: 'Unlike the original TP, we do not constrain the intermediate stages'. What constraints are the authors referring to?\n- line 137: FA is used before it has been defined as feedback alignment\n- line 156: 'on MNIST, discrimination performance is exceedingly high'. What is meant here?\n- line 162: 'In FFA, like FA, the feedforward and feedback weights aligned over training'. Could the authors show this?\n- I advise the authors to use consistent references to supplementary figures (cf. line 162 vs 178). Moreover, remove the space between the figure number and subpanel (e.g. Figure 2C in line 177)\n- line 182: z has not been defined. Is this the random sample as in algorithm 1?\n- Figure 4: the meaning of the left-to-right in this selection of images is unclear. Is each x position a different iteration? which iteration? the difference between high/low noise also appears very small. The authors argue that low noise \"took fewer iterations to converge\" but I do not find it very clear from the figure (except the second shown iteration for the 5 example). Similarly, in Figure 6 the authors state that \"Figure 6 shows that as noise in the inference goes higher, so does the quality of the imagined digits\": this appears true for the 3 digit but not 5. Is there a way to quantify the quality over all digits?\n- In section 8.1 the model architecturers are not very readable. A schematic on even textual description would be easier to intepret. \n- Equation in section 8.3: is it not supposed to be epsilon in the equation, not sigma\n", "limitations": "The authors do address some limitations, though as I say above I think it should also be highlighted in the paper the lack of theoretical study/support", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- the paper is generally well written\n- The proposed model architecture/learning rules are interesting and creative\n", "weaknesses": "- there is a distinct lack of specificity in the formal model description. Its illustration and algorithm is helpful, but I think the clarity of its exact computations would significantly benefit from equations\n- The authors do not address analytical aspects of their model. What types of solution should one expect in weight space compared to discrimination/auto-encoder only models? What is the extent of alignment compared to fixed random weights (feedback alignment). The authors write that \"co-optimizing discrimination and reconstruction objectives can lead to a mutualistic symbiotic interaction\", would it be possible to quantify this? Relately, are there any necessary conditions for this architecture to work, or where it'll break? for example, feedback alignment struggles with harder tasks, should one expect this with FFA?\n- The functional benefit of FFA is not very clear to me. For example, backprop still outperforms models even with noise on harder CIFAR task (fig 2), and why are several iterations necessary for discrimination if performance gets worse anyway (fig 3)? Is it not simpler just to train with backprop using two loss functions (discrimination and reconstruction), or is the idea that that's less bio-plausible?\n- I'm a little confused about the bio-plausibility of FFA: If I understand correctly, the forward weights W_f affect the activity in the post-synaptic neuron and this activity propagates forward, but the same activity doesn't propogate it the feedback pathway? How is this possible biologically?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciated the detailed evaluation of the reviewer and their comments to improve the manuscript.\n\n> \"Weaknesses:”\n\nWe now added a pseudocode to the paper (see general response). \n\n> \"The authors ... “\n\nIn the general response, we outline the theoretical motivation behind the work and the limitations in theoretical understanding regarding the learning algorithms.\n\n> \"What is ...”\n\nWe included the alignment analysis to the general response pdf.  Quantification of the mutualistic alignment between forward and backward means they can work with each other’s generation. In Fig.3, using iterative inference, we showed that FFA Feedforward and feedback together induce noise robustness, although neither of the loss functions for these two pathways were trained to be robust to noise.\n\n\n> \" Relately, ...”\n\n\nAs mentioned in the limitation section of the work, FFA and in essence FA, both have problems with more tapered architectures, that is the architectures where the dimension of representation reduces drastically over depth. In the original FA paper (Lillicrap et al., 2016), they mentioned FA specifically had trouble training an autoencoder which might be related to reconstruction loss and bottleneck nature of the architecture, leading to less degrees of freedom for alignment in the solution space.\n\n> \"The functional ...”\n\nWe apologize for the lack of clarity regarding the functional benefit of FFA. BP is indeed superior in performance, however, as we mentioned in the Introduction, the implementation of BP is not bio-plausible because it needs symmetric weights in feedforward and feedback pass. In the experiments regarding performance of FFA in discrimination and reconstruction, we included BP as a control to gauge the performance of FFA in comparison with the most performant but bio-implausible algorithm. \n\n> \"I'm ...”\n\nWe have included the pseudocode for FFA to clarify. When updating the feedback pass weights, the activity that propagated through the forward weights all the way to the top, propagates through the feedback weights all the way down to generate activity the same size of the image (for reconstruction). Thus, forward and backward pathways are distinct, just as there is evidence from feedforward and feedback streams in different lamina in the visual cortex  (Markov et al., 2013; Rockland, 2022).\n\n> \"Ques...”\n\nIn the literature for bio-plausible learning algorithms (Goudarzi, 2017/2017; Liao et al., 2016; Lillicrap et al., 2016; Meulemans et al., 2020; Moskovitz et al., 2018; Nøkland, 2016), it’s common practice to compare learning algorithms on the same feedforward architecture. There are certainly computational models of top-down feedback as we mentioned in the introduction (and now we added more), however, virtually all of them train the top-down connections using classical error backpropagation. Moreover, to compare the effectiveness of learning, the identical architecture should be used when comparing various training algorithms. Rather, other top-down models such as bidirectional variational autoencoders, have both architectural (additional encoder) and loss function (probabilistic component) complexities on top of a bio-implausible learning algorithm (BP). We would like to clarify that we didn’t aim to model the visual inference properties of feedback connections, rather, these properties fell out of a bio-plausible implementation of our learning algorithm (FFA) that does not rely on a weight symmetry constraint.  \n\n\n> \"in fig 1...”\n\nThe reviewer is correct in all those instances, we fixed the figure and the caption. Thank you.\n\n> \" line 99: ...”\n\nWe meant the visual ambiguities and occlusions or lack of resolution but we agree that was not evident from what we wrote. We now changed it to “ ..was not geared to improve our understanding of how feedback or recurrence supports visual inference outside of output discrimination performance.”\n\n> \"For discri...”\n\nFixed now, thank you.\n\n> \"Fig 2B ...”\n\nYes, by customized architecture we meant fully-conn. as opposed to the resnet-like conv. architecture as we explained in supplementary 8.1. We now modified the caption of Fig. 2 for clarity.\n\n> \"l. 132...?”\n\nHere, we referred to the layer-wise nature of target propagation in contrast to the end-to-end approach in FFA. But we acknowledge that this was not readily clear from what we wrote. We now modified it to: Unlike the layerwise implementation of TP, our training approach is end-to-end.  \n\n> \"l. 137...”\n\nFixed it, thank you. \n\n> \"l. 156: ...”\n\nThere was a typo and we meant reconstruction performance was much higher. \n\n> \"l. 162...\"\n\nWe now added the alignment between feedforward and feedback weights (see pdf above).\n\n> \"I advise ...”\n\nThank you for  reminding us to correct these points to improve the readability of the paper. Now fixed. \n\n> \"line 182...”\n\nThanks for catching the mistake, it was left from an old notation. We now change to \\hat{x} as in Fig.1.\n\n> \"Fig 4: ...”\n\nWe apologize for the lack of clarity in the Fig. 4 and the caption. Please see Fig. 2 in pdf above. We now added the number of iterations to the text for further clarity. Regarding the Fig. 6 statement on the quality of the imagined digits, quantification of the imagined digits would be very beneficial. However, because there is no ground truth, the ultimate gold standard would be human judgment and we would need to ask human subjects to rate the quality of the imagined digits. However, similar to many other studies on generative models, we relied on providing the generated samples here for the reader.\n\n> \"In section 8.1 ...”\n\nThe model architectures were added to the suppl. for full transparency. We now added more textual description to sec. 8.1 to make the code easier to interpret.\n\n> \"Equa ...”\n\nThank you for pointing out this typo.\n\n> \"Limitations”\n\nWe added theoretical motivation to the main text (see general response above) and have also added difficulties related to the theory to the Limitation section. \n"}]

Author rebuttal:
We thank the reviewers for their helpful comments to improve the work. Below, we address the general theme of concerns which were shared between reviewers. 

**FFA algorithm in pseudo code**  


Weight updates in a three-layer linear network:
parameters: $W_{f_1}$, $W_{f_2}$, $W_{b_1}$, $W_{b_2}$ \
for $epoch<n_{epoch}$: \
&nbsp;&nbsp;&nbsp;&nbsp;$Y = W_{f_2} \cdot h_f; h_f= W_{f_1} \cdot X$ \
&nbsp;&nbsp;&nbsp;&nbsp;$e_f = T - Y$ \
&nbsp;&nbsp;&nbsp;&nbsp;$Loss_f = \frac{1}{2} e_f^T \cdot e_f$ \
&nbsp;&nbsp;&nbsp;&nbsp;$\Delta W_{f_2} = -e_f^T \cdot h_f^T,$ $\Delta W_{f_1} = -W_{b_2} \cdot e_f \cdot X^T$
    (forward updates) \
&nbsp;&nbsp;&nbsp;&nbsp;$\hat{X} = W_{b_1} \cdot h_b; h_b = W_{b_2} \cdot Y$\
&nbsp;&nbsp;&nbsp;&nbsp;$e_b = X - \hat{X}$ \
&nbsp;&nbsp;&nbsp;&nbsp;$ Loss_b = \frac{1}{2} e_b^T \cdot e_b$ \
&nbsp;&nbsp;&nbsp;&nbsp;$\Delta W_{b_1} = -e_b^T \cdot h_b^T,  \Delta W_{b_2} = -W_{f_1} \cdot e_b \cdot Y^T$
    (backward updates)

**The Theoretical motivation for FFA** 

FFA at its core uses Feedback alignment (FA). As pointed out by Reviewer KkB4, the theory behind why FA works is still under active research (Cheng & Brown, 2023; Jan, n.d.; Refinetti et al., 2021; Robertson & Koyejo, 2023)
As we highlight in our abstract, this work aims to provide a proof-of-concept for how bio-plausible implementation of learning through feedback connections can support versatile visual inference. However, there were theoretical motivations behind the creation of this algorithm.


**Regularized autoencoders arrive at symmetric solutions.** 
*(rationale on the link between the choice of reconstruction loss and bio-plausibility of training)*
As discussed in the Introduction, the weight transport problem of bio-plausibility of backpropagation points to the fact that backpropagation requires symmetric weights in the forward and the backward computational graph ($W_f$ and $W_f^T$ in Figure 1). If there was a way to learn to copy the weights from the feedforward to the feedback connections in between weight updates, then this problem would be alleviated. A theoretical work (Kunin et al., 2019) shows that in autoencoders which were trained under $L_2$ regularization, decoder and encoder are the transposes of each other. 
We reasoned that if instead of the fixed random weights in the backward pass of FA, we let the backward pass to reconstruct what the forward pass generates, then essentially we are encouraging the learned weights to be transposes of each other and thus be a better approximation to backpropagation.
Thus, apart from the unsupervised nature of a reconstruction loss, and the structural similarity of feedback and feedforward, arriving at symmetric solutions to better estimate backpropagation was another motivation for reconstruction loss for the feedback pass.


**Theoretical Analogy of structural regularization to input noise robustness**
*(why FFA training induces noise robustness?)*


In the paper, we showed that although FFA didn’t use any noise during training, it still exhibits robustness to input noise. We later showed that this robustness to noise enables FFA to exhibit properties of a denoiser autoencoder in using the learned implicit prior (Kadkhodaie & Simoncelli, 2021) to support various generative properties. Thus, it is important to understand the link between FFA and robustness to input noise. A classical work by Bishop (Bishop, 1995) showed that training neural networks under Tikhonov regularization is mathematically equivalent to training these networks to be robust to input noise. In essence, (Bishop, 1995) provides the link between a structural regularization and input noise robustness. In FFA, feedforward and feedback connections updates are highly constrained to each other and we think that this could have served as a regularization on the weights. However, proving such a link would be a significant undertaking and we acknowledge that such a theoretical link should be proven in our future work.    




**Scalability.** As addressed in Limitations, FFA has problems with the architectures where the dimension of representation reduces drastically over depth, bearing some resemblances to what was previously reported for FA. Some form of Transformers could probably do better, however, the challenges of implementing the modules with a trainable backward computational graph hinders this line of reach from benefiting from autograd packages. 


**“Bio-plausible” learning algorithms and “Brain-like” visual inferences**
 Bio-plausible is an umbrella term used for structural, algorithmic, or conceptual theories about the brain. In our work, we used bio-plausible to refer to the plausibility of implementation of learning in biological neural networks, i.e. avoiding the requirement to copy the weights of feedforward to feedback after each weight update (weight transport problem).


* no weight transport is needed in our learning algorithm 


* the reciprocal connections between layers of the hierarchy (Markov et al., 2013; Rockland, 2022)


Among the bio-plausible algorithms proposed in past works, only FA has both of these criteria. However, to be able to have a reference to compare to other non-bio-plausible algorithms, we picked BP, the main algorithm for training DNNs. It is worth mentioning that this is a typical choice of control algorithms in the literature for bio-plausible learning algorithms (Lillicrap et al., 2016; Bengio, 2014; Goudarzi, 2017)


By brain-like, we referred to solving inference problems that humans appear to do effortlessly such as accurate perception even in presence of occlusions, and forming mental imagery, hallucination, etc. Predictive coding came up as a bio-plausible algorithm in several reviews. As we wrote in detail in addressing Reviewer 3rRy’s concerns, most existing implementations of the predictive coding framework use BP and BP-through-time to update the weights which violates the bio-plausibility of implementation we aimed for. 


Meta-review: Inspired by the feedback connection in the brain, this paper proposes a biologically plausible learning method that concurrently optimizes feedforward and feedback weights with different but dependent objectives for discrimination and reconstruction tasks to induce robustness to image noise and adversarial attacks. After rebuttal and discussions, most reviewers agreed that the topic of this paper is very important for both AI and neuroscience research and the proposed idea is interesting and novel. Two reviewers giving a final rating of 4 also acknowledged their concerns addressed and improved their original ratings. The AC looked through the paper and all the comments and agreed that the paper should be accepted. The AC suggested the authors fix all the issues mentioned in the discussions, including but not limited to: (1) unclear presentation – there should be enough formulations or pseudo code to clarify the detailed steps of the method, and the figures should be polished to be more informative; (2) complementing the literature review, especially for the ones listed by reviewers; (3) adding more explanations or discussions regarding the questions raised by reviewers.
Decision: Accept (poster)