Circular Learning Provides Biological Plausibility

Authors: mohammadamin  tavakoli
ian  domingo
pierre  baldi
Venue: NeurIPS_cc_2024_Workshop_UniReps
Published: 2024-09-24 13:56:31

Abstract: Training deep neural networks in biological systems is faced with major challenges such as scarce labeled data and obstacles for propagating error signals in the absence of symmetric connections. We introduce Tourbillon, a new architecture that uses circular autoencoders trained with various recirculation algorithms in a self-supervised mode, with an optional top layer for classification or regression. Tourbillon is designed to address biological learning constraints rather than enhance existing engineering applications. Preliminary experiments on small benchmark datasets show that Tourbillon performs comparably to models trained with backpropagation and may outperform other biologically plausible approaches. The code and models are available at https://github.com/IanRDomingo/Circular-Learning.

Main paper: 
 introduction: 
 Decades of machine learning have taught us that gradient descent is the sole effective optimization method in high-dimensional spaces. Other strategies, like random search, are bound to fail. Backpropagation, the algorithm behind gradient computation in artificial neural networks, has been incredibly successful. It powers advancements in Artificial Intelligence, from protein folding (e.g., AlphaFold (1)) to natural language understanding and generation (e.g., GPT-4 (2; 3)). Backpropagation efficiently computes the gradient in a network with W weights using O(W ) operations. Considering that at least O(W ) operations are necessary to adjust W synapses, backpropagation demonstrates optimal efficiency. Consequently, if learning is viewed as an optimization problem in a high-dimensional space of synaptic weights, this suggests that the brain likely employs learning algorithms based on gradient computation, either exact or approximate. Yet there are several well-known reasons in the literature why backpropagation is implausible in biological systems (4; 5; 6; 7; 8) . Thus, in short, we hypothesize that biological systems must strive to approximate gradient descent methods without being able to compute exact gradients by backpropagation. Here, we set out to propose a plausible strategy for achieving this goal.To begin, we outline the key factors that make existing neural architectures biologically implausible: 1) Symmetry of Connections (weight transport): Backpropagation requires precisely symmetric connections between the forward and backward passes. This constraint cannot be satisfied in a biological neural system and might be hard to realize in some physical neural systems. 2)Forward Nonlinearities (F prime): Backpropagation relies on an exact memory in the backward pass of the nonlinearities applied in the forward pass, such as activation functions, to compute weight derivatives. However, there is no evidence supporting the existence of such precise memory in biological or physical neural systems. 3) Locality: In a biological neural system, the learning rule for adjusting synaptic weights must be local, i.e. it must rely solely on variables available locally, both in space (spatial locality) and time (temporal locality), at each synapse. 4) Clocked Computation: In backpropagation, the forward and backward passes are manually clocked to compute activations and update weights. In contrast, in a biological system, neurons communicate stochastically, lacking the precise clocking mechanism observed in backpropagation. 5) Labeling: Training classifiers rely on large amounts of labeled data for supervised learning. However, biological systems do not seem to have access to large amounts of labeled data. 6) Spike: While biological neurons communicate using noisy spikes, artificial neurons typically communicate using deterministic analog values (with known exceptions such as dropout). 7) Distances: The architecture of deep neural networks necessitates propagating signals over considerable neural distances in deep models, which can result in signal dilution and lead to distorted or unstable gradients. 8) Developmental Modularity: Backpropagation in general, requires having a complete architecture in place before training can begin, which may not be realistic for biological systems undergoing development and other changes.Several solutions have been suggested to try to address these problems, in isolation or small combinations, but no approach addresses all of them at once. Here we propose a neural architecture called Tourbillon and its training algorithms to address all the implausibility discussed above by combining different ideas, including stacked autoencoders, recirculation, and asynchronous training. We emphasize that the primary goal here is to address the obstacles listed above for biological (or neuromorphic) neural systems and not to derive a new architecture or algorithm that is practically useful for digital applications of deep learning. 
biological plausibility: 
 Several approaches have been proposed to address the biological implausibilities enumerated above. The most notable ones include Feedback Alignment (FA) (9; 10; 11; 12) , Difference Target Propagation (DTP) (5) , Stacked Autoencoders (13; 14) , and the Forward-Forward (FF) algorithm (15) . However, each of these methods addresses only a limited subset of the biological implausibilities (Section A.1 and Table 1 ). Self-supervised learning, in particular stacked autoencoders, provides one way of addressing the data labeling issue. However, standard autoencoders suffer from several other issues which we now address. Circular Autoencoders. In a standard feed-forward autoencoder (AE), the data itself provides the targets (self-supervised learning). The data and hence the targets are available in the input layer. However, they are not available in the output layer, in the sense that they are not physically local (spatial-locality) to the output layer. This problem is addressed in circular autoencoders (CAE) (16) where the output layer is physically equal (or physically adjacent) to the input layer (Figure 1 ). With the circular layout, targets and errors can be computed at the level of the input/output layer.𝐻 ! " 𝐻 ! # 𝐻 !#$ " 𝐻 !#$ # 𝑊 ! 𝐻 ! % 𝐻 ! & 𝐻 !#$ % 𝐻 !#$ & 𝑊 ! 𝐻 ! " : Hidden Representation at t=0 𝐻 ! # : Hidden Representation at t=T 𝐻 ! $ : Positive Data 𝐻 ! % : Negative Data Δ𝑊 ! ∝ (𝐻 ! " -𝐻 ! # )𝐻 !$% " Δ𝑊 ! ∝ | 𝐻 ! & -𝐻 ! $ | ' 𝐻 !$% & 𝑌: Target % 𝑌 !: Local Target 𝑌: Target 𝐻 ! : Hidden Representations 𝐺 !"$ 𝐺 ! 𝑊 ! 𝐻 !#$ 𝑌 𝐻 ' 𝑊 ' 𝑅 !#$ 𝑅 ! Δ𝑊 ! ∝ ) 𝑌 ! -𝐻 ! 𝐻 !$% Δ𝑊 ! ∝ 𝑅 ( (𝑌 -𝐻 ( )𝐻 !$% & 𝑌 ! 𝐻 ! 𝑌 𝐻 ! 𝐻 !#$ Δ𝑊 ! ∝ (𝐻 ! " -𝐻 ! # )𝐻 !$% # Or ) 𝑌 ! = ) 𝑌 !&% 𝐺 !&%Recirculation Algorithms. Standard backpropagation, or even FA, of these targets, would require a channel (wires) running backward from the output layer to the hidden layer. However, because of the circular layout, it is possible to use the forward connections to propagate target and error information during learning. This is the fundamental idea behind recirculation, a family of algorithms for training CAEs that do not require backward connections (17; 18; 7) .Consider a CAE with layers numbered from 0 to L, where 0 corresponds to the input layer. We use the index t to denote different cyclic passes through the autoencoder, with the first pass indexed by t = 0. After the first pass, one can locally compute the error T -H 0 L , where T is the target located at the input layer. This error could be used to train the top layer of the CAE by gradient descent, and then train the other layers by using a form of random backpropagation where the error signal is obtained by propagating the error T -H 0 L using the forward weights of the CAE. This however requires propagating two different kinds of signals, activities, and errors, through the CAE. Thus rather than recirculating the error, a more uniform approach can be obtained by recirculating activities. If H t i denotes the activation of layer i during the forward pass indexed by t, the main idea behind the recirculation family of algorithms is to use H t i as the target for the output H t ′ i taken at a later time t ′ to produce the post-synaptic term for the weight update. The intuition is that the data may become increasingly corrupted as it is being recycled, thus earlier pass serve as targets for later passes. Different variations can be obtained, by varying, for instance, the post-and pre-synaptic terms. Equation 1 describes the update rule of the weights in a circular autoencoder.EQUATIONThis rule follows a Hebbian-product form, resembling backpropagation but with a postsynaptic recirculation error, denoted as [H 0 i -H 1 i ] post . This error term is both spatially and temporally local, assuming that consecutive passes through the circular autoencoder fall within the proper time window. In the input layer, the vector H 0 0 represents the input data, including the targets for an autoencoder. Consequently, the recirculation learning equation for the top layer of weights is identical to backpropagation. Although in this work we are not using spiking neurons, such learning rules are closely related to the concept of spike time-dependent synaptic plasticity (STDP) (19) . STDP Hebbian or anti-Hebbian learning rules have been proposed using the temporal derivative of the activity of the post-synaptic neuron (20) to encode error derivatives. 
tourbillon: a cae stack: 
 We propose the Tourbillon architecture as a stack of circular autoencoders, capped by a classification or regression layer connecting the hidden representation of the top circular autoencoder and the output layer. Each circular autoencoder has an encoder and decoder components. The hidden layer that is shared by the encoding and decoding components is called the hinge layer. In the stack, the hinge layer of the ith circular autoencoder becomes the input layer of the i+1th circular autoencoder (Figure 2 (b) ). The Tourbillon architecture addresses the issues of target labels and spatial locality. With the recirculation algorithms, it also addresses the issues of weight transport, forward non-linearities, temporal locality, and distances. Using a novel training algorithm, we set out to address issues of clocking and modularity.Asynchronous Training. To fully address modularity and provide non-clocked computations, we consider asynchronous training. In this case, each CAE can be viewed as a "spinning wheel" and these wheels can spin independently of each other. At any random time, a CAE may elect to recirculate whatever happens to be in its input layer and adapt its synapses accordingly. The algorithm for asynchronous training is given in the Appendix. 
experiments and results: 
 We begin by training CAEs and investigate the effects of various parameters, including the number of cycles (t), the CAE size (i.e., the number of hidden layers in the CAE except the input and output 1), on training dynamics. Then, using the best set of these parameters, we develop and train several Tourbillon architectures, both with and without a top classifier layer, using different stack depths and training algorithms. The goal of these experiments is not to outperform existing deep learning models but to show that Tourbillon architectures can learn complex tasks while satisfying the plausibility constraints. Similar to recently proposed plausible architectures (15) , we use relatively small datasets and models, leaving the scaling up to future studies. Details on the hyperparameters, hardware, and CAE implementation are in the appendix and the gitHub repository.We train CAEs using the learning rule in Equation 1. We optimize each architecture using a meansquared reconstruction loss. In all experiments, we use symmetric CAEs, where the number of hidden layers in the encoder and decoder are equal. To satisfy distance plausibility, we use CAEs with a small number of hidden layers (CAE size = one and three). Additionally, to maintain the temporal locality of the variables, we limit the number of cycles (t) to less than four. We train CAEs with fully connected layers for the MNIST and Fashion MNIST datasets. Table 2 displays the reconstruction loss on the test datasets. Notably, using a CAE size of one and one cycle (t = 1) yields the lowest testing loss. This corresponds to the greatest level of spatial and temporal locality.Using the best values for the CAE size and number of cycles, we further show the viability of training CAEs with the learning rule above. We compare the mean-squared loss of the trained CAE with the same autoencoder trained with backpropagation, FA, and DFA. Figure 2 (a) shows the training and test error curves for the MNIST and Fashion MNIST datasets. Our results show that recirculation achieves comparable, and possibly superior, reconstruction errors compared to backpropagation and FA. We construct stacks of two, three, and four compressive CAEs and train them using asynchronous algorithms. Table 3 presents the reconstruction error of the stacks, indicating the depth and training algorithm applied to each CAE.Experimenting with different learning rate schedules for each CAE in the stack reveals that decreasing the learning rates from the bottom to the top layers is crucial. The training algorithm's inherent randomness also leads to a higher test reconstruction error. However, the focus of this study is not on performance but on introducing a biologically plausible training algorithm and demonstrating its feasibility. After training the stacks, we add a top classifier layer. Based on Table 3 , we use three CAEs for MNIST and Fashion MNIST. We conduct classification experiments to compare Tourbillon's performance with neural networks of similar architecture trained using backpropagation, FA, and DFA. Tourbillon outperforms FA and matches backpropagation, particularly in fully connected architectures. A key advantage of Tourbillon is its ability to leverage unlabeled data for unsupervised training of the stack, allowing the top classifier to be trained with less labeled data. This reduces reliance on labeled data, enhancing biological plausibility.  
conclusion: 
 Tourbillon represents a systematic approach toward addressing the major biological implausibilities in both structures and training algorithms of existing neural networks. In essence, it is a stack of circular autoencoders, each trained by recirculation at random times. Hence we chose the name Tourbillon (associated with turbulence in French) the turbulent topology of the architecture. Moreover, in horology, a tourbillon is an addition to the mechanics of a watch escapement to increase its accuracy. While we do not claim to have increased accuracy, we have shown that the Tourbillon approach shows similar performance to existing neural networks, at least on small datasets, while being more biologically plausible. In conclusion, Tourbillon serves as a framework to investigate the biological implausibilities of artificial neural architectures and aims to advance the field of biologically plausible deep learning. By addressing key implausibilities, Tourbillon opens up new possibilities for studying neural networks in accordance with biological principles. 
 
 Appendix: In this appendix, we provide additional details regarding the algorithms and the experiments. All the experiments are conducted using a single NVidia Titan X GPU.We describe the learning equation (weight update) using post-and pre-synaptic terms. For a forward weight W i at layer i, the backpropagation learning equation can be written as:EQUATIONB post i-1 = F ′ i-1 • W T i B post iwhere η denotes the learning rate, B post i denotes the postsynaptic backpropagated error at layer i, and H pre i-1 denotes the pre-synaptic activity. F ′ i-1 • denotes the component wise multiplication by the vector of activation function derivatives in layer i -1. T and H L are the targets and the activation at the top layer L. Note that in order to avoid further cluttering the notation, we omit the transpose sign for all the presynatpic terms throughout this document.Feedback Alignment (FA) or Random Backpropagation (RBP) refers to a family of algorithms (9; 10; 11; 12) that address the weights transport problem by using non-symmetric, and usually random, weights in the backward pass as follows:EQUATIONB post i-1 = F ′ i-1 • R i B post iwhere R i denotes the random fixed matrices (random backward channels) to fix the issue of weights transport.While FA and its variants address the weight transport problem, by themselves they do not address the other problems. Among several flavors of FA (10; 21), Direct Feedback Alignment (DFA) (11) , backpropagates the error signal obtained at the top layer to each of the lower layers independently using direct fixed random matrices. By this means, DFA can address the issues of spatial locality and distance implausibilities. The learning equation of DFA can be written as follows:EQUATIONA version with component-wise multiplication by the derivatives of the corresponding activation functions is also possible. Experiments reported in the literature suggest that FA and its variants do not work well with convolutional layers (22; 23; 24) . A few methods have been proposed to address this apparent weakness of FA algorithms, however, most of them introduce more constraints and dependence on the forward pass which may make them less biologically plausible (23; 25; 26) .Difference Target Propagation (DTP) (5) as another biologically plausible model, trains the weights using a local target at each layer Ŷi that is propagated from the original target Y to each of the lower layers using learnable weights G i .EQUATIONThe G i s are trained in the forward pass to approximate the inverse of the forward operation at each layer. Propagating the target using G i s at the top two layers is dependent on the backpropagation and weight transport (24) . Also, the forward and backward passes through the network are completely clocked to learn G i s. However, since the information flows through layers independently, the variables are local in space, thus, this architecture can address space-locality and distance implausibilities. To evaluate the performance of the Tourbillon architecture when adding the top classification layer, we conducted a comparison with similar architectures trained using backpropagation, FA, and DFA. Figure 3 presents the results of this experiment specifically for the Tourbillons, trained sequentially with a stack of three fully connected CAEs using the Fashion MNIST dataset. 


Decision: In light of the positive reviewers' feedback and relevancy of the submission, we are pleased to accept this paper for presentation at UniReps 2024. We kindly ask the authors to incorporate the reviewers' suggestions and feedback in the final camera-ready version of the manuscript.

Reviews: review: title: title: The paper should provide more detailed explanations of Tourbillon's approach to biological implausibilities, a comparison with a standard baseline model, and correcting a missing figure number in a specific section.
main_review: main_review: Improvements: 
1. Please provide more detailed explanations of how Tourbillon addresses each biological implausibility.
2. The paper does not directly compare the Tourbillon with a standard baseline model. Please include the baseline model. 
3. In Section "3 Tourbillon: A CAE Stack", the figure number is missing in line 101. Please attach the correct figure number.
title: title: The authors introduce Tourbillon, a biologically plausible architecture using circular autoencoders trained asynchronously with different recirculation algorithms in a self-supervised manner. They provide preliminary results on the MNIST and Fashion MNIST datasets.
main_review: main_review: The authors introduce Tourbillon, a biologically plausible architecture using circular autoencoders trained asynchronously with different recirculation algorithms in a self-supervised manner. They provide preliminary results on the MNIST and Fashion MNIST datasets.
 
Strengths:
- The paper is well-written, with clear motivation and is easy to follow.
- Tourbillon is more biologically plausible than other bio-compatible methods.

Weaknesses:
- Several inconsistencies and errors are present, such as:
   1) The caption of Table 2 mentions CIFAR10 results, which are not included in the table.
   2) Line 101 references a figure, but it’s unclear which one.

- The experiments use small networks on simple datasets. While the authors acknowledge that scaling up is not their current goal, it would still be interesting to explore how this approach could perform in larger architectures. However, I don't consider this a major drawback.
title: title: Review of Circular Learning Provides Biological Plausibility
main_review: main_review: - The first paragraph seems unnecessarily argumentative. For instance, "Decades of machine learning have taught us that gradient descent is the **sole** effective optimization method in high-dimensional spaces" (emphasis mine). I agree that gradient descent is highly effective, but maybe it isn't the only optimization method that is so effective. Maybe we use it because it works and thus we haven't needed to find a better alternative. I don't think anything would be lost by instead saying "Decades of machine learning have taught us that gradient descent is an extremely effective optimization method in high-dimensional spaces"
- nit: Why are there no citations for " Feedback Alignment (FA), Difference Target Propagation (DTP), Stacked Autoencoders, and the Forward-Forward (FF) algorithm."
- nit: Fix the incorrect figure reference on Line 101 "(Figure ??)."
- Overall, I think this is an interesting idea. Its possible advantages aren't well demonstrated empirically, but I think it's worth discussing at the workshop.