BioLCNet: Reward-modulated Locally Connected Spiking Neural Networks

Recent studies have shown that convolutional neural networks ( CNNs ) are not the only feasible solution for image classification . Furthermore , weight sharing and backpropagation used in CNNs do not correspond to the mechanisms present in the biological visual system . To propose a more biologically plausible solution , we designed a locally connected spiking neural network ( SNN ) trained using spike-timing-dependent plasticity ( STDP ) and its reward-modulated variant ( R-STDP ) learning rules . The use of spiking neurons and local connections along with reinforcement learning ( RL ) led us to the nomenclature BioLCNet for our proposed architecture . Our network consists of a rate-coded input layer followed by a locally connected hidden layer and a decoding output layer . A spike population-based voting scheme is adopted for decoding in the output layer . We used the MNIST dataset to obtain image classification accuracy and to assess the robustness of our rewarding system to varying target responses . 1 INTRODUCTION . For many years , deep convolutional neural network ( DCNN ) has dominated the field of computer vision and object recognition Goodfellow et al . ( 2016 ) ; LeCun et al . ( 2015 ) . Although novel methods , such as visual transformers Carion et al . ( 2020 ) and very recent MLP-based models Tatsunami & Taki ( 2021 ) are threatening its reign , CNN is still the most popular architecture employed for solving visual tasks . However , CNNs lack biological plausibility . First of all , neuron activations in an artificial neural network ( ANN ) are static real-numbered values , that are modeled by differentiable , non-linear activation functions . This is in contrast to biological neurons that use discrete , and mostly sparse spike trains to transmit information between each other , and in addition to the rate of spikes ( spatial encoding ) , they also use spike timing to encode information temporally Tavanaei et al . ( 2019 ) . Therefore , a spiking neural network ( SNN ) is more akin to the neural networks in the brain . Spiking neural networks also require fewer labeled data and operations , which makes them compatible with energy-efficient neuromorphic hardware . Secondly , the brain is incapable of error backpropagation , as done in traditional ANNs . One issue with error backpropagation in ANNs is the weight transport problem , i.e. , the fact that weight connectivity in feedforward and feedback directions is symmetric Liao et al . ( 2016 ) ; Bartunov et al . ( 2018 ) . Additionally , error feedback propagation that does not affect neural activity is not compliant with the feedback mechanisms that biological neurons use for communication Lillicrap et al . ( 2020 ) . Furthermore , although convolutional neural networks has shown great potential in solving any translation-invariant task , its use of weight sharing is biologically problematic . There is no empirical support for explicit weight sharing in the brain Pogodin et al . ( 2021 ) . However , local connections between neurons is biologically plausible , since neurons in the biological visual system exploit them to have local visual receptive fields Gregor & LeCun ( 2010 ) . To be compatible with this fact , we also used a locally-connected scheme without explicit weight sharing to design our network . Despite the biological nature of local connections , they mostly underperform convolution-based methods with weight sharing in the visual domain , especially on large-scale datasets Bartunov et al . ( 2018 ) . This weaker performance may be mainly attributed to the smaller number of parameters and better generalization in CNNs . Fewer parameters in CNNs would also require less memory and computational cost , and would lead to faster training Poggio et al . ( 2017 ) . Studies are being done to bridge the performance gap between convolutional and locally-connected networks Lillicrap et al . ( 2020 ) ; Bartunov et al . ( 2018 ) . Noting the above considerations , in this paper , we are proposing BioLCNet , a reward-modulated locally-connected spiking neural network . Our network is trained using the unsupervised spiketiming-dependent plasticity and its semi-supervised variant reward-modulated STDP . The input images are encoded proportional to the pixels intensity using Poisson rate-coding that converts intensity to average neuron firing rate in Hertz . In the output layer , there are neuronal groups for each class label , and decision making is based on aggregated number of spikes during the decision period . Our novel dynamic reward prediction error ( RPE ) mechanism exploits strongly supported empirical findings to improve classification performance . We test the classification capabilities of our network with different sets of hyperparameters on the MNIST dataset LeCun et al . ( 1999 ) . We also conduct a classical conditioning experiment to prove the effectiveness of our decoding scheme and rewarding mechanisms . 2 RELATED WORK . Neuroscientists and deep learning researchers have long been searching for more biologically plausible deep learning approaches in terms of neuronal characteristics , learning rules , and connection types . Regarding neuronal characteristics , researchers have turned to biological neuronal models and spiking neural networks . The vanishing performance gap between deep neural netwroks ( DNNs ) and SNNs , and the compatibility of SNNs with neuromorphic hardware and online on-chip training Schemmel et al . ( 2010 ) has piqued the interest of researchers Mozafari et al . ( 2019 ) . For comprehensive reviews on deep learning in spiking neural networks , see Tavanaei et al . ( 2019 ) ; Pfeiffer & Pfeil ( 2018 ) . Spiking neurons are activated by discrete input spike trains . This differs from artificial neurons used in an ANN that have differentiable activation functions and can easily employ backpropagation and gradient-based optimization . There are works that use gradient-based methods with SNNs Kheradpisheh & Masquelier ( 2020 ) ; Wu et al . ( 2018 ) ; Neftci et al . ( 2019 ) ; Bellec et al . ( 2020 ) and some of them have achieved great performances . On the other hand , many works in this area use derivations of the Hebbian learning rule where changes in connection weights depend on the activities of the pre and post-synaptic neurons Hebb ( 1949 ) . Spike-timing-dependent plasticity ( STDP ) and its variants , apply asymmetric weight updates based on the temporal activities of neurons . Normal STDP requires an external read-out for classification Mozafari et al . ( 2018 ) , and have been applied to image reconstruction and classification tasks by many researchers . Some have employed fully-connected architectures Beyeler et al . ( 2013 ) ; Tavanaei & Maida ( 2015 ) ; Allred & Roy ( 2016 ) , while others used convolutional layers for feature extraction Masquelier & Thorpe ( 2007 ) ; Panda & Roy ( 2016 ) ; Kheradpisheh et al . ( 2016 ; 2018 ) . Reward-modulated STDP ( R-STDP ) uses a reward ( or punishment ) signal to directly modulate the STDP weight change , and can be used to decode the output without an external cue . Izhikevich ( 2007 ) solved the distal reward problem in reinforcement learning by using a version of R-STDP with decaying eligibility traces that gives recent spiking activity more importance . Around the same time , Florian ( 2007 ) showed that R-STDP can be employed to solve a simple XOR task with both rate and temporal encoding of the output . Also , Caporale & Dan ( 2008 ) used R-STDP to generate specific spiking patterns in the output of their spiking network . Historically , R-STDP was first adopted with temporal ( rank-order ) encoding for image classification Mozafari et al . ( 2018 ) . They employed a convolutional architecture based on Masquelier & Thorpe ( 2007 ) and a time-to-first-spike decoding scheme . An extended architecture was later developed which had multiple hidden layers Mozafari et al . ( 2019 ) . The use of R-STDP with Poisson ratecoding has been mostly limited to fully-connected architectures for solving reinforcement learning robot navigation tasks Shim & Li ( 2017 ) ; Bing et al . ( 2019 ) . To our knowledge , image recognition problems have not yet been addressed by combining R-STDP and rate-based encoding . The most prevalent architectures used for image classification in deep learning with both DNNs and SNNs are based on convolutional layers and weight sharing . However , there are arguments against the biological plausibility of these approaches Bartunov et al . ( 2018 ) ; Pogodin et al . ( 2021 ) . Locally connected ( LC ) networks are an alternative to the convolutional ones . Illing et al . ( 2019 ) show that shallow networks with localized connectivity and receptive fields perform much better than fully-connected networks on the MNIST benchmark . However , Bartunov et al . ( 2018 ) showed that the lower generalization of LC networks compared to CNNs results in their underperforming CNNs in most image classification tasks , and prevents their scalability to larger datasets such as ImageNet Deng et al . ( 2009 ) . Very recently , Pogodin et al . ( 2021 ) proposed bio-inspired dynamic weight sharing and adding lateral connections to locally-connected layers to achieve the same regularization goals of weight sharing and normal convolutional filters . The first work to integrate a locally-connected ( LC ) layer into an SNN Saunders et al . ( 2019 ) used a network with no hidden layers where the rate-coded input is passed to the output layer via local connections . They exploited recurrent inhibitory connections similar to the ones employed by Diehl & Cook ( 2015 ) to simulate a winner-take-all ( WTA ) inhibition mechanism in their output . Their learning rule is STDP , and therefore an external readout , in this case n-gram voting , is required for classification . Their network scheme was inspiring in designing our locally connected hidden layer . 3 THEORY . In this section , we will outline the theoretical foundations underlying our proposed method . Specifically , the dynamics of the spiking neuronal model , the learning rules used , and the connection type employed in our network will be described . 3.1 ADAPTIVE LIF NEURON MODEL . The famous leaky and integrate fire neuronal model is governed by the following differential equation Gerstner et al . ( 2014 ) , τm du dt = − [ u ( t ) − urest ] +RI ( t ) , ( 1 ) where u ( t ) denotes the neuron membrane potential and is a function of time , R is the membrane resistance , I ( t ) is any arbitrary input current , and τm is the membrane time constant . Equation ( 1 ) dictates that the neuron potential exponentially decays to a constant value urest over time . When a pre-synaptic neuron fires ( spikes ) , it generates a current that reaches its post-synaptic neurons . In the simple leaky integrate and fire ( LIF ) model , a neuron fires when its potential surpasses a constant threshold uthr . After firing , the neuron ’ s potential resets to a constant ureset and will not be affected by any input current for a period of time known as the refractory period ( ∆tref ) . A variant of the LIF model uses adaptive firing thresholds . In this model , uthr can change over time based on the neuron ’ s rate of activity Diehl & Cook ( 2015 ) . When a neuron fires , its tolerance to the input stimuli and consequently its firing threshold increases by a constant amount , g0 , otherwise the threshold decays exponentially with a time constant τg to the default threshold uthr0 . Equations ( 2 ) to ( 4 ) explain the dynamics of the adaptive LIF model , uthr ( t ) = uthr0 + g ( t ) , ( 2 ) where , τgdg/dt = −g ( t ) , ( 3 ) and spike⇒ g ( t ) = g ( t− 1 ) + g0 , ( 4 )

Summary: The paper discusses learning in biological networks, and proposes a model that combines spiking networks, local connections (convolutions without weight sharing) and reward-modulated spike-timing dependent plasticity. All three components are chosen to be more realistic compared to artificial neural networks. The model is tested on MNIST and on a classical conditioning task.