BioLCNet: Reward-modulated Locally Connected Spiking Neural Networks

Recent studies have shown that convolutional neural networks ( CNNs ) are not the only feasible solution for image classification . Furthermore , weight sharing and backpropagation used in CNNs do not correspond to the mechanisms present in the biological visual system . To propose a more biologically plausible solution , we designed a locally connected spiking neural network ( SNN ) trained using spike-timing-dependent plasticity ( STDP ) and its reward-modulated variant ( R-STDP ) learning rules . The use of spiking neurons and local connections along with reinforcement learning ( RL ) led us to the nomenclature BioLCNet for our proposed architecture . Our network consists of a rate-coded input layer followed by a locally connected hidden layer and a decoding output layer . A spike population-based voting scheme is adopted for decoding in the output layer . We used the MNIST dataset to obtain image classification accuracy and to assess the robustness of our rewarding system to varying target responses . 1 INTRODUCTION . For many years , deep convolutional neural network ( DCNN ) has dominated the field of computer vision and object recognition Goodfellow et al . ( 2016 ) ; LeCun et al . ( 2015 ) . Although novel methods , such as visual transformers Carion et al . ( 2020 ) and very recent MLP-based models Tatsunami & Taki ( 2021 ) are threatening its reign , CNN is still the most popular architecture employed for solving visual tasks . However , CNNs lack biological plausibility . First of all , neuron activations in an artificial neural network ( ANN ) are static real-numbered values , that are modeled by differentiable , non-linear activation functions . This is in contrast to biological neurons that use discrete , and mostly sparse spike trains to transmit information between each other , and in addition to the rate of spikes ( spatial encoding ) , they also use spike timing to encode information temporally Tavanaei et al . ( 2019 ) . Therefore , a spiking neural network ( SNN ) is more akin to the neural networks in the brain . Spiking neural networks also require fewer labeled data and operations , which makes them compatible with energy-efficient neuromorphic hardware . Secondly , the brain is incapable of error backpropagation , as done in traditional ANNs . One issue with error backpropagation in ANNs is the weight transport problem , i.e. , the fact that weight connectivity in feedforward and feedback directions is symmetric Liao et al . ( 2016 ) ; Bartunov et al . ( 2018 ) . Additionally , error feedback propagation that does not affect neural activity is not compliant with the feedback mechanisms that biological neurons use for communication Lillicrap et al . ( 2020 ) . Furthermore , although convolutional neural networks has shown great potential in solving any translation-invariant task , its use of weight sharing is biologically problematic . There is no empirical support for explicit weight sharing in the brain Pogodin et al . ( 2021 ) . However , local connections between neurons is biologically plausible , since neurons in the biological visual system exploit them to have local visual receptive fields Gregor & LeCun ( 2010 ) . To be compatible with this fact , we also used a locally-connected scheme without explicit weight sharing to design our network . Despite the biological nature of local connections , they mostly underperform convolution-based methods with weight sharing in the visual domain , especially on large-scale datasets Bartunov et al . ( 2018 ) . This weaker performance may be mainly attributed to the smaller number of parameters and better generalization in CNNs . Fewer parameters in CNNs would also require less memory and computational cost , and would lead to faster training Poggio et al . ( 2017 ) . Studies are being done to bridge the performance gap between convolutional and locally-connected networks Lillicrap et al . ( 2020 ) ; Bartunov et al . ( 2018 ) . Noting the above considerations , in this paper , we are proposing BioLCNet , a reward-modulated locally-connected spiking neural network . Our network is trained using the unsupervised spiketiming-dependent plasticity and its semi-supervised variant reward-modulated STDP . The input images are encoded proportional to the pixels intensity using Poisson rate-coding that converts intensity to average neuron firing rate in Hertz . In the output layer , there are neuronal groups for each class label , and decision making is based on aggregated number of spikes during the decision period . Our novel dynamic reward prediction error ( RPE ) mechanism exploits strongly supported empirical findings to improve classification performance . We test the classification capabilities of our network with different sets of hyperparameters on the MNIST dataset LeCun et al . ( 1999 ) . We also conduct a classical conditioning experiment to prove the effectiveness of our decoding scheme and rewarding mechanisms . 2 RELATED WORK . Neuroscientists and deep learning researchers have long been searching for more biologically plausible deep learning approaches in terms of neuronal characteristics , learning rules , and connection types . Regarding neuronal characteristics , researchers have turned to biological neuronal models and spiking neural networks . The vanishing performance gap between deep neural netwroks ( DNNs ) and SNNs , and the compatibility of SNNs with neuromorphic hardware and online on-chip training Schemmel et al . ( 2010 ) has piqued the interest of researchers Mozafari et al . ( 2019 ) . For comprehensive reviews on deep learning in spiking neural networks , see Tavanaei et al . ( 2019 ) ; Pfeiffer & Pfeil ( 2018 ) . Spiking neurons are activated by discrete input spike trains . This differs from artificial neurons used in an ANN that have differentiable activation functions and can easily employ backpropagation and gradient-based optimization . There are works that use gradient-based methods with SNNs Kheradpisheh & Masquelier ( 2020 ) ; Wu et al . ( 2018 ) ; Neftci et al . ( 2019 ) ; Bellec et al . ( 2020 ) and some of them have achieved great performances . On the other hand , many works in this area use derivations of the Hebbian learning rule where changes in connection weights depend on the activities of the pre and post-synaptic neurons Hebb ( 1949 ) . Spike-timing-dependent plasticity ( STDP ) and its variants , apply asymmetric weight updates based on the temporal activities of neurons . Normal STDP requires an external read-out for classification Mozafari et al . ( 2018 ) , and have been applied to image reconstruction and classification tasks by many researchers . Some have employed fully-connected architectures Beyeler et al . ( 2013 ) ; Tavanaei & Maida ( 2015 ) ; Allred & Roy ( 2016 ) , while others used convolutional layers for feature extraction Masquelier & Thorpe ( 2007 ) ; Panda & Roy ( 2016 ) ; Kheradpisheh et al . ( 2016 ; 2018 ) . Reward-modulated STDP ( R-STDP ) uses a reward ( or punishment ) signal to directly modulate the STDP weight change , and can be used to decode the output without an external cue . Izhikevich ( 2007 ) solved the distal reward problem in reinforcement learning by using a version of R-STDP with decaying eligibility traces that gives recent spiking activity more importance . Around the same time , Florian ( 2007 ) showed that R-STDP can be employed to solve a simple XOR task with both rate and temporal encoding of the output . Also , Caporale & Dan ( 2008 ) used R-STDP to generate specific spiking patterns in the output of their spiking network . Historically , R-STDP was first adopted with temporal ( rank-order ) encoding for image classification Mozafari et al . ( 2018 ) . They employed a convolutional architecture based on Masquelier & Thorpe ( 2007 ) and a time-to-first-spike decoding scheme . An extended architecture was later developed which had multiple hidden layers Mozafari et al . ( 2019 ) . The use of R-STDP with Poisson ratecoding has been mostly limited to fully-connected architectures for solving reinforcement learning robot navigation tasks Shim & Li ( 2017 ) ; Bing et al . ( 2019 ) . To our knowledge , image recognition problems have not yet been addressed by combining R-STDP and rate-based encoding . The most prevalent architectures used for image classification in deep learning with both DNNs and SNNs are based on convolutional layers and weight sharing . However , there are arguments against the biological plausibility of these approaches Bartunov et al . ( 2018 ) ; Pogodin et al . ( 2021 ) . Locally connected ( LC ) networks are an alternative to the convolutional ones . Illing et al . ( 2019 ) show that shallow networks with localized connectivity and receptive fields perform much better than fully-connected networks on the MNIST benchmark . However , Bartunov et al . ( 2018 ) showed that the lower generalization of LC networks compared to CNNs results in their underperforming CNNs in most image classification tasks , and prevents their scalability to larger datasets such as ImageNet Deng et al . ( 2009 ) . Very recently , Pogodin et al . ( 2021 ) proposed bio-inspired dynamic weight sharing and adding lateral connections to locally-connected layers to achieve the same regularization goals of weight sharing and normal convolutional filters . The first work to integrate a locally-connected ( LC ) layer into an SNN Saunders et al . ( 2019 ) used a network with no hidden layers where the rate-coded input is passed to the output layer via local connections . They exploited recurrent inhibitory connections similar to the ones employed by Diehl & Cook ( 2015 ) to simulate a winner-take-all ( WTA ) inhibition mechanism in their output . Their learning rule is STDP , and therefore an external readout , in this case n-gram voting , is required for classification . Their network scheme was inspiring in designing our locally connected hidden layer . 3 THEORY . In this section , we will outline the theoretical foundations underlying our proposed method . Specifically , the dynamics of the spiking neuronal model , the learning rules used , and the connection type employed in our network will be described . 3.1 ADAPTIVE LIF NEURON MODEL . The famous leaky and integrate fire neuronal model is governed by the following differential equation Gerstner et al . ( 2014 ) , τm du dt = − [ u ( t ) − urest ] +RI ( t ) , ( 1 ) where u ( t ) denotes the neuron membrane potential and is a function of time , R is the membrane resistance , I ( t ) is any arbitrary input current , and τm is the membrane time constant . Equation ( 1 ) dictates that the neuron potential exponentially decays to a constant value urest over time . When a pre-synaptic neuron fires ( spikes ) , it generates a current that reaches its post-synaptic neurons . In the simple leaky integrate and fire ( LIF ) model , a neuron fires when its potential surpasses a constant threshold uthr . After firing , the neuron ’ s potential resets to a constant ureset and will not be affected by any input current for a period of time known as the refractory period ( ∆tref ) . A variant of the LIF model uses adaptive firing thresholds . In this model , uthr can change over time based on the neuron ’ s rate of activity Diehl & Cook ( 2015 ) . When a neuron fires , its tolerance to the input stimuli and consequently its firing threshold increases by a constant amount , g0 , otherwise the threshold decays exponentially with a time constant τg to the default threshold uthr0 . Equations ( 2 ) to ( 4 ) explain the dynamics of the adaptive LIF model , uthr ( t ) = uthr0 + g ( t ) , ( 2 ) where , τgdg/dt = −g ( t ) , ( 3 ) and spike⇒ g ( t ) = g ( t− 1 ) + g0 , ( 4 )

Summary: This paper presents a spiking neural network (SNN) architecture for image classification, designed to be more biologically plausible than comparable existing architectures. In particular, it eschews convolutional connectivity in favour of local connectivity, and uses STDP (vanilla and reward-modulated) instead of gradient-based learning. The paper describes in detail the proposed architecture (providing introductory material on topics such as spiking artificial neurons and STDP), and then compares its performance on an MNIST benchmark to other SNN architectures. Finally, the paper presents the performance of the architecture on a conditioning task, which is used to highlight the capacity of the network to track a dynamically changing reward.