Biologically Plausible Neural Networks via Evolutionary Dynamics and Dopaminergic Plasticity

1 INTRODUCTION . In his Turing award lecture , neural networks pioneer Geoff Hinton opined that “ evolution can ’ t get gradients because a lot of what determines the relationship between the genotype and the phenotype is outside your control ” ( Hinton , 2019 ) . We beg to differ . Evolution does have what amounts to an effective oracle access to the ( indeed , complex and intractable ) mapping from genotype to phenotype . The well-established equations of population genetics governing evolution under recombination ( Bürger , 2000 ; Chastain et al. , 2014 ) describe the way whereby the distribution of genotypes in the population is updated from one generation to the next , informed by the empirical fitness of the phenotypes during lifetime ; and these equations do bear a similarity to gradient descent and , even closer , to no-regret learning ( Chastain et al. , 2014 ) . In this paper , we show that , in fact , quite effective training of neural nets can be carried out , without backpropagation , in evolutionary time . The towering empirical success of ANNs has brought into focus their profound incongruity with what we know about the brain : backpropagation requires that and synaptic weights and plasticity be informed by downstream events . Clever versions of ANNs have been proposed recently that avoid this criticism : ANNs whose backward weights are random and fixed ( Lillicrap et al. , 2016 ) and a variant that also uses random feedback weights but with zero initial conditions ( Nø kland , 2016 ) , a backpropagation interpretation of STDP ( a widely accepted theory of plasticity ) ( Bengio et al. , 2015 ) , unsupervised learning using STDP ( Diehl & Cook , 2015 ) , ANNs driven by neural competition ( Krotov & Hopfield , 2019 ) , or ANNs with target value propagation at each layer rather than the loss gradient ( Lee et al. , 2015 ) . Here we take a very different approach . We believe that , while forward neural computation is coterminous with life , backpropagation ( i.e. , feedback to individual neurons and synapses about their contribution to the overall performance of the circuit ) can be effectively carried out over evolutionary time . Suppose that the brain circuitry for a particular classification task , such as “ food/not food ” , is encoded in the animal ’ s genes , assuming each gene to have two alleles 0 and 1 . A ( haploid ) genotype is a bit string . Crucially , we assume that the weight of each link of the neural network is a fixed sparse linear function of the genes . Evolution proceeds in generations . At each generation , a gene is an independent binary variable with fixed probability of 1 . A population is sampled from this distribution of genotypes , and it experiences a sequence of inputs to the brain circuit . Fitness of each genotype depends , to some small degree , on the animal ’ s success over its lifetime in the specific classification task . In the next generation , the allele frequencies will change slightly , depending on how each allele of each gene fared cumulatively ( over both all inputs and all genotypes containing it ) in the classification task . These changes follow the standard population genetics equations for the weak selection regime , see Bürger ( 2000 ) ; Chastain et al . ( 2014 ) ; weak selection means that the classification task is only one of the many biological functions ( digestion , locomotion , other brain circuits , etc . ) that affect the animal ’ s fitness . The question is , can competent brain circuits evolve this way ? We offer theoretical evidence that this is indeed the case1 . In Section 2 we prove that , if the classifier to be learned is linear , then evolution does indeed succeed in creating a neural network that classifies well , almost certainly and within polynomial ( in the dimension of the problem ) number of generations , individuals per generation , and neurons per individual . We also validate our theorem through experiments on the MNIST data set . Our experiments are not meant to compete with what is happening in the ANN world . We want to make the point that competent learning can happen in life : NNE with a single hidden layer already gives surprisingly good accuracy rates ( more than 90 % accuracy for classifying the MNIST digits 0 to 4 ) . There is a different way of looking at , and motivating , our results , namely from the point of view of the study of the brain in connection to evolution . “ Nothing in biology makes sense except in the light of evolution , ” Theodosius Dobzhansky famously proclaimed . Neuroscientists have espoused this point of view , and evolutionary arguments come up often in the study of the brain , see for example Bosman & Aboitiz ( 2015 ) . However , we are not aware of a technical discussion in the literature of the obvious existential question : Is the architecture of the brain susceptible to evolution through natural selection ? Can brain circuits evolve ? Our mathematical and empirical results in this paper on NNE strongly suggest that , indeed , effective brain circuits specializing in classification tasks could have evolved . We also propose a second biologically plausible ANN-like mechanism , based on dopaminergic plasticity . It was recently established experimentally ( Yagishita et al. , 2014 ) that weights in certain synapses ( in this case from the cortex to the striatum in the mouse brain , but not only ) are increased if dopamine was released within 0.5-2 seconds after the synapse ’ s firing . Intuitively , this is a reinforcement plasticity mechanism that “ rewards ” synapses whose firing led to a favorable outcome . Inspired by this experiment , we define dopaminergic neural nets ( DNN ) , in which the weight of a link that fired ( that is , both nodes fired during the current training example ) is modified by a multiple of ( 14 − err 2 ) , where err is the error of the current training example . That is , links that fired are rewarded if the result was good , and actually also punished if it was not . Our experiments show that such DNNs can also learn to classify quite well , comparable to SGD . Our Contributions . In Section 2 , we give a rigorous proof that NNE with a single hidden layer succeeds in learning arbitrary linear target functions . In Section 3 , we discuss experiments with NNE and DNN on MNIST . 2 ANALYSIS OF NNE . A genotype can be viewed as a vector x ∈ { 0 , 1 } n. A probability distribution over the genotypes is given by a vector p ∈ [ 0 , 1 ] n ; a genotype x is sampled by setting x ( i ) = 1 with probability p ( i ) , independently for each i . The neural network corresponding to a genotype x is a feed-forward neural network ( FFNN ) whose weights are computed as follows . For a prediction network having m links , the weights of the links are given by Wx , where W is an m× n sparse weight generation 1We note incidentally that NNE is of course very much distinct from neuroevolution ( see the recent survey Stanley et al . ( 2019 ) ) , which optimizes ANN architecture and hyperparameters through genetic algorithms . matrix . We choose the entries of W to be random and i.i.d . : with probability β , W ( i , j ) is chosen uniformly at random from [ −1 , 1 ] , and is 0 with probability 1− β . The input to the network is a vector y drawn from a distribution D and has a label ( possibly realvalued ) ` ( y ) . The output of the network on an input y is NNEx ( y ) . In the simplest linear case ( Section 2.1 ) , y ∈ Rm and NNEx ( y ) = xTWT y . In our experiments ( Section 3.1 ) , we study the case whenD is the uniform distribution over MNIST , and NNEx ( · ) is a 1-layer neural network with a ReLU output gate ( see Section 3.1 for formal definition ) . For each genotype x , we measure its performance by computing the loss L ( NNEx ( y ) , l ( y ) ) ( this could be squared loss , cross-entropy loss , etc. ) . For a probability distribution over genotypes p , we define the loss as L ( p ) : = Ex∼pEy∼DL ( NNEx ( y ) , ` ( y ) ) . We calculate the rewards f t ( i ) and f̄ t ( i ) as the expected negative loss whenever the allele is present and absent respectively . f t ( i ) = Ex∼pt [ Ey∼D [ −L ( NNEx ( y ) , ` ( y ) ) ] |x ( i ) = 1 ] . ( 1 ) and f̄ t ( i ) = Ex∼pt [ Ey∼D [ −L ( NNEx ( y ) , ` ( y ) ) ] |x ( i ) = 0 ] . ( 2 ) For the next generation we calculate , p = pt ( i ) ( 1 + f t ( i ) ) and q = ( 1− pt ( i ) ) ( 1 + f̄ t ( i ) ) We normalize p and q to make it a probability distribution . Thus the allele probabilities for the next generation will be , pt+1 ( i ) = p p+ q = pt ( i ) ( 1 + f t ( i ) ) 1 + f̄ t ( i ) + pt ( i ) ( f t ( i ) − f̄ t ( i ) ) . ( 3 ) This is the standard update rule in population genetics under the weak selection assumption . The multiplier captures the small degree to which the performance of this task by the animal confers an evolutionary advantage leading to larger progeny . Our first observation is that perfomance per allele is in fact a function of the gradient of the loss function . Lemma 1 L ( pt ) = −f̄ t ( i ) − pt ( i ) ( f t ( i ) − f̄ t ( i ) ) and ∂ ∂pt ( i ) ( L ( pt ) ) = − ( f t ( i ) − f̄ t ( i ) ) . Proof . L ( pt ) = Ex∼pEy∼D [ L ( NNEx ( y ) , ` ( y ) ) ] = pt ( i ) Ex∼pt [ Ey∼D [ L ( NNEx ( y ) , ` ( y ) ) ] |x ( i ) = 1 ] + ( 1− pt ( i ) ) Ex∼pt [ Ey∼D [ L ( NNEx ( y ) , ` ( y ) ) ] |x ( i ) = 0 ] = pt ( i ) ( −f t ( i ) ) + ( 1− pt ( i ) ) ( −f̄ t ( i ) ) = −f̄ t ( i ) − pt ( i ) ( f t ( i ) − f̄ t ( i ) ) . Here , the last line follows from equations 1 and 2 . Now , taking the derivative w.r.t . pt ( i ) we get ∂ ∂pt ( i ) ( L ( pt ) ) = − ( f t ( i ) − f̄ t ( i ) ) . We use this to prove the following theorem . Theorem 1 Fix δ > 0 . Suppose ∇2L ( z ) H · I ∀z ∈ [ 0 , 1 ] n. Let U : = supp∈ [ 0,1 ] n L ( p ) and St : = { i ∈ [ n ] |δ ≤ pt ( i ) ≤ 1− δ } . For ≤ min { 1/ ( max { 2U , 1 } ) , 2/H , 1 } , there is an η > 0 s.t . E ( L ( pt+1 ) ) ≤ L ( pt ) − η ∑ i∈St ( ∇iL ( pt ) ) 2 . Proof . Using Equation 3 and Lemma 1 we get pt+1 ( i ) − pt ( i ) = · p t ( i ) ( 1− pt ( i ) ) ( f t ( i ) − f̄ t ( i ) ) 1 + f̄ t ( i ) + pt ( i ) ( f t ( i ) − f̄ t ( i ) ) = − · p t ( i ) ( 1− pt ( i ) ) 1− L ( pt ) ∇iL ( pt ) = −γi · ∇iL ( pt ) where γ is as defined above . For our choice of , we have 1− L ( pt ) ≥ 1− U ≥ 1 2 and · pt ( i ) ( 1− pt ( i ) ) ≤ 4 ≤ 1 2H . Therefore , γi ≤ 1/H . Using Taylor ’ s theorem , there exists a z ∈ [ pt , pt+1 ] s.t . L ( pt+1 ) = L ( pt ) − ( pt+1 − pt ) T∇L ( pt ) + 1 2 ( pt+1 − pt ) T∇2L ( z ) ( pt+1 − pt ) ≤ L ( pt ) − ∑ i γi ( ∇iL ( pt ) ) 2 + H 2 ∑ i γ2i ( ∇iL ( pt ) ) 2 ( Using∇2L ( z ) HI ) = L ( pt ) − ∑ i ( γi − H 2 γ2i ) ( ∇iL ( pt ) ) 2 . Since , γi ≤ 1/H , we have γi − H2 γ 2 i ≥ γi/2 . Therefore , L ( pt+1 ) ≤ L ( pt ) − ∑ i γi 2 ( ∇iL ( pt ) ) 2 = L ( pt ) − 2 ( 1− L ( pt ) ) ∑ i pt ( i ) ( 1− pt ( i ) ) ( ∇iL ( pt ) ) 2 ≤ L ( pt ) − δ ( 1− δ ) 2 ( 1− B ) ∑ i∈St ( ∇iL ( pt ) ) 2 , where B : = infp∈ [ 0,1 ] n L ( p ) . Therefore , the conclusion of the theorem holds for η = δ ( 1 − δ ) / ( 2 ( 1− B ) ) .

Summary: This paper argues that Artificial Neural Network (ANN) lack in biological plausibility because of the back-propagation process. Therefore, the authors provide an alternative approach, named neural net evolution (NNE) that follows evolutionary theory. This approach uses a large number of genotypes (in the form of vector with binary logits) that will evolve overtime during training. It does not require to calculate the gradient explicitly. The authors have conducted some experiments on MNIST using ANN with only one hidden layer. The experimental results show that the NNE can learn the classification task reasonably well considering that no explicit back propagation is used.