Biologically Plausible Neural Networks via Evolutionary Dynamics and Dopaminergic Plasticity

1 INTRODUCTION . In his Turing award lecture , neural networks pioneer Geoff Hinton opined that “ evolution can ’ t get gradients because a lot of what determines the relationship between the genotype and the phenotype is outside your control ” ( Hinton , 2019 ) . We beg to differ . Evolution does have what amounts to an effective oracle access to the ( indeed , complex and intractable ) mapping from genotype to phenotype . The well-established equations of population genetics governing evolution under recombination ( Bürger , 2000 ; Chastain et al. , 2014 ) describe the way whereby the distribution of genotypes in the population is updated from one generation to the next , informed by the empirical fitness of the phenotypes during lifetime ; and these equations do bear a similarity to gradient descent and , even closer , to no-regret learning ( Chastain et al. , 2014 ) . In this paper , we show that , in fact , quite effective training of neural nets can be carried out , without backpropagation , in evolutionary time . The towering empirical success of ANNs has brought into focus their profound incongruity with what we know about the brain : backpropagation requires that and synaptic weights and plasticity be informed by downstream events . Clever versions of ANNs have been proposed recently that avoid this criticism : ANNs whose backward weights are random and fixed ( Lillicrap et al. , 2016 ) and a variant that also uses random feedback weights but with zero initial conditions ( Nø kland , 2016 ) , a backpropagation interpretation of STDP ( a widely accepted theory of plasticity ) ( Bengio et al. , 2015 ) , unsupervised learning using STDP ( Diehl & Cook , 2015 ) , ANNs driven by neural competition ( Krotov & Hopfield , 2019 ) , or ANNs with target value propagation at each layer rather than the loss gradient ( Lee et al. , 2015 ) . Here we take a very different approach . We believe that , while forward neural computation is coterminous with life , backpropagation ( i.e. , feedback to individual neurons and synapses about their contribution to the overall performance of the circuit ) can be effectively carried out over evolutionary time . Suppose that the brain circuitry for a particular classification task , such as “ food/not food ” , is encoded in the animal ’ s genes , assuming each gene to have two alleles 0 and 1 . A ( haploid ) genotype is a bit string . Crucially , we assume that the weight of each link of the neural network is a fixed sparse linear function of the genes . Evolution proceeds in generations . At each generation , a gene is an independent binary variable with fixed probability of 1 . A population is sampled from this distribution of genotypes , and it experiences a sequence of inputs to the brain circuit . Fitness of each genotype depends , to some small degree , on the animal ’ s success over its lifetime in the specific classification task . In the next generation , the allele frequencies will change slightly , depending on how each allele of each gene fared cumulatively ( over both all inputs and all genotypes containing it ) in the classification task . These changes follow the standard population genetics equations for the weak selection regime , see Bürger ( 2000 ) ; Chastain et al . ( 2014 ) ; weak selection means that the classification task is only one of the many biological functions ( digestion , locomotion , other brain circuits , etc . ) that affect the animal ’ s fitness . The question is , can competent brain circuits evolve this way ? We offer theoretical evidence that this is indeed the case1 . In Section 2 we prove that , if the classifier to be learned is linear , then evolution does indeed succeed in creating a neural network that classifies well , almost certainly and within polynomial ( in the dimension of the problem ) number of generations , individuals per generation , and neurons per individual . We also validate our theorem through experiments on the MNIST data set . Our experiments are not meant to compete with what is happening in the ANN world . We want to make the point that competent learning can happen in life : NNE with a single hidden layer already gives surprisingly good accuracy rates ( more than 90 % accuracy for classifying the MNIST digits 0 to 4 ) . There is a different way of looking at , and motivating , our results , namely from the point of view of the study of the brain in connection to evolution . “ Nothing in biology makes sense except in the light of evolution , ” Theodosius Dobzhansky famously proclaimed . Neuroscientists have espoused this point of view , and evolutionary arguments come up often in the study of the brain , see for example Bosman & Aboitiz ( 2015 ) . However , we are not aware of a technical discussion in the literature of the obvious existential question : Is the architecture of the brain susceptible to evolution through natural selection ? Can brain circuits evolve ? Our mathematical and empirical results in this paper on NNE strongly suggest that , indeed , effective brain circuits specializing in classification tasks could have evolved . We also propose a second biologically plausible ANN-like mechanism , based on dopaminergic plasticity . It was recently established experimentally ( Yagishita et al. , 2014 ) that weights in certain synapses ( in this case from the cortex to the striatum in the mouse brain , but not only ) are increased if dopamine was released within 0.5-2 seconds after the synapse ’ s firing . Intuitively , this is a reinforcement plasticity mechanism that “ rewards ” synapses whose firing led to a favorable outcome . Inspired by this experiment , we define dopaminergic neural nets ( DNN ) , in which the weight of a link that fired ( that is , both nodes fired during the current training example ) is modified by a multiple of ( 14 − err 2 ) , where err is the error of the current training example . That is , links that fired are rewarded if the result was good , and actually also punished if it was not . Our experiments show that such DNNs can also learn to classify quite well , comparable to SGD . Our Contributions . In Section 2 , we give a rigorous proof that NNE with a single hidden layer succeeds in learning arbitrary linear target functions . In Section 3 , we discuss experiments with NNE and DNN on MNIST . 2 ANALYSIS OF NNE . A genotype can be viewed as a vector x ∈ { 0 , 1 } n. A probability distribution over the genotypes is given by a vector p ∈ [ 0 , 1 ] n ; a genotype x is sampled by setting x ( i ) = 1 with probability p ( i ) , independently for each i . The neural network corresponding to a genotype x is a feed-forward neural network ( FFNN ) whose weights are computed as follows . For a prediction network having m links , the weights of the links are given by Wx , where W is an m× n sparse weight generation 1We note incidentally that NNE is of course very much distinct from neuroevolution ( see the recent survey Stanley et al . ( 2019 ) ) , which optimizes ANN architecture and hyperparameters through genetic algorithms . matrix . We choose the entries of W to be random and i.i.d . : with probability β , W ( i , j ) is chosen uniformly at random from [ −1 , 1 ] , and is 0 with probability 1− β . The input to the network is a vector y drawn from a distribution D and has a label ( possibly realvalued ) ` ( y ) . The output of the network on an input y is NNEx ( y ) . In the simplest linear case ( Section 2.1 ) , y ∈ Rm and NNEx ( y ) = xTWT y . In our experiments ( Section 3.1 ) , we study the case whenD is the uniform distribution over MNIST , and NNEx ( · ) is a 1-layer neural network with a ReLU output gate ( see Section 3.1 for formal definition ) . For each genotype x , we measure its performance by computing the loss L ( NNEx ( y ) , l ( y ) ) ( this could be squared loss , cross-entropy loss , etc. ) . For a probability distribution over genotypes p , we define the loss as L ( p ) : = Ex∼pEy∼DL ( NNEx ( y ) , ` ( y ) ) . We calculate the rewards f t ( i ) and f̄ t ( i ) as the expected negative loss whenever the allele is present and absent respectively . f t ( i ) = Ex∼pt [ Ey∼D [ −L ( NNEx ( y ) , ` ( y ) ) ] |x ( i ) = 1 ] . ( 1 ) and f̄ t ( i ) = Ex∼pt [ Ey∼D [ −L ( NNEx ( y ) , ` ( y ) ) ] |x ( i ) = 0 ] . ( 2 ) For the next generation we calculate , p = pt ( i ) ( 1 + f t ( i ) ) and q = ( 1− pt ( i ) ) ( 1 + f̄ t ( i ) ) We normalize p and q to make it a probability distribution . Thus the allele probabilities for the next generation will be , pt+1 ( i ) = p p+ q = pt ( i ) ( 1 + f t ( i ) ) 1 + f̄ t ( i ) + pt ( i ) ( f t ( i ) − f̄ t ( i ) ) . ( 3 ) This is the standard update rule in population genetics under the weak selection assumption . The multiplier captures the small degree to which the performance of this task by the animal confers an evolutionary advantage leading to larger progeny . Our first observation is that perfomance per allele is in fact a function of the gradient of the loss function . Lemma 1 L ( pt ) = −f̄ t ( i ) − pt ( i ) ( f t ( i ) − f̄ t ( i ) ) and ∂ ∂pt ( i ) ( L ( pt ) ) = − ( f t ( i ) − f̄ t ( i ) ) . Proof . L ( pt ) = Ex∼pEy∼D [ L ( NNEx ( y ) , ` ( y ) ) ] = pt ( i ) Ex∼pt [ Ey∼D [ L ( NNEx ( y ) , ` ( y ) ) ] |x ( i ) = 1 ] + ( 1− pt ( i ) ) Ex∼pt [ Ey∼D [ L ( NNEx ( y ) , ` ( y ) ) ] |x ( i ) = 0 ] = pt ( i ) ( −f t ( i ) ) + ( 1− pt ( i ) ) ( −f̄ t ( i ) ) = −f̄ t ( i ) − pt ( i ) ( f t ( i ) − f̄ t ( i ) ) . Here , the last line follows from equations 1 and 2 . Now , taking the derivative w.r.t . pt ( i ) we get ∂ ∂pt ( i ) ( L ( pt ) ) = − ( f t ( i ) − f̄ t ( i ) ) . We use this to prove the following theorem . Theorem 1 Fix δ > 0 . Suppose ∇2L ( z ) H · I ∀z ∈ [ 0 , 1 ] n. Let U : = supp∈ [ 0,1 ] n L ( p ) and St : = { i ∈ [ n ] |δ ≤ pt ( i ) ≤ 1− δ } . For ≤ min { 1/ ( max { 2U , 1 } ) , 2/H , 1 } , there is an η > 0 s.t . E ( L ( pt+1 ) ) ≤ L ( pt ) − η ∑ i∈St ( ∇iL ( pt ) ) 2 . Proof . Using Equation 3 and Lemma 1 we get pt+1 ( i ) − pt ( i ) = · p t ( i ) ( 1− pt ( i ) ) ( f t ( i ) − f̄ t ( i ) ) 1 + f̄ t ( i ) + pt ( i ) ( f t ( i ) − f̄ t ( i ) ) = − · p t ( i ) ( 1− pt ( i ) ) 1− L ( pt ) ∇iL ( pt ) = −γi · ∇iL ( pt ) where γ is as defined above . For our choice of , we have 1− L ( pt ) ≥ 1− U ≥ 1 2 and · pt ( i ) ( 1− pt ( i ) ) ≤ 4 ≤ 1 2H . Therefore , γi ≤ 1/H . Using Taylor ’ s theorem , there exists a z ∈ [ pt , pt+1 ] s.t . L ( pt+1 ) = L ( pt ) − ( pt+1 − pt ) T∇L ( pt ) + 1 2 ( pt+1 − pt ) T∇2L ( z ) ( pt+1 − pt ) ≤ L ( pt ) − ∑ i γi ( ∇iL ( pt ) ) 2 + H 2 ∑ i γ2i ( ∇iL ( pt ) ) 2 ( Using∇2L ( z ) HI ) = L ( pt ) − ∑ i ( γi − H 2 γ2i ) ( ∇iL ( pt ) ) 2 . Since , γi ≤ 1/H , we have γi − H2 γ 2 i ≥ γi/2 . Therefore , L ( pt+1 ) ≤ L ( pt ) − ∑ i γi 2 ( ∇iL ( pt ) ) 2 = L ( pt ) − 2 ( 1− L ( pt ) ) ∑ i pt ( i ) ( 1− pt ( i ) ) ( ∇iL ( pt ) ) 2 ≤ L ( pt ) − δ ( 1− δ ) 2 ( 1− B ) ∑ i∈St ( ∇iL ( pt ) ) 2 , where B : = infp∈ [ 0,1 ] n L ( p ) . Therefore , the conclusion of the theorem holds for η = δ ( 1 − δ ) / ( 2 ( 1− B ) ) .

Summary: In this paper the authors propose a method for training neural networks using evolutionary methods. The aim of developing this method is to provide a biological alternative to back-propagation. The authors prove that their method converges and with high probability succeeds in learning linear classification problems. Another method is also proposed which is linked to dopaminergic neurons.