Original: Biologically Plausible Neural Networks via Evolutionary Dynamics and Dopaminergic Plasticity
Revised: Biologically Plausible Neural Networks via Evolutionary Dynamics and Dopaminergic Plasticity

Original: Abstract
Revised: Abstract

Original: Artiﬁcial neural networks (ANNs) lack in biological plausibility, chieﬂy because backpropagation requires a variant of plasticity (precise changes of the synaptic weights informed by neural events that occur downstream in the neural circuit) that is profoundly incompatible with the current understanding of the animal brain.
Revised: Artiﬁcial neural networks (ANNs) lack in biological plausibility, chieﬂy because backpropagation requires a variant of plasticity (precise changes of the synaptic weights informed by neural events that occur downstream in the neural circuit) that is profoundly incompatible with the current understanding of the animal brain.

Original: Here we propose that backpropagation can happen in evolutionary time, instead of lifetime, in what we call neural net evolution (NNE).
Revised: Here we propose that backpropagation can happen in evolutionary time, instead of lifetime, in what we call neural net evolution (NNE).

Original: In NNE the weights of the links of the neural net are sparse linear functions of the animal’s genes, where each gene has two alleles, 0 and 1.
Revised: In NNE the weights of the links of the neural net are sparse linear functions of the animal’s genes, where each gene has two alleles, 0 and 1.

Original: In each generation, a population is generated at random based on current allele frequencies, and it is tested in the learning task through minibatches.
Revised: In each generation, a population is generated at random based on current allele frequencies, and it is tested in the learning task through minibatches.

Original: The relative performance of the two alleles of each gene is determined, and the allele frequencies are updated via the standard population genetics equations for the weak selection regime.
Revised: The relative performance of the two alleles of each gene is determined, and the allele frequencies are updated via the standard population genetics equations for the weak selection regime.

Original: We prove that, under assumptions,
Revised: We prove that, under assumptions,

Original: NNE succeeds in learning simple labeling functions with high probability, and with polynomially many generations and individuals per generation.
Revised: NNE succeeds in learning simple labeling functions with high probability, and with polynomially many generations and individuals per generation.

Original: Finally, NNE is also tested on MNIST with encouraging results.
Revised: NNE is also tested on MNIST with encouraging results.

Revised: Finally, we explore a further version of biologically plausible ANNs (replacing backprop) inspired by the recent discovery of dopaminergic plasticity.

Original:  Introduction
Revised:  Introduction

Original: In his Turing award lecture [5] neural networks pioneer Geoff Hinton opined that “evolution can’t get gradients because a lot of what determines the relationship between the genotype and the phenotype is outside your control” .
Revised: In his Turing award lecture [6] neural networks pioneer Geoff Hinton opined that “evolution can’t get gradients because a lot of what determines the relationship between the genotype and the phenotype is outside your control” .

Original: We beg to differ.
Revised: We beg to differ.

Original: The well-established equations of population genetics governing evolution under recombination and weak selection [3, 4] do bear a similarity to gradient descent – and, even more, to no-regret learning [4].
Revised: The well-established equations of population genetics governing evolution under recombination and weak selection [3, 4] do bear a similarity to gradient descent – and, even more, to no-regret learning [4].

Original: In this paper, we show that, in fact, quite effective training of neural nets can be carried out without backpropagation, and in evolutionary time, through a variant of gradient descent.
Revised: In this paper, we show that, in fact, quite effective training of neural nets can be carried out without backpropagation, and in evolutionary time, through a variant of gradient descent.

Original: In other words, we argue both theoretically and through experiments that brain circuits specializing in classiﬁcation tasks could have evolved.
Revised: In other words, we argue both theoretically and through experiments that brain circuits specializing in classiﬁcation tasks could have evolved.

Original: The towering empirical success of ANNs has brought into focus their profound incongruity with what we know about the brain: backpropagation requires that plasticity be informed by downstream events.
Revised: The towering empirical success of ANNs has brought into focus their profound incongruity with what we know about the brain: backpropagation requires that plasticity be informed by downstream events.

Original: Clever versions of ANNs have been proposed recently that avoid this criticism: ANNs whose backward weights are random and ﬁxed [7], a backpropagation interpretation of STDP (a widely accepted theory of plasticity) [1], or ANNs driven by neural competition [6].
Revised: Clever versions of ANNs have been proposed recently that avoid this criticism: ANNs whose backward weights are random and ﬁxed [9], a backpropagation interpretation of STDP (a widely accepted theory of plasticity) [1], or ANNs driven by neural competition [8].

Original: Here we take a very different approach.
Revised: Here we take a very different approach.

Original: We believe that, while forward neural computation is coterminous with life, backpropagation (i.e., local feedback on the performance of the circuit) can be effectively carried out over evolutionary time.
Revised: We believe that, while forward neural computation is coterminous with life, backpropagation (i.e., local feedback on the performance of the circuit) can be effectively carried out over evolutionary time.

Original: Suppose that the brain circuitry for a particular classiﬁcation task, such as “food/not food”, is encoded in the animal’s genes, assuming each gene to have two alleles 0 and 1.
Revised: Suppose that the brain circuitry for a particular classiﬁcation task, such as “food/not food”, is encoded in the animal’s genes, assuming each gene to have two alleles 0 and 1.

Original: A (haploid) genotype is a bit string, and the weight of each link of the neural network (assumed to be ﬁxed for simplicity) is a sparse linear function of the genes.
Revised: A (haploid) genotype is a bit string, and the weight of each link of the neural network (assumed to be ﬁxed for simplicity) is a sparse linear function of the genes.

Original: Evolution proceeds in generations.
Revised: Evolution proceeds in generations.

Original: At each generation, a gene is an independent binary variable with ﬁxed probability of 1.
Revised: At each generation, a gene is an independent binary variable with ﬁxed probability of 1.

Original: A population is sampled from this distribution of genotypes, and it experiences a sequence of inputs to the brain circuit.
Revised: A population is sampled from this distribution of genotypes, and it experiences a sequence of inputs to the brain circuit.

Original: Fitness of each genotype depends, to some small degree, on the animal’s success over its lifetime in the speciﬁc classiﬁcation task.
Revised: Fitness of each genotype depends, to some small degree, on the animal’s success over its lifetime in the speciﬁc classiﬁcation task.

Original: In the next generation, the allele frequencies will change slightly, depending on how each allele of each gene fared cumulatively (over both all inputs and all genotypes containing it) in the classiﬁcation task.
Revised: In the next generation, the allele frequencies will change slightly, depending on how each allele of each gene fared cumulatively (over both all inputs and all genotypes containing it) in the classiﬁcation task.

Original: These changes follow the standard population genetics equations for the weak selection regime, see [3, 4]; weak selection means that the classiﬁcation task is only one of the many biological functions (digestion, locomotion, etc.)
Revised: These changes follow the standard population genetics equations for the weak selection regime, see [3, 4]; weak selection means that the classiﬁcation task is only one of the many biological functions (digestion, locomotion, etc.)

Original: that affect the animal’s ﬁtness.
Revised: that affect the animal’s ﬁtness.

Original: The question is, can competent brain circuits evolve this way?
Revised: The question is, can competent brain circuits evolve this way?

Original: We offer both theoretical and experimental evidence that this is indeed the case.
Revised: We offer both theoretical and experimental evidence that this is indeed the case.

Original: 1 Our experiments are on the MNIST data set, where we apply Neural Network Evolution (NNE) to learn to categorize digits.
Revised: 1 Our experiments are on the MNIST data set, where we apply Neural Network Evolution (NNE) to learn to categorize digits.

Original: In fact, we use the simplest possible version corresponding to a single hidden layer of activations.
Revised: In fact, we use the simplest possible version corresponding to a single hidden layer of activations.

Original: We ﬁnd that this already gives surprisingly good accuracy rates.
Revised: We ﬁnd that this already gives surprisingly good accuracy rates.

Original: We then use a simple model of arbitrary linear target functions and show that NNE will converge to the target.
Revised: We then use a simple model of arbitrary linear target functions and show that NNE will converge to the target.

Original: We show in Section 2 that NNE shadows the gradient of the squared error function over a mini-batch, and converges to a 0 / 1 allele distribution.
Revised: We show in Section 2 that NNE shadows the gradient of the squared error function over a mini-batch, and converges to a 0 / 1 allele distribution.

Original: We also propose a totally different alternative to backprop — biologically plausible ANNs based on dopaminergic plasticity.
Revised: We also propose a totally different alternative to backprop — biologically plausible ANNs based on dopaminergic plasticity.

Original: It was recently established experimentally [9] that weights in certain synapses (from the cortex to the striatum but not only) are increased if dopamine was released within 0.5-2 seconds after the synapse’s ﬁring.
Revised: It was recently established experimentally [14] that weights in certain synapses (from the cortex to the striatum but not only) are increased if dopamine was released within 0.5-2 seconds after the synapse’s ﬁring.

Original: Inspired by this experiment, we deﬁne dopaminergic neural nets (DNN), in which the weight of a link that ﬁred (that is, both nodes ﬁred during the current minibatch) is modiﬁed by a multiple of ( 14 − err 2 ) , where err is the error of the current minibatch.
Revised: Inspired by this experiment, we deﬁne dopaminergic neural nets (DNN), in which the weight of a link that ﬁred (that is, both nodes ﬁred during the current minibatch) is modiﬁed by a multiple of ( 14 − err 2 ) , where err is the error of the current minibatch.

Original: That is, links that ﬁred are rewarded if the result was good, and punished if it was not.
Revised: That is, links that ﬁred are rewarded if the result was good, and punished if it was not.

Original: We show experimentally that such ANNs can also learn to classify quite well.
Revised: We show experimentally that such ANNs can also learn to classify quite well.

Original: Our Contributions.
Revised: Our Contributions.

Original: In Section 2, we study a 1 -layer NNE with squared loss as loss function.
Revised: In Section 2, we study a 1 -layer NNE with squared loss as loss function.

Original: In Section 3, we discuss preliminary experiments showing that NNE performs reasonably well on
Revised: In Section 3, we discuss preliminary experiments showing that NNE performs reasonably well on

Original: MNIST.
Revised: MNIST.

Original:  A rigorous analysis for the case of learning linear functions
Revised:  A rigorous analysis for the case of learning linear functions

Original: A genotype can be viewed as a vector x ∈ { 0 , 1 } n .
Revised: A genotype can be viewed as a vector x ∈ { 0 , 1 } n .

Original: A probability distribution over the genotypes is given by a vector p ∈ [0 , 1] n ; a genotype x is sampled by setting x ( i )
Revised: A probability distribution over the genotypes is given by a vector p ∈ [0 , 1] n ; a genotype x is sampled by setting x ( i )

Original: = 1 with probability p ( i ) , independently for each i .
Revised: = 1 with probability p ( i ) , independently for each i .

Original: The neural network corresponding to a genotype x is a feed-forward neural network (FFNN) whose weights are computed as follows.
Revised: The neural network corresponding to a genotype x is a feed-forward neural network (FFNN) whose weights are computed as follows.

Original: For a prediction network having m links, the weights of the links are given by Wx , where W is an m × n sparse weight generation matrix .
Revised: For a prediction network having m links, the weights of the links are given by Wx , where W is an m × n sparse weight generation matrix .

Original: We choose the entries of W to be random and i.i.d.
Revised: We choose the entries of W to be random and i.i.d.

Original: : with probability β , W ( i, j ) is chosen uniformly at random from [ − 1 , 1] , and is 0 with probability 1 − β .
Revised: : with probability β , W ( i, j ) is chosen uniformly at random from [ − 1 , 1] , and is 0 with probability 1 − β .

Original: The input to the network is a vector y drawn from a distribution D and has a label (possibly realvalued) (cid:96) ( y ) .
Revised: The input to the network is a vector y drawn from a distribution D and has a label (possibly realvalued) (cid:96) ( y ) .

Original: The output of the network on an input y is NNE x ( y ) .
Revised: The output of the network on an input y is NNE x ( y ) .

Original: In the simplest linear case, y ∈ R m and NNE x ( y ) =
Revised: In the simplest linear case, y ∈ R m and NNE x ( y ) =

Original: x T W T y .
Revised: x T W T y .

Original: In our experiments (Section 3), we study the case when D is the uniform distribution over MNIST, and NNE x ( · ) is a 1 -layer neural network with a ReLU output gate (see Section 3 for formal deﬁnition).
Revised: In our experiments (Section 3), we study the case when D is the uniform distribution over MNIST, and NNE x ( · ) is a 1 -layer neural network with a ReLU output gate (see Section 3 for formal deﬁnition).

Original: For each genotype x , we measure its performance by computing the loss L ( NNE x ( y ) , l ( y ))
Revised: For each genotype x , we measure its performance by computing the loss L ( NNE x ( y ) , l ( y ))

Original: (this could be squared loss, cross-entropy loss, etc.).
Revised: (this could be squared loss, cross-entropy loss, etc.).

Original: For a probability distribution over genotypes p , we deﬁne the loss as
Revised: For a probability distribution over genotypes p , we deﬁne the loss as

Original: [Equation]
Revised: [Equation]

Original: We calculate the rewards f t ( i ) and ¯ f t ( i ) as the expected negative loss whenever the allele is present and absent respectively.
Revised: We calculate the rewards f t ( i ) and ¯ f t ( i ) as the expected negative loss whenever the allele is present and absent respectively.

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: For the next generation we calculate,
Revised: For the next generation we calculate,

Original: [Equation]
Revised: [Equation]

Original:  We normalize p and q to make it a probability distribution.
Revised: and [Equation] We normalize p and q to make it a probability distribution.

Original: Thus the allele probabilities for the next generation will be,
Revised: Thus the allele probabilities for the next generation will be,

Original: [Equation]
Revised: [Equation]

Original: This is the standard update rule in population genetics under the weak selection assumption.
Revised: This is the standard update rule in population genetics under the weak selection assumption.

Original: The multiplier (cid:15) captures the small degree to which the performance of this task by the animal confers an evolutionary advantage leading to larger progeny.
Revised: The multiplier (cid:15) captures the small degree to which the performance of this task by the animal confers an evolutionary advantage leading to larger progeny.

Original: Our ﬁrst observation is that perfomance per allele is in fact a function of the gradient of the loss function.
Revised: Our ﬁrst observation is that perfomance per allele is in fact a function of the gradient of the loss function.

Original: Lemma
Revised: Lemma

Original: [Equation]
Revised: [Equation]

Original: We use this to prove the following theorem.
Revised: We use this to prove the following theorem.

Original: Theorem 1 Fix δ > 0 .
Revised: Theorem 1 Fix δ > 0 .

Original: Suppose ∇ 2 L ( z ) (cid:22) H · I ∀ z ∈ [0 , 1] n .
Revised: Suppose ∇ 2 L ( z ) (cid:22) H · I ∀ z ∈ [0 , 1] n .

Original: Let U :=
Revised: Let U :=

Original: sup p ∈ [0 , 1] n L ( p ) and
Revised: sup p ∈ [0 , 1] n L ( p ) and

Original: S t :=
Revised: S t :=

Original: { i ∈ [ n ] | δ ≤ p t ( i ) ≤ 1 − δ } .
Revised: { i ∈ [ n ] | δ ≤ p t ( i ) ≤ 1 − δ } .

Original: For (cid:15) ≤ min { 1 / (max { 2 U, 1 } ) , 2 /H, 1 } , there is an η > 0 s.t.
Revised: For (cid:15) ≤ min { 1 / (max { 2 U, 1 } ) , 2 /H, 1 } , there is an η > 0 s.t.

Original: [Equation]
Revised: E ( L ( p t +1 )) ≤ L ( p t ) − η (cid:88) i ∈

Original: i ∈ S t [Equation] ∇ [Equation]
Revised: S t (cid:0) ∇ i L ( p t ) (cid:1) 2 .

Original: 2.1 Learning linear functions
Revised: 2.1 Learning linear functions

Original: In this section, we show that in the case of a linear target functions, with high probability, NNE converges to an allele distribution p which is arbitrarily close to the correct linear labeling.
Revised: In this section, we show that in the case of a linear target functions, with high probability, NNE converges to an allele distribution p which is arbitrarily close to the correct linear labeling.

Original: Our NNE has m input gates connected to one output gate (i.e., no hidden layers).
Revised: Our NNE has m input gates connected to one output gate (i.e., no hidden layers).

Original: For a genotype x , the weights of the connections are given by Wx .
Revised: For a genotype x , the weights of the connections are given by Wx .

Original: On input y , the NNE outputs x T W T y .
Revised: On input y , the NNE outputs x T W T y .

Original: Theorem 2 Let D be the uniform distribution over vectors in an n -dimensional unit ball.
Revised: Theorem 2 Let D be the uniform distribution over vectors in an n -dimensional unit ball.

Original: Let a be a ﬁxed vector with (cid:107) a (cid:107) ≤ 1 , such that the label of y is (cid:96) ( y ) :
Revised: Let a be a ﬁxed vector with (cid:107) a (cid:107) ≤ 1 , such that the label of y is (cid:96) ( y ) :

Original: = a T y .
Revised: = a T y .

Original: Let W have i.i.d.
Revised: Let W have i.i.d.

Original: entries with
Revised: entries with

Original: W ij = ± (cid:112) m/
Revised: W ij = ± (cid:112) m/

Original: d with probability d/m and 0 with probability 1 − ( d/m ) .
Revised: d with probability d/m and 0 with probability 1 − ( d/m ) .

Original: Then, for any δ ∈ (0 , 1] , with n = O ( m + log(1 /δ )) , with probability at least 1 − δ , there exists an allele distribution p s.t.
Revised: Then, for any δ ∈ (0 , 1] , with n = O ( m + log(1 /δ )) , with probability at least 1 − δ , there exists an allele distribution p s.t.

Original: Wp = a .
Revised: Wp = a .

Original: Moreover, with probability at least 3 / 4 , for any (cid:15) ∈ (0 , 1] , with n = Ω( m (log(1 /(cid:15) ) /(cid:15) 2 ) , there is an x ∈ { 0 , 1 } n s.t.
Revised: Moreover, with probability at least 3 / 4 , for any (cid:15) ∈ (0 , 1] , with n = Ω( m (log(1 /(cid:15) ) /(cid:15) 2 ) , there is an x ∈ { 0 , 1 } n s.t.

Original: ( Wx ) · a (cid:107)
Revised: ( Wx ) · a (cid:107)

Original: Wx (cid:107)(cid:107) a (cid:107) ≥ 1 − (cid:15) .
Revised: Wx (cid:107)(cid:107) a (cid:107) ≥ 1 − (cid:15) .

Original: We remark that the above guarantee works for every linear target function in R m .
Revised: We remark that the above guarantee works for every linear target function in R m .

Original: To learn, with high conﬁdence, the target function from among d unknown (arbitrary) linear functions, m above can be replaced by log d .
Revised: To learn, with high conﬁdence, the target function from among d unknown (arbitrary) linear functions, m above can be replaced by log d .

Original:  Experiments 
Revised:  Experiments 3.1 NNE on MNIST

Original: We study the effectiveness of NNE by evaluating its classiﬁcation performance on the MNIST dataset.
Revised: We study the effectiveness of NNE by evaluating its classiﬁcation performance on the MNIST dataset.

Original: To train an NNE via evolution of T generations of genotypes, we ﬁx a sufﬁciently large population size N .
Revised: To train an NNE via evolution of T generations of genotypes, we ﬁx a sufﬁciently large population size N .

Original: Each generation t ∈ [ T ] consists of a sample of N independently sampled genotypes from the allele distribution p t , we denote this sample by P t .
Revised: Each generation t ∈ [ T ] consists of a sample of N independently sampled genotypes from the allele distribution p t , we denote this sample by P t .

Original: This distribution is updated based on the average performance f t ( i ) and ¯ f t ( i ) of all the genotypes on a task, in our case, MNIST handwirtten digit recognition task.
Revised: This distribution is updated based on the average performance f t ( i ) and ¯ f t ( i ) of all the genotypes on a task, in our case, MNIST handwirtten digit recognition task.

Original: We let the allele distribution p t evolve over T generations in this manner. 
Revised: We let the allele distribution p t evolve over T generations in this manner. [Table]

Original: Experimental setup.
Revised: Experimental setup.

Original: We use 200 training samples for each of the digits, drawn uniformly at random from MNIST; we denote this set of training examples by S .
Revised: We use 200 training samples for each of the digits, drawn uniformly at random from MNIST; we denote this set of training examples by S .

Original: p 1 , the allele distribution for the ﬁrst generation, is sampled uniformly at random from [0 , 1] n .
Revised: p 1 , the allele distribution for the ﬁrst generation, is sampled uniformly at random from [0 , 1] n .

Original: We evaluate the performance of the alleles over N = 1000 genotypes.
Revised: We evaluate the performance of the alleles over N = 1000 genotypes.

Original: Our network has 784 input units, one hidden layer of | h 1 | = 1000 units with ReLU activation and an output layer of 10 units with softmax activation.
Revised: Our network has 784 input units, one hidden layer of | h 1 | = 1000 units with ReLU activation and an output layer of 10 units with softmax activation.

Original: We add a sparse random graph between the input layer and the hidden layer: between a neuron in the input layer and a neuron in the hidden layer, we independently add an edge with probability 0 .
Revised: We add a sparse random graph between the input layer and the hidden layer: between a neuron in the input layer and a neuron in the hidden layer, we independently add an edge with probability 0 .

Original: The hidden layer is fully connected to the output layer.
Revised: The hidden layer is fully connected to the output layer.

Original: We choose β = 0 .
Revised: We choose β = 0 .

Original: 0025 for our experiments, i.e., each edge weight is a sparse random function of only β fraction of the alleles.
Revised: 0025 for our experiments, i.e., each edge weight is a sparse random function of only β fraction of the alleles.

Original: For the input sample y , (cid:96) ( y ) is now a one-hot encoding of the label, and NNE x ( y ) is the soft-max output of the network.
Revised: For the input sample y , (cid:96) ( y ) is now a one-hot encoding of the label, and NNE x ( y ) is the soft-max output of the network.

Original: We use the cross-entropy loss function, L ( NNE x ( y ) , (cid:96) ( y ))
Revised: We use the cross-entropy loss function, L ( NNE x ( y ) , (cid:96) ( y ))

Original: = − (cid:80)
Revised: = − (cid:80)

Original: c
Revised: c

Original: ∈ [ C ] (cid:96) ( y ) c log ( NNE x ( y ) c ) .
Revised: ∈ [ C ] (cid:96) ( y ) c log ( NNE x ( y ) c ) .

Original: If a classiﬁer were to randomly guess the label of an input intance, its loss function value would be α :=
Revised: If a classiﬁer were to randomly guess the label of an input intance, its loss function value would be α :=

Original: − log (1 / 10) .
Revised: − log (1 / 10) .

Original: We use the relative performance of the genotype w.r.t.
Revised: We use the relative performance of the genotype w.r.t.

Original: to a random guess for our updates.
Revised: to a random guess for our updates.

Original: To this end, we deﬁne for a genotype x , δ x := 1 | S | (cid:80) s ∈ [ S ] max { 0 , α 2 −
Revised: To this end, we deﬁne for a genotype x , δ x := 1 | S | (cid:80) s ∈ [ S ] max { 0 , α 2 −

Original: L ( NNE x ( y ) , (cid:96) ( y )) 2 } .
Revised: L ( NNE x ( y ) , (cid:96) ( y )) 2 } .

Original: For each allele, we calculate the rewards f t ( i ) and ¯ f t ( i ) whenever the allele is present and absent respectively.[Equation] and
Revised: For each allele, we calculate the rewards f t ( i ) and ¯ f t ( i ) whenever the allele is present and absent respectively.

Original: [Equation]
Revised: [Equation]

Original: The allele distribution for the next generation is updated using equation 3.
Revised: The allele distribution for the next generation is updated using equation 3.

Original: NNE as described above achieves 78.8% test accuracy on the full MNIST test set.
Revised: NNE as described above achieves 78.8% test accuracy on the full MNIST test set.

Original: While this is somewhat far from the state of art in classiﬁcation of MNIST images, our results demonstrate that very basic NNEs can perform reasonably well in this task.
Revised: While this is somewhat far from the state of art in classiﬁcation of MNIST images, our results demonstrate that very basic NNEs can perform reasonably well in this task.

Revised: See experimental results in Appendix showing the effect of number of genes on performance.

Original: NNE with output layer training.
Revised: with output layer training.

Original: The biological implausibility objection of using stochastic gradient based updates is less acute for the output layer, since in animal brains synaptic changes due to plasticity happen at the post-synaptic neuron, and for the output layer this is the output neuron.
Revised: The biological implausibility objection of using stochastic gradient based updates is less acute for the output layer, since in animal brains synaptic changes due to plasticity happen at the post-synaptic neuron, and for the output layer this is the output neuron.

Original: Even then, computing exact (or approximate) gradients is a nontrivial computational task; instead we consider using just the sign of the gradient for only the output layer as a lifetime training mechanism.
Revised: Even then, computing exact (or approximate) gradients is a nontrivial computational task; instead we consider using just the sign of the gradient for only the output layer as a lifetime training mechanism.

Original: For the same network described as above, we randomly initialize the network weights using allele distribution learned using the NNE.
Revised: For the same network described as above, we randomly initialize the network weights using allele distribution learned using the NNE.

Original: We then calculate the sign of the gradient of the output layer weights and update the weights in the opposite direction ( SignSGD ), using a sufﬁciently small learning rate (cid:15) (cid:48) , similar to stochastic gradient descent.
Revised: We then calculate the sign of the gradient of the output layer weights and update the weights in the opposite direction ( SignSGD ), using a sufﬁciently small learning rate (cid:15) (cid:48) , similar to stochastic gradient descent.

Original: For i in the hidden layer and j in the output layer, the update is
Revised: For i in the hidden layer and j in the output layer, the update is

Original: [Equation]
Revised: [Equation]

Original:  where h i is output of the neuron i , and z j softmax output of neuron j .
Revised: (4) where h i is output of the neuron i , and z j softmax output of neuron j .

Original: SignSGD has been shown to be effective for traning large deep neural networks (for e.g., see [2]).
Revised: SignSGD has been shown to be effective for traning large deep neural networks (for e.g., see [2]).

Original: We perform a few hundred iterations of this training using batch size 50 .
Revised: We perform a few hundred iterations of this training using batch size 50 .

Original: In this experiment (NNE + SignSGD ), we obtain 86.3 % accuracy on full MNIST test set.
Revised: In this experiment (NNE + SignSGD ), we obtain 86.3 % accuracy on full MNIST test set.

Original: This further demonstrates that biologically plausible neural networks can perform reasonably well in this task.
Revised: This further demonstrates that biologically plausible neural networks can perform reasonably well in this task.

Original: Number of genes.
Revised: Table 1 compares the results of all the models along with the baseline, stochastic gradient descent trained on the same subset of MNIST. [Table] 3.2 Dopaminergic Neural Nets (DNNs) DNNs are biologically plausible ANNs based on dopaminergic plasticity.

Revised: They learn by a weak form of immediate reinforcement - “rewarding" synapses whose ﬁring led to a favourable outcome.

Revised: If a connection between two neurons has ﬁred during a training step, then its weight is increased if the square error was low (less than 14 ).

Revised: In this section, we demonstrate that simple DNNs can perform reasonably well for tasks like classifying the images in the MNIST dataset.

Revised: Experimental setup.

Revised: For our experiments we use a network consisting of an input layer, a single hidden layer, and an output layer consisting of 784 , h , and 10 neurons respectively.

Revised: Each neuron in the input layer has a link to each neuron in the hidden layer, and its weight is initialised by the popularly used Kaiming Uniform (more commonly called He initialisation [5]).

Revised: These weights are unchanged through out the learning process.

Revised: Recent theoretical results suggest that a large enough random layer is sufﬁciently rich and efﬁciently trainable [13] (see also [11]).

Revised: Each neuron in the hidden layer has a link to each neuron in the output layer.

Revised: The output layer outputs the softmax score.

Revised: The weights of this layer are learned using plasticity based updates.

Revised: On seeing an input y , the DNN tries to predict the label of y ; let us denote this by DNN W ( y ) .

Revised: If the DNN got the prediction correct, i.e. the loss L ( DNN W ( y ) , (cid:96) ( y )) is at most (cid:15) 0 , then weight w ij get increased by a small amount, provided the output neuron j has low error (i.e. | z j − l ( y ) j | 2 ≤ 1 / 4 ) where z j is the jth coordinate of DNN W ( s ) .

Revised: Formally, the update rule is as follows for i in the hidden layer and j in the output layer.

Revised: [Equation]

Revised: Experimental results.

Revised: To study the effectiveness of our DNN in the 10 -class MNIST digit classiﬁcation, we compare its peformance with some other standard baselines.

Revised: SGD : In this we use the standard stochastic gradient descent (with the Adam optimiser [7]) based updates to train our network.

Revised: SignSGD : As before, we use the sign of the gradient for updates (equation 4).

Revised: Table 2 shows the results for different h values.

Revised: All results are after 500 epochs of training.

Revised: As with NNE we use the cross-entopy loss for all the models.

Original: {+We found that (cid:15) 
+}
Revised: {+We found that (cid:15) 
+}

Revised: = 0 .

Revised: 75 and (cid:15) 1 = 1 for the DNN gives reasonable performance.

Revised: Our DNN gives encouraging results and is comparable to

Revised: SignSGD in performance.

Revised: Acknowledgements

Revised: We would like to acknowledge support for this project.

Revised: CP was supported by NSF grant CCF-1763970.

Revised: SV was supported in part by NSF awards CCF-1563838, CCF-1717349 and CCF-1909756.

Revised: AL was supported in part by SERB Award ECR/2017/003296 and a Pratiksha Trust Young Investigator

Original:  A crucial choice for an NNE is the number of genes.
Revised: Award. Appendix Number of genes. A crucial choice for an NNE is the number of genes.

Original: In our experiments, we use a few thousand genes; this is not unreasonable as it is estimated that about 5 , 000 genes are expressed in the cells of the mammalian brain.
Revised: In our experiments, we use a few thousand genes; this is not unreasonable as it is estimated that about 5 , 000 genes are expressed in the cells of the mammalian brain.

Original: To investigate further, we compare the performance of our algorithm with increasing values of n (the number of genes).
Revised: To investigate further, we compare the performance of our algorithm with increasing values of n (the number of genes).

Original: Figure 1 presents the validation accuracy trends on the same network described above for ﬁve class [0 − 4] classiﬁcation and for full MNIST dataset.
Revised: Figure 1 presents the validation accuracy trends on the same network described above for ﬁve class [0 − 4] classiﬁcation and for full MNIST dataset.

Original: We observe that the accuracy rate of the network improves signiﬁcantly with increase in the number of genes.
Revised: We observe that the accuracy rate of the network improves signiﬁcantly with increase in the number of genes.

Original: However, it requires much longer training time to achieve a desired accuracy rate.
Revised: However, it requires much longer training time to achieve a desired accuracy rate.

Original: Table 1 compares the results of all the models along with the baseline, stochastic gradient descent trained on the same subset of MNIST. 50k 100k 150k 200k 250k 300k acc u r ac y
Revised: [Figure]

Original: r a t e generations5k 10k 15k 20k 25k 30k 35k n =n =n =n =n =n =generations (b) (a)
Revised: n =n =n =n =n =

Original: [Table]
Revised: [Figure]

Original: We would like to acknowledge support for this project.
Revised: Convergence of allele distributions.

Original: CP was supported by NSF grant CCF-1763970.
Revised: We repeat training NNE for many (hundreds of thousands) generations.

Original: SV was supported in part by NSF awards CCF-1563838, CCF-1717349 and CCF-1909756. AL was supported in part by SERB Award ECR/2017/003296 and  a Pratiksha Trust Young Investigator
Revised: As our theoretical results predict (see also [10]), the vast majority of genes have allele probabilities that are very close to 0 and 1 . Figure 2 shows the fraction of allele probabilities that are at a distance [ x, 1 − x ] from 0 or 1, i.e., y is calculated as y = 1 − |{ i :min { p t ( i ) , 1 − p t ( i ) }≤ x }| n

Original: Award.
Revised: [Figure]