  Plasticity circuits in the brain are known to be influenced by the
distribution of the synaptic weights through the mechanisms of synaptic
integration and local regulation of synaptic strength. However, the complex
interplay of stimulation-dependent plasticity with local learning signals is
disregarded by most of the artificial neural network training algorithms
devised so far. Here, we propose a novel biologically inspired optimizer for
artificial (ANNs) and spiking neural networks (SNNs) that incorporates key
principles of synaptic integration observed in dendrites of cortical neurons:
GRAPES (Group Responsibility for Adjusting the Propagation of Error Signals).
GRAPES implements a weight-distribution dependent modulation of the error
signal at each node of the neural network. We show that this biologically
inspired mechanism leads to a systematic improvement of the convergence rate of
the network, and substantially improves classification accuracy of ANNs and
SNNs with both feedforward and recurrent architectures. Furthermore, we
demonstrate that GRAPES supports performance scalability for models of
increasing complexity and mitigates catastrophic forgetting by enabling
networks to generalize to unseen tasks based on previously acquired knowledge.
The local characteristics of GRAPES minimize the required memory resources,
making it optimally suited for dedicated hardware implementations. Overall, our
work indicates that reconciling neurophysiology insights with machine
intelligence is key to boosting the performance of neural networks.
