Title: A Bio-inspired Gradient-Free Learning Framework for CPG-based Neural Networks in Robot Locomotion
Primary area: reinforcement learning
Keywords: synaptic plasticity, centre pattern generator, parameter optimization, quadrupedal locomotion

Abstract: Central Pattern Generators (CPGs) for animal locomotion have been adopted for robot locomotion control for three decades, but have rarely been used in reinforcement learning. It is partly due to the inherent recursive connections causing gradient vanishment or explosion during optimisation. In this paper, we propose a framework with a bio-inspired learning rule to optimize a recurrent neural network for robot reinforcement learning. The learning rule utilises weight fluctuation for parameter exploration and reward for convergence. With the framework, we trained a minimal network containing CPG and full connection layers for quadrupedal locomotion. Experiments suggest that our framework can train the network, while the Proximal Policy Optimization (PPO) fails. Compared to PPO with a feedforward network, our framework with a CPG is more robust to sensor failure. This work lowers the barrier to exploring the potential advantages of recurrent neural networks in robot reinforcement learning.