  This paper proposes a modular framework to generate robust biped locomotion
using a tight coupling between an analytical walking approach and deep
reinforcement learning. This framework is composed of six main modules which
are hierarchically connected to reduce the overall complexity and increase its
flexibility. The core of this framework is a specific dynamics model which
abstracts a humanoid's dynamics model into two masses for modeling upper and
lower body. This dynamics model is used to design an adaptive reference
trajectories planner and an optimal controller which are fully parametric.
Furthermore, a learning framework is developed based on Genetic Algorithm (GA)
and Proximal Policy Optimization (PPO) to find the optimum parameters and to
learn how to improve the stability of the robot by moving the arms and changing
its center of mass (COM) height. A set of simulations are performed to validate
the performance of the framework using the official RoboCup 3D League
simulation environment. The results validate the performance of the framework,
not only in creating a fast and stable gait but also in learning to improve the
upper body efficiency.
