Before revision:  . We first constructed a net with the same structure as an actor net in the PPO2 and pre-fit it using the data collected from a robot using conventional model-based controller . Next, the trained weights are transferred into the PPO2 and be optimized further . Moreover, target on the symmetrical and periodic characteristic during bounding, we designed a reward function based on contact points . We also used feature engineering to improve the input features of the DRL model and improve performanceon flat ground.
After revision:  . We first constructed a net with the same structure as an actor net in the PPO2 and pre-fit it using the data collected from a robot using conventional model-based controller . Next, the trained weights are transferred into the PPO2 and be optimized further . Moreover, target on the symmetrical and periodic characteristic during bounding, we designed a reward function considering contact points to enforce the gait symmetry and periodicity, and used feature engineering to improve the input features of the DRL model and improve performanceon flat ground.