Before revision:  . We first constructed a net with the same structure as an actor net in the PPO2 and pre-fit it using the data collected from a robot using conventional model-based controller . Next, the trained weights are transferred into the PPO2 and be optimized further . Moreover, target on the symmetrical and periodic characteristic during bounding, we designed a reward function based on contact points .
After revision:  . We first constructed a net with the same structure as an actor net in the PPO2 and pre-fit it using the data collected from a robot using conventional model-based controllers . Next, the trained weights are transferred into the PPO2 and be optimized further . Moreover, target on the symmetrical and periodic characteristic during bounding, we designed a reward function based on contact points .