Model Learning for TwoWheeled Robot SelfBalance Control Twowheeled robots have many advantages over other mobile robots, but they are difficult to selfbalance compared with other wheeled robots. Reinforcement learning (RL) is a general framework for sequential decisionmaking problems. So far, there are many applications of reinforcement learning to solve robot control problems, but most of them are used in simulators because of the large amount of data required. In addition, due to the reality gap, the policy learned in a simulation environment cannot be transferred directly to a real robot. Real robot data often expensive due to the potential damage to the robot. Modelbased methods require far fewer robot data than modelfree methods, but these methods have the problem of model bias. In this paper, we use a modelbased reinforcement learning method to achieve selfbalance of a twowheeled robot. We present a model learning method that can reduce the problem of model bias. Our method combines the simulator and a few real robot data to learn a probabilistic dynamics model of the robot through an iterative way, which requires no expertise and can learn from scratch. Then the control policy is optimized based on the learned model.

Trajectory Optimization for Unknown Constrained Systems using Reinforcement Learning In this paper, we propose a reinforcement learningbased algorithm for trajectory optimization for constrained dynamical systems. This problem is motivated by the fact that for most robotic systems, the dynamics may not always be known. Generating smooth, dynamically feasible trajectories could be difficult for such systems. Using samplingbased algorithms for motion planning may result in trajectories that are prone to undesirable control jumps. However, they can usually provide a good reference trajectory which a modelfree reinforcement learning algorithm can then exploit by limiting the search domain and quickly finding a dynamically smooth trajectory. We use this idea to train a reinforcement learning agent to learn a dynamically smooth trajectory in a curriculum learning setting. Furthermore, for generalization, we parameterize the policies with goal locations, so that the agent can be trained for multiple goals simultaneously. We show result in both simulated environments as well as real experiments, for a 6DoF manipulator arm operated in positioncontrolled mode to validate the proposed idea. We compare the proposed ideas against a PID controller which is used to track a designed trajectory in configuration space. Our experiments show that our RL agent trained with a reference path outperformed a modelfree PID controller of the type commonly used on many robotic platforms for trajectory tracking.