Trajectory Optimization for Unknown Constrained Systems using Reinforcement Learning In this paper, we propose a reinforcement learningbased algorithm for trajectory optimization for constrained dynamical systems. This problem is motivated by the fact that for most robotic systems, the dynamics may not always be known. Generating smooth, dynamically feasible trajectories could be difficult for such systems. Using samplingbased algorithms for motion planning may result in trajectories that are prone to undesirable control jumps. However, they can usually provide a good reference trajectory which a modelfree reinforcement learning algorithm can then exploit by limiting the search domain and quickly finding a dynamically smooth trajectory. We use this idea to train a reinforcement learning agent to learn a dynamically smooth trajectory in a curriculum learning setting. Furthermore, for generalization, we parameterize the policies with goal locations, so that the agent can be trained for multiple goals simultaneously. We show result in both simulated environments as well as real experiments, for a 6DoF manipulator arm operated in positioncontrolled mode to validate the proposed idea. We compare the proposed ideas against a PID controller which is used to track a designed trajectory in configuration space. Our experiments show that our RL agent trained with a reference path outperformed a modelfree PID controller of the type commonly used on many robotic platforms for trajectory tracking.

ChanceConstrained Trajectory Optimization for Nonlinear Systems with Unknown Stochastic Dynamics Iterative trajectory optimization techniques for nonlinear dynamical systems are among the most powerful and sampleefficient methods of modelbased reinforcement learning and approximate optimal control. By leveraging timevariant local linearquadratic approximations of system dynamics and reward, such methods can find both a targetoptimal trajectory and timevariant optimal feedback controllers. However, the local linearquadratic assumptions are a major source of optimization bias that leads to catastrophic greedy updates, raising the issue of proper regularization. Moreover, the approximate modelsxe2x80x99 disregard for any physical stateaction limits of the system causes further aggravation of the problem, as the optimization moves towards unreachable areas of the stateaction space. In this paper, we address the issue of constrained systems in the scenario of onlinefitted stochastic linear dynamics. We propose modeling state and action physical limits as probabilistic chance constraints linear in both state and action and introduce a new trajectory optimization technique that integrates these probabilistic constraints by optimizing a relaxed quadratic program. Our empirical evaluations show a significant improvement in learning robustness, which enables our approach to perform more effective updates and avoid premature convergence observed in stateoftheart algorithms.