  In this paper, we propose the Model Reference Adaptive Control &
Reinforcement Learning (MRAC-RL) approach to developing online policies for
systems in which modeling errors occur in real-time. Although reinforcement
learning (RL) algorithms have been successfully used to develop control
policies for dynamical systems, discrepancies between simulated dynamics and
the true target dynamics can cause trained policies to fail to generalize and
adapt appropriately when deployed in the real-world. The MRAC-RL framework
generates online policies by utilizing an inner-loop adaptive controller
together with a simulation-trained outer-loop RL policy. This structure allows
MRAC-RL to adapt and operate effectively in a target environment, even when
parametric uncertainties exists. We propose a set of novel MRAC algorithms,
apply them to a class of nonlinear systems, derive the associated control laws,
provide stability guarantees for the resulting closed-loop system, and show
that the adaptive tracking objective is achieved. Using a simulation study of
an automated quadrotor landing task, we demonstrate that the MRAC-RL approach
improves upon state-of-the-art RL algorithms and techniques through the
generation of online policies.
