This paper studies the  safe reinforcement learning (RL) problem without assumptions about prior knowledge of the system dynamics and the constraint function . We employ an uncertainty-aware neural network ensemble model to learn the dynamics, and we infer the unknown constraint function through indicator constraint violation signals. We use model predictive control (MPC) as the basic control framework and propose the robust cross-entropy method (RCE) to optimize the control sequence considering the model uncertainty and constraints. We evaluate our methods in the Safety Gym environment. The results show that our approach achieves better constraint satisfaction than baseline safe RL methods while maintaining good task performance . Additionally, we are able to achieve several orders of magnitude better sample efficiency when compared to constrained model-free RL approaches. The code is available at URL
This paper studies the constrained/ safe reinforcement learning (RL) problem with sparse indicator signals for constraint violations. We propose a model-based approach to enable RL agents to effectively explore the environment with unknown system dynamics and environment constraints given a significantly small number of violation budgets . We employ the neural network ensemble model to estimate the prediction uncertainty and use model predictive control as the basic control framework . We propose the robust cross-entropy method to optimize the control sequence considering the model uncertainty and constraints. We evaluate our methods in the Safety Gym environment. The results show that our approach learns to complete the tasks with a much smaller number of constraint violations than state-of-the-art baselines . Additionally, we are able to achieve several orders of magnitude better sample efficiency when compared with constrained model-free RL approaches. The code is available at URL
A
constrained/
R
without assumptions about prior knowledge of the
with sparse indicator signals for constraint violations. We propose a model-based approach to enable RL agents to effectively explore the environment with unknown
R
the constraint function
environment constraints given a significantly small number of violation budgets
R
an uncertainty-aware
the
R
learn the dynamics, and we infer the unknown constraint function through indicator constraint violation signals. We
estimate the prediction uncertainty and
D
(MPC)
R
and
. We
D
(RCE)
R
achieves better constraint satisfaction than baseline safe RL methods while maintaining good task performance
learns to complete the tasks with a much smaller number of constraint violations than state-of-the-art baselines
R
to
with