Standard reinforcement learning (RL) algorithms train agents to maximize given reward functions . However, many real-world applications of RL require agents to also satisfy certain constraints which may, for example, be motivated by safety concerns. Constrained RL algorithms approach this problem by training agents to maximize given reward functions while respectingexplicitly defined constraints. However, in many cases, manually designing accurate constraints is a challenging task . In this work, given a reward function and a set of demonstrations from an expert that maximizes this reward function while respectingunknown constraints , we propose a framework to learn the most likely constraints that the expert respects. We then train agents to maximize the given reward function subject to the learned constraints  . Previous works in this regard have either mainly been restricted to tabular settings or specific types of constraints or assume knowledge of transition dynamics of the environment . In contrast, we empirically show that our framework is able to learn arbitrary \textit{Markovian} constraints in high-dimensions in a  model-free setting. 
In real world settings, numerous constraints are present which are hard to specify mathematically . However, for the real world deployment of reinforcement learning (RL), it is critical that RL agents are aware of these constraints, so that they can act safely . In this work, we consider the problem of learning constraints from demonstrations of a constraint-abiding agent's behavior. We experimentally validate our approach and show that our framework can successfully learn the most likely constraints that the agent respects. We further show that these learned constraints aretransferable to new agents that may have different morphologies and/or reward functions . Previous works in this regard have either mainly been restricted to tabular (discrete) settings, specific types of constraints or assume the environment's transition dynamics . In contrast, our framework is able to learn arbitrary \textit{Markovian} constraints in high-dimensions in a completely model-free setting. The code can be found it: URL
R
Standard reinforcement learning (RL) algorithms train agents to maximize given reward functions
In real world settings, numerous constraints are present which are hard to specify mathematically
D
many real-world applications of RL require agents to also satisfy certain constraints which may, for example, be motivated by safety concerns. Constrained RL algorithms approach this problem by training agents to maximize given reward functions while respecting
D
explicitly
R
defined constraints. However, in many cases, manually designing accurate constraints is a challenging task
for the real world deployment of reinforcement learning (RL), it is critical that RL agents are aware of these constraints, so that they can act safely
D
given a reward function and a set of demonstrations from an expert that maximizes this reward function while respecting
D
unknown
R
constraints , we propose a framework to
we consider the problem of learning constraints from demonstrations of a constraint-abiding agent's behavior. We experimentally validate our approach and show that our framework can successfully
R
expert
agent
R
then train agents to maximize the given reward function subject to the learned constraints
further show that these learned constraints are
A
transferable
A
to new agents that may have different morphologies and/or reward functions
R
settings or
(discrete) settings,
R
knowledge of transition dynamics of the environment
the environment's transition dynamics
D
we empirically show that
A
completely
A
The code can be found it: URL