Before revision: This paper studies the  safe reinforcement learning (RL) problem without assumptions about prior knowledge of the system dynamics and the constraint function .
After revision: This paper studies the safe reinforcement learning (RL) problem with sparse indicator signals for constraint violations. We propose a model-based approach to enable RL agents to effectively explore the environment with unknown system dynamics and the constraint function .