  This paper studies the problem of developing an approximate dynamic
programming (ADP) framework for learning online the value function of an
infinite-horizon optimal problem while obeying safety constraints expressed as
control barrier functions (CBFs). Our approach is facilitated by the
development of a novel class of CBFs, termed Lyapunov-like CBFs (LCBFs), that
retain the beneficial properties of CBFs for developing minimally-invasive safe
control policies while also possessing desirable Lyapunov-like qualities such
as positive semi-definiteness. We show how these LCBFs can be used to augment a
learning-based control policy so as to guarantee safety and then leverage this
approach to develop a safe exploration framework in a model-based reinforcement
learning setting. We demonstrate that our developed approach can handle more
general safety constraints than state-of-the-art safe ADP methods through a
variety of numerical examples.
