This paperformulates an application of reinforcement learning for an evader in a confinement escape problem . An evader's objective is to attempt escaping a confinement region patrolled by multiple defenders, with minimum use of energy . Meanwhile, the defenders aim to reach and capture the evader without any communication between them. The problem formulation uses the actor-critic approach for the defender. In this paper, the novel Scaffolding Reflection in Reinforcement Learning ( SR2L ) framework is proposed, using a potential field method as a scaffold to assist the actor's action-reflection. Through the user's clearly articulated intent, the action-reflection enables the actor to learn by observing the probable actions and their values based on experience. Extensive Monte-Carlo simulations show the performance of a trained SR2L against the baseline approach. The SR2L framework achieves at least one order fewer episodes to learn the policy than the conventional RL framework .
In this paper, a novel Scaffolding Reflection in Reinforcement Learning (SR2L) is proposed for solving the confinement escape problem (CEP). In CEP, an evader's objective is to attempt escaping a confinement region patrolled by multiple pursuers . Meanwhile, the pursuers aim to reach and capture the evader . The inverse solution for pursuers to try and capture has been extensively studied in the literature. However, the problem of evaders escaping from the region is still an open issue. The SR2L employs an actor-critic framework to enable the evader to escape the confinement region. A time-varying state representation and reward function have been developed for proper convergence. The formulation uses the sensor information about the observable environment and prior knowledge of the confinement boundary. The conventional Independent Actor-Critic (IAC) method fails to converge due to sparseness in the reward. The effect becomes evident when operating in such a dynamic environment with a large area. In SR2L, along with the developed reward function, we use the scaffolding reflection method to improve the convergence significantly while increasing its efficiency. In SR2L , a motion planner is used as a scaffold for the actor-critic network to observe, compare and learn the action-reward pair. It enables the evader to achieve the required objective while using lesser resources and time. Convergence studies show that SR2L learns faster and converges to higher rewards as compared to IAC. Extensive Monte-Carlo simulations show that a SR2L consistently outperforms conventional IAC and the motion planner itself as the baselines .
R
This paperformulates an application of reinforcement learning for an evader in a
In this paper, a novel Scaffolding Reflection in Reinforcement Learning (SR2L) is proposed for solving the
R
. An
(CEP). In CEP, an
R
defenders, with minimum use of energy
pursuers
R
defenders
pursuers
R
without any communication between them. The problem formulation uses the
. The inverse solution for pursuers to try and capture has been extensively studied in the literature. However, the problem of evaders escaping from the region is still an open issue. The SR2L employs an
R
approach for the defender. In this paper, the novel Scaffolding Reflection in Reinforcement Learning (
framework to enable the evader to escape the confinement region. A time-varying state representation and reward function have been developed for proper convergence. The formulation uses the sensor information about the observable environment and prior knowledge of the confinement boundary. The conventional Independent Actor-Critic (IAC) method fails to converge due to sparseness in the reward. The effect becomes evident when operating in such a dynamic environment with a large area. In SR2L, along with the developed reward function, we use the scaffolding reflection method to improve the convergence significantly while increasing its efficiency. In
R
) framework is proposed, using a potential field method
, a motion planner is used
R
to assist the actor's action-reflection. Through the user's clearly articulated intent, the action-reflection enables the actor to learn by observing the probable actions and their values based on experience.
for the actor-critic network to observe, compare and learn the action-reward pair. It enables the evader to achieve the required objective while using lesser resources and time. Convergence studies show that SR2L learns faster and converges to higher rewards as compared to IAC.
R
the performance of a trained
that a
R
against the baseline approach. The SR2L framework achieves at least one order fewer episodes to learn the policy than the conventional RL framework
consistently outperforms conventional IAC and the motion planner itself as the baselines