$Pursuit-Evasion Game of Under-actuated ASVs via Model-based Deep Reinforcement Learning$

Authors: st chao pan
anqing  wang
Venue: IEEE_org_ICIST_2024_Conference
Published: 2024-08-14 11:44:15

Abstract: This paper investigates a pursuit-evasion game scenario involving multiple under-actuated autonomous surface vehicles (ASVs) pursuing a single ASV in a closed environment in the presence of moving and stationary obstacles. The pursuer and evader ASVs suffer from entirely unknown models. An entirely data-driven learning and control scheme is designed to achieve pursuit-evasion game behavior by leveraging deep reinforcement learning (DRL) and model predictive path integral(MPPI) control. Specifically, a gated recurrent unit (GRU) neural network is first trained to learn the state transition function for each ASV by taking advantage of the logged input and output data. Then, by considering inter-vehicle collision avoidance as well as collision avoidance with moving and stationary obstacles, the MPPI control laws are designed to achieve the hunting of the pursuers against the evader based on the learned state transition functions. It is noted that after learning by exploiting random and new data, pursuit-evasion game is capable of being achieved in the absence of model information. Simulation results are conducted to validate the efficacy of the presented GRU neural network-based DRL method for pursuit-evasion game of underactuated ASVs.

Main paper: 
 school of marine electrical engineering: 
 Dalian Maritime University Dalian, China dwangdl@gmail.com Abstract-This paper investigates a pursuit-evasion game scenario involving multiple under-actuated autonomous surface vehicles (ASVs) pursuing a single ASV in a closed environment in the presence of moving and stationary obstacles. The pursuer and evader ASVs suffer from entirely unknown models. An entirely data-driven learning and control scheme is designed to achieve pursuit-evasion game behavior by leveraging deep reinforcement learning (DRL) and model predictive path integral(MPPI) control. Specifically, a gated recurrent unit (GRU) neural network is first trained to learn the state transition function for each ASV by taking advantage of the logged input and output data. Then, by considering inter-vehicle collision avoidance as well as collision avoidance with moving and stationary obstacles, the MPPI control laws are designed to achieve the hunting of the pursuers against the evader based on the learned state transition functions. It is noted that after learning by exploiting random and new data, pursuit-evasion game is capable of being achieved in the absence of model information. Simulation results are conducted to validate the efficacy of the presented GRU neural network-based DRL method for pursuit-evasion game of underactuated ASVs.Index Terms-autonomous surface vehicles, pursuit-evasion game, gated recurrent unit, deep reinforcement learning. 
 
 Appendix: