A multi-AUVs Bio-inspired Cooperative Hunting Algorithm for Environment with Ocean Current and Obstacles

Authors: qingqin  liu
bing  sun
chunhua  gu
dinghua  zhang
daqi  zhu
Venue: IEEE_org_ICIST_2024_Conference
Published: 2024-08-08 07:29:12

Abstract: As a result of the complexity and indeterminacy of the underwater environment, multi-AUV cooperative hunting is a very challenging research area. In this paper, the focus is on the underwater environment with ocean currents and obstacles, and a novel multi-AUV bio-inspired cooperative hunting algorithm is presented. Firstly, a bio-inspired Neural Network model (BINN) is established to represent the AUV's underwater working environment. Each point in the grid map corresponds to the activity of a neural node in the BINN, and the next target point of the hunting AUV can be planned autonomously according to the neural activity of the neurons in the BINN. Secondly, the direction decision algorithm is embedded into the BINN hunting model for the ocean current environment, and the integrated algorithm proposed is compared with the hunting method without the direction decision. Finally, simulation studies are presented to demonstrate that the proposed algorithm is effective for cooperative hunting missions.

Main paper: 
 i. introduction: 
 Nowadays, multi-robot systems have been widely used in many areas [1] . For example, the multi-robot formation [2] , cooperation hunting [3] - [5] , task assignment and cooperation [6] - [8] , etc. An autonomous underwater vehicle (AUV) is a robot that has intelligence and can complete tasks in the ocean by itself without the guidance of an operator. Because one This project is supported by the National Natural Science Foundation of China (62033009) and the Creative Activity Plan for Science and Technology Commission of Shanghai (23550730300).single AUV has limited capacity when working in the ocean, the multi-AUV system is an important research direction for the technology of current robots.Many studies have been made on multi-AUV systems, including underwater cooperative search [9] , [10] , mine sweeping [11] , autonomous navigation [12] , formation control [13] , dynamic task assignment [14] , collaborative plotting [15] , cooperative hunting [16] - [18] and so on. Among these studies, multi-AUV cooperative hunting is interesting, comprehensive, and worthy of study. But there are only a few researches on the underwater hunting problem now. Hunting in the ocean is different from the ground-hunting. Ding and Wang et al. proposed a virtual potential method for the path planning of AUV in [19] . Wang et al. [20] applied a potential field method to conduct the hunting of multi-AUV, where an ant colony optimization method is combined with an artificial potential field method to avoid collision in the hunting process. Zhu et al. [21] , [22] studied the multi-AUVs hunting problem with the BNN algorithm, where the negotiation method is put forward to hunting task allocation. Simulation is completed in both 2-D and 3-D environments [23] , and the proposed algorithm can deal with various situations automatically and catch the moving target [24] efficiently. However, these researches above has not considered the ocean current effects on the underwater environment. In fact, it is very important for multi-AUV cooperative hunting to consider the ocean current [25] .This paper focuses on situations in which the dynamic environment with ocean currents and obstacles as well as the target is intelligent with their motions unpredicted and irregular. The multi-AUV hunting algorithm based on the bioinspired neural network and the direction decision is presented. The hunting AUVs' paths are guided through a bio-inspired neural network and the direction decision model, and the results show that it can achieve the desired hunting result efficiency.This paper is organized as follows: Section 2 describes the map of the hunting process, and the simulation of ocean currents and kinds of hunting final states are also given. In Section 3, the bio-inspired neural network algorithm and direction decision algorithm are designed. And the hunting algorithm is proposed in the environment of constant ocean currents. Simulations are conducted in Section 4, and Section 5 concludes the whole paper.In this paper, the multi-AUVs hunting problem is studied in the underwater environment. Underwater environments replaced by discrete grid maps, the work area will be divided into the same small cells, each cell has two states: the free space and occupied space, as shown in Figure 1 . The black cells indicate obstacles, and the white ones represent free space. The whole space of the hunting task is discrete, therefore, the hunting workspace can be used to simplify by grid map. The numbers of AUVs is denoted as P c = [P c1 , P c2 , . . .], the evader is labeled as Ev, and the obstacles are denoted as Obs = [Obs1Obs2......]. The evader has the same intelligent abilities as the hunting AUVs, which can automatically avoid obstacles and update the location in real time to avoid being arrested.A bio-inspired Neural Network model (BINN) is established to represent the AUV underwater working environment. Each neuron corresponds to the position in the grid map one by one in the neural network, as shown in Figure 2 . Based on the distribution of neuron activity output in the neural network, plan the moving path of AUV. First, calculates all adjacent activity output of neurons to the current neuron, and then finds the maximum activity of these neurons as the next position of AUV. This model will help AUV to update the motion position in real-time and avoid obstacles automatically.When the hunting task begins, the hunting AUV will move to the target, meanwhile, the target AUV updates the location as the escaped trajectory. In the process of hunting, hunting AUVs can avoid obstacles and find the shortest path between the AUV with the evader AUV. However, the evader needs to judge whether the contiguous positions are occupied by hunting AUVs or not. If not, continue to escape. If so, it needs to judge whether the surrounding positions are all occupied or not. If the positions are not occupied, determine a free space as the next step. Otherwise, stop moving and the hunting task is successful in the end. As shown in Fig. 3 and Fig. 4 , four statuses of hunting the target successfully. For the underwater environment, obstacles and ocean currents are very important factors that cannot be ignored. Obstacles are divided into static and dynamic obstacles, and the ocean current is defined as the constant current with direction (45°) and speed (a grid each second).Theoretically, AUV is a complex dynamic system that cannot move fully. However, the small turning radius does not affect the accuracy of the working space dimensions of AUV. Meanwhile, the size of AUV is much smaller than its working space in path planning, which can be thought of as a point. The ultimate principle of the path planning algorithm is on the basis of the grid map and establishes the biologically inspired neural network model, which represents the working environment. Each neural network neuron corresponds to the position in the grid map one by one. The connection between neurons to each other is shown in Figure 2 . AUV will occupy a grid of two-dimensional environment at any moment, the adjacent 8 grids as the next optional positions of AUV. The specific navigation location depends on the activity of eight adjacent neurons and regards the maximum activity grid as the AUV navigation direction.Dynamic model of neural distributions represented as a two-dimensional network of neurons, each neuron and its surrounding adjacent neurons are connected, as shown in Figure 2 . Biologically inspired neural dynamics model can be converted to a shunting equation as follows [21] : The "shunting model" proposed by Grossberg is shown in the following formulation (1) :dx i dt = -Ax i + (B -x i )S + i -(D + x i )S - i (1)where x i is the neural activity of the i-th neuron, A, B and D represent the passive decay rate, and the upper and lower bounds of the neural activity respectively which are nonnegative constants. S + i and S - i are the excitatory and inhibitory inputs to the neuron. In the process of hunting, the hunting AUVs' motions are guided by the dynamic landscape of a neural network. The excitatory input S + i results from the target and its neighboring AUVs and the input S - i only results from the obstacles. In this context, the dynamic of the i-th neuron in the neural network can be characterized by a shunting equation asdx i dt = -Ax i + (B -x i )([I i ] + + k j=1 w ij [x j ] + ) -(D + x i )[I i ] -(2)where k is the number of neural connections of the i-th neuron to its neighboring neurons. The terms[I i ] + + k j=1 w ij [x j ]+ and [I i ] -are the S + i and S - i in equation ( 1), respectively. The terms [I i ] + is a linear-above-threshold function defined as[I i ] + = max{I i , 0} , similarly the term [I i ] -= min{-I i , 0}. [I i ] + and [I i ] -arethe variables that represent the input to the i -th neuron from target and obstacle, respectively. They defined as:I i = I    E If it is a neighboring cell to objective 0 Otherwise(free space) -E If it is an obstacle (3)where E >> B, which is a very large positive constant. In equation ( 2), the term w ij is defined as:EQUATIONwhere x i and x j are two vectors, is the Euclidean distance between them. Function f (a) is a monotonically decreasing function which is defined as:EQUATIONwhere µ and R n are positive constants. Obviously, the weight connection coefficients are symmetrical, that is w ij = w ji . Fig. 2 shows the neural network in a 2-D environment model [12] , where the green circles represent neurons. In this structure, each neuron is connected by adjacent ones which form the whole network for transmission of activity.Discrete the navigation path of AUV, and the path planning problem will evolve into seeking the next position of AUV. Therefore, it is significant to link the specific dynamic environment with the preceding and next position of the AUV moments, and comprehensively the next driving position of the AUV. Using the distribution of the neuron activity output, find out the next time position of AUV. Firstly, calculate all the neurons adjacent to the current neuron activity output values, and then find out the maximum activity of these neurons, AUV began moving to the position where the largest output of the neural activity. When the same size of neuronal activity appears, the AUV will randomly select one of the neurons as their next position, which does not affect the process of path planning. The specific decisions of next position x Pn are:EQUATIONx i represents the neuron's activity around the AUV, P p is the previous step of AUV, P c represents the current position of AUV, P n represents the next position. Under the process of AUV path planning, selecting the largest neural activity as its next movement repeatedly until reach the goal. As without considering the effect of ocean currents on AUV path planning, the introduction about ocean currents influencing the path planning of AUV will be expounded in the following section, and raise the corresponding solutions suitably.Under the effect of ocean currents, there is a certain deviation in the movement of AUV. In order to overcome the deviation and make AUV navigate as the predetermined trajectory, the direction decision mechanism is presented to solve this problem. During the process of path selection, reducing the ocean currents' effect on the navigational angle of AUV is the main ideal, it can be shown as formula (7) :EQUATIONwhere x P n means the active value of neurons surrounding at the current location, P p represents a previous step on the AUV, P c shows the current location of AUV, P n express the next location of AUV, g i is a function of an associated with a deflection angle which come from the ocean current effect, acting on AUV, which is defined as (8):EQUATIONc is a constant greater than zero, setting the range as [0,1]. ∆θ is the absolute value of the difference of θ l and φ l , defined as (9) . θ l denotes the angle that there is no current on AUV with a horizontal axis, defined as equation (10) . φ l indicates the angle of AUV under the current direction and the horizontal axis, defined as (11) .EQUATIONθ l = a tan(y Pc -y P p , x Pc -x Pp ) (10)EQUATIONθ l indicates the angle of value currents direction and level axis, for currents size a, x Pp , y Pp , (x Pc , y Pc ) and (x Pn , y Pn ) respectively express previous mobile location, current location and the final next mobile location, shown as Figure 5 . When the currents direction is consistent with the scheduled mobile direction, the formula ∆θ = 0 and cos ∆θ = 1 means AUV keeping movement direction with the scheduled direction. When the direction of the current is contrary to mobile direction with no currents effect instead, ∆θ = π and cos ∆θ = -1, after the path select of AUV in reduction off anti-direction of currents role and still keep the scheduled direction to path planning. Another situation uses direction decision-making to steer the direction of AUV under a constant current environment, shown as formula (7) , which in order to accordance with the path planning of biologically inspired neural network, after losing steering effect of currents.In view of the path conflict between AUV caused by neuronselective conflict, as shown in Fig. 6 , this paper proposes a solution called the location prediction method. This method records the location information between the hunting AUVs. Before each AUV selects the next moving position, if the moving grid is occupied by other AUV then choose the other grid to move. It can avoid the problem of path planning coordination between hunting AUVs effectively. The algorithm steps as follow.Step 1: initialized the position of hunting AUVs.Step 2: defines a vector type variable L to storage the initial position of each hunting AUV, and make a declaration in the program of AUV's path planning.Step 3: updates the position information of AUV in variable L, the passed position has been clear.Step 4: traverses the current position of other hunting AUV in variable L, and judges the position is equal with other position or not. If yes, then set this positional value as -E and choose other grid as the next position of hunting AUV. If not, follow step 5.Step 5: AUV finds the next position and store this position information in the variable L.It can be seen that the next path selection will prejudge the location information of other AUV. If the grid to be moved has been occupied by other AUV, then the occupied grid can be seen as an obstacle and the hunting AUVs will not move towards this grid, avoiding the conflict between the AUVs. The simulation in Fig. 7 is that the hunting AUVs successfully resolute the conflict with the method proposed above. The angle of ocean current is 45°, and the ocean current speed is about √ 2 gate grid/seconds. Red five-pointed star represents the evader and the remaining 4 different shapes are represented the hunting AUVs. The four AUVs in the hunting team, starting move respectively from (1,1), (20, 16) , (1,20) (17,7) four initial position to hunt the evader, whose initial position was (11, 6) .Fig. 8 shows the different time state of the whole hunting task for static obstacles and ocean current environment. During the whole process of hunting, ocean currents will significantly affect the movement of AUVs, lengthen the hunting distance. We can see clearly that each hunting AUV was obviously influenced by the ocean current, moved more steps and cannot chose an appropriate path to complete the hunting task. The hunting trajectory of AUV was desultory which cannot pick out the concrete path of each hunting AUV accurately. During the whole hunting process, the evader AUV moved 5 steps and was hunt at (15,7) finally. Fig. 9 indicates the hunting simulation results with direction decision for static obstacles and ocean current environment. Fig. 9 shows the different time state of the whole hunting task. In the process of the hunting mission, each AUV selected the applicable path to hunt the evader AUV precisely. The evader moved 3 steps and was hunt at (11, 9) finally. Compared with Fig. 8 , the intention of each hunting AUV was more clearly and the paths of hunting AUV were obviously shortened after adding the direction decision mechanism, which can appropriately reduce the influence of ocean currents on AUVs. Under the same conditions, the direction decision mechanism can reduce the steps of hunting AUVs that make the hunting task more efficient under the current setting. For instance, the path of AUV2 was clearly with pertinence and directive to move to the evader precisely. This direction decision mechanism can greatly reduce the effect from ocean current.Simulation results of comparison experiments above are shown in Table 1 . Sailing steps of each hunting AUV in the hunting process for different hunting algorithm are listed. As it is shown in Table 1 , the proposed algorithm can help the AUVs to capture the evaders with less cost and higher efficiency. It takes the total steps of 40 for four hunting AUVs to catch the evader successfully. However, that of hunting algorithm without direction decision scheme is 73. The simulation above has shown the context of static obstacles with the decisions mechanism. The cooperative hunting with dynamic obstacles is simulated in here. Ocean current environment remains for 45°, size √ 2 grid/ seconds of constant current. Red five-pointed star represents the evader AUV and the other four different shapes are represented the hunting AUVs. The four AUVs in the hunting team, moving from (1,1), (20, 16) , (1, 20) (17, 7) respectively to hunt the evader. And the initial position of evader AUV was (11, 6) , the whole hunting process was shown in Figure 8 and Figure 9 .Fig. 10 shows the different time state of the total hunting task. During the process of hunting, ocean currents were distinctly affected the movement of AUVs, lengthen the hunting distance and prolong the time duration. Each hunting AUV was obviously influenced by the ocean current with more steps and cannot chose the shortest path to complete the hunting task. The hunting trajectory of AUV was desultory, it cannot find out the concrete path of each hunting AUV accurately. During the whole hunting process, the evader AUV moved 5 steps and was hunt at (14,7) finally, the trail of the moving obstacle started at (3, 12) and it stopped at (9,12) ultimately.Fig. 11 indicates the hunting simulation results with direction decision and shows the different time state of the whole hunting task. In the process of the hunting mission, each AUV selected the applicable path to hunt the evader AUV precisely. During the hunting process, as seen in Fig. 11 , a moving obstacle passing through and just affect the path of AUV1, but cannot impact the final hunting task. The evader moved 5 steps and was hunted at (11, 11) successfully, the trail of the moving obstacle stopped at (18, 12) in the end.In comparison with Fig. 10 , the intention of each hunting AUV was more clearly and the paths of hunting AUV were obviously shortened after adding the direction decision mechanism, and can appropriately reduce the influence of ocean currents on AUVs. Under the same conditions, the direction decision mechanism can reduce the steps of hunting AUVs that make the hunting task more efficient under the current setting. This direction decision mechanism can greatly reduce the effect from ocean current.Results of comparison experiments about the different environment are shown in Table 2 . The navigating steps of each AUV in the hunting task in both current and dynamic obstacle environments. As it is shown in Table 2 , the algorithm raised above can greatly improve the efficiency of hunting AUVs. With the two algorithms, the steps of evader AUV were both 5 steps, the hunting total steps with direction decision scheme was 49 steps which was 10 steps lesser than the algorithm without direction decision scheme. V. CONCLUSIONThe multi-AUV cooperative hunting problem is studied in this paper mainly. The direction decision mechanism based on bio-inspired neural network algorithm, can maintain the hunting path of each AUV unaffected by the ocean current and plan the effective path to hunt the evader AUV. The direction decision mechanism is proposed to reduce the influence from ocean current on AUV movement and improve the efficiency of hunting task. Because of the complicated underwater environment, the ocean current changes with the time actually which is a problem that need to be solved in the following research work.  
 
 Appendix:  



Reviews: review: title: title: As the complexity and indeterminacy of the underwater environment, multi-AUV cooperative hunting is a very challenging research area. In this paper, it is focused on the underwater environment with ocean current and obstacles, and a novel multi-AUVs bio-inspired cooperative hunting algorithm is presented. Firstly, a bio-inspired Neural Network model (BINN) is established to represent the AUV underwater working environment. Each point in the grid map corresponds to the activity of a neural activity in the BINN, and the next target point of the hunting AUV can be planned autonomously according to the neuron activity of the neurons in the BINN. Secondly, the direction decision algorithm is embedded into the BINN hunting model for the ocean current environment, and the integrated algorithm proposed is compared with the hunting method without direction decision. Finally, simulation studies are presented to demonstrate that the proposed algorithm is effective for cooperative hunting mission. Comments for this submission are given as follows.
main_review: main_review: (1)、The pictures in this paper are clearly drawn, but there are some minor problems; the sizes of “obstacles” and “free” in Figure 1 are different, which should be revised by the authors, and the rest of the pictures should be double-checked by the authors.
(2)、The layout of this paper is very neat, although the authors are asked to revise the formatting of equation (7) if it is different from the formatting of the other equations at their discretion.
(3)、The literature citations in this paper are very appropriate, but whether the year of some of the literature is too old, for example, literature [15], the authors are requested to modify them at their discretion.
title: title: he paper makes a valuable contribution to the field of autonomous underwater vehicles and multi-robot systems.
main_review: main_review: The work is significant as it addresses the complexities of multi-AUV cooperative hunting in challenging underwater environments, where factors like ocean currents and dynamic obstacles can greatly influence the success of the mission. The bio-inspired approach is innovative, and the inclusion of the direction decision algorithm adds robustness to the system. However, further validation in real-world scenarios would strengthen the findings, as the current study relies heavily on simulations. Additionally, exploring the algorithm's scalability to larger fleets of AUVs and more complex environments could provide deeper insights into its practical applicability. Overall, the paper makes a valuable contribution to the field of autonomous underwater vehicles and multi-robot systems. Below is a list of comments that should be taken into account further when revising the paper.

1.	There are a few typos in this paper which should be corrected. And there are some notions missed. Please make some corrections.
2.	Please add the necessary comments for Figures.
3.	Look out for the following grammatical and spelling errors: misspelling，redundant or unnecessary words，subject-predicate consistency problem，punctuation error，preposition error.
title: title: This article is very interesting and a good one
main_review: main_review: This study presented a novel multi-AUVs bio-inspired cooperative hunting algorithm in underwater environment with ocean current and obstacles. The obtained result is valuable and can be accepted if the following problems can be clarified.
 (1) In the introduction, it is not enough to state the current work. It should be expended and reconstructed. 
(2) Where are ${[I_i^e]^ + }$, ${[I_i^o]^ + }$, ${[a]^ + }$ and ${[a]^ - }$  specifically reflected in formulas (1) and (2)? Please explain 
(3) The simulation description can be simplified appropriately.
(4) There are some typos and grammar errors. The authors should have a native English speaker or software packages to perform the editing check. 
(5) The conclusion of the article suggests using the present perfect tense for description. 
(6) The references need to be updated and the format needs to be standardized.

Comments: title: This article is very interesting and a good one
comment: This study presented a novel multi-AUVs bio-inspired cooperative hunting algorithm in underwater environment with ocean current and obstacles. The obtained result is valuable and can be accepted if the following problems can be clarified.
(1) In the introduction, it is not enough to state the current work. It should be expended and reconstructed.
(2) Where are ${\left[ {I_i^e} \right]^ + },{\left[ {I_i^o} \right]^ + },{\left[ a \right]^ + }{\left[ a \right]^ - }$ specifically reflected in formulas (1) and (2)? Please explain$
(3) The simulation description can be simplified appropriately.
(4) There are some typos and grammar errors. The authors should have a native English speaker or software packages to perform the editing check.
(5) The conclusion of the article suggests using the present perfect tense for description.
(6) The references need to be updated and the format needs to be standardized.