# Data-Driven Policy Learning Methods from Biological Behavior: A Systematic Review
## 6 Discussion
### Application Scenario Analysis  
For the reviewed policy learning approaches, their applications in autonomous driving, robot control, and biological behavior understanding were analyzed. In this section, we highlight how the diverse characteristics of these algorithms lead to varied applications across these domains, summarizing the relevant policy learning methods for each application scenario.  
1. Autonomous driving: Autonomous driving tasks often involve autonomous driving systems and driver behavior in various traffic situations. Considering the visual information input, drivers' style modeling, and system safety of the autonomous driving system, policy learning methods such as DAgger, IRL, GAIL, and CPL are often used for autonomous driving. In contrast, DAgger is often used for automatic driving strategy planning from visual information [14; 15] and can be used for simple lane-keeping tasks [19]. However, modeling drivers based on their gender, age, and driving style is a difficult task. IRL and GAIL model drivers' driving preferences from driver data, which enables optimal driving strategies in different driving situations (e.g., car following, lane changing, and overtaking) [2; 35; 43; 44]. In addition, safety is also an important factor, as CPL improves the safety of autonomous driving strategies by modeling the causal relationship between variables such as speed and distance to the vehicle in front [69; 73].
2. Robot control: Robot control applications, including robot manipulation, motion control, and navigation, are discussed. According to different test conditions, BC and IRL can be used in robot manipulation with trajectory reproduction. For applications that require high environmental interaction conditions, BC combined with RL methods, IRL, and GAIL are often used to implement motion control and navigation tasks. Robot manipulation requires an agent to learn the presentation trajectories of humans. BC is an effective and simple method for low-dimensional robot manipulation without interactions [20]. However, for complex and high-dimensional robot manipulation trajectories, IRL performs significantly better than the BC [57; 66; 67]. For dexterous robot motion control, we found that using BC or DAgger as pre-training and combining it with RL's policy learning method can flexibly mimic animal motion and learn the ideal motion strategy, which is more practical [4]. Both IRL and GAIL were used for robot navigation tasks. The former focuses on reproducing human navigation trajectories [45; 55; 56], whereas the latter focuses on autonomous robot navigation in dynamic environments [33].
3. Biological behavior understanding: Reward functions reproduced through behavioral data can explain behavioral preferences; therefore, with the biological behavior data input, IRL can thus be used to help understand biological behaviors. The study of behavioral strategies has been widely discussed as a fundamental topic in neuroscience and behavior [1]. IRL has been used to study foraging strategies in worms [60], olfactory searching of silk moths [46], migratory routes of migratory birds [47], and the behavior of group animals [59; 68]. The recognition of behavior preferences is fundamental to understanding the mechanisms of animal behavior and analyzing the control activities of the nervous system [1; 117; 118]. It is not limited to animal behavior but also includes human behavior. IRL has also been used to learn user preferences from user data to help relevant organizations adjust user strategies to maximize their benefits [42; 49; 62].  
In addition to the three typical examples mentioned above, these policy-learning methods are also used in strategy learning for game agents, HCI, and helicopter aerobatics. Practitioners and researchers can select appropriate strategy learning methods depending on the requirements of different application contexts.