# Data-Driven Policy Learning Methods from Biological Behavior: A Systematic Review
## 3 Strategy Learning from Demonstration: Imitation Learning
### Section Summary  
Through direct imitation, strategies can be learned from demonstrations using IL. This section describes two specific methods by which behavior can be imitated directly from demonstrations, namely BC and GAIL, and their improved methods. Supervised-learning-based simple BC uses labeled demonstration data for learning. Depending on the needs of the demonstration subject and the experimental environment, the BC can use demonstration data not only from a first-person viewpoint but also from a third-person viewpoint. The first-person-view BC is designed for occasions where the accuracy of the measurement data is important, and the sensor can be easily mounted directly on the expert. When customizing the experimental environment is costly, and it is difficult to fix the sensor to the experimental subject, third-person-view BC based on video measurement is easier to apply. BC is theoretically feasible as the simplest IL method. It performs well in imitating demonstrative robotic manipulations in simple environments. However, relying on the properties of supervised learning and the policy-learning process without environmental interaction, it was found in early practice that simple BC often fails to mimic expert behaviors in real-world high-dimensional environments. This is because of problems such as covariate shifts and causal confusion [97]. To address the problem of environmental interaction, DAgger employs the policy learned by the BC to interact with the environment and generate new data, iterating online to augment the dataset. The algorithm considersthe differences between demonstration and test results. Thus, it exhibits a generalization performance superior to simple BC in complex environments. However, DAgger does not inherently depart from the BC algorithm. Owing to its online learning mechanism, it requires long hours of expert assistance to supplement the demonstration, which can result in high workloads. Moreover, BC is often used for pre-training and is combined with RL owing to its simplicity. This approach of combining IL and RL introduces a reward obtained from the imitation of an expert into the RL framework, which can be used to obtain an agent that exceeds that of a demonstration expert. It has been extensively applied in robot motion control. By contrast, GAIL uses a generative adversarial approach to extract abstract features from training samples, and its stronger representation ability alleviates the dependence of the algorithm on large datasets. However, the original GAIL experiences the problems of GAN modal collapse and inefficiency of stochastic policy search methods. To solve the modal collapse problem, cGAIL and AC-GAIL constrain the explicit modal labels. In contrast, through unsupervised learning, InfoGAIL and VAE-GAIL determine the implicit modal labels in the presentation data. In addition, Wasserstein GAIL improves the smoothness of the training process. MGAIL and Actor-Critic-based GAIL improve the environment sample interaction utilization by introducing environment models and changing the strategy search method, respectively. GAIL is widely employed in automatic driving, motion control, robot manipulation in high-dimensional environments, and game strategy learning. It has become a popular strategy learning algorithm in recent years.