## 1 Introduction
### Methods and Applications of Inverse Reinforcement Learning  
Since the formulation of the IRL problem, various approaches have been proposed. Foundational methods include those based on optimizing some margin between the expert and agent, such as with feature-based value functions [6], or focusing on aligning hand-crafted feature expectations [9], as seen in AL-IRL. This last method has shown favorable results in robotic applications like autonomous driving [9; 10] and robotic quadruped locomotion [11]. More recently, it has been applied to swarm-guidance shepherding [12] and path planning for autonomous quadcopters [13]. However, a key issue with IRL is that multiple reward functions can explain the expert's behavior, leading to ambiguity and bias. To address this, the maximum entropy principle was introduced [7], ensuring that the learned reward function is both unbiased and robust. This principle identifies a distribution over behaviors that maximizes entropy while still aligning with the expert's performance and has been successfully applied to tasks such as modeling driving routes. Furthermore, it has been extended to deep learning, resulting in Deep Max-Entropy IRL [14], which enables robots to learn directly from raw sensory data. This eliminates the need for manual feature engineering and facilitates the learning of control policies in continuous, high-dimensional spaces.  
Another direction in robotic IRL involves probabilistic methods. These methods use approaches like Bayesian updates [15], Gaussian processes [16; 17], or maximum likelihood estimation [18; 19] to infer reward functions in a way that accounts for uncertainty, making them particularly suited for tasks involving noisy or incomplete data, such as simulations of autonomous highway driving and robotic control tasks like grasping.  
A more recent and promising development in IRL is the application of generative adversarial techniques. These approaches involve training a discriminator to distinguish between the expert's behavior and that of the agent, allowing the agent to learn a reward function based on this feedback. Key examples of this technique include Generative Adversarial Imitation Learning (GAIL) [20] and Adversarial Inverse Reinforcement Learning (AIRL) [21], both of which have shown excellent results in physics-based control tasks but also in autonomous highway driving [22] and robotic manipulation [23].  
Other recent developments in IRL have introduced new methods that expand its capabilities. For instance, gradient-based approaches [24; 25] have improved the efficiency of reward estimation, particularly in high-dimensional spaces. Additionally, alternative optimization criteria, such as average reward accumulation [26], address IRL without relying on the assumption of discounted expert behavior. Another promising direction involves active exploration strategies [27], where agents use a greedy exploration approach based on defined errors to refine their learned policies.  
Another key advancement concerns how to handle expert demonstrations in order to extract useful information. For instance, one challenge is managing multiple experts [28],where agents learn from diverse sources of expertise. This also includes cases where experts are sub-optimal [29], which has been demonstrated to help mitigate the ambiguity problem. Furthermore, recent research has focused on estimating a model of the expert's decision-making process [30; 31], which is then used to provide additional data to the IRL algorithm. In tasks that require time-dependent decision-making, IRL has also been applied with time-dependent reward parameterization [32], enabling more accurate modeling of complex, high-task environments.