## 1 Introduction
Assuming that an underlying reward function guides the user's choices, the goal is to unveil this function. Here, the reward function is modeled as a weighted sum of predefined features, simplifying the learning process to determine these weights. Central to this approach is the selection of solutions presented to the user, so the reward function can be deduced from a limited number of queries and the reliability of the user's ability to discern between these alternatives. Designing reward functions in artificial intelligence and robotics presents a substantial challenge. It is inherently difficult for humans to explicitly articulate all the favorable behaviors a robot should aim to optimize. The crux of the challenge in leveraging reinforcement learning for intricate autonomous driving navigation tasks lies in crafting a reward function that genuinely reflects the intentions of the human operator. Tradition suggests that assigning a high reward for reaching a destination or meeting a significant penalty for collisions would sufficiently train an agent to adopt an optimal policy. However, we have found that such an approach can be myopic. Merely emphasizing a high reward for reaching the endpoint or harshly penalizing collisions can result in myopic strategies; the trained policy may, for instance, lead the agent to repeatedly return to the same spot or exhibit unwarranted risk-taking behaviors. Navigating through visual input alone is particularly daunting in algorithms with heightened complexity and without effective routing mechanisms. Consequently, we adopted a human-in-the-loop reward-based learning method to address these challenges. An alternative approach is inverse reinforcement learning (IRL) [12; 13; 14], which involves deducing a reward function from observed expert behavior. While it has shown promise, IRL is not without its drawbacks. It is computationally intensive, requiring repeated solutions of reinforcement learning problems to approximate the reward function, typically exceeding the complexity of conventional reinforcement learning. Moreover, solutions derived from IRL can be ambiguous, often yielding multiple reward function candidates that rationalize the observed behavior. This ambiguity complicates the identification of a single \"correct\" reward function. The accuracy of the inferred function is heavily contingent on the quality of the expert's demonstrations, and poor or insufficient data can result in flawed outcomes. Furthermore, IRL's applicability is constrained by its susceptibility to overfitting, especially with smaller datasets, and its limited adaptability to certain domains. The intricacies of reward structures in real-world scenarios, such as the necessity to consider long-term benefits and multiple objectives, frequently transcend the assumptions underpinning IRL. The need for reward function updates in dynamic environments further adds to the computational overhead and resource demands. Additionally, incorrect assumptions within the model can lead to inaccurate inferences. Privacy and ethical considerations are also paramount when dealing with personal behavioral data. In contrast, active preference reward learning (APRL) [15; 16; 17] is an approach that elicits users' relative preferences on various trajectories, which is more intuitive and accessible than IRL. APRL strategically prompts users with questions that significantly impact decision-making, substantially enhancing learning efficiency and effectiveness. It integrates a reinforcement learning framework, making it adaptable across various environments and tasks. When faced with scarce or expensive labeling data, APRL turns to preference learning to determine action values by engaging user preferences, thereby overcoming the limitations of limited labeling resources. This study used the active preference-based reinforcement learning (APRL) methodology to construct a reward function tailored for autonomous driving navigation tasks.