## 2 Problem Statement
### Why Reinforcement Learning?  
Traditional path planning methods, such as A* and Dijkstra's algorithm, are primarily designed for static environments where obstacles and goals are fixed and known a priori [5]. While extensions to these methods, such as dynamic A* (D*), allow for some level of adaptability [6], they often require substantial computational overhead for replanning. In contrast, RL inherently learns a policy \(\pi(a|s)\) that maps states to actions, enabling real-time decision-making without explicit replanning. Indeed, the optimal policy in RL, \(\pi^{*}(a|s)\), which is obtained by solving  
\[\pi^{*}(a|s)=\arg\max_{\pi}\mathbb{E}_{\pi}\left[\sum_{t=0}^{\infty}\gamma^{t }r(s_{t},a_{t})\right], \tag{22}\]  
directly accounts for environment dynamics through interaction during the training phase [1]. This allows the robot to handle scenarios where the obstacle set \(\mathcal{O}_{t}\) or the goal position \([x_{g},y_{g}]^{\top}\) varies over time.  
Another advantage of RL stems from its optimality in applications involving environments with complex geometries or high-dimensional state spaces. In such applications, traditional methods often struggle due to the exponential growth of the search space [7]. For instance, sampling-based methods like Rapidly-Exploring Random Trees (RRT) can provide feasible paths but are not guaranteed to find optimal paths without significant computational effort. Reinforcement Learning, particularly with Deep RL, excels in such scenarios by approximating the value function \(V^{\pi}(s)\) or action-value function \(Q^{\pi}(s,a)\) using neural networks [3, 8]. Algorithms such as Deep Q-Networks (DQN) or Proximal Policy Optimization (PPO) leverage the Bellman optimality principle (see Equation (5)) to find near-optimal policies efficiently, even in high-dimensional settings [4].  
Finally, in real-world applications, robots face significant uncertainties, such as sensor noise, actuator variability, and unpredictable environmental dynamics. While traditional methods can handle some uncertainties through deterministic approximations, they lack the robustness to stochastic effects. In RL, stochasticity is inherently modeled in the state transition function \(\mathcal{P}(s^{\prime}|s,a)\) and reward function \(r(s,a)\). For example, the dynamics of a differential-drive mobile robot are represented as \(\mathcal{P}(s^{\prime}|s,a)=\mathcal{N}(s^{\prime};f(s,a),\Sigma)\), where \(\Sigma\) captures uncertainties in motion. The RL agent learns to optimize its policy under such probabilistic dynamics, making it robust to real-world noise [2].  
Reinforcement Learning is particularly advantageous in several applications and scenarios. In navigation through unknown environments, where the environment is partially known or entirely unknown, RL allows the robot to learn an effective policy through exploration and interaction with the environment [9]. This capability is crucial for tasks such as search-and-rescue operations. In scenarios involving dynamic obstacle avoidance, RL enables the robot to predict and react to dynamic changes in the environment by leveraging the policy \(\pi(a|s)\) trained on similar conditions. RL is also effective in multiobjective path planning, allowing for the optimization of complex reward functions that balance competing objectives, such as minimizing energy consumption, maximizing safety, and achieving timely goal completion:  
\[r(s,a)=-\alpha\|[x\;y]^{\top}-[x_{g}\;y_{g}]^{\top}\|-\beta\|a\|^{2}-\gamma_{ \text{collision}}\mathbb{I}\{(x,y)\in\mathcal{O}\}, \tag{23}\]  
where \(\mathbb{I}\) is an indicator function penalizing collisions. Additionally, RL can be integrated with imitation learning, enabling robots to learn policies from human demonstrations or expert-generated trajectories, which reduces the need for manual design of path planning algorithms [10].