## 3 Research Method
### Proposed Approach  
This research focuses on location-aware RL-enabled UAV path planning aiming to autonomously learn/find best-fitted and cost-effective flight directions/paths to optimize the results of UAV path planning for remote sensing such as reduced data collection delay and UAV power consumption. There are two operational strategies under RL-enabled UAV route planning: exploration and exploitation. The former allows the UAV to take a random path to fly toward the closest SP, whereas the latter forwards the UAV according to what was learned (e.g., achieved reward values) from previous flights. Figure 2 shows how a UAV takes the benefits of RL (Q-learning) to explore and exploit the environment (e.g., farm). RL assumes UAV as an agent to find optimal routes through constant \"trial-and-error\" training/interaction with the environment (e.g., the farm and location of SPs). The following outlines the key components of RL enabled UAV path planning approach:  
* Objective: the UAV flies over a farm aiming to visit a constant number of SPs through the shortest path (minimum Euclidean distance) and with minimized delay. Each journey/path has a maximum flight time (e.g., 20 min) to meet the battery constraints. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**State** & \(Action_{1}\) & \(Action_{2}\) & \(Action_{n}\) \\ \hline \(S_{1}\) & Q(\(S_{1}\), \(A_{1}\)) & Q(\(S_{1}\), \(A_{2}\)) & \(\cdots\) & Q(\(S_{1}\), \(A_{n}\)) \\ \(S_{2}\) & Q(\(S_{2}\), \(A_{1}\)) & Q(\(S_{2}\), \(A_{2}\)) & \(\cdots\) & Q(\(S_{2}\), \(A_{n}\)) \\ \(\cdots\) & \(\cdots\) & \(\cdots\) & \(\cdots\) & \(\cdots\) \\ \(S_{n}\) & Q(\(S_{n}\), \(A_{1}\)) & Q(\(S_{n}\), \(A_{2}\)) & \(\cdots\) & Q(\(S_{n}\), \(A_{n}\)) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Q-table. Figure 2: Exploration–exploitation for the RL enabled UAV. * States: UAV's location and residual battery. * Action: flight direction and/or route. * Environment: a map of labeled regions (e.g., SP or no-fly), the reward algorithm, and mission termination function. * Reward: two rewards, including a positive if a SP is visited and a negative if the UAV moves into no-fly zones or loop. * Termination: if the UAV visits all SPs, or runs out of battery. The RL-enabled UAV path planning algorithm aims to explore/exploit the farm and update the Q-table with the rewards achieved. The UAV gets positive rewards if it flies over SP zones and captures the remote sensing recordings, whereas it receives a penalty if moves to no-fly areas. To avoid loops, the UAV receives a negative reward if it continuously moves over safe-fly zones for a while with no SP-visit success. This results in reduced UAV power consumption, and data collection delay. The UAV explores the farm and moves until no SP is left or residual battery drops below a threshold. Otherwise, it returns to the base station to re-charge the battery, deliver the remote sensing recordings and plan the next journey. The remote sensing journey is finished if all SPs are visited. Figure 3 depicts the UAV route planning approach. The UAV returns the base station to deliver the results in three cases: (1) all SPs are visited and the collected data should be delivered to the base station. The UAV might be able to transmit remote sensing recordings (e.g., photos) to the base station via internet links. However, this results in increased communication overhead and consequently power consumption, especially if the farm is wide, data volume (e.g., image quality) is large and/or SPs are crowded and dense. By this, UAV should return the sink to deliver the data with minimized power consumption, and (2) the UAV's battery meets the threshold. For this, the UAV cannot move forward and visit the next SPs, but it should only return home for battery re-charge, and (3) a (part of) remote sensing mission in a particular/target field/farm is finished. The proposed approach has the potential to support remote sensing with multiple UAVs.