# Data-Aware Path Planning for Autonomous Vehicles Using Reinforcement Learning +
## 3 Problem Formulation
### Experiment Setup  
In the setup of our experiment, we adopted a structured approach to define the environment for our agent's operation, along with predetermined constants that framed the experimental conditions. An important stipulation in our experimental design is that the agent is required to transfer a specified quantity of data before reaching its destination. Data transfer by the agent, denoted by \(D\) (in bits), is influenced by the bandwidth \(B\) available on a road link (in bits per second) and the duration \(T\) the agent spends on that link (in seconds). For a given road link \(i\), the potential data transfer \(D_{i}\) can initially be approximated as follows:  
\[D_{i}=B_{i}\cdot T_{i} \tag{1}\]  
Equation (1) assumes optimal conditions in which the agent uses the maximum bandwidth rate of the road. However, this model requires refinement due to the presence of other agents in the network, which share the road and its available bandwidth. Thus, a more realistic approach to model the data transfer on road \(i\) incorporates the traffic density or congestion \(\gamma\), modifying the equation to  
\[D_{i}=\left(\frac{B_{i}}{\alpha\cdot\gamma_{i}}\right)\cdot T_{i} \tag{2}\]  
This adjustment scales the data transfer rate of each link inversely with congestion, reflecting the reduced bandwidth available to an agent as traffic density increases. To guarantee that congestion can only decrease effective bandwidth, we require \(\alpha\gamma_{i}\geq 1\) for every road link and choose \(\alpha=1+\sigma_{\gamma}\), where \(\sigma_{\gamma}\) is the standard deviation of the congestion values across the map. For our dataset, this gives \(\alpha\approx 1.5\), which keeps the denominator safely above one and normalizes congestion impact network-wide. Sweeping \(\alpha\) over [1.2, 2.0] altered the mean trip completion time by less than 3%, indicating that the learned policy is robust to reasonable choices of \(\alpha\). Thus, the adjustment both normalizes traffic effects and ensures that the effective bandwidth \(B_{i}/\) doesn't increase, fully consistent with the model's design premise.  
To quantify road congestion, we used our preprocessed data, including the average speed and time spent traveling between nodes, along with the distances between these nodes, to find a normalized congestion factor for each road link \(i\). This factor inversely impacts the bandwidth available to each agent, with higher traffic densities leading to a reduced bandwidth per user.  
Bandwidth availability across the road network is modeled by introducing strategically positioned base stations in the city. These base stations create zones of varying bandwidth intensity, with roads within the inner radius of a station receiving maximum bandwidth. The bandwidth allocation decreases for each subsequent radius away from a station until the outer radius is reached. In cases where a road falls within the coverage area of multiple base stations, the bandwidth is determined by the strongest signal or highest value from among the relevant stations, thus optimizing data transfer capabilities in areas with dense coverage. An illustration of this can be seen in Figure 4.  
With the bandwidth and traffic parameters carefully defined and quantified, our experimental framework was equipped to assess the amount of data that an agent could feasibly transfer while traversing each road link within the urban network. We noted that the same MDP structure can be extended to additional objectives, such as electric vehicle battery management, by appending an energy state variable and corresponding actions; details are discussed in the Future Work Section.  
This comprehensive setup provides a robust foundation for testing and validating our reinforcement learning-based approach to autonomous vehicle path planning and data transfer in complex urban environments. Moreover, the methodology is highly generalizable and can be applied to other maps and datasets, provided that the relevant type of data is available.