# Data-Aware Path Planning for Autonomous Vehicles Using Reinforcement Learning +
## 3 Problem Formulation
### Constructing the Environment  
We modeled the environment as a graphical road network, constructed using the GraphML framework. This model allows for the integration of real-world map data and vehicle mobility patterns, providing a realistic and dynamic setting for the evaluation of our RL-based path planning strategy. In this graphical network, roads are depicted as connections (edges) between nodes (intersections), where each node has a series of attributes. These include their precise coordinates, lengths of adjoining roads, the average duration vehicles occupy each road, vehicles' average speed on each road, and a congestion factor that measures traffic density levels on each road.  
Figure 2 illustrates the environment creation process. To achieve this, we used a vehicle mobility dataset (This dataset was obtained from the University of Southern California's Autonomous Networks Research Group [[[https://anrg.usc.edu/www/downloads/](https://anrg.usc.edu/www/downloads/)]([https://anrg.usc.edu/www/downloads/](https://anrg.usc.edu/www/downloads/)) (accessed on 5 July 2023)]) consisting of time-stamped location coordinates of 3000 taxis in Beijing over a 24 h period. This dataset served as the basis for modeling this graphical road network.  
We began preprocessing by extracting the full trajectory of each taxi and map-matching every GPS fix to OpenStreetMap using Valhalla ([[https://github.com/valhalla/valhalla](https://github.com/valhalla/valhalla)]([https://github.com/valhalla/valhalla](https://github.com/valhalla/valhalla)) (accessed on 13 November 2023)). For each vehicle, we recorded the sequence of visited nodes (intersections) and computed the distance, speed, and travel time between consecutive nodes. A point was tagged stationary when its instantaneous speed remained below 3 km/h for at least 60 s; such points, typically parking or long red light stops, were removed so that the analysis reflected only in-motion segments. Congestion factors were calculated as the ratio of observed-to-free-flow speed, aggregated in 10 min windows. Assuming bidirectional travel on all roads, we consolidated nodes that lay on the same heading (within \(\pm 5^{\circ}\)) and were separated by less than 20 m, thereby reducing graph complexity without altering connectivity.  
Figure 2: Environment creation process.  
After preprocessing the user data in the dataset, we created a JSON file that recorded the number of edges connected to each node. This JSON file, together with the preprocessed data, was used to build a detailed representation of vehicle mobility patterns on roads using the GraphML framework. Figure 3 displays the entire graphical network, which represents not only the physical layout of the road network but also the dynamic factors such as traffic congestion and speed patterns. The top graph shows a birds-eye overview of the entire Beijing graph, confirming that the extracted topology matches the street grid. The bottom graph shows a zoomed-in portion of the map, highlighting node consolidation and edge connections after preprocessing. This graphical model forms the basis of our reinforcement learning environment, facilitating the training and testing of reinforcement learning algorithms under realistic traffic conditions.  
Figure 3: Bird’s-eye overview of the entire Beijing graph (**top**) and a zoomed-in portion of the graph (**bottom**). Both panels serve purely as visual context.