# Environment-Aware Adaptive Reinforcement Learning-Based Routing for Vehicular Ad Hoc Networks
## 3 Preliminaries
### Motivation Scenario  
To address the limitations of the previous research, the study introduced an adaptive perception V2V routing protocol based on Q-learning. In this protocol, we broadcast our own state through beacon packets, and receiving nodes extract information about neighboring nodes, including vehicle IDs, positions, received signal strengths, available bandwidth, and other relevant parameters. By extracting beacon data from neighboring vehicles, training the Q table, and selecting vehicles with the maximum Q values, this study identifies the optimal or suboptimal paths. This routing approach eliminates the need for additional RSUs or infrastructure, thereby reducing the hardware requirements in the working environment. Compared to RL routing protocols that partially rely on the geographical environment as states [16; 20; 39; 41], the entire algorithm is relatively easy to implement. For example, the protocol needs to divide the network environment into grids and choose the next hop based on forwarding policies within the optimal next grid. However, grid division heavily relies on the urban environment, as intersections and buildings within the grids affect signal transmission. Alternatively, it depends on forwarding data at intersections, assessing road conditions, and then forwarding packets to the evaluated optimal road segment. This method mitigates sudden changes in vehicle direction and signal attenuation caused by building obstruction. However, its reliance on RSUs and traffic-aware messages restricts its applicability. This study describes a VANET scenario that comprises a vehicle network and a base station (BS) for real-time traffic information, road conditions, and accident reports, as depicted in Figure 2. The BS serves as the destination node, while the vehicles distributed across various road segments are considered as the source nodes for data transmission. Data packets are relayed through other vehicle nodes. Vehicles report their own driving status, traffic conditions on their respective road segments, and any sudden accidents they encounter. The BS is responsible for collecting vehicle and road-condition data and uploading it to a traffic management server, which facilitates various research or application platforms such as traffic-flow optimization, emergency assistance, and route planning. This application scenario considering the design of a distributed data collection server that aligns with geographical distribution characteristics. For instance, a city map can be divided into multiple regions, with each region assigned to an RSU or BS for data collection. Each server would be connected to RSUs or BSs and be responsible for one or more regions. Figure 2: VANET scenario. The division of regions can follow a cellular structure design; further design details are beyond the scope of this paper's discussion. The working environment of VANETs inherently exhibits high levels of randomness and complexity. Designing routes that align with the current application scenarios is extremely challenging, and these challenges require special consideration when devising effective communication and data-transmission solutions. For instance, the high dynamism of vehicles necessitates routing algorithms to rapidly adapt to changes in the network topology. Routing algorithms must effectively address unreliable communication environments, including issues such as signal obstruction, multipath propagation, and signal attenuation. Sensitive information is involved in self-organizing networks; hence, routing algorithms must account for privacy protection and security concerns. Effective spectrum resource management is crucial to support communication among a large number of vehicles. Additionally, meeting the unique requirements of vehicular networks may necessitate crosslayer design and other related challenges. This research places a particular emphasis on the dynamism of vehicles, shadow fading, and effective bandwidth to identify the most suitable paths between source and destination nodes.