# Environment-Aware Adaptive Reinforcement Learning-Based Routing for Vehicular Ad Hoc Networks
## 1 Introduction  
With the increasing demand for intelligent transportation systems (ITSs) to enhance road-traffic efficiency and reduce traffic accidents, vehicular ad hoc networks (VANETs) are expected to attract more interest in research and industry. They leverage communication between vehicles for information exchange and collaborative operations, thereby offering significant benefits in areas such as traffic management, sharing road-condition information, traffic-flow optimization, and emergency communications [1; 2; 3; 4; 5]. Over the past decade, many studies have attempted to optimize the performance of VANET routingprotocols [6; 7; 8; 9]. Despite the utility of VANETs, they come with limitations and challenges. In VANETs, data transmission faces various difficulties, including frequent changes in network topology, intermittent link connectivity, abundant environmental interference, limited channel resources, and more. These challenges can significantly impact the user experience of vehicular applications and may not meet diverse communication service-quality requirements. While there are many traditional mobile ad hoc network (MANET) routing protocols to choose from, the challenges posed by VANETs are often insurmountable for traditional routing approaches. On complex urban roadways, vehicles typically move freely at speeds of approximately 50 km/h. The transmission paths in VANETs require support for multihop communication [10], and, as a result, many traditional routing protocols need to maintain paths from source to destination. When vehicles move rapidly, the frequent changes in network topology further increase the maintenance cost of the entire network, thereby exacerbating network congestion and making it difficult to establish effective communication links. The applications of VANETs demand reliable and robust data transmission. Routing protocols have a crucial impact on the network's stability and performance. Location-based routing protocols [11; 12] hold significant development potential in dynamic network environments. By utilizing the global positioning system (GPS) or digital map to acquire their location coordinates and enable the sharing of location information among nodes, routing based on location can mitigate the impact of high node mobility. It possesses simplicity and scalability features. However, in urban networks, location-based routing protocols also suffer from frequent link failures and routing voids, thereby limiting the achievement of better performance. In routing, forwarding nodes need to compromise multiple factors to select an appropriate next-hop relay towards the destination node. Choosing the right relay nodes can reduce the number of hops, transmission delay, and packet loss from the source node to the destination. Traditional location-based routing employs a greedy gf strategy [13], which satisfies the requirements for low hop count and low latency. However, the high-speed movement of vehicles leads to nonuniform node distribution and communication-link disruptions. Additionally, the urban road structure imposes constraints on the topology of vehicular networks. The presence of intersections, as well as shadow attenuation caused by vegetation and buildings, significantly reduces signal strength, thereby impacting routing performance. Therefore, location-based routing protocols face challenges in adapting to the highly dynamic topology and complex working environment of VANETs. Figure 1 shows the routing challenges present in VANETs. Therein, vehicle A and vehicle B are located in the effective communication range with each other, but the building restricts the possibility of nodes communicating with each other and needs to be relayed through vehicle B. Vehicle E waits to pass due to the presence of traffic lights and is disconnected from vehicle B in the routing table. However, the routing table may not be updated in time, thereby resulting in packet loss.