# A Reinforcement Learning-Based Congestion Control Approach for V2V Communication in VANET
## 2 Related Work  
VANETs are constantly evolving to support a diverse range of applications, including both critical safety applications, such as forward collision warnings and traffic signal violation warnings, and more comfort-oriented applications, such as weather information systems and restaurant recommendations [14]. These safety applications require the reliable delivery of both alert messages and the periodic broadcast of BSMs containing information, such as a vehicle's position, speed, and heading. However, with limited channel capacity and high transmission rates required for maintaining awareness, the reliable delivery of BSMs can pose a challenge in VANETs. In this section, we will explore both classical approaches and recently proposed approaches. The nominal rate for transmitting BSMs is 10 packets per second. One well-known rate-based congestion control method is the linear message rate integration control (LIMERIC) algorithm [15]. Its objective is to fairly distribute the available channel bandwidth among all vehicles by dynamically adjusting the BSM transmission rate in each iteration. LIMERICuses linear feedback to adapt the transmission rate instead of the limit cycle behavior. In LIMERIC, vehicles in a given region, i.e., a single collision domain, where all the nodes measure the same channel load, sense and adapt their transmission rates to meet a specified CBR. LIMERIC is a distributed algorithm, executed independently by each station in a neighborhood. While LIMERIC effectively reduces congestion, it can result in an increased IPD from neighboring vehicles. The error model-based adaptive rate control (EMBARC) algorithm [16] improves LIMERIC with the capability to preemptively schedule messages based on the vehicle's movement, i.e., a kind of suspected tracking error technique. This approach can provide extra transmission opportunities for highly dynamic vehicles, which can therefore reduce instances of large tracking errors. Another rate-based approach, BSM rate control over IEEE802.11p vehicular networks (BRAEVE) [17], uses the estimated number of vehicles as the input parameter, leading to smoother convergence and lower packet error ratio, IPD, and tracking error compared to other algorithms. In a previous study [18], researchers introduce a congestion-aware message (CAM) for beacon signals in the vehicle environment that utilizes vehicle IDs. The CAM model incorporates the unique automobile IDs into the back-off procedure, weighting the randomized back-off numbers chosen by each vehicle. This results in the generation of car ID-based randomized back-off codes that reduce the risk of collisions caused by identical back-off numbers. In this paper, BSMs as the standard safety messages are not considered. The proposed scheme in [19] first analyzes congestion detection schemes and then utilizes a priority model to adjust the transmission rate of beacon messages. The Tabu search algorithm was employed to control network congestion. This paper does not show the performance in terms of CBR which is a critical metric to evaluate the channel load. In their paper [20], the authors present a solution to the message rate control problem by formulating it as a Markov decision process and using on-policy control with function approximation in reinforcement learning (RL). This approach enables vehicles to take appropriate actions quickly, with low computational cost, and converge in a short time. However, the action space consists of only three actions which severely limits the vehicle's ability to mitigate congestion. In addition to the transmission rate, another adjustable parameter for congestion control is transmission power. By increasing the transmission power, a packet can reach a greater distance and potentially more vehicles, enabling control over the number of vehicles a BSM can reach. Transmission power is typically calculated based on the measured or estimated level of channel load. A widely cited power control algorithm is the distributed fair transmit power adjustment for VANETs (D-FPAV) [21].