# A Reinforcement Learning-Based Congestion Control Approach for V2V Communication in VANET
## 1 Introduction  
Road traffic safety is a persistent issue that has been the subject of study for over 80 years [1]. The safety of motor vehicles, as the most widely used mode of transportation, is of utmost importance. Despite the global pandemic in 2020, an estimated 38,680 people lost their lives in motor vehicle crashes, marking the highest projected number of fatalities since 2007 [2]. In addition to driver behavior and attitudes, improving vehicle safety through inter-vehicular communication is a crucial aspect. Vehicular ad hoc networks (VANETs) [3] have gained recognition from government agencies, automobile industries, and academia as key components of intelligent transportation systems (ITS) [4] to enhance safety and efficiency on the road. VANETs enable direct communication between vehicles through onboard units (OBUs) or with infrastructure nodes such as roadside units (RSUs), thereby facilitating the dissemination of safety-related information. This information, including a vehicle's position, speed, and acceleration, is periodically broadcast as basic safety messages (BSMs) [5] (also referred to as cooperative awareness messages (CAMs) [6] in Europe) to surrounding vehicles, enabling safety-critical applications, such as collision avoidance, lane change warnings, and hard braking warnings. The timely and accurate dissemination of these messages is essential for the effective operation of various safety applications. The 5.9 GHz band has been allocated 75 MHz of spectrum for vehicle-to-vehicle (V2V) communication in VANET through dedicated short-range communications/wireless access for the vehicular environment (DSRC/WAVE) [7]. An additional 30 MHz of the spectrum is reserved for cellular vehicle-to-everything (C-V2X) communication [8]. Channel 172, with a bandwidth of 10 MHz, has been designated for vehicle safety and uses the 802.11p protocol [7], a contention-based random access MAC layer protocol. This type of communication can lead to simultaneous transmissions, causing packet collisions and reducing the reliability of communication [9]. As the number of vehicles increases, thebroadcasting of BSMs can easily lead to congestion on this single channel, resulting in lower reception probability and decreased transmission ranges. Traditional methods attempt to address this by controlling transmission parameters, such as the transmission power and transmission rate (also known as beacon rate), but this can also lead to reduced awareness and increased inter-packet delay (IPD) [10]. Although an optimal combination of transmission power and rate could potentially provide sufficient awareness while keeping the channel load below a specified threshold, the optimization problem is non-convex, making it challenging to solve, as noted in [11]. In vehicular ad hoc networks (VANETs), it is challenging to find optimal solutions for congestion and awareness due to the highly mobile nodes and dynamic environments. As an alternative, this paper approaches the problem as a decision-making task, where each vehicle must make the appropriate choice of transmission parameters for its safety messages based on the information it gathers from its surroundings. The transmission rate selection is formulated as a Markov Decision Process (MDP) [12], where each vehicle is modeled as an independent agent that interacts with its environment and selects its transmission parameters based on the current conditions. Reinforcement learning (RL) [13] is utilized to train the vehicles to make the right decisions, and the learning is based solely on the observations of the surrounding environment. After training, the vehicles will have learned to make optimal decisions in dynamic conditions, adapting to changes in vehicle density and channel load. The key contributions of this paper are:  
* A framework to solve the MDP using RL methods, with a focus on discrete action and state spaces.