Enhancing the Minimum Awareness Failure Distance in V2X Communications: A Deep Reinforcement Learning Approach  
[PERSON]  
1 Department of Electrical Engineering, Pusan National University, Busan 46241, Republic of Korea;  
[EMAIL_ADDRESS] (A.K.G.L. ); [EMAIL_ADDRESS] (H.-H.N.) [PERSON]  
1 Department of Electrical Engineering, Pusan National University, Busan 46241, Republic of Korea;  
[EMAIL_ADDRESS] (A.K.G.L. ); [EMAIL_ADDRESS] (H.-H.N.) [PERSON]  
2 Intelligent Systems Research Lab, Intel Labs, Intel Tecnologia de Mexico, Jalisco 45017, Mexico;  
[EMAIL_ADDRESS]  
[PERSON]  
4 Department of Automobile and IT Convergence, Kookmin University, Seoul 02/07, Republic of Korea;  
[EMAIL_ADDRESS]  
enhance situational awareness. Our research focuses on cooperative awareness via beaconing, emphasizing its critical importance in preventing vehicle-to-vehicle (V2V) collisions through the accurate and immediate sharing of information among neighboring vehicles. Despite the extensive research on the beaconing problem over the past two decades, there is still no consensus on the precise definition of awareness in V2X communications [6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20]. Typically, awareness is viewed monolithically as a vehicle's ability to perceive and share real-time kinematic information about itself and neighboring vehicles to enhance safety and efficiency [1; 2]. Based on this view, the initial approaches either control transmit power (TP) [6] or beacon rate (BR) in [7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20] to maintain an appropriate beacon load in dynamic vehicular environments. Later, these approaches are extended to a joint approach with BR and TP in [11; 12; 13; 14]. However, all these approaches focus on access-layer performance neglecting the key requirements of vehicle awareness. Some approaches have used direct [17] or indirect [15; 16] tracking metrics to measure discrepancies between the actual and estimated kinematics of vehicles. While these schemes help ensure that surrounding vehicles are adequately tracked, they fail to consider neighbor vehicles that do not receive any beacon messages, which poses a significant threat to driving safety by potentially leaving undetected vehicles unaccounted for in the V2V collision avoidance process. To address these limitations, we argue that awareness should be understood as a multifaceted concept involving vehicle detection, vehicle tracking, and the minimum awareness failure distance (mAFD), defined as the shortest distance within which all neighbor vehicles are detected and successfully tracked. The mAFD is crucial as it represents the closest distance to any undetected or untracked vehicles that pose a high risk of V2V collisions. By this definition, increasing the mAFD is essential for enhancing driving safety. To achieve this, we propose a novel deep reinforcement learning (DRL) framework for the joint control of beacon rate and transmit power (DRL-JCBRTP) that models beaconing as a Markov decision process (MDP). Based on locally observable state information, a vehicle (agent) determines its BR and TP (action) to optimize our intricately crafted reward functions (return), considering factors such as awareness, beacon aging, and channel congestion. This DRL framework is applicable to any model-free DRL scheme, making it a versatile tool for enhancing beaconing performance. We employ the Soft Actor-Critic (SAC) [21] algorithm due to its sample efficient learning and stability. Furthermore, we present and analyze different neural network (NN) architectures within the SAC framework to evaluate their impact on the performance of the DRL-JCBRTP scheme. The contributions of this paper are summarized as follows:  
Figure 1: Example of beaconing and event notification. * We present a new definition of awareness in V2X communications, framing it as a multi-faceted concept involving vehicle detection, tracking, and safe distances.