## 5 Numerical Results
### Simulation Framework, Training, Testing, and Deployments  
We developed the SLMLab-Gym-VEINS simulation framework to train and evaluate the DRL-JCBRTP policy, as illustrated in Figure 5. This comprehensive setup merges the VEINS simulation framework--which itself is a fusion of the SUMO mobility simulator and the OMNeT++ network simulator--with the OpenAI-Gym framework to support SLMLab [32; 33; 34; 35; 36]. Our simulation environment, designed for both training and testing phases, features a 2 km highway segment with multiple vehicle density scenarios. Training was conducted across various vehicle densities and inter-vehicle distances, with vehicles operating under a car-following model. To accurately capture wireless channel characteristics in high-mobility scenarios, the Nakagami-m fading model was employed.  
During the training phase, a single agent (the red vehicle in Figure 5) was responsible for policy updates, while all other agents (white vehicles) acted based on this policy. This was facilitated by dual SAC networks: one network updated its weights based on observations from the red vehicle at the beginning of each frame, incorporating recent policy adjustments from the red vehicle's experiences. Subsequently, the second network in each white vehicle adopted these updated weights and performed inference using its own state information.  
Training was conducted over 32,000 frames, with each frame representing a step in the simulation. Each scenario was treated as an episode lasting 160 frames, after which the system randomly selected a new scenario to expose the agent to a diverse set of conditions. This approach helped generalize the policy across different densities and inter-vehicle distances.  
In the testing phase, our scenarios were executed using the dual SAC networks exclusively for inference, with no further training. The networks' beaconing performance was assessed by comparing it to other state-of-the-art schemes across various metrics, including BR, TP, and CBR adaptability. These metrics, which will be discussed in detail later, were used to evaluate the strengths and weaknesses of our approach and to demonstrate its ability to increase mAFD relative to existing methods. This ensures that our networks are robust, efficient, and effective across a variety of densities and inter-vehicle distances.  
Regarding the deployment of DRL-JCBRTP in real vehicles, it is important to note that while exploration and exploitation are integral during the training phase, the deployment phase focuses solely on inference. The training phase is conducted using the SLMLab-Gym-VEINS simulation framework, which allows for safe and comprehensive exploration within a simulated environment. In a real-world deployment, the trained NN is integrated into the vehicle's systems for real-time decision-making based on the learned policies. The NN processes real-time data from the vehicle's sensors and V2X communications to make quick, accurate decisions regarding beaconing strategies. This allows the vehicle to adapt to dynamic traffic conditions and optimize communication efficiency without the need for further exploration, which mitigates the high costs and risks associated with exploration in real environments. Through this approach, DRL-JCBRTP enhances the vehicle's ability to maintain robust and reliable communication, ultimately contributing to improved traffic management and collision avoidance.  
Figure 5: SLMLab-Gym-VEINS simulation framework.