# A Hybrid Multi-Agent Reinforcement Learning Approach for Spectrum Sharing in Vehicular Networks
## 5 Simulation Results and Findings  
This section presents the simulation results, thus demonstrating the effectiveness of our proposed multi-agent RL-based spectrum sharing scheme. We utilized a custom-built simulator adhering to the evaluation methodology outlined in Annex \(A\) of 3 GPP TR 36.885, thus considering critical parameters such as vehicle drop models, traffic densities, speeds, directional movements, and channel characteristics for comprehensive analysis. Table 1 lists the simulation parameters reflecting realistic vehicular communication scenarios. The parameters like power levels, frequencies, noise characteristics, and speed were precisely chosen for their alignment with real-world scenarios of vehicular communication. Additionally, Table 2 specifies the channel models for both V2I and V2V links. Moreover, in Table 2, \"B1 Manhattan\" refers to the typical environment or scenario being considered, especially the metropolitan environment of Manhattan. In the context of wireless communication modeling, assuming the unique characteristics of urban surroundings such as Manhattan is fundamental due to aspects like high population density, tall buildings, and complex propagation conditions. All the parameters in Tables 1 and 2 were set to default values for the simulation. However, meticulous attention to specific configurations for each figure ensures precision and coherence in the results. The simulations were carefully designed to authenticate the effectiveness and efficiency of our proposed resource sharing scheme in dynamic and realistic vehicular environments. The neural architecture for each V2V agent's DQN comprises three fully connected hidden layers with 500, 250, and 120 neurons. Inside the hidden layers, the ReLU activation function, expressed as \(f(x)=\max(0,x)\) is utilized. This activation function was selected to introduce non-linear transformations, thereby enabling the network to effectively capture \begin{table}
\begin{tabular}{l l} \hline \hline
**Parameter** & **Value** \\ \hline Number of V2I links \(M\) & 4 \\ Number of V2V links \(K\) & 4 \\ Carrier frequency & 2 GHz \\ Bandwidth & 4 MHz \\ BS antenna height & 25 m \\ BS antenna gain & 8 dBi \\ BS receiver noise figure & 5 dB \\ Vehicle antenna height & 1.5 m \\ Vehicle antenna gain & 3 dBi \\ Vehicle receiver noise figure & 9 dB \\ Absolute vehicle speed \(v\) & 36 km/h \\ Vehicle drop and mobility model & Urban case of A.1.2 in [29] \\ V2I transmit power \(P^{c}\) & 23 dBm \\ V2V transmit power \(P^{d}\) & [23, 10, 5, 0] dBm \\ Noise power \(\sigma^{2}\) & \(-\)114 dBm \\ Time constraint of V2V payload \(T\) & 100 ms \\ V2V payload size \(B\) & [1, 2,...] \(\times 10^{6}\) bytes \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation parameters [29]. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Parameter** & **V2I Link** & **V2V Link** \\ \hline Path loss model & 128.1 + 37.6 log\({}_{10}\)\(d\), \(d\) in km & LOS in WINNER + B1 Manhattan \\ Shadowing distribution & Log-normal & Log-normal \\ Shadowing standard deviation \(\xi\) & 8 dB & 3 dB \\ Decorrelation distance & 50 m & 10 m \\ Path loss and shadowing update & A.1.4 100 ms & A.1.4 100 ms \\ Fast fading & Rayleigh fading & Rayleigh fading \\ Fast fading update & Every 1 ms & Every 1 ms \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Modeling the wireless channels for V2I and V2V communications [29]. intricate patterns and features from the input data. Throughout the training process, the network parameters were updated using the RMSProp optimizer with a learning rate set at 0.001. Additionally, the network was trained using the Huber loss function, thus enhancing its ability to handle outliers and noisy data during learning. In Figures 5 and 6, we compared the QMIX resource sharing scheme with the following three baseline methods:  
1. The multi-agent RL-based algorithm where multiple agents update their actions in a distributed way. 2.