# Reinforcement Learning Approach for Adaptive C-V2X Resource Management
## 3 Proposed Reinforcement Learning Approach for Adaptive Resource Management  
### Impact of MCS Index  
This work simulated the C-V2X network and communications using the LTEV2 Vsim version 5.4 simulator [36]. The LTEV2 Vsim simulator is based on MATLAB R2022a software designed to investigate the C-V2X resource management research. The radio resource parameters are briefly explained in Table 1. We simulated the 1 km highway scenario with one lane in each direction. The vehicle's speed was set at 120 km/h with a 40 km/h standard deviation. Two kinds of vehicle density were simulated, one with the total number of vehicles, \(n_{V}=100\) vehicles, and another one with \(n_{V}=200\) vehicles. The simulation time (\(T\)) was set to 60 s, with the position update time resolution (\(t_{r}\)) updated every 0.1 s. The channel bandwidth (\(B_{c}\)) was set to 10 MHz. We set the transmission power to 23 dBm for this work. The packet received ratio (PRR) was used to investigate the overall statistical performance. The PRR performance was observed against the vehicle's 10-150 m distance. The overall PRR performance statistic can be obtained by:  
\[PRR=\frac{Number\ of\ successfully\ received\ packets}{Total\ number\ of\ packets}, \tag{4}\]  
PRR values reflect the statistical data of the total number of successfully received packets divided by the total number of packets. The total number of packets includes the retransmission packets. The transmitting vehicle and the designated receiving vehicle are assigned randomly. For the given simulation time (\(T\)), we recorded the average total number of packets for \(n_{V}=100\) vehicles, which was 60,000 packets, and for the \(n_{V}=200\) vehicles scenario, which was 120,000 packets. The vehicle position update (\(t_{r}\)) was carried out at each 0.1 s. Suppose a packet transmission occurs within the 0.1 s space. In that case, the transmission number will be wrapped up for a more straightforward calculation. Following the 3 GPP mode-4 mechanism, every vehicle will be assigned a random resource reselection counter. This reselection counter will then decrease by one for every packet transmission. When the counter reaches zero, the vehicle will be given a chance to select a new radio resource for the following transmissions. The probability resource keep (\(P_{rk}\)) value dictates the chance of selecting the new radio resource. The higher the \(P_{rk}\) value, the greater the chance to select the same radio resource previously assigned and vice versa. Considering the randomly of the simulation works, we have yet to track the performance of individual vehicles. In the simulation, an RB matrix (RRtable) represented the LTE frame grid. The RBtable consisted of how many RBs were allocated for transmission according to the MCS index. The radio resource management uses a slot-based approach. Other matrices were also created to accommodate the radio calculation and the position update. Every vehicle selected for a packet transmission was paired with the corresponding transmitting and receiving flag. The transmission direction flag was then used to represent the uplink (UL) and downlink (DL). Every vehicle shared the same RBtable to simplify the mode-4 sensing mechanism purposely. In this study, we focused on how radio resource management works to provide successful C-V2X communication. The radio resource management emphasized how many resource blocks would be provided to each vehicle in the system, which was governed by the MCS index, to reach the highest PRR. Some related works used particular MCS index values in their investigation. We conducted our initial investigation referencing some works that we have collected so far. Table 2 shows the corresponding related works and the MCS index value used within the report. The 3 GPP stated that the MCS index values range from 0 to 28. Thus, the RL algorithm we used was the MCS index value from 0 to 28 in this work. We investigated various Q-learning parameters in this work.