# An Adaptive Hybrid Automatic Repeat Request (A-HARQ) Scheme Based on Reinforcement Learning
## 5 Performance Evaluation
### Performance Analysis  
This section presents an analysis of the performance evaluation of the proposed A-HARQ scheme. The evaluation encompasses simulations conducted within a heterogeneous network environment. The assessment is performed in terms of the BLER, re-transmission times, and RTT. The simulation results of the proposed A-HARQ scheme are compared with that of the 5G HARQ scheme. This part only analyzes the results of selecting the optimal transmission scheme for the state that does not meet the requirements of the BLER (i.e., actions A1-A8), because the state that meets the requirements of the BLER maintains the transmission scheme of 5G HARQ (i.e., action A0).  
This study guides agents to optimize effective information by calculating rewards according to the feedback of system BLER changes after performing actions and adjusting learning rate strategies. The design of the reward mechanism can overcome the problem of sparse returns and can converge. By utilizing the available prior knowledge data for offline  
Figure 5: State selection diagram.  
training, a convergent basic Q-Table is derived. After tens of thousands of times of training, the agent learns the action strategy that can obtain the biggest reward, and the training data in each state will have an obvious tendency to converge to a certain action. Through the offline training simulation, the optimal Q-Table is obtained, and the action corresponding to the maximum Q value in each state is the optimal action. Figure 6 illustrates the ultimate optimal information transmission scheme corresponding to states S1 to S400, respectively.  
According to the final optimal action table, an online decision is made, and performance analysis is conducted on the entire sample space, which consists of 400 states. Figure 7 illustrates the downlink average BLER performance of the A-HARQ scheme and the 5G HARQ scheme. The simulation results demonstrate that the BLER performance of the proposed Adaptive Hybrid Automatic Retransmission Request Technical Solution (A-HARQ) is significantly better than that of the 5G HARQ. The A-HARQ scheme achieved a reduction in the maximum BLER from 11.48% to 10.22% compared with the HARQ scheme. The overall downlink average BLER decreased by 4.8 percentage points from 10.35% to 5.55%, which also means that the average information transmission accuracy performance increased by 46.41%.  
Figure 8 depicts the performance comparison of RTT and retransmission times between the A-HARQ scheme and the HARQ scheme. The simulation results demonstrate that that the A-HARQ scheme enables a reduction in the RTT and retransmission instances compared with the HARQ scheme, as shown in Figure 8. The final RTT is shortened by 35.23% and 56.82%, respectively, on average, i.e., from 2.4 ms and 3.6 ms to 1.55 ms, respectively. The average reduction in the number of retransmissions is 2.31% and 51.16% for the first and second cases, respectively. Specifically, the number of retransmissions decreased from 1 to 0.97 for the first case, and from 2 to 0.97 for the second case.  
Figure 6: Action distribution diagram for each state.  
Figure 8: RTT and the number of retransmissions performance comparison between 5G HARQ and A-HARQ.  
Figure 7: Average downlink BLER performance comparison between 5G HARQ and A-HARQ.