# Reinforcement Learning for Energy Optimization with 5G Communications in Vehicular Social Networks
## 5 Results and Discussion
The cumulative distribution functions (CDFs) of SINR received by vehicles for different D2D distance thresholds are compared in Figure 5. Figure 5a-d shows the distribution of the received SINR in each algorithm. In all panels, the proposed algorithm has more vehicles that receive higher SINR compared to BA, Compare1, and Compare2. The proposed algorithm and Compare1 can select the communication mode and transmission power to maximize the system energy efficiency, allowing vehicles to select more D2D modes. It can increase spectral efficiency and reduce system energy consumption, but it can also decrease the received SINR. However, the proposed algorithm is designed to select the mode and transmission power to ensure the received SINR. Therefore, the proposed algorithm has more vehicles, receives higher SINR, and performs better with an achievable data rate compared to Compare1. The BA and Compare2 decide the communication mode based only on the expected SINR of the vehicle, so they can achieve higher performance in a longer D2D distance threshold. The comparisons of system energy efficiency as a reward function with an increasing number of learning iterations are shown in Figure 6. In Figure 6a it can be seen that as the number of learning iterations increases, the value of system energy efficiency stabilizes after 100 iterations. In heavy traffic scenario, as shown in Figure 6b, the value of the system energy efficiency stabilizes after 100 iterations. The system energy efficiency is overserved with a low value at the beginning of the learning iterations  
Figure 4: Comparison of average achievable data rate with various D2D distances and system loads: (**a**) 250 m with light traffic; (**b**) 350 m with light traffic; (**c**) 250 m with heavy traffic; and (**d**) 350 m with heavy traffic. and increases until it reaches a stable point. This means that the proposed algorithm updates its policy optimally and converges faster and in a more stable manner than the other algorithms. The convergence time of the proposed algorithm is longer than those of Compare1 and Compare2. This is because the proposed algorithm has longer decision process and considers more states in decision process. It finally decides an action by comparing two recommended actions. In addition, it considers available resources and interference level as state to make a decision. However, Compare1 and Compare2 consider only interference level as state. Although the convergence time of the proposed algorithm becomes a little bit longer, the proposed algorithm shows better performance in terms of system energy efficiency. To show the comparison with the optimal solution, the performance values are shown in Tables 2 and 3. In Table 2, the average system energy efficiencies with system load variance are described. Figure 5: Comparison of signal-to-interference noise ratio (SINR) with various D2D distances and system loads: (**a**) 250 m with light traffic; (**b**) 350 m with light traffic; (**c**) 250 m with heavy traffic; and (**d**) 350 m with heavy traffic. Figure 6: Convergence of system energy efficiency as a reward function with D2D distance = 350 m: (**a**) light traffic scenario; (**b**) heavy traffic scenario. Based on the optimal solution, the proposed algorithm, BA, Compare1, and Compare2 obtain 44%, 20%, 35%, and 20% of the system energy efficiency, respectively. In Table 3, the average achievable data rate with system load variance is described. Based on the optimal solution, the proposed algorithm, BA, Compare1, and Compare2 obtain 79%, 68%, 50%, and 66% of the achievable data rate, respectively. This shows that the proposed algorithm performs closest to the optimal solution among other compared algorithms because it adopts the system energy efficiency and received SINR as rewards. However, as the system load increases, the differences in performance increase. This is because the interference increases according to increasing system load environment.