# Delay-Tolerance-Based Mobile Data Offloading Using Deep Reinforcement Learning
## 3 Evaluation and Discussion
### Performance Comparison  
We evaluated the cellular infrastructure utilization efficiency of the proposed method. In this evaluation, the proposed method, the timewise-offloading of MDOP, and no-control are compared. We created a scenario different from the basic evaluation for this evaluation. Table 5 shows the scenario used for comparing the performance, and Figure 7 presents the topology of the scenario for comparing the performance. We opine that multiple applications have a delay tolerance, such as local synchronization of map data on the cloud and life-logging data collected by wearable devices. We assumed that the UE \begin{table}
\begin{tabular}{c c c} \hline \hline \multicolumn{1}{c}{**Excess Amount from \(L_{ideal}\) [MB]**} & \multicolumn{1}{c}{**Transmission Amount Sent within \(L_{ideal}\) [MB]**} \\ \hline No-control & 24.7 & 21.4 \\ Proposed method & 5.4 & 40.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Excess volume and transmission volume for basic evaluation. Figure 5: Learning result in basic evaluation: (**a**) total reward; (**b**) average q-value. Figure 6: QoS level assignment. is a smartphone or wearable device. Moreover, we assumed that there are four-types of delay tolerance contents from content-A to content-D in the mobile network. We created these contents according to [2,27]. The delay tolerance was provisionally set between 30 s to 120 s. In addition, we set the maximum transmission rate as 500 Kbyte/s. The UE's mobility is stationary, and the initial position is determined based on a uniform distribution. The total mobile data traffic volume is 1920 Mbytes in the evaluation environment. This mobile data traffic volume was 80% of the network topology capacity. We set the episode end time \(t_{end}\) as 600 in Algorithm 1 as the simulation time is 600 s.  
Figure 8 shows the training curves tracking the total reward and average q-value. We made the RL server learn the performance comparison scenario until 800 epochs or approximately 9,600,000 data. One epoch means that the scenario for comparing performance was executed once in the simulation. Based on Figure 8a, we confirmed that the total reward value increased as the number of epochs increased. Similarly, we confirmed that the average Q-value also tends to converge. From Figure 8, we confirmed that the RL server learned the performance comparison scenario. Therefore, we evaluated the load balancing performance using the learned model as compared with the no-control case and the timewise-offloading of MDOP. Table 6 shows the excess amount from \(L_{ideal}\) and the transmission amount sent within \(L_{ideal}\). As a result, the amount of traffic exceeds \(L_{ideal}\) significantly with no-control and causes the occurrence of the temporal locality. The occurrence of the temporal locality degrades the quality of the communication because \(L_{ideal}\) is set to avoid situations wherein the demand of the traffic causes packet loss. In contrast to no-control, the proposed method and the timewise-offloading of the MDOP reduce the traffic excess, and the proposed method reduces the traffic excess amount most. Furthermore, we confirmed that the proposed method reduced the excess data amount by 35% as compared with the time-wise offloading of MDOP.