## 6 Performance Evaluation  
A realistic mobility trajectory collected in Korea Advanced Institute of Science & Technology (KAIST), Korea is devised to model mobility patterns of users. The mobility patterns of users are observed through Garmin GPS 60 CSx receivers. We exploit the \(\epsilon\) greedy policy to maintain a balance between exploration and exploitation of action in order to maximize reward. The \(\epsilon\) can be adjusted from (0, 1) [35]. The value of \(\epsilon=0.5\) is set to explore the actions to maximize the reward. Moreover, \(\psi=0.6\) and \(\alpha=0.7\) is set. All the simulations are performed with Matlab R2017a. All simulation parameters are mentioned in Table 2.  
The performance of our proposed RL-based caching mechanism is validated against state-of-the-art caching schemes:  
* _Learning based Branch & Bound (LB&B)_: LB&B enables learning from patterns of users' data demands based on their data viewing preferences. * _Least Frequently Used Contents Caching (LFUCC)_: The frequency of users' data requests is followed by a diverse data popularity profile. Therefore, LFUCC devised data demands frequency and content popularity to improve data access via cache resources. * _Highly Popular Contents Caching (HPCC)_: HPCC enables caching highly demanded contents by the users to conserve limited cache resources. * _Randomized Contents Caching (RCC)_: RCC exploits caching randomly to reduce the complexity without reducing data service quality. In Figure 3, an impact of data library size is observed on data access delay via cache resources. The obtained results depict a low delay for small numbers of contents because there are data requests for only these contents. These small number of contents can be accommodated in the limited cache resources to minimize data access delay. However, when number of contents is large, then users have more choices of contents that cause diversity of data demands. This creates a challenge in optimizing data caching to improve data access \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Notation** & **Value** \\ \hline Square area of small cell & \(A_{sc}\) & 250 m \(\times\) 250 m \\ Bandwidth & \(W\) & 20 MHz \\ Transmission power of SBS & \(P_{g}\) & 30 dBm \\ Background noise & \(\sigma_{\mu}^{2}\) & \(-174\,\mathrm{dBm}/\mathrm{Hz}\) \\ Discount factor & \(\alpha\) & 0.7 \\ Learning rate & \(\psi\) & 0.8 \\ Popularity skewness & \(\gamma\) & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: System Parameters. via cache resources. In the case of failure of data access via cache resources, contents will be fetched from the core network with a large delay. Our proposed cache scheme achieves low delay of data access by exploiting Q-learning and users' mobility patterns. When there are 300 contents, our proposed scheme achieves 4%, 8%, 9.8%, and 13.5% lower delay than LB&B, LFUCC, HPCC, and RCC, respectively. Hence, performance gain of the proposed caching scheme is high by employing intelligent cache decisions based on available cache resources and users' data demands. In Figure 4, the influence of the data popularity profile is observed on the cache decision of our proposed caching mechanism. Popularity skewness is utilized for the aforementioned reason. From the obtained results it is obvious that when popularity skewness value is high, then data access delay is low. This low delay is due to the fact that fewer contents are responsible for constituting data traffic that can be placed in the cached resources. This improves the potential of data access via cache resources. From the obtained results it is evident that our proposed caching mechanism outperforms all the baseline schemes. The delay is greatly reduced when popularity skewness is varied from 0.5 to 5. However, after popularity skewness equal to 2.5, the delay response is stable because contents generating data requests are already cached.