###### Abstract
Furthermore, the performance of a caching scheme is affected by the fluctuation in data traffic load over time, which leads to waste of bandwidth [14]. To counter demand fluctuation issues, data placement during off-peak hours is a promising approach when it comes to meeting future users' data demands for smoothing data traffic load. This appears to be an efficient way to optimize bandwidth utilization. Hence, it is imperative to shift the backhaul data traffic burden from peak to off-peak hours and perform data placement close to the users based on the predicted data popularity profile. There are two categories of data caching schemes, i.e., static and dynamic caching schemes. In static caching schemes, cached contents remain the same [15]. Moreover, all the static caching schemes perform data caching based on the data popularity profile and the inter-file correlation is not exploited to improve the network performance [16]. However, dynamic caching schemes such as Least Frequently Used (LFU) and Least Recently Used (LRU) schemes update their cached contents with respect to time and new arriving contents to meet diverse users' data demands [17]. Also, stochastic network information can be exploited to optimize data placement. For instance, the diversity of data caching can be optimized by modeling the cache-enabled SBSs through homogeneous Poisson Point Process (PPP) [18]. In addition, users' mobility also poses significant challenges due to dynamic network connectivity and varying channel conditions; therefore, it is crucial to take users' mobility patterns into account [19]. Machine learning based caching schemes can be a potential approach to optimize 5G cellular network performance by learning and tracking users' mobility patterns and spatio-temporal data popularity profiles [20]. Motivated by the above mentioned prospects of data caching in a wireless network, in this paper we design an intelligent data caching approach to offload more data traffic through cache resources to minimize network delay. We implement data caching while considering users' mobility and diverse data demands that introduce significant challenges. In addition, the RL approach is devised to optimize data caching without requiring any prior information. Moreover, we validated our proposed caching scheme performance under various network parameters such as varying cache capacity, data popularity profiling, etc. The prominent contributions of our work are summarized as follows:* We design a data caching approach in a 5G network comprised of a hierarchical heterogeneous network to minimize network delay. Moreover, we consider practical assumptions such as inter-cell interference and users' mobility, which make our problem more arduous. * We model the data caching problem at the SBS level by considering evolving users' data demands patterns. The data demands and cache states are taken as system states. Both the transmission of cached contents and replacement of old contents with new and more popular contents constitute the system action. * We propose a Q-learning based data caching scheme following a dynamic data popularity profile while taking cache capacity constraints and users' mobility into account. It is notable that we formulate a tractable MDP in a mobility scenario, which is a highly challenging task. * Extensive simulations are conducted to demonstrate the performance of the propose caching scheme in minimizing network delay. The performance of our proposed Q-learning based caching scheme is compared with the four baseline caching schemes. These include learning based branch and bound (LB&B) [21], least frequently used content caching (LFUCC) [22], highly popular content caching (HPCC) [23], and random content caching (RCC) [24] schemes under different network parameters such as varying number of users, cache capacities, data popularity skewness, and data library sizes. The rest of the paper is organized as follows. Section 2 discusses the related works on data caching. Section 3 provides detail of the system model.