## 2 Related Works  
There is an unprecedented growth in data traffic on online social apps and platforms that strained the limited backhaul resources of wireless networks. Therefore, [PERSON] et al. [25] proposed a proactive data caching mechanism to support interest-based user grouping for data sharing, while taking into account the complete information from the social graph for one-to-one interaction between users. The data caching entities are selected based on users' profiles and revenue status. Then, the users' group membership is approximated by utilizing users' data demands logs without any prior information from the  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Description** \\ \hline \(s\) & Small base stations \\ \(u\) & Users \\ \(l_{s,u}\) & Binary element to determine user \(u\) connectivity with serving SBS \(s\) \\ \(F\) & Number of contents \\ \(\gamma\) & Popularity skewness \\ \(x_{s,f}\) & Decision variable to determine whether content \(f\) is cached by SBS \(s\) or not \\ \(t_{s,u}\) & Communication time between user \(u\) and SBS \(s\) \\ \(R_{u}\) & Data transmission rate \\ \(W\) & Bandwidth \\ \(P_{s}\) & Transmission power of SBS \(s\) \\ \(d_{u,s}\) & Distance between user \(u\) and SBS \(s\) \\ \(T_{uf}\) & Time slots required for successful transmission of content \(f\) \\ \(D_{uf}\) & Delay experienced by user \(u\) in receiving content \(f\) from SBS \(s\) \\ \(D^{\prime}\) & Data transmission delay from core network to SBS \(s\) \\ \(V\) & State value function \\ \(\psi_{st}\) & Transition reward of SBS \(s\) at time \(t\) \\ \(\alpha\) & Discount factor \\ \hline \hline \end{tabular}
\end{table}
Table 1: Notations.  
overlaid users' social graph. The obtained results demonstrated a 30% to 34% performance gain in data access via cache resources. In [26], the authors proposed a concept of data caching at aerial base stations and defined delay tolerant and delay sensitive users which have different data transmission rate requirements. The authors proposed an iterative algorithm that applies a decomposition method. The obtained results showed a significant improvement in utilization of backhaul resources.  
Nowadays, many researchers are exploiting the potential of machine learning to optimize data caching in wireless networks. [PERSON] et al. [27] provided a comprehensive survey about several machine learning approaches used in next generation edge networks. [PERSON] et al. [28] proposed a caching mechanism to predict data popularity profiles and estimation of popularity of new contents based on features extraction of existing contents and similarity with newly arrived contents. The authors extracted spatio-temporal features of existing and newly arrived contents with a convolution neural network (CNN) with several stage pooling. Then, a clustering algorithm based on Jaccard and cosine distance is devised to extract contents features to handle the dimensional. Afterwards, contents are classified into various categories by employing support vector machine (SVM). The obtained results showed significant performance gain.  
However, in a realistic network, users are mobile, and the network condition keeps evolving. Hence, a mechanism is necessary that does not require prior information to train the system and can learn through interaction with the environment. Therefore, researchers consider implementation of RL to optimize data caching in wireless networks. One such effort is made by [PERSON] et al. [29], where the RL approach is devised for content caching and wireless network slicing. The authors used a Zipf distribution to model users' data demands, and the wireless network channel is designed as a finite state Markov channel. In [30], the authors proposed data caching at the base station level. RL is devised to optimize cache allocation without any previous information in an online fashion. The proposed approach learns through direct perturbation of the environment and observing variation in the network performance.