# A Survey of Deep Learning for Data Caching in Edge Network
## 4 Deep Learning for Data Caching
### Drl
In Reference [90], the authors generate an optimization model with the aim of maximizing network operator's utility in mobile social networks under the framework of mobile edge computing, in-network caching and D2D communications (3C). The trust value which if estimated through social relationships among users are also considered. Then, a DQN model is utilized for solving the optimization problem, including determining video provider and subscriber association, video transcoding offloading, and the video cache allocation for video providers. The DQN employs two CNNs for the training process, where one generates target Q value and the other is for estimated Q value. Unlike the conventional DQN, the authors in Reference [90] introduces a dueling DQN structure, which helps achieve a more robust result. The authors also consider utilizing dueling DQN model in different scenarios, like cache-enabled opportunistic interference alignment [91] and orchestrating 3C in vehicular network [92]. Reference [93] provides a DDPG model to cope with continuous valued control decision for 3C in vehicular edge networks. The work in Reference [102] focuses on the cooperative caching policy at FBSs with maximum distance separable coding in ultra dense networks. A Q-learning model is utilized to determine caching categories and the content quantity at FBSs during the off-peak duration. The authors also consider the performance of their algorithm on continuous action cases because of action discretization. Compared with DDPG, there is only a tiny performance gap in small-scale case. A very interesting result is that in large-scale case, the proposed Q-learning model even slightly outperforms DDPG due to underfitting, which express the complicated network structure of DDPG. Reference [94] provides an optimization model which takes what to cache and content delivery into consideration in the fog-enabled IoT network in order to minimize service latency. Since the wireless signals and user requests are stochastic, an actor-critic model is engaged where the actor makes decisions for requesting contents, while the critic estimates the reward. Especially, the action space \(S\) consists of decision variables and reward function is a variant of the objective function. In order to avoid converging to local optima, the nature policy gradient method is employed in Reference [94]. A caching replacement strategy and dynamic multicast scheduling strategy are studied in Reference [95]. In order to get a suboptimal result, an auto encoder is used to approximate the state. Further, a weighted double DQN scheme is utilized for avoiding overestimation of Q value. Reference [96] applies an RNN to predict content popularity by collecting historical requests and the output represents the popularity in the near future. Then, the prediction is employed for cooperative caching and computation offloading among MEC servers, which is modeled as an ILP problem. For the purpose of solving it efficiently, a multi-agent DQN is applied where each user is viewed as an agent. The action space consists of task local computing and offloading decision, as well as local caching and cooperative caching determination. The reward is measured by accumulated latency. The agent chooses its own action based on current state without cooperation. The where to cache, what to cache, and content delivery decision of D2D network are jointly modeled in Reference [97]. Two RNNs, ESN and LSTM, are considered to predict mobile users' location and requested content popularity. Then, the prediction result is used for determining content categories and cache locations. The content delivery is formulated as the actor-critic based DRL framework. The state spaces include CSI, transmission distances, and communication power between the requested user and other available candidates. The function of DRL is determining the communication link among users with the aim of minimizing power consumption and content delay.