## 2 Related Works  
Research on the cache-enabled cloud-edge collaborative architecture can be broadly categorized into two groups.  
The first group employs traditional methods based on convex optimization or probabilistic modeling to solve the content cache placement problem in cloud-edge environments. This category includes classic algorithms, such as least recently used (LRU) [17], first in first out (FIFO) [18], and least frequently used (LFU) [19]. However, these methods disregard the popularity trend of the content, which can result in significant performance degradation [12]. [PERSON] et al. [20] optimize the hit rate by jointly optimizing content placement and user association. [PERSON] et al. [21] propose a greedy service placement algorithm that maximizes the number of service requests by considering collaborative service placement, but it is not able to account for random requests in a dynamic environment. Similarly, [PERSON] et al. [22] address the joint service caching and load scheduling problem to minimize the average request delay. [PERSON] et al. [23] employ probabilistic caching in heterogeneous networks to enhance traffic offloading and optimize edge device cache resources. Since cloud-edge collaboration is constantly evolving, the content cache problem in cloud-edge environments requires continuous optimization. Nevertheless, these strategies are often challenging to adapt to the dynamic cloud-edge environment.  
The second category of research pertains to machine learning algorithms, such as machine learning, deep learning, and federated learning [24]. This approach involves learning key attribute features in the cloud-edge environment, including user request behavior, content popularity, and user mobility distribution, in order to optimize content cache strategies. Reinforcement learning [25] and deep reinforcement learning [15] exhibit significant potential in designing content cache strategies. Specifically, [PERSON] et al. [26] propose a proactive caching framework based on deep learning that achieves significant improvements in feedback link and QoE. [PERSON] et al. [27] employ an improved DRL method to optimize caching for enhancing video streaming quality of service. To mitigate the excessive network resource consumption of deep reinforcement learning during the transmission of training and testing data, [PERSON] et al. [13] propose a caching framework based on federated deep reinforcement learning. [PERSON] et al. [12] present the CVC video caching framework based on federated deep reinforcement learning to reduce communication costs.  
While machine-learning-based methods can achieve higher caching efficiency by adapting to the caching environment, they themselves consume significant amounts of energy. Reducing training energy consumption offers three benefits: (1) enhancing system efficiency alleviates base station energy burden, and lowers operational costs; (2) improving device reliability and stability, and reducing maintenance and replacement costs; and (3) reducing the system's carbon footprint, supporting environmentally responsible edge computing. To address this issue, we propose a novel federated deep reinforcement learning mechanism that incorporates energy-saving properties to reduce the energy consumption associated with machine learning.