## 1 Introduction
Metaheuristic algorithms are often limited by current search spaces and do not readily obtain globally optimal solutions. Given the dynamic nature and a large number of services in a multi-cloud environment, deep reinforcement learning offers better application value. By continuously learning and optimizing, deep reinforcement learning can provide users with higher-quality service composition solutions. There are several issues with current research on cloud service composition: (1) although some researchers have used DRL-based methods for service composition in web services [8], DRL is rarely used in cloud service composition in a multi-cloud environment; (2) when using deep reinforcement learning for service composition, most existing research has not considered energy consumption. Researchers typically use the characteristics of DRL to find the best service composition plan while overcoming adaptability issues but do not consider service energy consumption; (3) the increasing number of cloud services increases the search space significantly, which substantially increases the computation time of the algorithm. However, few researchers have optimized the time complexity of the deep reinforcement learning algorithm. To address the energy-aware cloud service composition problem in a multi-cloud environment, we propose the skyline-enhanced deep reinforcement learning approach (SkyDRL). In this algorithm, we consider both energy consumption and dynamic environmental issues in a multi-cloud environment. We added energy consumption terms to both the objective and reward functions to ensure that the service composition has excellent QoS performance while reducing energy usage and achieving a balance between QoS and energy consumption. As previously discussed, while energy consumption is a significant factor in evaluating cloud services, optimizing QoS remains the primary focus of service composition. A singular focus on reducing energy consumption without considering QoS attributes can lead to a decline in service quality and user satisfaction. Therefore, by balancing QoS with energy consumption, data center energy consumption can be further reduced, and resource utilization can be improved while ensuring that users obtain the required service quality and reliability, thus enhancing user satisfaction. We leverage the advantages of deep reinforcement learning's adaptive learning policy to solve the dynamic environment problem. At the same time, we combined the skyline algorithm with deep reinforcement learning to effectively narrow the search space, reduce deep reinforcement learning training time, and improve algorithm execution efficiency. Overall, the main contributions of this paper are as follows:  
1. We propose a novel method called SkyDRL based on deep reinforcement learning models to solve the cloud service composition problem in a multi-cloud environment, providing a new solution to this problem;
2. In multi-cloud environments, service composition generates more energy consumption. In addition, cloud services are usually not static. Therefore, we mainly focus on the energy consumption generated during the cloud service composition process in multi-cloud environments and consider dealing with dynamic cloud environments rather than static ones. We propose an energy-efficient QoS-guaranteed cloud service composition algorithm;
3. Additionally, a large amount of data can lead to excessively long training times and increased optimization difficulty. Therefore, we combined the branch and bound skyline algorithm with deep reinforcement learning, using the skyline algorithm to reduce the search space and computational complexity;
4. We compared our proposed method with other methods in experiments to demonstrate its effectiveness. In the upcoming sections, we will review related work in this research area, including traditional research on cloud service composition and recent machine learning-based service composition research. Section 3 will describe and model the problem studied in this paper.