## 1 Introduction
Most current research on cloud service composition restricts the cloud environment to a single cloud, assuming that all the cloud services in the composition come from a single cloud environment. However, multi-cloud environments have become a trend because no single cloud provider can establish its data centers in all possible locations worldwide. Moreover, complex user requirements are likely to involve multiple cloud service providers distributed across different clouds [5]. As the number of clouds increases, the availability of better and more diverse services also increases, which may be missed if only one cloud provider is used. Limiting cloud service composition to a single cloud environment hampers the potential to access the best available services offered by other cloud providers. Furthermore, some consumers require high network transmission speeds, making clouds with excellent data transfer rates more suitable for their needs. Other consumers prioritize data privacy and security and may exclude untrusted clouds. Therefore, considering multi-cloud environments is essential when researching cloud service composition, as it provides better support for consumer needs. Cloud service composition in a multi-cloud environment presents several challenges. For example, multi-cloud environments often generate more data transmission and computation, which increases energy consumption and communication costs for cross-cloud service composition. Specifically, composite services involving multiple clouds will inevitably result in longer response times and higher network bandwidth usage. This is because invoking multiple services across clouds requires a large amount of data transmission and exchange, causing increased energy consumption [5]. Given the current global environmental situation, reducing carbon dioxide emissions and energy consumption is crucial to promoting the sustainable development of cloud computing and maintaining an ecological environment. Therefore, reducing the energy consumption of cloud service composition is a significant challenge when composing services in a multi-cloud environment. Moreover, the performance, price, and energy consumption of cloud services can frequently fluctuate in a dynamic multi-cloud environment. For example, a service that currently offers the best QoS performance may not remain optimal in the future. Additionally, the QoS attribute value may change, or the cloud service may suddenly become unavailable during runtime. These variables can have an impact on applications and services in a multi-cloud environment, necessitating the need to consider how to adapt to this dynamic nature. To address this, algorithms must have good adaptability and continuously adjust strategies based on environmental changes. Researchers have proposed various methods to address the cloud service composition problem, including traditional methods, such as 0-1 programming, and metaheuristic methods, such as particle swarm optimization (PSO) [6] and genetic algorithm (GA) [7]. While these approaches have yielded satisfactory results, they lack adaptability in dynamic environments. In other words, when changes occur in the environment, these algorithms may require adjustment or even redesign. Additionally, these methods are not suitable for large-scale space scenarios, which can be challenging given the large number and dimensions of services involved. Recently, deep reinforcement learning-based (DRL-based) methods have gained attention for web service composition [8]. DRL-based methods offer a quick adjustment of strategies based on the current scenario, provide the adaptive updating of weights, and dynamically select optimal service composition solutions. DRL-based methods can adjust the network structure and other means to deal with different scales and types of problems. In contrast, metaheuristic algorithms require manual feature representation and rules, knowledge and experience guidance, and tedious tuning processes and are not ideal for large-scale and high-dimensional optimization problems.