## 2 Related Work
### Service Composition Based on Machine Learning  
In recent years, there has been a growing trend in using machine learning methods for service composition. Compared to non-heuristic and heuristic algorithms, machine learning algorithms have the ability to automatically learn and adapt to various data distributions and changes in real time, which allows them to update and optimize their models. Additionally, machine learning algorithms can efficiently process data adaptively. Furthermore, they can be applied to different types of cloud service composition problems and can be adjusted and optimized according to specific situations. This makes them more widely applicable and eliminates the need to define complex models and rules in advance. Currently, there are three main types of machine learning-based service composition methods: deep learning; reinforcement learning; and deep reinforcement learning. For deep-learning-based service composition, [PERSON] et al. [33] proposed a combination of long short-term memory (LSTM) and particle swarm optimization (PSO) algorithms to accurately recommend cloud services based on QoS properties. [PERSON] et al. [34] suggested combining traditional machine learning methods with metaheuristic methods to provide a comprehensive approach to service composition problems under cloud manufacturing mode. Additionally, [PERSON] et al. [35] proposed using Word2 Vec and LSTM-based neural network models to identify suitable candidate sets for each submitted manufacturing subtask in the cloud manufacturing platform, followed by an approach to optimal composition services using genetic algorithms. [PERSON] et al. [36] introduced DeepQSC, a deep supervision learning framework based on graph convolutional networks and attention mechanisms that can form high-QoS composition services within a limited computing time. However, this method does not account for user QoS constraints. Reinforcement learning has been widely explored for service composition. [PERSON] et al. [37] proposed a multi-agent reinforcement learning model that combines multi-agent technology and reinforcement learning for adaptive service composition, but communication cost reduction between agents remains an issue. [PERSON] et al. [38] introduced the CSSC-MDP model, which accounts for the uncertainty of service quality and behavior. [PERSON] et al. [39] combined reinforcement learning with skyline computation to improve the efficiency of compositional computation. However, without integrating deep learning, this method may not be efficient in dealing with large-scale problems. [PERSON] et al. [40] proposed a reinforcement-learning-based method for service composition in WoT environment, which does not require prior knowledge of user preferences for QoS attributes. Several researchers have proposed combining reinforcement learning with deep learning for service composition. [PERSON] et al. [8] introduced an adaptive deep reinforcement learning method for service composition, combining recurrent neural networks and heuristic behavior policies. [PERSON] et al. [41] proposed a QoS-prediction-based reinforcement learning method for service composition that combines recurrent neural networks with reinforcement learning, achieving better results in the effectiveness of service composition. [PERSON] et al. [42] introduced the CSSC-DQN model to address dynamic service composition problems. The model handles missing candidate services and fluctuations in QoS values during the execution phase, particularly for services with fine-grained QoS attributes. [PERSON] et al. [43] proposed a DRL-based QoS-aware cloud manufacturing service composition model that considers logistic issues and established a logistics-based DRL model. [PERSON] et al. [44] proposed a deep reinforcement learning-based combination method for selecting and composing quality parameter-aware mobile IoT services and developed a parallel group-based service discovery algorithm as the basis for measuring the accuracy of the proposed method. [PERSON] et al.