## 1 Introduction  
In recent years, with the development and widespread application of machine learning technology, it has demonstrated powerful performance across various fields, bringing significant changes and opportunities to numerous industries. Deep Reinforcement Learning (DRL), as a subfield of machine learning, exhibits unique advantages in dynamic network environments. Unlike traditional rule-based routing or single-agent reinforcement learning,DRL leverages deep neural networks for high-dimensional state representation and employs multi-agent collaboration to achieve distributed decision-making. Moreover, it can adaptively optimize long-term cumulative rewards through end-to-end learning, thereby enabling more efficient routing selection and resource allocation in complex, time-varying network topologies. In the network environment, machine learning makes decisions based on real-time data, continuously learns, and adjusts models to adapt to changes in the network environment. This makes it widely used in the networking field to optimize network performance and improve network security [1]. Traditional machine learning methods often assume that the environment is static or determined by external factors. However, in practical network environments, system behavior is frequently influenced by multiple autonomous entities. Multi-Agent Systems (MAS) provide a powerful framework for addressing such challenges by enabling multiple agents to interact and collaborate in pursuit of shared or individual objectives. In the context of machine learning, the application of MAS allows algorithms to more effectively handle uncertainty and dynamic changes in distributed environments. Each agent can independently learn and adapt to the environment while simultaneously accounting for the behaviors and influences of other agents [2]. Reinforcement learning (RL) is a fundamental branch of machine learning that enables an agent to interact with its environment and learn an optimal policy by leveraging feedback in the form of reward signals. The agent interacts with the environment by taking a sequence of actions, upon which the environment provides corresponding rewards or penalties. The agent's objective is to maximize the long-term cumulative reward. In network routing scenarios, the agent can be represented by a network node or a routing algorithm, while the environment encompasses network conditions such as the topology and link states. The reward function can be defined based on network performance metrics, such as throughput and latency. This reward-driven learning paradigm endows reinforcement learning with significant potential in network routing optimization, which enables it to adapt to dynamic network conditions and enhance overall network performance [3]. Machine learning brings revolutionary breakthroughs to network routing by enabling autonomous decision-making and dynamic adaptation. Unlike traditional rule-based algorithms, which rely on static protocols and struggle with real-time topology changes, machine learning algorithms--such as DRL--can continuously optimize routing policies through iterative interactions with network environments [4]. Intelligent routing employs machine learning algorithms to achieve load balancing and improve network performance and reliability. Machine learning demonstrates advantages such as intelligence, efficiency, and security in intelligent routing applications, thereby enhancing network performance and reducing network operation and maintenance costs. However, network topology often undergoes changes due to the physical and logical connection methods of each node to meet network requirements, enable fault recovery, optimize performance, and other factors. For example, equipment failure can cause changes in network paths [5]. After a network topology change, routing algorithms based on reinforcement learning may need to be retrained to avoid performance loss, as these algorithms typically make decisions based on the network status and topology.