## 3 ERA-MADDPG Intelligent Routing Solution
### ERA-MADDPG Intelligent Routing Interacts with the Environment  
The ERA-MADDPG algorithm architecture employs a CNN for MADDPG network training. When using the MADDPG algorithm for training, it is usually necessary to input high-dimensional observation data. The observation data contains information about the data layer environment state. When processing high-dimensional data, the original MADDPG neural network often faces problems such as high computational complexity and low training efficiency. This paper adopts a CNN to leverage high-dimensional data processing, which can effectively accelerate the convergence efficiency of the algorithm. At the same time, the convolutional layers in a CNN can share weight parameters, and the neural network can reuse the same weights to process different areas of the input data, reducing the number of network parameters and model complexity. A CNN can better process high-dimensional observation data, extract effective features, and pass them as input to the MADDPG network for decision-making and value function estimation. This architecture enhances the representation ability of the network, enabling agents to better understand the state of the environment to learn appropriate strategies and value functions. The interaction between the ERA-MADDPG intelligent routing and the environment is illustrated in Figure 4.  
The intelligent agent interacts with the data layer environment through the controller layer. The state is extracted by CNN and input into the decision network. Actions are jointly generated by multiple intelligent agents and executed through a stream table. Reward values are dynamically calculated based on real-time performance indicators. In the ERA-MADDPG intelligent routing algorithm, a CNN is employed to construct the decision-making model of the intelligent agent. This model enables the agent to autonomously carry out path planning and decision-making by learning the mapping relationship between states, actions, and reward values. At the same time,  
Figure 4: Mapping process. the mutual influence and collaboration among multiple agents enable collaborative learning and decision-making among agents. The ERA-MADDPG algorithm uses a reinforcement learning model that consists of an agent and the environment. The model collects the environmental state \(S\), makes a decision \(A\) based on \(S\), and obtains the reward value \(R\) according to the quality of the decision. The mapping process is as follows:  
(1) State Mapping: The state mapping in multi-agent routing is a real-time mapping of the current network state, mapping the network topology structure, link bandwidth, throughput, switch resource utilization, link delay, packet loss rate, and other state information of the traffic into the neural network. Through state mapping, multi-agent routing can divert, optimize, and secure traffic according to different conditions. In addition to information input such as network link bandwidth, link throughput, link delay, packet loss rate, and traffic, the algorithm in this paper also allows multiple agents to interact with each other. Agents can perceive the status of other agents and collaborate in training, effectively improving the efficiency of algorithm training. (2) Action Mapping: Action mapping is in different states; the agent selects actions and the values or probabilities associated with those actions. These actions are determined based on state information input and cumulative reward values. Under different network information or traffic requirements, each link presents different weights. In multi-agent routing, each agent needs to choose an optimal action based on the current state to achieve overall routing optimization. By learning the action mapping relationship, the agent can find the best action strategy that should be taken in a specific state to maximize its expected reward value.