## 4 Training and Simulation  
### Training Process of the MARL-Based Controller  
The learning process of the collaborative control strategy for IRW vehicles is detailed in Figure 6. The random selections of the operating conditions of the vehicle include different speeds, curve radii, and cant, in addition to lateral and vertical random excitations based on AAR5 track spectra, as shown in Table 1. In each episode, agents utilize virtual sensors installed on the wheelsets to measure the lateral displacement, yaw angle, and speed differential of the IRWs, thereby obtaining the necessary local observations for control decisions. The front and rear IRW active controllers then execute actions based on their current policies and these observed states, managing the torque differential applied to the IRWs. The dynamic response of the vehicle, as defined by the model described in Section 2, is computed along with the corresponding reward functions. The environment then transitions to the next state, during which the experience tuples are stored in the PER buffer. During the centralized training phase, each agent updates its network parameters: the critic network parameters are refined by minimizing the loss function, reflecting the action-value function's estimation error, while the actor network parameters are optimized using the policy gradient method to maximize the expected reward. The training process involves continuous interaction with the environment, with networks being updated according to the PER-MADDPG mechanism until a terminal state is encountered, signaling the end of the current episode and the initialization of a new episode. The reward function quantifies the steering capability of the IRWs, where higher rewards indicate more effective active steering control. Early termination of agent-environment interactions occurs if the lateral displacement or yaw angle exceeds the predefined thresholds in Table 2, prompting an environmental reset and agent penalization. This mechanism can guide agents toward more effective control strategies. If controlled IRWs do not exceed the state threshold or wheel flange contact occurs within the maximum time step \(N_{stops}\), the centering and curve guidance abilities of the IRWs are successfully restored, thereby validating the effectiveness of the MARL controller. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Tracker** & **Type** & **Speed** & **Curve** & **Cant [mm]** & **Track** \\
**No.** & & **[km/h]** & **Radius [m]** & & **Irregularities** \\ \hline
1 & Straight track & 120 & — & — & AAR5 \\
2 & Straight track & 200 & — & — & AAR5 \\
3 & Curved track & 100 & 600 & 80 & AAR5 \\
4 & Curved track & 150 & 1500 & 60 & AAR5 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation conditions. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Value** & **Description** \\ \hline \(\alpha\) & 0.5 & Balance coefficient for prioritized sampling in PER \\ \hline \(\beta\) & 0.5 & Bias correction coefficient for importance sampling \\ \hline \(\gamma\) & 0.99 & Discount factor for future rewards \\ \hline \(\xi\) & 0.005 & Soft update coefficient for target networks \\ \hline \(\epsilon\) & \(10^{-4}\) & Small positive constant ensuring nonzero sampling probability \\ \hline \(E_{m}\) & \(5\times 10^{5}\) & Total number of training episodes \\ \hline \(\mid\!\!D\!\mid\) & \(1\times 10^{6}\) & Capacity of the experience replay buffer \\ \hline \(\mid\!\!D_{s}\!\mid\) & 512 & Default batch size for gradient descent \\ \hline \(N\) & 2 & Number of agents, including front and rear wheelsets \\ \hline \hline \end{tabular}
\end{table} 
Table 2: MARL training parameters. The PER-MADDPG algorithm was implemented using Python 3.8 and the PyTorch 1.12 framework. We conducted the training on a workstation powered by an Intel Gold 6132 CPU and equipped with a GPU accelerator.