## 3 Active Guidance Controller Design for IRWs Based on MARL
### MARL-Based Controller Design
#### 3.4.3 Reward Shaping  
The design of the reward function is essential for guiding agents to optimize toward the control objective efficiently. For an IRW vehicle, the MARL-based controller aims to improve straight-line stability and curve-negotiation capabilities, which can be achieved by implementing control methodologies, including zero lateral clearance [23] and radial control [24]. Specifically, zero lateral clearance aims to maintain the geometric center alignment of the IRW with the track centerline, minimizing lateral displacement. Radial control ensures that each wheelset achieves a radial position in a curved line with a minimized attack angle. Thus, the reward function should reward agents more when wheelsets exhibit less lateral displacement and yaw angles. Moreover, considering the impact of actuator output on the energy efficiency of guidance control, the reward decreases as the control torque increases, promoting environmentally friendly control actions.  
Based on the above discussion, the definition of the reward function is constructed using the weighted sum of the lateral displacement, yaw angle, and motor torque of the wheelsets for the IRW vehicles. Since the control torque and wheelset dynamics state are not of the same order of magnitude, normalization is needed between the parameters. In the fully cooperative multiagent environment of our proposed MARL-based controller, all agents share a global reward function \(R\), and \(R\) is formulated as (16).  
\[R=\kappa^{2}\sum_{i=1}^{N=2}\left[\eta_{i1}(y_{\text{max}}-|y_{wi}|)^{2}+\eta_{ i2}(\psi_{\text{max}}-|\psi_{wi}|)^{2}\right]+\sum_{i=1}^{N=2}\eta_{i3}(T_{ \text{max}}-|T_{wi}|)^{2} \tag{16}\]where \(y_{\max}\) and \(\psi_{\max}\) are the set upper bounds for the lateral position and attack angle of the wheelsets, respectively. The lateral displacement reward coefficients are denoted by \(\eta_{11}\) and \(\eta_{21}\), those for the yaw angle by \(\eta_{12}\) and \(\eta_{22}\), and those for the control output by \(\eta_{13}\) and \(\eta_{23}\). \(\kappa\) denotes the conversion factor from meters to millimeters, and \(\kappa=1000\). \(T_{\max}\) denotes the maximum torque output for the actuators.  
The first term of \(R\) represents the reward associated with the lateral displacement and attack angle, aiming to ensure that under active guidance control, the IRW vehicle maintains optimal steering performance while avoiding wheel flange contact on various tracks, which implies that the lateral movement and yaw angle of the wheelsets should remain within the predefined limits. The second term of the reward function concerning the control torque indicates that the controller should constrain the output torque of the motors without compromising the guidance performance. Furthermore, the front wheelset, which typically encounters larger yaw angles during curve negotiation, has higher reward weights than does the rear wheelset. Therefore, this approach prioritizes the improvement of the dynamic performance of the front wheelset, taking into account its more severe wheel-rail contact conditions in curved lines.