## 4 Methodology
### Reinforcement Learning-Based Response Decision Model  
Following the risk score computation in the previous section, this section presents reinforcement learning (RL) techniques to construct a Q-learning-based response decision module to optimize the system's adaptive response strategies based on different risk levels and attack types specific to T2G communication system, as described in Section 3. The RL environment is represented as a Markov Decision Process (MDP), in which the state space consists of two main factors: risk score and attack type confidence. To reduce training complexity, the risk score is discretized into 20 levels, and 10 types of attacks are treated as independent categorical dimensions, creating \(S=20\times 10=200\) finite states. The action space consists of five types of defensive responses targeted at the onboard T2G devices, as listed in Table 4. \begin{table}
\begin{tabular}{c c} \hline \hline
**Risk Level** & **Range** \\ \hline Low & [0, 25] \\ Medium & (25, 50] \\ High & (50, 75] \\ Extreme & (75, 100] \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Risk level threshold classification. A further explanation of these decisions is provided as follows:  
1. Monitoring and Logging: This action records and monitors train status and communication activities to detect threats such as eavesdropping and traffic analysis or unauthorized access, which correspond to backdoor attacks in the TON_IoT dataset. By tracking data flows between the train and ground systems, it helps identify unusual communication patterns or unauthorized access attempts in real-time. 2. Limiting Train Communication: In response to DoS/DDoS or communication jamming attacks, which are related to DDoS or flooding attacks in the TON_IoT dataset, this action limits the transmission rate of non-critical data, such as diagnostic messages, ensuring that crucial T2G control data, like operational commands or fault information, are prioritized and protected from overload. 3. Train Deceleration: If a MitM or message tampering attack is detected, which could correspond to MitM or injection attacks in the TON_IoT dataset, this action automatically initiates deceleration to ensure the train operates safely by reducing speed. This allows time to verify data integrity and control the situation before a critical failure occurs. 4. Emergency Braking: In cases of replay attacks or data integrity attacks, similarly to replay or ransomware attacks in the TON_IoT dataset, this action triggers emergency braking to rapidly stop the train, ensuring that any compromised operational data, such as false fault reports or location data, do not lead to unsafe situations, thereby protecting passengers and infrastructure. 5. System Reset: If a device compromise or routing exploit (similar to backdoor or command injection attacks in the TON_IoT dataset) is detected, this action performs a secure reboot or restoration of the affected T2G devices, such as MCG, to ensure the systems are cleared of any malicious modifications and return to a secure state, allowing communication to resume safely. The training process uses a classical \(\varepsilon\)-greedy policy to dynamically balance exploration and exploitation. The specific steps are as follows:  
Stage 1: Initialize the Q-table as a zero matrix. Stage 2: Select a state randomly and observe the current state \(s_{t}\); choose an action \(a_{t}\) based on the \(\varepsilon\)-greedy probability. Stage 3: Execute the selected action and evaluate its impact on system risk, receiving the corresponding reward \(r_{t}\) and transition to a new state \(s_{t+1}\). Stage 4: Update the Q-table using the Q-learning update rule:  
\[Q(s_{t},a_{t})\gets Q(s_{t},a_{t})+\alpha[r_{t}+\gamma\cdot\max_{a}Q(s_{t+ 1},a)-Q(s_{t},a_{t})] \tag{22}\]  
where \(\alpha\) is the learning rate and \(\gamma\) is the discount factor. Once training is complete, the optimal policy is extracted by selecting actions with maximum Q-values for each state: \(\pi^{*}(s)=\arg\max_{a}Q(s,a)\).