# Interpretable Reinforcement Learning for Sequential Strategy Prediction in Language-Based Games
## 5 Results
### Model Comparison  
To further evaluate the performance of Enhanced-DDPG algorithm, we use the Enhanced-DDPG algorithm to conduct comparison experiments with RFR, XGBoost, METRA, SQIRL, and LightGBM. The hyperparameter settings of RFR, XGBoost, METRA, SQIRL, and LightGBM are shown in Table 1.  
We train and evaluate the above model on the original dataset and the dataset with Gaussian noise, and we use the two indicators of MSE and R\({}^{2}\) to compare the results, as shown in Table 2.  
On the original dataset, Enhanced-DDPG has an MSE of 0.0134 and an R\({}^{2}\) of 0.8439, which are significantly better values than those for RFR (MSE = 0.0425, R\({}^{2}\) = 0.6554), XGBoost (MSE = 0.0276, R\({}^{2}\) = 0.7960), LightGBM (MSE = 0.0221, R\({}^{2}\) = 0.8342), METRA (MSE = 0.0242, R\({}^{2}\) = 0.6977), and SQIRL (MSE = 0.0371, R\({}^{2}\) = 0.7153). This reflects that Enhanced-DDPG has better performance in processing raw data. This is due to the dynamic learning mechanism of Enhanced-DDPG, which makes it better able to capture nonlinear and complex relationships in the data. On the dataset with Gaussian noise, Enhanced-DDPG has an MSE = 0.0241 and R\({}^{2}\) = 0.7853, which is still better than RFR (MSE = 0.0512, R\({}^{2}\) = 0.6133), XGBoost (MSE = 0.0314, R\({}^{2}\) = 0.7215), LightGBM (MSE = 0.0325, R\({}^{2}\) = 0.7215), METRA (MSE = 0.0211, R\({}^{2}\) = 0.6991), and SQIRL (MSE = 0.0281, R\({}^{2}\) = 0.7186). The introduction of noise in the dataset leads to the performance degradation of Enhanced-DDPG algorithm, but the decline is \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Hyperparameters** & **Value** \\ \hline \multirow{4}{*}{RFR} & max\_depth & 10 \\  & min\_samples\_leaf & 4 \\  & min\_samples\_split & 10 \\  & n\_estimators & 100 \\ \hline \multirow{4}{*}{XGBoost} & max\_depth & 5 \\  & learning\_rate & 0.01 \\  & n\_estimators & 100 \\  & subsample & 0.5 \\  & colsample\_bytree & 0.5 \\ \hline \multirow{4}{*}{LightGBM} & max\_depth & 3 \\  & learning\_rate & 0.01 \\  & n\_estimators & 100 \\  & num\_leaves & 20 \\ \hline \multirow{4}{*}{METRA} & learning\_rate & \(3\times 10^{-4}\) \\  & discount\_factor & 0.99 \\  & latent\_dimension & 32 \\  & random\_exploration-steps & - \\ \hline \multirow{4}{*}{SQIRL} & learning\_rate & \(10^{-3}\) \\  & discount\_factor & 0.95 \\ \cline{1-1}  & latent\_dimension & - \\ \cline{1-1}  & random\_exploration-steps & 10,000 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Hyperparameter settings of the models, including RFR, XGBoost, LightGBM, METRA, and SQIRL. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Model** & **MSE** & **R\({}^{2}\)** \\ \hline Enhanced-DDPG (Raw data) & 0.0134 & 0.8439 \\ RFR (Raw data) & 0.0425 & 0.6554 \\ Enhanced-DDPG (Gaussian noise data) & 0.0241 & 0.7853 \\ RFR (Gaussian noise data) & 0.0512 & 0.6133 \\ XGBoost (Raw data) & 0.0276 & 0.7960 \\ LightGBM (Raw data) & 0.0221 & 0.8342 \\ XGBoost (Gaussian noise data) & 0.0314 & 0.7215 \\ LightGBM (Gaussian noise data) & 0.0325 & 0.7749 \\ METRA (Raw data) & 0.0242 & 0.6977 \\ SQIRL (Raw data) & 0.0371 & 0.7153 \\ METRA (Gaussian noise data) & 0.0211 & 0.6991 \\ SQIRL (Gaussian noise data) & 0.0281 & 0.7186 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance comparison of different models, including Enhanced-DDPG, RFR, XGBoost, LightGBM, METRA, and SQIRL, on raw and Gaussian noise datasets, using MSE and R\({}^{2}\) metrics. significantly smaller than that of traditional methods, indicating that Enhanced-DDPG has stronger robustness in noisy environments and can better adapt to fluctuations in data. Ablation experiments compare the original DDPG (without the SHAP module) and DDPG with representative features removed against our proposed Enhanced-DDPG. As shown in Table 3, when using the original reward function instead of the SHAP-based reward and removing the highest-contribution features, the algorithm's performance degrades significantly.