## 4 Experiment
### Comparative Experiments  
On the ReClor dataset and LogiQA dataset, using the same experimental setup, the DaGATN model was compared with baseline models and the other logical inference machine reading comprehension models on leaderboard, i.e., graph neural network \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Value** \\ \hline Dropout & 0.1 \\ Learning rate & \(5\times 10^{-5}\) \\ Optimizer & AdamW \\ Batch size & 1 \\ Epoch & 10 \\ Weight decay & 0.01 \\ Max sequence length & 200 \\ Graph attention head & 5 \\ Graph reasoning iteration step & 1 \\ Gradient accumulation step & 8 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The training hyperparameters. based DAGN [18], AdaLoGN [16], Logiformer [34], LoCSGN [35], the pre-training-based MERIt [4] and the data enhancement-based LReasoner [17]. For a fair comparison with our approach, we selected the experimental results of MERIt using deberta-v2-xlarge and the best results of LReasoner under non-ensemble conditions, and the experimental results are shown in Table 3.  
The experiments show that on the ReClor dataset, our proposed DaGATN achieves an overall accuracy of 74.3% on the Test set and 65.71% on the Hard test subset, which is closer to the performance of humans on the Hard test set. Compared with other graph neural networks (AdaLoGN, Logiformer, DAGN, LoCSGN), pre-training based (MERIt), and data enhancement based (LReasoner) methods, our model has a higher accuracy on the Hard test set. And by comparing the accuracy difference between the validation set and the test set, we can find that the degree of overfitting is lower compared to others. On the LogiQA dataset, DaGATN also demonstrated competitive performance. The accuracy of the validation set is better than competitors, slightly behind the MERIt in the test set. Due to the more detailed categorization in the ReClor dataset, we have chosen this dataset for the analysis of experimental results, facilitating the discussion of the strengths and weaknesses of DaGATN. The detailed analysis results for each type of logical reasoning type are shown in Table 4. The ReClor dataset inherits 17 logical reasoning types, and we selected DeBERTa-xlarge as the baseline model to compare and analyze the performance of our system on different types of questions. We also list the detailed performance of different types of approaches in logical inference machine reading comprehension tasks, i.e., MERIt, which represents the pre-training perspective and captures the logical relationships of text based on heuristic rules, and LReasoner, which represents the data enhancement based and captures the logical symbolic relationships in text and options, to compare and analyze our strengths and weaknesses. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Val (\%)**} & \multicolumn{2}{c}{**ReClor**} & \multicolumn{2}{c}{**LogiQA**} \\  & & **Test (\%)** & **Test-E (\%)** & **Test-H (\%)** & **Val (\%)** & **Test (\%)** \\ \hline Human Performance * & - & 63.00 & 57.10 & 67.20 & - & 86.00 \\ ChatGPT(zero-shot) * & - & 60.90 & 64.55 & 58.04 & - & 38.44 \\ DeBERTa-xlarge * & 72.60 & 71.00 & 83.41 & 61.25 & 44.40 & 41.50 \\ \hline AdaLoGN & 65.80 & 60.20 & 79.32 & 45.18 & 39.94 & 40.71 \\ Logiformer & 68.40 & 63.50 & 79.09 & 51.25 & 42.24 & 42.55 \\ DAGN(deberta-large) & 72.40 & 66.80 & 79.80 & 56.58 & 42.40 & 41.70 \\ LoCSGN & 78.60 & 73.20 & 84.77 & 64.11 & 43.70 & 43.20 \\ \hline MERIt & 78.00 & 73.10 & 85.23 & 63.57 & 43.90 & 45.30 \\ \hline LReasoner & 76.40 & 70.70 & 81.10 & 62.50 & 41.60 & 41.20 \\ \hline
**DaGATN** & **74.60** & **74.30** & **85.22** & **65.71** & **44.71** & **44.50** \\ \hline \hline \end{tabular}  
* Baseline, The results of human performance is come from [PERSON] et al. (2020) [6] [PERSON] et al. (2020) [5] and leaderboard.  
\end{table} 
Table 3: Overview of the results from the comparative experiment.