# _Applied Sciences_
## 5 Results and Discussion
### Comparison between XNLI and ES-Contradiction  
In order to demonstrate the generality of our proposal, a series of experiments has been performed using the XNLI dataset and ES-Contradiction dataset in different training-test configurations. The XNLI dataset is divided into training, development, and test set. The training set is developed in the English language. The development and test sets are in 15 languages, including Spanish. To carry out this experiment, machine translation of the training set into Spanish and the test set in Spanish were used. Table 7 presents the results of each trained system. The best results are highlighted in italic. The models of line 1 and 2 are trained using the XNLI training set, the difference being that the first line predicts the ES-Contradiction dataset test set, and the other one, the XNLI test set. The prediction results are quite close for both of them, but the Contradiction class is detected with a higher accuracy and \(F_{1}m\). Comparing lines 1 and 4, considering our dataset as the test set with the four types of contradictions, as expected the system trained on our dataset is substantially better than the system trained on the XNLI training set. The result indicates that the XNLI dataset does not manage to cover all the contradictions contained in our dataset, even though it is more than 40 times the size of the training set of ES-Contradiction dataset and is composed of examples from different genres. The XNLI training set is exactly the same as the MultiNLI training set. It has been developed manually by parsing a sentence from a non-fiction article and creating three sentence variants: definitely correct, might be correct, and definitely incorrect [7]. The procedure for creating the training set of MultiNLI dataset follows an annotation guide that is sufficiently general to avoid bias in the dataset. However, this lack of specificity may cause a shortage of examples of various types of contradiction, resulting in an imbalance of contradiction types. Table 8 shows the accuracy by type of contradiction of the model trained in row 1 of Table 7.  
In the prediction of the type of contradiction (Con_Neg, Con_Ant, and Con_Num), this model achieves significantly good results; even in the class Con_Neg they are very good (88.50% accuracy). However, in the prediction of the class Con_Str, they are very \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{\(F_{1}\) Score (\%)} & \(F_{1}m\) (\%) & Acc (\%) \\ \hline
**System** & **Compatible** & **Contradiction** & & \\ \hline Training set (XNLI) and test set (ES-Contradiction) & 72.57 & 75.00 & 73.78 & 73.84 \\ Training and test set (XNLI) & 71.73 & 77.53 & 74.63 & 74.97 \\ Training set (ES-Contradiction) and test set (XNLI) & 65.21 & 32.28 & 48.75 & 54.04 \\ BETO-Contra\_Comp & 88.63 & 88.75 & 88.69 & 88.69 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Results obtained from Experiment 5: Results of different training-test configurations between XNLI and ES-Contradiction. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Acc (\%)**} \\ \hline
**System** & **Con-Neg** & **Con-Ant** & **Con-Num** & **Con-Str** \\ \hline Training set (XNLI) and test set (ES-Contradiction) & 88.50 & 75 & 70.10 & 48.48 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Accuracy obtained for detecting each specific type of contradiction with the model in row 1 of Table 7.  
low (48.48% accuracy); this result could be due to the lack of examples of this type in the XNLI dataset. Finally, the system trained on the ES-Contradiction dataset failed to obtain good enough results in order to predict the XNLI test set. This system only obtains 32.28% \(F_{1}m\) to predict the contradictions of the XNLI test set. The need to include new types of contradictions in the ES-Contradiction dataset is evidenced, specifically those that allow the creation of robust contradiction detection systems for the real-world and enable prediction with higher accuracy in the XNLI dataset.