# Automatic Construction of Fine-Grained Paraphrase Corpora System Using Language Inference Model
## 5 Fine-Grained Label Correctness and Accuracy Investigation
### String Property Analysis  
Table 11 shows the average word length difference between the first sentence and second sentence of each class. The sentence pairs in paraphrase datasets have very close word length, as shown in Table 11a. In both datasets, the second sentence length is slight longer than the first sentences in the negative class. For the positive class, almost all the sentence pairs share the same length. As for the language inference datasets, most sentence 1 in the sentence pairs tend to be relatively longer than sentence 2. Compared to SNLI, MNLI has a much bigger length difference in the sentence pair samples. However, for both datasets, the length difference in each class is negligible. On the contrary, the length difference of each fine-grained label is substantial. For the bidirectional entailment class (\(\equiv\)), all datasets except MNLI has a length difference close to 0. In terms of the elaboration class (\(\sqsupseteq\)), the second sentences tend to have longer word length than the first sentences except for MNLI. In contrast to elaboration, the entailment class (\(\sqsupseteq\)) has longer first sentences. Due to the considerable difference in sentences length of MNLI, the average length of sentence1 is longer in all fine-grained classes. However, the difference of each class can still be reflected in Table 11c. The bidirectional entailment paraphrases preserve all the amount of information in the given sentences. Thus, the sentence pair should theoretically have the same sentence length. On the other hand, unidirectional entailment paraphrases, including elaboration class and the entailment class, exhibit a certain amount of content loss [11]. A typical example of content loss by deletion can be observed from the following sentence pair:  
* [noitemsep]
* Yesterday I went to the park. * Yesterday I went to Victoria Park. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & & & \\ \hline
**Data** & **0** & **1** & **All** & \\ \hline QQP & \(-\)0.4 & 0 & \(-\)0.3 & \\ MRPC & \(-\)0.1 & 0 & 0 & \\ \hline \hline \multicolumn{5}{c}{**(b) Language Inference Datasets**} \\ \hline
**Data** & **Contradiction** & **Neutral** & **Entailment** & **All** & \\ \hline MNLI & 12.3 & 11.1 & 12.5 & 12 \\ SNLI & 5.8 & 5 & 6.6 & 5.8 \\ \hline \hline \multicolumn{5}{c}{**(c) Fine-grained Labelled Datasets**} \\ \hline
**Data** & **Negation (\(\wedge\))** & **Entailment (\(\sqsubseteq\))** & **Elaboration (\(\sqsupseteq\))** & **Equivalence (\(\equiv\))** & **Independence (\#)** \\ \hline QQP & 0 & 3.3 & \(-\)3.6 & 0 & 1 \\ MRPC & 0 & 3.1 & \(-\)3 & 0 & 0.1 \\ MNLI & 10 & 16 & 2.4 & 4.3 & 13.1 \\ SNLI & 5.6 & 7.3 & \(-\)1 & 0.8 & 6 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Sentence pair length difference. This explains the sentence length difference of the two unidirectional entailment class in Table 11c. In addition to sentence word length, we also examine the Rouge-L score of each fine-grained class. Rouge-L takes into account sentence level structure similarity naturally and identifies the longest co-occurring in sequence n-grams automatically [22]. Table 12c shows the Rouge-L F1 score of each fine-grained class. For paraphrase datasets, the positive class has a higher Rouge-L score. MRPC sentences are mainly extracted from news while QQP contains duplicated question pairs in the knowledge sharing forum Quora. Thus, the sentences of MRPC are normally longer than QQP. This explains why MRPC tends to have higher Rouge-L score in all classes compared with QQP. As for the language inference datasets, the entailment class has a higher score than other classes.The MNLI contradiction class has a higher score compared with neutral class while the SNLI contradiction class has a lower score. In terms of fine-grained labels, the paraphrase classes, including entailment, elaboration and equivalence, have a higher score than the other fine-grained classes. Among the paraphrase classes, the equivalence class has the highest Rouge-L score of all the original labels and the other fine-grained classes.