# Automatic Construction of Fine-Grained Paraphrase Corpora System Using Language Inference Model
## 3 Auto Relabelling Methods
### Auto Relabel Rules  
Most fine-grained paraphrase relations are symmetric; these include: equivalence, alternation, independence and negation. The forward and reverse entailment are directional. This suggests: If a sentence pair \((s_{1},s_{2})\) has symmetric relation, the swapped pair \((s_{2},s_{1})\) should have the same relation; if a sentence pair \((s_{1},s_{2})\) has forward entailment relationship, the swapped pair \((s_{2},s_{1})\) should have reverse entailment relation and could be labelled as \"neutral\".  
We utilise the labels of swapped pairs to identify directional and symmetric entailment relations. The rules are defined as follows: for a sentence pair, \((s_{1},s_{2})\):  
1. If \((s_{1},s_{2})\) is \"contradiction\" and \((s_{2},s_{1})\) is \"contradiction\", \((s_{1},s_{2})\) is of negation or contradiction relation and is labelled as 0.
2. If \((s_{1},s_{2})\) is \"entailment\" and \((s_{2},s_{1})\) is \"neutral\", \((s_{1},s_{2})\) is a \(\sqsubseteq\) relation and is labelled as 1.
3. If \((s_{1},s_{2})\) is \"neutral\" and \((s_{2},s_{1})\) is \"entailment\", \((s_{1},s_{2})\) is a \(\sqsupset\) relation and is labelled as 2.
4. If \((s_{1},s_{2})\) is \"entailment\" and \((s_{2},s_{1})\) is \"entailment\", \((s_{1},s_{2})\) is of equivalent or bidirectional entailment relation (\(\equiv\)) and is labelled as 3.  
\begin{table}
\begin{tabular}{c l l} \hline \hline
**Relation** & **Premise** & **Hypothesis** \\ \hline \multirow{5}{*}{\(\sqsubseteq\)} & Finally, the FDA will conduct & \\  & workshops, issue guidance manuals & \\  & and videotapes and hold & \\  & teleconferences to aid small entities in & \\  & complying with the rule. & \\ \hline \multirow{2}{*}{\(\equiv\)} & Postal Service were to reduce delivery & The postal service could deliver less \\  & frequency. & frequently. \\ \hline \multirow{2}{*}{\(\sqsupset\)} & A smiling costumed woman is holding & A happy woman in a fairy costume \\  & an umbrella. & holds an umbrella. \\ \hline \multirow{2}{*}{\#} & He went down on his knees, examining & \\  & it minutely, even going so far as to & It smelled like eggs. \\  & smell it. & \\ \hline \multirow{5}{*}{\(|\)} & The company once assembled, [PERSON] & \\  & rose from his seat with the air of a & \\ \cline{1-1}  & popular lecturer, and bowed politely to & \\ \cline{1-1}  & his audience. & \\ \hline \hline \end{tabular}
\end{table}
Table 4: Sample “entailment” and “neutral” pairs from MNLI.  
If \((s_{1},s_{2})\) is \"neutral\" and \((s_{2},s_{1})\) is \"neutral\", the sentence pair could be of alternation or independence relation. We label it as 4.  
The rules cover a few combinations of original and swapped pair labels. Not all combinations are semantically possible. The definition of \"entailment\" and \"contradiction\" make it impossible for a sentence pair and swapped pair to have these combinations. However, the fuzzy boundary between \"contradiction\" and \"neutral\" caused by entity or event coreference [18,20] may introduce cases of such combinations in the dataset.  
It is semantically not correct to have the original pair classified as \"contradiction\" and the swapped pair classified as any relation other than \"contradiction\". If some impossible combination appears in the dataset, it could be caused by either the classification error or labelling error. If each dataset contains some pairs with an impossible combination, this could be used to optimise the classifier.