# Transfer Learning for Multi-Premise Entailment with Relationship Processing Module
###### Abstract  
Using the single premise entailment (SPE) model to accomplish the multi-premise entailment (MPE) task can alleviate the problem that the neural network cannot be effectively trained due to the lack of labeled multi-premise training data. Moreover, the abundant judgment methods for the relationship between sentence pairs can also be applied in this task. However, the single-premise pre-trained model does not have a structure for processing multi-premise relationships, and this structure is a crucial technique for solving MPE problems. This paper proposes adding a multi-premise relationship processing module based on not changing the structure of the pre-trained model to compensate for this deficiency. Moreover, we proposed a three-step training method combining this module, which ensures that the module focuses on dealing with the multi-premise relationship during matching, thus applying the single-premise model to multi-premise tasks. Besides, this paper also proposes a specific structure of the relationship processing module, i.e., we call it the attention-backtracking mechanism. Experiments show that this structure can fully consider the context of multi-premise, and the structure combined with the three-step training can achieve better accuracy on the MPE test set than other transfer methods. transfer learning; multi-premise entailment; natural language inference; attention mechanism +
Footnote †: journal:  
0000-0001-9991-4000-0000-0000-0000-0000-0000-0000-00000-0000-0000-0000-0000-00000-0000-00000-0000-0000-0000-00000-00000-00000-00000-00000-00000-00  
There is no sequential relationship among multiple premises;
3. The information contained in each premise is unbalanced. In other words, for the semantic scenario constituted by all premises, each premise does not necessarily provide unique information for this scenario, and the description of one premise may also be a subset of another premise. This means that each premise may have a different level of contribution to the final classification. 4. Compared with the SPE data, the MPE data is more scarce, so that the corresponding network model cannot be effectively trained. **Premises:**:  
_A soccer game with multiple males playing._  
_Hypothesis:_  
_Some men are playing a sport._  
_Premises:_  
_An older and younger man smiling._  
_Hypothesis:_  
_Two men are smiling and laughing at the cats playing on the floor._  
_NEUTRAL_  
**Premises:**:  
_A man inspects the uniform of a figure in some East Asian country._  
_Hypothesis:_  
_The man is sleeping_  
_CONTRADCTION_  
**Figure 1.** Single-premise entailment task. **Premises:**:  
1. Two girls sitting down and looking at a book. 2. A couple laughs together as they read a book on a train. 3. Two travelers on a train or bus reading a book together. 4. A woman wearing glasses and a brown beanie next to a girl with long brown hair holding a book. **Hypothesis:**:  
_Women smiling._  
_Premises:_  
1. Three men are working construction on top of a building. 2. Three male construction workers on a roof working in the sun. 3. One man is shittless while the other two men work on construction. 4. Two construction workers working on infrastructure, while one worker takes a break. **Hypothesis:**:  
_A man smoking a cigarette._  
_Premises:_  
1. _A group of individuals performed in front of a seated crowd._
2. _Woman standing in front of group with black folders in hand._
3. _A group of women with black binders stand in front of a group of people._
4. _A group of people are standing at the front of the room, preparing to sing._  
**Hypothesis:**:  
_A group having a meeting._  
_CONTRADCTION_  
This paper proposes a method suitable for transferring from SPE pre-trained model to the MPE model. This method has the following characteristics:  
* Considering the existing SPE and MPE corpus, we think the biggest difference between the latter and the former is that the latter needs to deal with the relationship between multiple premises.