# Transfer Learning for Multi-Premise Entailment with Relationship Processing Module
## 3 Methods
### Attention-Backtracking Mechanism  
The overall mechanism is shown in Figure 5. First of all, the Siamese-based framework is committed to building a sentence embedding vector after obtaining each sentence representation of premise sentence with an encoder. We hope to use a sentence embedding vector to build a preliminary contextual representation, which is a general description of the semantics of multiple premises:  
\[c=Add(P) \tag{1}\]  
\(P\in\mathbb{R}^{d\times n}\), its column vectors is the sentence representation of \(n\)\(d\)-dimensional premises, \(Add(. )\) is the sum of all column vectors element-by-element, \(c\in\) means obtained the preliminary contextual representation that can represent all premise sentences. Next, vector \(c\) and all context-related word vectors in each premise are calculated for attention. Let \(Y\in\mathbb{R}^{d\times l}\) be the context-related word vector matrix of a premise sentence after the encoding layer, and \(l\) is the number of words:  
\[M=tanh(W^{y}Y+W^{c}c\otimes e_{l}) \tag{2}\]  
\[\alpha=softmax(w^{T}M) \tag{3}\]  
\[r=Y\alpha^{T} \tag{4}\]  
\(W^{y},W^{c}\in\mathbb{R}^{d\times d},w\in\mathbb{R}^{d}\) are weight parameters to be learned, \(e_{l}\in\mathbb{R}^{l}\) is a vector and its elements are all 1, note that the outer product \(W^{c}c\otimes e_{l}\) is repeating the linearly transformed \(c\) as many times as there are words in this premise (i.e., \(l\) times). The column vector in \(M\in\mathbb{R}^{d\times l}\) is the attention representation of each word in the premise sentence, which is essentially a nonlinear combination of each word with the preliminary contextual representation. \(\alpha\in\mathbb{R}^{l}\) is an attention weight vector, which means the importance of each word in one premise. Therefore, the \(r\in\mathbb{R}^{d}\) obtained represents a new sentence representation combining attention information. The above operation is that when a single premise sentence is first encoded into a sentence vector, the resulting sentence vector contains biased information because there is no context information of other premise sentences. The idea of the attention-backtracking mechanism is that, after one premise is informed of other premises, the model hopes to confirm whether there is missing information or whether there is more critical information in the original sentence. Therefore, all the details will be encoded again in this section for more exciting information. Therefore, we will get a matrix \(R\in\mathbb{R}^{d\times n}\) by doing the above operations in each premise sentence, in which the column vector is composed of \(r\) calculated by each premise sentence. Figure 5: Attention-backtracking mechanism. Finally, we need to compress all the obtained premise vectors into a single premise vector equivalent to SPE data to input the inference layer, and we can still use pooling operation to achieve this:  
\[\overline{p}=AveragePooling(R) \tag{5}\]  
\(\overline{p}\in\mathbb{R}^{d}\) is the representation of multiple premises input to inference layer corresponding to SPE model. Since there is no sentence representation in the general sense in the \"matching-aggregation\" framework, as shown in Figure 6, we call the vector after the comparison module as aligned evaluation representation and the vector after the aggregate module as comprehensive evaluation representation. In our method, multiple premise sentences will have multiple comprehensive evaluation representations. The comprehensive evaluation representation can be equivalent to the sentence representation above. The aligned evaluation representation can be equivalent to the context-related word embedding representation in the previous text. In other words, after obtaining the information of other premises, the comprehensive evaluation representation of one premise sentence will go back to its own aligned evaluation representation, and the operation process is basically consistent with that described above.