# Few-Shot Relation Extraction on Ancient Chinese Documents
## 4 Methodology
### Paired Attention  
On the basis of the soft-align elements [55] for input representation and the enhancement of local inference information [35], we used a paired instance attention mechanism in this study, which consists of three parts. The first part is an input sentence representation attention, \(S\) and \(Q\), which are generated by the sentence encoder, for the support and query instance representations. Similar to MLMAN, we first reshape \(S\) from \(\mathbb{R}^{N\times K\times H}\) to \(\mathbb{R}^{T_{s}\times H}\). Similar to \(S\), query instance \(Q\) also has the shape \(\mathbb{R}^{T_{q}\times H}\), where \(T_{q}\) represents the total number of query instances. Then, attention weights between the support and query instances can be calculated by  
\[e_{ij}=Q_{i}^{\top}\cdot S_{j}, \tag{6}\]  
where \(Q_{i}\) and \(S_{j}\) represent the \(i\)-th query instances and the \(j\)-th support instances, respectively; and \(e_{ij}\) is the attention weight matrix.  
With the attention weight matrix, the attention-weighted support and query instance representations can be calculated as follows:  
\[\widetilde{S}_{i}=\sum_{j=1}^{T_{q}}\frac{exp\big{(}e_{ij}\big{)}}{\sum_{j^{ \prime}=1}^{T_{q}}exp\big{(}e_{ij^{\prime}}\big{)}}\cdot S_{j}, \tag{7}\]  
\[\widetilde{Q}_{j}=\sum_{i=1}^{T_{q}}\frac{exp\big{(}e_{ij}\big{)}}{\sum_{i^{ \prime}=1}^{T_{q}}exp\big{(}e_{ij}\big{)}}\cdot Q_{i}, \tag{8}\]  
Figure 2: MASCOT encoder structure of our model.  
where \(\widetilde{S}_{i}\) collects the attention weights of each query instance and attempts to identify related and irrelevant parts; these relations will be magnified by a product operation. The same process is performed with \(\widetilde{Q}_{j}\).  
The second part is an aggregate of attention-weighted and original instance representations, and it is designed to enhance instance information. In support instance embedding \(S\), the attention-weighted embedding \(\widetilde{Q}_{j}\) has attention information about \(S\) and \(Q\), and the instance discrepancy metric between \(S\) and \(Q\) can be measured using the difference \(|S-\widetilde{Q}|\) and element-wise product \(S\odot\widetilde{Q}\). Thus, the aggregate operation for support instance embedding will be expressed as follows:  
\[\widetilde{S}=f_{\phi}([S;\widetilde{Q};|S-\widetilde{Q}|;S\odot\widetilde{ Q}]), \tag{9}\]  
and it is the same for query instance embedding:  
\[\widetilde{Q}=f_{\phi}([Q;\widetilde{S};|Q-\widetilde{S}|;Q\odot\widetilde{ S}]), \tag{10}\]  
where the function \(f_{\phi}:\mathbb{R}^{T_{s}\times dH}\rightarrow\mathbb{R}^{T_{s}\times D}\) compressed aggregated instance embedding into a denser space. Then, the support instance embedding will be split into \(N\) classes. Next, an instance-encoded sample, which is a conjunction of the support and query instance embedding, will be sent to instance pair unit.  
\[\textit{Paired}=\textit{MASCOT}(\textit{Concat}([\widetilde{S};\widetilde{Q}])) \tag{11}\]  
The third part is a paired instance relation extractor, which is designed to discover the inner relations between two paired instances. The extractor uses two layers of MASCOT to learn the relation connection of each word in the support and query instances, which is the same as Equation (11). Then, the paired instance embedding will be compressed using the max and average pooling operations in Equation (12).  
\[\textit{Out}=\textit{Concat}([\textit{MaxPooling}(\textit{Paired});\textit{ ArgPooling}(\textit{Paired})]) \tag{12}\]  
Finally, the adjusted instance vector will be normalized by layer normalization and sent to a class predictor to find their corresponding labels.