## 3 Dual Pointer Network Model for Relation Extraction
### Dual Pointer Network Decoder  
In a pointer network, attentions show the position distributions of an encoding layer. Since attention is highlighted at only one position, the pointer network has a structural limitation when one entity forms relations with several entities (for instance, \"Lee\" in Figure 1). The proposed model adopts a dual pointer network decoder (see Figure 2) to overcome this limitation. The first decoder called an object decoder, learns the position distribution from subjects to objects as follows:  
\[ \begin{array}{l}h_{t}=[e_{t};s_{t}],\\ g_{t}=\text{LSTM}(h_{t},g_{t-1}),\\ score_{t}^{obj}=v^{obj}\text{tanh}(w^{obj}[\text{O};g_{t}]),\\ a_{t}^{obj}=\text{softmax}(score_{t}^{obj}),\\ \hat{p}_{t}^{obj}=\text{argmax}(a_{t}^{obj}),\\ \hat{r}_{t}^{obj}=\text{argmax}(w^{obj}\text{tan}\ln(z^{obj}[a_{t}^{obj} \text{O};g_{t}])),\end{array}  \tag{4}\]  
where \(h_{t}\) is a concatenation of the entity embedding vector \(e_{t}\) and the LSTM-encoded entity embedding vector \(s_{t}\), and the decoding vector \(g_{t}\) (i.e., the \(t\)-th entity to determine its objects) is calculated by the forward LSTM. Then, \(a_{t}^{obj}\) is the position distribution based on the attention scores \(score_{t}^{obj}\) between \(g_{t}\) and the other entities \(o_{1},\dots,o_{t-1},o_{t+1},\dots,o_{m}\) in the context-to-entity attention layer. \(\hat{p}_{t}^{obj}\) and \(\hat{r}_{t}^{obj}\) represent a position and a relation name of \(g_{t}\)'s object, respectively. The weighting parameters \(v,w,u,\text{ and }z\) are set during the training phase. Conversely, the second decoder, called a subject decoder,learns the position distribution from objects to subjects in the same manner as the object decoder, as shown below. \[ \begin{array}{c}score_{t}^{sub}=v^{sub}\text{tanh}(w^{sub}[\text{O};g_{t}]),\\ a_{t}^{sub}=\text{softmax}(score_{t}^{sub}),\\ \hat{p}_{t}^{sub}=\text{argmax}(a_{t}^{sub}),\\ \hat{p}_{t}^{sub}=\text{argmax}(u^{sub}\text{tanh}(z^{sub}[a_{t}^{sub}O;g_{t}] )),\end{array}  \tag{5}\]  
where \(\hat{p}_{t}^{sub}\) and \(\hat{p}_{t}^{sub}\) represent a position and a relation name of \(g_{t}\)'s subject, respectively. In Figure 1, \"[PERSON]\" should point to both \"ABC mart\" and \"his father.\" This problem cannot be solved using the conventional forward decoder because it cannot point to multiple targets. However, the subject decoder (a backward decoder) resolves this problem, because \"ABC mart\" and \"his father\" can point to \"[PERSON]\" Additionally, we adopt a multi-head attention mechanism to improve the performance of the dual pointer network; this is shown in the following equation. \[\begin{array}{c}q_{j}=w^{l}*splitsplitsplit(q,n)_{j},\\ a_{j}=softmax(\frac{q\hat{r}_{j}}{\sqrt{d}}),\\ \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\  
As a result, we obtained a dataset annotated with multiple triples. We divided the new dataset into a training set (5023 sentences), a development set (629 sentences), and a test set (627 sentences) by a ratio of 8:1:1. Table 1 shows the composition of the preprocessed ACE-2005 corpus in detail. NYT corpus [16]: This is a news corpus sampled from news articles published in the New York Times (NYT). The training data is automatically labeled using distant supervision. The NYT corpus was manually converted to a relation extraction dataset by [PERSON] et al. [17]. We excluded sentences without relation facts from [PERSON]'s corpus. Finally, we obtained 66,202 sentences in total. We used 59,581 sentences for training and 6621 for testing. Table 2 shows the composition of the NYT corpus in detail. Table 3 shows sample sentences and their tripe relations in the ACE-2005 corpus and the NYT corpus. \begin{table}
\begin{tabular}{c c} \hline \hline \# of entities per sentence (avg/max) & 3.5/22 \\ \hline \# of triples per sentence (avg/max) & 1.5/11 \\ \hline \# of entity types & 7 \\ \hline \# of relation types & 7 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Composition of the preprocessed ACE-2005 corpus.