## 3 Dual Pointer Network Model for Relation Extraction
### Experimental Results  
In the first experiment, we evaluated the effectiveness of the multi-head attention in the dual pointer network decoder; the results are summarized in Table 4. The evaluation was performed using the ACE-2005 corpus. In Table 4, single-head refers to a conventional attention mechanism proposed by [PERSON] et al. (2011). As shown in Table 4, the multi-head attention mechanism used in the proposed model demonstrated better performance than the single-head one. Then, using the ACE-2005 corpus, we evaluated the effectiveness of multi-head attention in the context and entity encoder; the results are summarized in Table 5.  
In Table 5, BIDAF ([PERSON] et al., 2018) refers to a machine-reading and comprehension (MRC) model based on a co-attention mechanism between a query and a context. C2Q and Q2C are referring to mean context-to-query attention and query-to-context attention used in the BIDAF model, respectively. As shown in Table 5, the multi-head attention mechanism used in the proposed model showed the best F1-score. The \(p\)-values of F1-scores between multi-head and the comparison models were from 5.0E-4 to 0.0039. This implies that the performance differences are statistically significant at the 0.05 level. In the second experiment, we compared the proposed model with previous state-of-the-art models. Table 6 compares the performance of the proposed model and with other models for the ACE-2005 corpus. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Recall** & **Precision** & **F1-Score** \\ \hline Single-head & 0.800 & 0.759 & 0.779 \\ Multi-head & 0.832 & 0.787 & 0.808 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance for different attention mechanisms in the dual pointer network decoder. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **Sentence** & **Triple** \\ \hline \multirow{3}{*}{ACE-2005} & Do you travel to meet up with & \multirow{3}{*}{\begin{tabular}{c} \{you, PER-SOC, family], \\ \{you, PER-SOC, friends}\} \\ \end{tabular} } \\  & family or friends during the & \multirow{3}{*}{\begin{tabular}{c} \{you, PER-SOC, friends] \\ \{you, PER-SOC, friends}\} \\ \end{tabular} } \\ \hline \multirow{3}{*}{NYT} & [PERSON] was & \multirow{3}{*}{\begin{tabular}{c} \{Lancaster County,/location/location/contains, Mount \\ \# of correct predict \\ \end{tabular} } \\  & born on Jan. 18, 1923, in the & \multirow{3}{*}{\
\begin{tabular}{c} \{Lancaster County town of Mount \\ \{[PERSON] \\ \end{tabular} } } \\  & Lancaster County town of Mount & \multirow{3}{*}{
\begin{tabular}{c} Newcomer,/people/person/place\_of\_birth, Mount \\ Joy \\ \end{tabular} } \\  & Joy, Pa. & & Joy \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Sample of the ACE-2005 corpus and the NYT corpus. PER-SOC: person-social. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Recall** & **Precision** & **F1-Score** \\ \hline \multirow{2}{*}{
\begin{tabular}{c} Single-head \\ Multi-head \\ \end{tabular} } & 0.800 & 0.759 & 0.779 \\  & 0.832 & 0.787 & 0.808 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance for different attention mechanisms in the dual pointer network decoder. In Table 6, SPTree [6] is a model that applies the dependency information between the entities. In FCM [19], handcrafted features are combined with word embeddings. DYGIE [9] dynamically generates spans between entities and spans' representations. Span-Level [20] jointly performs entity mention detection and relation extraction. HRCNN [21] is a hybrid model of CNN, RNN, and FNN. Walk-Based [22] is a graph-based neural network model. As shown in Table 6, the proposed model outperformed all models across all metrics. The \(p\)-values of F1-scores between the proposed model and the comparison models were from 5.81E-8 to 1.37E-5. This implies that the performance differences are statistically significant at the 0.001 level. Table 7 compares the performance of the proposed model with existing models for the NYT corpus.