# Entity-Centric Fully Connected GCN for Relation Classification
## 3 Experiment
### Ablation Study  
To demonstrate the effectiveness of various components of the propose model, we implement a series of ablation experiments on the TACRED dataset. The results of ablation study are shown in Table 4, from which we can observe that the performance of the proposed model drops with removing the components. Especially removing the pruning strategy, the F1 score decreases 2.2, which indicates the importance of the pruning mechanism. The performance drops by 0.7 and 0.9 respectively when removing the Bi-LSTM layer and multi-head attention layer. It is worth noting that the performance drops by 1.5 when masking the entities with random tokens, which indicates that the semantic information contained in the entity mentions is crucial for relation classification. \begin{table}
\begin{tabular}{l l} \hline \hline
**Model** & **Dev F1** \\ \hline FCGCN & **67.1** \\ -Pruning category & 64.9 \\ -Bi-LSTM layer & 66.4 \\ -Multi-head layer & 66.2 \\ -Mask-entity & 65.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Ablation study on TACRED. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Model** & **P** & **R** & **F1** \\ \hline LR & 73.5 & 49.9 & 59.4 \\ CNN & **75.6** & 47.5 & 58.3 \\ SDP-LSTM & _66.3_ & 52.7 & 58.7 \\ Tree-LSTM & _66.0_ & 59.2 & 62.4 \\ PA-LSTM & _65.7_ & **64.5** & 65.1 \\ C-GCN & 69.9 & 63.3 & 66.4 \\ AGGCN & 69.9 & 60.9 & 65.1 \\ \hline FCGCN (ours) & 72.2 & 62.0 & **67.1** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results on TACRED. \begin{table}
\begin{tabular}{l l} \hline \hline
**Model** & **F1** \\ \hline CNN ([PERSON] et al., 2014) & _83.7_ \\ SDP-LSTM ([PERSON] et al., 2015b) & _84.4_ \\ Tree-LSTM ([PERSON] et al., 2015) & _82.7_ \\ LR ([PERSON] et al., 2017) & _82.2_ \\ PA-LSTM ([PERSON] et al., 2017) & _84.8_ \\ C-GCN ([PERSON] et al., 2018) & _84.8_ \\ C-AGGCN ([PERSON] et al., 2020) & _85.7_ \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results on SemEval 2010 Task 8. **Effect of Pruning:** Figure 3 shows the effect of the FCGCN model with pruning tree, where K indicates that the distance between the token contained in the pruning tree and the dependent path in the LCA subtree is at most K. In order to show the path-centric pruning effect, when the pruning distance K changes, we compare the experimental results under different epochs. We conducted experiments on \(K\in\{0,1,2,\infty\}\) on the TACRED development set, and compared the models without pruning. As shown in Figure 3, when K = 1, the performance of all epochs reaches the peak, which also evaluates that pruning improves the proposed model. **Effect of Mask-entity:** Figure 4 shows the effect of the FCGCN model with and without concealed entities.To show the role of the entity itself in the FCGCN model, we compared the experimental results under different epochs. As shown in Figure 4, the entity is not covered, and the performance of all epochs reaches the peak, which also evaluates the importance of the entity in relation extraction. Figure 4: Experimental results in terms of F1 under different epochs with and without masking the entities. Figure 3: Experimental results in terms of F1 under different epochs whih different pruning strategies. **Analyze of Bi-LSTM & Multi-Head Attention:** In the relation classification model based on deep learning, Bi-LSTM and multi-head attention mechanism are widely used in the model. By capturing the forward and backward LSTM information, each word in the sentence can capture contextual semantic information. Multi-head attention mechanism can help us pay more attention to the important words in the sentence, rather than some words that can bring noise. This model uses multiple queries to calculate in parallel to select multiple pieces of information from the input information to assign attention weights to each word in the sentence, playing the role of global observation.