# Extraction of Joint Entity and Relationships with Soft Pruning and GlobalPointer
## 4 Experiment
### Result and Analysis  
#### 4.4.1 Main Results  
We compare the following baseline models with our proposed model:  
* **NovelTagging**[35] applies a novel tagging method to transform the joint extraction of entities and relations into a sequence tagging problem, but it cannot tackle the overlap problem. * **CopyRE**[32] uses seq2 seq to generate all triples to solve the overlap problem for a sentence, but such an approach only considers a single token and not multiple tokens. * **GraphRel**[24]: A model that generates a weighted relation graph for each relation type, and applies a GCN to predict relations between all entity pairs. * **OrderCopyRE**[23]: An improved model of CopyRE that uses reinforcement learning to generate multiple triples. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Category**} & \multirow{2}{*}{**Train**} & **NYT** & \multicolumn{2}{c}{**WebNLG**} \\  & & **Test** & **Train** & **Test** \\ \hline Normal & 37,013 & 3266 & 1596 & 246 \\ EPO & 9782 & 978 & 227 & 26 \\ SEO & 14,735 & 1297 & 3406 & 457 \\ ALL & 56,195 & 5000 & 5019 & 703 \\ \hline \hline \end{tabular}  
**NYT**: NYT is a widely used dataset for relation extraction tasks. This dataset is generated by aligning relations in freebase with the New York Times (NYT) corpus in a remotely supervised manner and it contains 24 predefined relations and 1.18M sentences. **WebNLG**: WebNLG is a general dataset for evaluating relation extraction models; the dataset uses triples in DBPedia, so it contains more relations, and it contains 246 predefined relations.  
\end{table} 
Table 1: Statistics of datasets. Note that a sentence can belong to both EPO class and SEO class. * **ETL-Span**[36] decomposes the joint extraction task into two subtasks. The first subtask is to distinguish all head entities that may be related to the target relation, and the second subtask is to determine the corresponding tail entity and relation for each extracted head entity. * **WDec**[48]: An improved model of CopyRE, which solves the problem that CopyRE misses multiple tokens. * **CasRel**[26] identifies the head entity first and then the tail entity under a particular relationship. * **DualDec**[49] designs an efficient cascaded dual-decoder approach to address the extraction of overlapping relation triplets, which consists of a text-specific relation decoder and a relation-corresponding entity decoder. * **RMAN**[50] not only considers the semantic features in the sentence but also leverages the relation type to label the entities to obtain a complete triple. Table 2 reports the main results of our model and the other baseline models on both datasets. As can be seen from the table, our model outperforms all baseline models in the F1 score. For the NYT dataset, our model achieves the best results in recall and F1 scores, with F1 scores improving by 1.7%, 0.8%, and 5.9% over the CasRel framework, DualDec model, and RSAN model, respectively. For the WebNLG dataset, our model performs well, with F1 scores improving by 0.1%, 1.0%, and 7.4% over the CasRel framework, DualDec model, and RSAN model, respectively. In the experiment, to verify the effectiveness of the graph module and the joint decoding module, we removed the graph module and only kept the BERT model and the joint decoding module, which were named SGNetw\({}_{\text{WG}}\). For the NYT dataset, the F1 score of the SGNetw\({}_{\text{WG}}\) model is 0.6% higher than that of CasRel, which only uses the BERT module and the decoding module. Furthermore, compared with SGNetw\({}_{\text{WG}}\), our model improves by 1.1% and 1.7% on the NYT and WebNLG, respectively. These encouraging results show that the graph module and joint decoding module can effectively assist the model to extract the relation triples. Although SGNetw\({}_{\text{WG}}\) does not perform very well for the WebNLG dataset, SGNet performs better. Previous studies have shown that it is already difficult to improve the F1 score obtained by the model on the dataset when it reaches 90+, which is close to the human limit [26].