SlowGuess/NAACL_2022_OCR huggingface dataset:
Dominican Republic. Association for Computational Linguistics.
Benfeng Xu, Quan Wang, Yajuan Lyu, Yong Zhu, and Zhendong Mao. 2021. Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16):14149-14157.
Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, and Maosong Sun. 2019. DocRED: A large-scale document-level relation extraction dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 764-777, Florence, Italy. Association for Computational Linguistics.
Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Peng Li, Maosong Sun, and Zhiyuan Liu. 2020. Coreferential Reasoning Learning for Language Representation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7170-7186, Online. Association for Computational Linguistics.
Adams Wei Yu, David Dohan, Quoc Le, Thang Luong, Rui Zhao, and Kai Chen. 2018. Fast and accurate reading comprehension by combining self-attention and convolution. In International Conference on Learning Representations.
Shuang Zeng, Runxin Xu, Baobao Chang, and Lei Li. 2020. Double graph based reasoning for document-level relation extraction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1630-1640, Online. Association for Computational Linguistics.
Ningyu Zhang, Xiang Chen, Xin Xie, Shumin Deng, Chuanqi Tan, Mosha Chen, Fei Huang, Luo Si, and Huajun Chen. 2021. Document-level relation extraction as semantic segmentation. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 3999-4006. International Joint Conferences on Artificial Intelligence Organization. Main Track.
Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, and Jiebo Luo. 2019. Learning deep bilinear transformation for fine-grained image representation. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
Hengyi Zheng, Rui Wen, Xi Chen, Yifan Yang, Yunyan Zhang, Ziheng Zhang, Ningyu Zhang, Bin Qin, Xu Ming, and Yefeng Zheng. 2021. PRGC: Potential relation and global correspondence based joint relational triple extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6225-6235, Online. Association for Computational Linguistics.
Dataset	No. of Instances w.r.t. Split	Entities (Avg.)	Relations	No. of Instances w.r.t. Multi-triples
Train	Dev	Test	N=1	1<N<=5	5<N<=25	N>25
DocRED	3053	1000	1000	19.5	96	48	561	3171	234
NYT*	56195	4999	5000	2.15	24	43397	22207	590	NA
WebNLG*	5019	500	703	3.15	171	2189	3969	64	NA
NYT	56196	5000	5000	2.16	24	43358	22237	601	NA
WebNLG	5019	500	703	3.26	216	2277	3862	83	NA
Table 4: Statistics of used datasets. * denotes task settings that only annotate the last word.  $N$  denotes the number of valid triples within an instance. We can see that these selected benchmarks all involve multiple triples, thus pose significant challenge for relation extraction systems.
A Benchmarks
We introduce the benchmarks used in this work. Table 4 gives their detailed statistics. DocRED is constructed from Wikipedia document. It provides comprehensive human annotations for entity mentions, entity types, relational triples, along with their supporting evidences. Each document is a semantically integrate unit that centers in one concept (the title of the wiki page), resulting multiple triples with rich correlations. NYT is constructed from New York Times news articles and annotated through distant supervision. WebNLG is originally created for natural language generation task, and the sentences are written by humans to cover given triples. Both datasets have the other version denoted as NYT* and WebNLG*. The texts in NYT and WebNLG are much shorter than DocRED documents. These two datasets also feature in multiple triples. In this paper, we solve all three datasets under a unified multi-triple extraction formulation with EmRel.
To provide comparable results, we set hyperparameters following previous works (Wang et al., 2020; Xu et al., 2021). On NYT / WebNLG, we set learning rate as 5e-5, batch size as 24 / 6, and epoch as 100, as each word is seen as a candidate entity, we directly take the word representation as entity representation. On DocRED, we set learning rate as 3e-5, batch size as 4, and search epochs in {40, 60, 80, 100}. Each document is truncated by 512 sequence length. Entity representation is constructed by pooling from all its mention positions. To produce more robust results, we further perform multiple searches using 5 different seeds, resulting a grid search on both epochs and random seeds. The mean and standard deviation results across different seed are reported on development set. All
experiments are conducted on a single NVIDIA V100 or A100 GPU machine.
C Grouped Alignment
The WebNLG dataset has up to 216 relations, which requires increased computational cost. Inspired by (Zheng et al., 2019), we split the alignment tensors into  $N$  groups across its dimensions to reduce the computational overhead, and re-write Eq. 7 as:
$$
\phi \left(s _ {i}, p _ {k}, o _ {j}\right) = \sum_ {n = 1} ^ {N} \mathcal {Z} ^ {n} \times_ {1} \hat {\mathrm {e}} _ {i} ^ {s, n} \times_ {2} \hat {\mathrm {r}} _ {k} ^ {n} \times_ {3} \hat {\mathrm {e}} _ {j} ^ {o, n} + b _ {k} \tag {9}
$$
$$
\hat {\mathbf {e}} _ {i} ^ {s, n} = \widehat {\mathbf {E}} _ {i, [ (n - 1) \frac {d _ {e}}{N}: n \frac {d _ {e}}{N} ]} ^ {s}
$$
$$
\hat {\mathbf {r}} _ {k} ^ {n} = \widehat {\mathbf {R}} _ {k, \left[ (n - 1) \frac {d _ {r}}{N}: n \frac {d _ {r}}{N} \right]} \tag {10}
$$
$$
\hat {\mathbf {e}} _ {i} ^ {o, n} = \widehat {\mathbf {E}} _ {j, [ (n - 1) \frac {d _ {e}}{N}: n \frac {d _ {e}}{N} ]} ^ {o}
$$
We set group  $N$  to 4 for WebNLG, and 1 for other datasets (that is, without further splitting).