SlowGuess/NAACL_2021_OCR huggingface dataset:
In Proceedings of the First Workshop on Computational Models of Reference, Anaphora and Coreference, pages 23-33.
Ina Rösiger, Arndt Riester, and Jonas Kuhn. 2018. Bridging resolution: Task definition, corpus resources and rule-based experiments. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3516-3528.
Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Francesca Delogu, Kepa Joseba Rodriguez, and Massimo Poesio. 2020. Annotating a broad range of anaphoric phenomena, in a variety of genres: the ARRAU corpus. Natural Language Engineering, 26(1):95-128.
Juntao Yu and Massimo Poesio. 2020. Multi-task learning based neural bridging reference resolution. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3534-3546.
A Rules for Bridging Resolution
Table 8 enumerates the list of heuristic rules manually designed for bridging resolution on ISNotes, BASHI, and ARRAU RST. As mentioned in Section 2, each rule is composed of a rule ID, an anaphor condition, and an antecedent condition. To enable the reader to better understand these rules, we describe in the last column of the table the motivation behind the design of each rule.
B Results on ARRAU RST
Results on ARRAU RST are shown in Tables 9 and 10. Specifically, Table 9 shows the performance of the three resolvers (Rules, MTL, and Hybrid) on different rule categories. This table is formatted in the same way as Table 4 and therefore can be interpreted in the same manner as Table 4. Comparing Table 4 and Table 9, we can see that Rules 11-17 are specifically designed for bridging resolution on ARRAU RST. Nevertheless, three of these seven rules are not fired on the ARRAU RST test set. Among these three rules, Rules 13 and 16 may have captured infrequent bridging phenomena, as they fail to cover any gold anaphors in the test set, whereas Rule 17 may have overfitted the training set, as it fails to recognize any gold anaphors in the test set. Overall, the Rules's results are somewhat disappointing: Rules outperforms MTL on only two of the rule categories, specifically the two defined by Rule 12 and Rule 14. In fact, our hypothesis that hand-crafted rules tend to have higher precision than machine-learned patterns fails on
Table 8: Complete set of hand-crafted rules for bridging resolution on ISNotes, BASHI, and ARRAU RST.
Rules 5, 6, 14, and 15. Consequently, the improvement of Hybrid over MTL on ARRAU RST is the smallest of the three evaluation datasets.
In Table 10, we attempt to quantify how different Rules and MTL are w.r.t. each rule category on ARRAU RST by showing the percentages of gold anaphors covered by each rule category that are correctly recognized/resolved correctly by both Rules and MTL (B), by Rules only (R), and by
MTL only (M). This table is formatted in the same way as Table 5 and therefore can be interpreted in the same way as Table 5. As we can see, the largest values in the "R" column for both recognition and resolution are associated with Rules 12 and 14, meaning that these are the rule categories in which Rules has unique strength. This observation is consistent with the results of Rules 12 and 14 in Table 9. Other than these two rule categories, Rules
Rule	Anaphors Mentions	Rules	MTL	Hybrid
Recognition	Resolution	Recognition	Resolution	Recognition	Resolution
P	R	F	P	R	F	P	R	F	P	R	F	P	R	F	P	R	F
1	1/17	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
5	8/92	56	62	59	11	12	12	100	75	86	50	38	43	64	88	74	18	25	21
6	1/2	50	100	67	0	0	0	100	100	100	0	0	0	50	100	67	0	0	0
10	56/110	53	42	47	38	30	34	53	67	59	38	49	43	48	74	59	32	49	39
11	215/2653	37	17	23	28	13	18	36	42	39	23	27	25	33	47	39	22	32	26
12	3/119	33	100	50	33	100	50	0	0	0	0	0	0	30	100	46	30	100	46
13	0/9	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
14	26/54	67	96	79	61	88	72	100	36	53	78	28	41	67	96	79	61	88	72
15	255/5163	16	8	11	11	6	8	32	27	29	20	17	18	25	31	28	17	20	18
16	0/0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
17	14/41	0	0	0	0	0	0	56	64	60	44	50	47	56	64	60	44	50	38
18	79/1987	0	0	0	0	0	0	30	21	24	20	14	17	30	21	24	20	14	17
Table 9: Performance of the three resolvers on different rule categories in ARRAU RST.
Rule	Recognition	Resolution
B	R	M	B	R	M
1	0	0	0	0	0	0
5	50	12	25	12	0	25
6	100	0	0	0	0	0
10	35	7	33	21	9	28
11	12	6	30	7	6	20
12	0	100	0	0	100	0
13	0	0	0	0	0	0
14	36	60	0	24	64	4
15	4	4	23	2	4	14
16	0	0	0	0	0	0
17	0	0	64	0	0	50
18	0	0	21	0	0	14
Table 10: Percentages of gold anaphors in each rule that are correctly recognized/resolved by both Rules and MTL (B), by Rules only (R), and by MTL only (M) in ARRAU RST.
manages to uniquely recognize/resolve just a few anaphors covered by rule categories 5, 10, 11, and 15. In contrast, the number of gold anaphors that are uniquely recognized/resolved by MTL is larger than that by Rules. Overall, we can infer from the results in Table 10 that the use of Rules does not add a lot of value to MTL on ARRAU RST.