SlowGuess/ACL_2020_OCR huggingface dataset:
In this section, we carry out experiments using our BARQA system with different training strategies. For every bridging anaphor  $a$ , we choose the span with the highest confidence score from its context  $c_{a}$  as the answer for the question  $q_{a}$  and use this span as the predicted antecedent. We report results on ISNotes and BASHI using lenient accuracy (see Table 2).
Looking at the results on ISNotes, we find that BARQA trained on a small number of in-domain dataset (BASHI) achieves an accuracy of  $38.16\%$  on ISNotes, which is better than the model trained on the other two large-scale datasets (SQuAD 1.1 and QuasiBridging). However, when using these two datasets to pre-train the model then fine-tuning it with the small in-domain dataset (BASHI), both settings (i.e., SQuAD 1.1 + BASHI and QuasiBridging + BASHI) achieve better results compared to using BASHI as the only training dataset. This verifies the value of the pre-training + fine-tuning strategy, i.e., pre-training the model with large scale out-of-domain or noisy dataset, then fine-tuning it with a small in-domain dataset.
Particularly, we notice that the performance of using QuasiBridging alone is worse than the one using SQuAD 1.1 only. However, combining QuasiBridging and BASHI achieves the best result on ISNotes, with an accuracy of  $47.21\%$ . It seems that the large-scale in-domain noisy training data (QuasiBridging) brings more value than the large-scale out-of-domain training data (SQuAD 1.1).
We observe similar patterns on the results on
$^{7}$ BASHI considers comparative anaphora as bridging anaphora. We exclude them from this study.  
 $^{8}$ Note that these WSJ articles are different from the ones in ISNotes.
Corpus	Genre	Bridging Type	# of Anaphors	# QA Pairs
ISNotes	WSJ news articles	referential bridging	663	1,115
BASHI	WSJ news articles	referential bridging	344	486
SQuAD 1.1 (train)	Wikipedia paragraphs	-	-	87,599
QuasiBridging	NYT news articles	quasi bridging	2,870,274	2,870,274
Table 1: Four datasets used for experiments.
BARQA	Lenient Accuracy on ISNotes	Lenient Accuracy on BASHI
Large-scale (out-of-domain/noisy) training data
SQuAD 1.1	28.81	29.94
QuasiBridging	25.94	17.44
Small in-domain training data
BASHI	38.16	-
ISNotes	-	35.76
Pre-training + In-domain fine-tuning
SQuAD 1.1 +BASHI	42.08	-
QuasiBridging +BASHI	47.21*	-
SQuAD 1.1 + ISNotes	-	35.76
QuasiBridging + ISNotes	-	37.79
Table 2: Results of BARQA on ISNotes and BASHI using different training strategies.  $\star$  indicates statistically significant differences over the other models (two-sided paired approximate randomization test,  $p < 0.01$ ).
BASHI. Pre-training the model on QuasiBridging then fine-tuning it on ISNotes achieves the best result with an accuracy of  $37.79\%$ . Furthermore, when evaluating on BASHI, it seems that using SQuAD 1.1 as the pre-training dataset does not bring additional values when combining it with ISNotes.
5.4 Results on ISNotes and BASHI Compared to Previous Approaches
Previous work for bridging anaphora resolution on ISNotes and BASHI use gold/system mentions as antecedent candidates and report results using strict accuracy (Hou et al., 2013b; Hou, 2018a).
In order to fairly compare against these systems, for every bridging anaphor  $a$ , we first map all top 20 span predictions of our system BARQA to the gold/system mentions, then we choose the gold/system mention with the highest confidence score as the predicted antecedent. Specifically, we map a predicted span  $s$  to a mention  $m$  if they share the same head and  $s$  is part of  $m'$  ( $m'$  is created by removing all postmodifiers from  $m$ ). For instance, "total potential claims" is mapped to the mention "the total potential claims from the disaster". If a predicted span can not be mapped to any gold/system mentions, we filter it out. Following
Hou (2018a), we only keep the predictions whose semantic types are "time" if  $a$  is a time expression. The above process is equal to using gold/system mentions and their semantic information to further prune BARQA's span predictions.
Table 3 and Table 4 compare the results of our system BARQA against previous studies for bridging anaphora resolution on ISNotes and BASHI, respectively. For both datasets, the BARQA model is trained using the best strategy reported in Table 2 (pre-training with QuasiBridging + fine-tuning with small in-domain data).
On ISNotes, previously Hou (2018a) reported the best result by adding the prediction from a deterministic algorithm (embeddings bridging (NP head + modifiers)) as an additional feature into the global inference model (MLN II) proposed by Hou et al. (2013b). The deterministic algorithm is based on word embeddings for bridging and models the meaning of an NP based on its head noun and modifications.
Our system BARQA, when using the gold mentions together with their semantic information to further prune the span predictions, achieves the new state-of-the-art result on ISNotes, with a strict accuracy of  $50.08\%$  (see BARQA with gold mentions/semantics, strict accuracy in Table 3). How-
System	Use Gold Mentions	Accuracy
Models from Hou et al. (2013b)
pairwise model III	yes	36.35
MLN model II	yes	41.32
Models from Hou (2018a)
embeddings_bridgeging (NP head + modifiers)	yes	39.52
MLN model II + embeddings_bridgeging (NP head + modifiers)	yes	46.46
This work
BARQA with gold mentions/semantics, strict accuracy	yes	50.08
BARQA without mention information, strict accuracy	no	36.05
BARQA without mention information, lenient accuracy	no	47.21
Table 3: Results of different systems for bridging anaphora resolution in ISNotes. Bold indicates statistically significant differences over the other models (two-sided paired approximate randomization test,  $p < 0.01$ ).
System	Use System Mentions	Accuracy
Model from Hou (2018a)
embeddings_bridgeing (NP head + modifiers)	yes	29.94
This work
BARQA with system mentions/semantics, strict accuracy	yes	38.66
BARQA without mention information, strict accuracy	no	32.27
BARQA without mention information, lenient accuracy	no	37.79
Table 4: Results of different systems for bridging anaphora resolution in BASHI. Bold indicates statistically significant differences over the other models (two-sided paired approximate randomization test,  $p < 0.01$ ).
ever, we argue that using gold mention information to construct the set of antecedent candidates is a controlled experiment condition, and our experiment setup BARQA without mention information, lenient accuracy is a more realistic scenario in practice.
On BASHI, Hou (2018a) reported an accuracy of  $29.94\%$  (strict accuracy) using automatically extracted mentions from the gold syntactic tree annotations. Our system BARQA without any mention/semantic information achieves an accuracy of  $32.27\%$  using the same strict accuracy evaluation. The result of BARQA is further improved with an accuracy of  $38.66\%$  when we integrate mention/semantic information into the model.
Note that Hou (2018a) also adapted their deterministic algorithm to resolve lexical bridging anaphors on ARRAU (Poesio et al., 2018) and reported an accuracy of  $32.39\%$  on the RST Test dataset. Although in this paper we do not focus on lexical bridging, our model BARQA can also be applied to resolve lexical bridging anaphors. We found that BARQA trained on the RST Train dataset alone with around 2,000 QA pairs achieves an accuracy of  $34.59\%$  on the RST Test dataset.
6 Error Analysis
In order to better understand our model, we automatically label bridging anaphors in ISNotes as either “referential bridging/world-knowledge” or “referential bridging/context-dependent”. We then analyze the performance of BARQA and the best model from Hou (2018a) on these two categories.
Rösiger et al. (2018) pointed out that although lexical and referential bridging are two different concepts, sometimes they can co-occur within the same pair of expressions. In Example 2, "Employees" is an anaphoric expression. At the same time, the relation between the antecedent entity " $\{Mobil Corp./the company's\}$ " and the bridging anaphor "Employees" corresponds to the common-sense world knowledge which is true without any specific context. We call such cases as "referential bridging/world-knowledge". Differently, we call a bridging anaphor as "referential bridging/context-dependent" if it has multiple equally plausible antecedent candidates according to the common-sense world knowledge about the NP pairs and we have to analyze the context to choose the antecedent (see Example 1). One may
# pairs	BARQA	MLN II + emb
Know.	256	71.88	88.28
Context	407	36.36	19.90
Table 5: Comparison of the percentage of correctly resolved anaphors between BARQA and the best model from Hou (2018a) on two bridging categories.
argue that  $"\{$  the exploration and production division - Employees\} in Example 2 is also a valid common-sense knowledge fact, however, we consider that it is less prominent than "the company's -Employees\}.
(2) Mobil Corp. is preparing to slash the size of its workforce in the U.S., possibly as soon as next month, say individuals familiar with the company's strategy. The size of the cuts isn't known, but they'll be centered in the exploration and production division, which is responsible for locating oil reserves, drilling wells and pumping crude oil and natural gas. Employees haven't yet been notified.
For a bridging anaphor  $a$ , the deterministic algorithm (embeddings_bridging) from Hou (2018a) uses a word representation resource learned from a large corpus to predict the most semantically related NP among all NP candidates as the antecedent. The predictions from this system reflect the common-sense world knowledge about the NP pairs. We thus use this algorithm to label bridging anaphors in ISNotes: if a bridging anaphor is correctly resolved by embeddings_bridging, we label it as "referential bridging/world-knowledge", otherwise the label is "referential bridging/context-dependent".
Table 5 compares the percentage of correctly resolved anaphors between BARQA with gold mentions and the best model from Hou (2018a) (MLN II + emb) on the two bridging categories. Note that MLN II + emb contains several context-level features (e.g., document span, verb pattern). Overall, it seems that our BARQA model is better at resolving context-dependent bridging anaphors.
In this paper, we model bridging anaphora resolution as a question answering problem and propose a QA system (BARQA) to solve the task.
We also propose a new method to automatically generate a large scale of "quasi-bridging" training data. We show that our QA system, when
trained on this "quasi-bridging" training dataset and fine-tuned on a small amount of in-domain dataset, achieves the new state-of-the-art results on two bridging corpora.
Compared to previous systems, our model is simple and more realistic in practice: it does not require any gold annotations to construct the list of antecedent candidates. Moreover, under the proposed QA formulation, our model can be easily strengthened by adding other span-based text understanding QA corpora as pre-training datasets.
Finally, we will release our experimental QA datasets (in the SQuAD json format) for bridging anaphora resolution on ISNotes and BASHI. They can be used to test a QA model's ability to understand a text in terms of bridging inference.
The author appreciates the valuable feedback from the anonymous reviewers.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, 2-7 June 2019, pages 4171-4186.
Kerstin Eckart, Arndt Riester, and Katrin Schweitzer. 2012. A discourse information radio news database for linguistic analysis. In Christian Chiarcos, Sebastian Nordhoff, and Sebastian Hellmann, editors, Linked Data in Linguistics, pages 65-76. Springer Berlin Heidelberg.
Kari Fraurud. 1990. Definiteness and the processing of noun phrases in natural discourse. Journal of Semantics, 7:395-433.
Claire Gardent and Hélène Manuelian. 2005. Création d'un corpus annote pour le traitement des descriptions définies. Traitément Automatique des Langues, 46(1):115-140.
Matt Gardner, Jonathan Berant, Hannaneh Hajishirzi, Alon Talmor, and Min Sewon. 2019. Question answering is a format; when is it useful? arXiv preprint arXiv:909.11291.
Yufang Hou. 2016. Incremental fine-grained information status classification using attention-based LSTMs. In Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Japan, 11-16 December 2016, pages 1880-1890.
Yufang Hou. 2018a. A deterministic algorithm for bridging anaphora resolution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October-4 November 2018, pages 1938-1948.
Yufang Hou. 2018b. Enhanced word representations for bridging anaphora resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, 1-6 June 2018, pages 1-7.
Yufang Hou, Katja Markert, and Michael Strube. 2013a. Cascading collective classification for bridging anaphora recognition using a rich linguistic feature set. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Wash., 18-21 October 2013, pages 814-820.
Yufang Hou, Katja Markert, and Michael Strube. 2013b. Global inference for bridging anaphora resolution. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, Georgia, 9-14 June 2013, pages 907-917.
Yufang Hou, Katja Markert, and Michael Strube. 2014. A rule-based system for unrestricted bridging resolution: Recognizing bridging anaphora and finding links to antecedents. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 25-29 October 2014, pages 2082-2093.
Yufang Hou, Katja Markert, and Michael Strube. 2018. Unrestricted bridging resolution. Computational Linguistics, 44(2):237-284.
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2019. Spanbert: Improving pre-training by representing and predicting spans. arXiv preprint arXiv:1907.10529.
Vid Kocijan, Ana-Maria Cretu, Oana-Maria Camburu, Yordan Yordanov, and Thomas Lukasiewicz. 2019. A surprisingly robust trick for the Winograd Schema Challenge. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 28 July-2 August 2019, pages 4837-4842.
Emmanuel Lassalle and Pascal Denis. 2011. Leveraging different meronym discovery methods for bridging resolution in French. In Proceedings of the 8th Discourse Anaphora and Anaphor Resolution Colloquium (DAARC 2011), Faro, Algarve, Portugal, 6-7 October 2011, pages 35-46.
Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019.
Entity-relation extraction as multi-turn question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 28 July-2 August 2019, pages 1340-1350.
Katja Markert, Yufang Hou, and Michael Strube. 2012. Collective classification for fine-grained information status. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Jeju Island, Korea, 8-14 July 2012, pages 795-804.
Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018. The natural language decathlon: Multitask learning as question answering. arXiv preprint arXiv:1806.08730.
Courtney Naples, Matthew Gormley, and Benjamin Van Durme. 2012. Annotated Gigaword. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction & Web-scale Knowledge Extraction (AKBC-WEKEX) Montreal, Quebec, Canada, 7-8 June 2012, pages 95-100.
Anna Nedoluzhko, Jiri Mirovský, and Petr Pajas. 2009. The coding scheme for annotating extended nominal coreference and bridging anaphora in the Prague dependency treebank. In Proceedings of the Third Linguistic Annotation Workshop at ACL-IJCNLP 2009, Suntec, Singapore, 6-7 August 2009, pages 108-111.
Malvina Nissim, Shipara Dingare, Jean Carletta, and Mark Steedman. 2004. An annotation scheme for information status in dialogue. In Proceedings of the 4th International Conference on Language Resources and Evaluation, Lisbon, Portugal, 26-28 May 2004, pages 1023-1026.
Juri Opitz and Anette Frank. 2018. Addressing the Winograd Schema Challenge as a sequence ranking task. In Proceedings of the First International Workshop on Language Cognition and Computational Models, pages 41-52.
Robert Parker, David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2011. English Gigaword Fifth Edition. LDC2011T07.
Massimo Poesio. 2004. The MATE/GNOME proposals for anaphoric annotation, revisited. In Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue, Cambridge, Mass., 30 April - 1 May 2004, pages 154-162.
Massimo Poesio, Yulia Grishina, Varada Kolhatkar, Nafise Sadat Moosavi, Ina Rosiger, Adam Roussel, Fabian Simonjetz, Alexandra Uma, Olga Uryupina, Juntao Yu, and Heike Zinsmeister. 2018. Anaphora resolution with the ARRAU corpus. In Proceedings of the Workshop on Computational Models of Reference, Anaphora and Coreference. New Orleans, Louisiana, June 6, 2018, pages 11-22.
Massimo Poesio, Rahul Mehta, Axel Maroudas, and Janet Hitzeman. 2004. Learning to resolve bridging references. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, 21-26 July 2004, pages 143-150.
Massimo Poesio and Renata Vieira. 1998. A corpus-based investigation of definite description use. Computational Linguistics, 24(2):183-216.
Massimo Poesio, Renata Vieira, and Simone Teufel. 1997. Resolving bridging references in unrestricted text. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Text, Madrid, Spain, July 1997, pages 1-6.
Altaf Rahman and Vincent Ng. 2012. Resolving complex cases of definite pronouns: The Winograd Schema Challenge. In Proceedings of the 2012 Conference on Empirical Methods in Natural Language Processing and Natural Language Learning, Jeju Island, Korea, 12-14 July 2012, pages 777-789.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,  $000+$  questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, USA, 1-4 November 2016, pages 2383-2392.
Ina Rösiger. 2018. BASHI: A corpus of wall street journal articles annotated with bridging links. In Proceedings of the 11th International Conference on Language Resources and Evaluation, Miyazaki, Japan, 7-12 May 2018, pages 382-388.
Ina Rösiger, Arndt Riester, and Jonas Kuhn. 2018. Bridging resolution: Task definition, corpus resources and rule-based experiments. In Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New-Mexico, USA, 20-26 August 2018, pages 3516-3528.
Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2011. OntoNotes release 4.0. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium.
Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. Coreference resolution as query-based span prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, Wash., 5-10 July 2020.