QA -Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions (2021)

Abstract: Multi-text applications, such as multi-document summarization, are typically required to model redundancies across related texts. Current methods confronting consolidation struggle to fuse overlapping information. In order to explicitly represent content overlap, we propose to align predicate-argument relations across texts, providing a potential scaffold for information consolidation. We go beyond clustering coreferring mentions, and instead model overlap with respect to redundancy at a propositional level, rather than merely detecting shared referents. Our setting exploits QA-SRL, utilizing question-answer pairs to capture predicate-argument relations, facilitating laymen annotation of cross-text alignments. We employ crowd-workers for constructing a dataset of QA-based alignments, and present a baseline QA alignment model trained over our dataset. Analyses show that our new task is semantically challenging, capturing content overlap beyond lexical similarity and complements cross-document coreference with proposition-level links, offering potential use for downstream tasks.

## introduction end-to-end neural methods have become the defacto standard for natural language understanding models. while these often work well for single document tasks, tasks that consider multiple textual inputs remain more challenging. a key difficulty concerns consolidating information from different, possibly redundant texts, which is crucial for applications such as multi-document summarization (mds), sentence fusion @xcite or multi-hop question-answering @xcite . previous works show that mds methods for example, often just concatenate rather than merge inputs @xcite , or erroneously consolidate on non-coreferring elements @xcite . recognizing such challenges, a few earlier mds works @xcite @xcite attempted at leveraging semantic structures for consolidation, such as abstract meaning representation @xcite or open knowledge graph @xcite , however, these initial means were found either too fine-grained (amr) or immature (okr) for efficient downstream consolidation. in this paper, we propose that a useful step toward effective consolidation is to detect and align minimal propositions that refer to the same information. therefore, we choose to capture propositions at the fine-grained level of predicate-argument relations, each representing a single predication involving two sentence elements, along with the semantic role that relates them. identifying these types of alignments at the propositional level would facilitate recognizing information redundancies and salience. aligning propositions across documents may also prove useful to a variety of cross-text tasks, such as knowledge-base population, multihop qa and cross document event extraction. consider the example in figure 1 ; our alignments represent that both sentences express similar propositional information -sentence a talks about a "buying" event while sentence b is framing the same event through the "seller" perspective, essentially capturing the same reversed roles. in previous predicate-argument alignment works @xcite , predicates and arguments were aligned individually, rather than aligning the predicate argument relation as a whole, along with its semantic role. our approach can be seen as a conceptual extension over the established coreference resolution framework, while handling content overlap at the more complex level of semantic relations (conveying stated information) rather than entity and event mentions (denoting referents). this difference in alignment scope also pertains to the cross-document coreference dataset used in these prior works, as we analyze and compare to in §5.2. unlike earlier work @xcite ) that leveraged structured srl to align predicateargument-structures, we leverage the question-answer driven semantic role labeling paradigm (qa-srl) @xcite . qa-srl captures predicate-argument relations using a naturally phrased question-answer pair, where the question type @xcite reflects the role being captured, while the answer denotes the argument. for example, in figure 1 sentence a, the agent role (a0) of the predicate "purchase" is captured through the question-answer who purchased something? -wade. once such relations are captured by qas, we can align these qas to capture similar propositional information expressed in a pair of sentences (see figure 1 ). since qas are naturally intelligible and do not require a pre-defined schema, they can be easily annotated and used by non-experts. our contributions are outlined as follows: we introduce the qa-align task, modeling crossdocument proposition-level content overlap as qabased alignments ( §3). we show that this task, though semantically challenging, is attainable via crowd annotations, and publish our guidelines and crowdsourcing methodology. in addition, we compile and release our crowdsourced qa-align dataset (accompanied by new qa-srl annotations), over semantically similar paired texts ( §4) that we collect. 1 we further analyze the quality of our data and compare it to an established crossdocument (cd) coreference benchmark, ecb+ @xcite . finally, we implement a baseline modeling approach for qa-align ( §6), and present an analysis of appealing potential downstream use, over a sentence fusion task ( §7). ## related work proposition alignment is closely related to the cross document coreference resolution (cdcr) task @xcite . this task concerns clustering together entity or event mentions across topically related documents that refer to the same "real world" element, (entity or event). it has drawn substantial recent attention @xcite @xcite , as its considered a fundamental intermediate task for cross-text language understanding. while the concept of coreference is essential for defining our alignment criteria ( §3), we consider matching predicate-argument relations as expressing alignments of propositional information, and accordingly as a useful component in capturing information correspondence between texts, unlike prior approaches which match individual event and entity mentions disjointly. our work is largely inspired by the earlier predicate-argument alignment task @xcite @xcite . these works used either a semantic role labeler to align predicates to predicates @xcite , or annotated coreference chains to align both predicates and arguments, though disjointly evaluated @xcite . unlike our work, these works did not take into consideration the relationship between the predicates and arguments, as expressed by the semantic role relating them. in this sense, these prior works are closer to the cdcr setting, whereas we directly align on propositional relations as the core units of interest, which, as we show later, captures more exhaustively information overlap across related texts ( §5.2). as potential uses for downstream tasks, various prior works presume that effectively handling multiple text sources, such as for multi-document summarization (mds), requires intermediate explicit representations for cross-text content consolidation @xcite . further, modeling redundant or overlapping text has been tackled through the sentence fusion task @xcite @xcite , and recently as "disparate" sentence fusion, targeting related sentences in a single document @xcite @xcite . in a recent series of investigations, @xcite highlight sentence fusion as a necessary step for improving summarization. in particular, pairs of sentences to be fused were empirically shown as a better source for generating summary sentences than single sentences @xcite . in section 7, we analyze the potential utility of our qa alignments as a redundancy signal in a sentence fusion model. qa-srl qa-srl has been shown to attain high quality annotation of predicate-argument structure for verbs, via crowdsourcing @xcite , achieving above 90% coverage of propbank arguments @xcite . in addition, @xcite showed that it captures much implicit information that is often missed by traditional srl schemes. for these reasons, we employ the qa-srl scheme for crowdsourcing propositional alignments and then modeling them in our qa-align parser ( §6). ## task definition setting the task of aligning semantic content across text requires defining the informational units that are to be aligned. moving from coreference resolution onto the propositional level, it is desired to pinpoint the minimal unit of propositional information, at which information overlap may be flexibly identified. inspired by the seminal neo-davidsonian approach @xcite and following previous works @xcite @xcite , we view a single predicateargument relation as an atomic propositional unit. as mentioned, we capture these using qa-srl @xcite question-answer (qa) pairs, as was shown in figure 1 . the current scope of our work considers alignments between two sentences. we define the task as follows: given a pair of texts concerning (partly) overlapping scenarios, along with their predicateargument structure representation -consisting of qa-srl qas in our setting, we want to find all cross-text alignments between qas that refer to the same fact within the context of their sentences. more concretely, our annotation guidelines, targeted for trained crowd workers (see §4), require aligned qas to correspond on the following elements: the vast majority of the alignments we observe are 1-to-1, that is, aligning a single qa from each sentence. however, we also allow for many-tomany alignments in our annotation protocol, which constitute about 4% of our data. these are required, for instance, to handle light-verb constructions, as in the following example: ## analysis of potential downstream use aiming at an initial analysis of the potential extrinsic utility of our alignments, we experiment with using them as a signal for the sentence fusion task @xcite . sentence fusion is a summarizing task applied for the single sentence level, thus providing a suitable use-case for exploring the extrinsic utility of our sentence-level alignments. indeed, it was shown that fusion models tend to generate naive extractive summaries that are copied from a single source @xcite , rather than merging information from multiple sources. in that vein, we aim to examine whether explicitly incorporating alignments into the fusion task would encourage a model to attend to corresponding information originated in multiple sentences. ## conclusion we present a new task for aligning propositional predicate-argument information, captured via the question-answer pairs of qa-srl. we compile and publish a qa-align dataset and present our crowdsourcing methodology, showing that a high inter-annotator agreement is achievable for this challenging semantic task even for non-expert annotators. we compare our annotation scheme to that of ecb+ with respect to aligning information across related texts, and show that alignments at the level of predicate-argument relations capture 13 in both models the majority is still single-sourced, probably due to the "extractive" nature of the this data @xcite . xiaodong yu, wenpeng yin, and dan roth. 2020. paired representation learning for event and entity coreference. arxiv preprint arxiv:2010.12808. yutao zeng, xiaolong jin, saiping guan, jiafeng guo, and xueqi cheng. 2020. event coreference resolution with their paraphrases and argument-aware embeddings. in proceedings of the 28th international conference on computational linguistics, pages 3084-3094, barcelona, spain (online). international committee on computational linguistics.