Explicating the Implicit: Argument Detection Beyond Sentence Boundaries (2024)

Abstract: Detecting semantic arguments of a predicate word has been conventionally modeled as a sentence-level task. The typical reader, however, perfectly interprets predicate-argument relations in a much wider context than just the sentence where the predicate was evoked. In this work, we reformulate the problem of argument detection through textual entailment to capture semantic relations across sentence boundaries. We propose a method that tests whether some semantic relation can be inferred from a full passage by first encoding it into a simple and standalone proposition and then testing for entailment against the passage. Our method does not require direct supervision, which is generally absent due to dataset scarcity, but instead builds on existing NLI and sentence-level SRL resources. Such a method can potentially explicate pragmatically understood relations into a set of explicit sentences. We demonstrate it on a recent document-level benchmark, outperforming some supervised methods and contemporary language models.

## introduction identifying which entities in a text play specific semantic roles with respect to a predicate word is a core ability of language comprehension @xcite . such basic semantic information is often surfaced via simple lexical and syntactical patterns in the sentence. readers however can perfectly interpret such semantic relations pragmatically in a wider context than a single sentence. consider the example in figure 1 . the location that the boat left, 'the house', and the destination where it was headed to, 'the port', can be deduced by associating the trip with the leave event in the preceding sentence. such examples showcase where our semantic interpretation departs from syntax, and allow us to systematically investigate how humans and machines reason over events in text in cases "this house on al zaharah street half a mile from the port is where investigators believe the bomb was built into the boat that carried it. … on the day of the bombing, neighbors saw the boat leaving. … the trip from the house to the harbor was only about a mile …" 1. local args the boat left on the day of the bombing. ## background and related works implicit arguments mainstream research efforts in semantic role labeling (srl) @xcite have focused on the problem of assigning semantic roles only to syntactically related phrases, e.g. the subject or object phrases of verbs, while neglecting constituents from the wider passage that are pragmatically interpreted as participants. the latter ones, referred to as implicit arguments @xcite despite being overtly understood by readers, constitute a sizeable portion of the potentially identified argument set @xcite @xcite @xcite . while some recent works @xcite have annotated large datasets with semantic arguments captured anywhere within the sentence scope, to this day, only a handful of limited resources for srl in the document scope exist @xcite @xcite . some resources contain only a few hundred instances, others lack diversity, capturing only a tiny set of predicates (5-10 unique verbs), and all focused only on semantic core roles (i.e. the numbered arguments in propbank), neglecting other meaningful information for the reader such as temporal or locative modifiers. @xcite annotated a dataset of cross-sentence arguments on top of amr graphs @xcite specifying arguments as amr concepts, without their exact location in the sentence. earlier supervised models for implicit srl relied on extensive feature engineering and also using gold features @xcite . many works additionally attempted to overcome data scarcity by creating artificial training data using coreference @xcite or aligning predicates in comparable documents @xcite . @xcite proposed to transform the problem into a narrative cloze task, creating synthetic datasets. more recently, @xcite improved upon the baseline model pro-posed for the rams dataset @xcite , and trained a supervised model that detects argument heads before expanding to the full constituent. qa-srl @xcite represents the label of each semantic argument as a simple wh-question that the argument answers, for example, 'who acquired something?' encodes the agent, and 'who did someone give something to?' encodes the recipient. these question-labels point at the syntactic position of the argument in a declarative form of the qa pair, e.g.: 'the agent acquired something' or 'someone gave something to the recipient' (see the example in figure 2 , top-left, where the position of the answer is apparent from the question). each question also encodes the tense of the event, the modality, and negation properties (might the event occur or has the event occurred?) which are used to instantiate our propositions. @xcite extended qa-srl to deverbal nominal predicates, recently leveraged for training a joint verbal and nominal qa-srl model @xcite . @xcite used qa pairs to represent implicit and explicit discourse relations, also across sentences @xcite . tne is a dataset for modeling semantic relations between noun phrases (nps) across a document and is annotated on top of wikinews. a relation consists of an anchor and complement phrases that are labeled with a preposition, i.e. [the investigation] anchor by [the police] complement . each document is first segmented into a list of non-overlapping nps and every np pair is annotated with either a preposition or a 'no-relation' tag. each np is also assigned to a cluster of co-referring within-document mentions. on5v @xcite is a dataset derived from news articles in the development and train partitions of ontonotes @xcite . the dataset contains 390 instances selected from 260 documents, and annotates five different verbal predicates. the original annotators have inspected only core roles (i.e. numbered: arg0, arg1, etc.) that were missing an explicit filler argument in ontonotes, and filled the role with the closest phrase to the predicate that fit the role description. we have re-annotated this dataset to close the coverage gap for modifier roles and retrieve all fitting phrases (see §6.1). ## task definition given a text document @xmath0, . . . , s n ], consisting of n sentences, and a target predicate word p from the document (a verb or deverbal noun), our task is to detect the set of semantic arguments a @xmath1. in this work, we adopt the qa-srl definition of a semantic argument and apply it to the document-level task. an argument of p is any phrase a in the document that pertains to the event referred by p and correctly answers a simple wh-question revolving around the predicate word. for example, what did something leave? asks about the house argument in figure 1 . we will mark the sentence where the predicate resides as s p and define the set of local, in-sentence arguments as l @xmath2, where the subset notation is overloaded here to represent a sub-sequence. while past works have focused on detecting l p , we will show a method that leverages l p to detect the rest of cross-sentence arguments. ## method our approach is based on synthesizing a simple sentence that encodes candidate phrases as arguments for the predicate p, and testing if the sentence is entailed from the document. the synthesized sentence h, named the semantic hypothesis, is created from templates by placing different phrases in subject or object positions around the predicate verb. 2 for example, in proposition no. 2 in figure 1 'the boat' is assigned to the subject position and represents the leaver role, and 'on the day of the bombing' is assigned to an adjunct and represents a temporal role. if the semantic hypothesis is entailed from the passage, we conclude that the predicate-argument relations encoded within h are manifested in the document. on the other hand, if the hypothesis is not entailed, we can conclude that the document does not express one or more of the encoded argument relations. for example, proposition no. 4 incorrectly assigns 'the investigators' to the leaver role, which should not be entailed from the passage. our method iterates over candidate phrases extracted from across the document (figure 2 , left). to determine if a candidate phrase c is a semantic argument of the predicate, we construct several hypotheses that incorporate the candidate and this house on al zaharah street half a mile from the port is where investigators believe the bomb was built into the boat that carried it… on the day of the bombing, neighbors saw the boat leaving. the trip from the house to the harbor was only about a mile …" q: when was something leaving? a: the day of the bombing q: what was leaving? a: the boat adj subj the boat was leaving the house on the day of the bombing. check for entailment against the document (figure 2, right). the hypotheses combine local, insentence, arguments of the predicate with the candidate phrase, and differ in the syntactic position that the candidate assumes. conceptually, each position encodes the candidate into a different semantic role. moreover, the hypothesis sentence associates the candidate with the specific target event by incorporating local arguments of the event, the remote candidate, and the target predicate into a single sentence. if h is entailed, it suggests that the candidate participates in the event in the document, and that the candidate's semantic role in the document is the same as its role in h. our synthetic hypothesis sentence is generated from syntactic templates. they accommodate the main verb (verb) with up to 4 argument phrases in the following unique syntactic positions: subject (subj), direct object (dobj), indirect object (iobj), and adjunct (adj). it can be modified using the following grammatical attributes: tense (ten), negation (neg), and modality (mod). these fields are inspired by the qa-srl question syntax that was similarly used to generate simple propositions in @xcite . they present high versatility and can capture any semantic role of a verbal predicate in english, yet maintain a compact representation requiring only a few assignments. in this work, unless specified otherwise, we consider as candidates noun phrases and named entities from the document that do not overlap with local arguments, including those from s p . each candidate is placed into different hypotheses in the subject, direct and indirect object positions. we use the probability of entailment as predicted by an nli model as the score of the hypothesis nli(d, h) and define the score of each candidate to be the maximal score of any of its hypotheses. our final predicted set of arguments includes all candidate phrases that scored above a configurable threshold and all local arguments predicted by the parser. in the next subsections, we will describe how we populate the hypothesis syntactic positions and attributes given the predicate and its sentence ( §4.1), and how we then generate the actual hypothesis sentence ( §4.2). ## predicate-argument aware nli dataset throughout our experiments, we noticed that the readily available nli models usually make poor decisions when considering different semantic hypotheses, assigning high probability to propositions with unrelated candidates -which resonates the findings of min et al. (2020); @xcite . we believe that this is caused by the inherent lexical overlap between the hypothesis and the premise texts since our proposition is built entirely from phrases found in the original document. to circumvent this, we train a semantics-aware entailment model from qa-srl data. we use the singlesentence training data and generate entailed and not-entailed propositions. each training instance includes a sentence and a proposition centered on a predicate in the sentence. positive instances include propositions built using the predicate's argument set. each true argument is placed in the hypothesis according to their syntactic position as determined by their qa-srl question. the positive propositions are then used to build the negative instances in the following two ways: the first inserts a noun phrase from the sentence that is not an argument into any position. the second switches between syntactic positions of true arguments in the positive proposition, replacing objects as subjects and viceversa. this training setup encourages the model to be more sensitive to the semantics of the hypothesis, as encoded in its argument structure. our training set contains 465k sentencehypothesis pairs extracted from the training partitions of qanom @xcite and @xcite , with 30% positive (entailed) instances. negative instances are split between subject-object swaps (14%), and insertions of non-argument phrases from the sentence (56%). we created multiple positive hypotheses for each predicate by omitting subsets of true arguments, anticipating low coverage conditions of the qa-srl parser at inference time. for negative examples, we sampled one positive hypothesis for each predicate and applied our augmentations. ## results tables 1 and 2 present the results of the argument detection task on nominal predicates from tne and verbal predicates from on5v, respectively. for tne, we report the results in two settings, (1) full document considering all semantic arguments in the entire document and (2) cross-sentence, focusing on arguments located in different sentences than the predicate. this separation allows us to analyze the parsers' performance beyond sentence boundaries. for on5v, we show results for the cross-sentence setting in across both datasets, our predicate-argumentaware entailment model (srl-nli), trained on a diverse mix of nli datasets and further fine-tuned on qa-srl-derived entailment data ( §5), exhibits superior overall performance (f1) compared to all evaluated approaches. our generic approach outperforms supervised models on tne as shown in predicate-argument-aware entailment model boost performance srl-nli outperforms nli (using the same deberta underlying model) by 6.6 f1 points on tne and 3.8 on on5v, indicating the benefit of an enhanced classifier that is sensitive to predicate-argument semantics. cross-sentence is more difficult when evaluated on tne, all examined models undergo a performance deterioration for the more challenging setting of cross-sentence argument detection. the drop in performance is especially detrimental for the qa-srl parser (-24.8 f1), which can be attributed to its single-sentence training scope. notably, nli-based models exhibit an on-par performance decrease with the tne parser, which was supervised over task-specific document-level data. hence, it seems that our srl-nli approach enjoys the best of both worlds -it learns document-level semantic understanding from nli, while specializing in predicate-argument semantics due to the sentence-level qa-srl supervision. llms: simple wins, complex stumbles directly asking mistral in the few-shot setting to identify all semantic arguments of a predicate within a paragraph leads to subpar performance (40.72 vs. 58.34 f1 on tne and 12.71 vs. 29.28 f1 on on5v for the best mistral configuration). interestingly, prompting mistral just for the arguments (arg) consistently achieves higher performance on both tne and on5v, than asking it to produce arguments in the form of qa pairs (qa), which could have been more fitting for an instruction tuned model. however, our approach of framing implicit argument identification as a series of entailment decisions, and leveraging mistral as a zero-shot entailment model (instruct-nli) already yields remarkable performance gains. this method surpasses directly prompting mistral for arguments, achieving a 5.9 f1-score improvement on tne and an impressive 10+ f1-score increase on on5v. these results highlight the benefit of decomposing complex tasks into simpler binary decisions for llms, potentially due to reduced reasoning burden and better alignment with their instruction fine-tuning data. ## analysis our evaluation against the tne datasets measures unlabelled argument detection, which leaves the role assignment accuracy of our system unexplored. since our approach is schema-independent, the argument's semantic role is not provided explicitly but is expressed through its syntactic position in the proposition. we thus tap into the labeling accuracy of our system through a manual analysis. specifically, we sample 50 deverbal nominal predicates from the tne test set along with their 260 gold cross-sentence complements and inspect the complements' highest-ranked proposition during inference. each proposition contains the complement in its most probable syntactic position as ranked by our srl-nli model. in order to align the setting of our analysis to a typical use case scenario of our method, we further run an ontonotes parser @xcite over the selected propositions to attain propbank labels of the arguments. an author of this paper then verified that the predicted semantic role label matches in definition against the semantic relation captured by tne annotators. omitting 14 tne complements that don't correspond to verbal arguments, and 20 arguments that are missed by the ontonotes parser, the extracted role is accurate at 161/226 (71%) of the cases. mistakes include both ontonotes parsing mistakes, as well as erroneous syntactic positions selected by the nli-based ranking. for further analysis refer to appendix d. ## conclusions we have demonstrated how to reformulate the problem of argument detection into an entailment task, and successfully used it to detect arguments across sentence boundaries where training data is scarce. moreover, we have explicated the meaning of these distant arguments in the form of simple and easyto-grasp propositions that keep the correct semantic role information without committing to a specific label schema. our proposed method can augment any srl or event-extraction schema with crosssentence arguments at test time, without additional annotation or training. given a sentence-level parser, one can apply it to the extracted proposition to get a label for the captured argument. the propositions can potentially serve applications that require information decomposition into smaller units, e.g. scus @xcite for the summarization task and many more.