Venue: NAACL 2016

# Automatic Generation and Scoring of Positive Interpretations

from Negated Statements

Eduardo Blanco

Zahra Sarabi

Human Intelligence and Language Technologies Lab

University of North Texas

Denton, TX, 76203

eduardo.blanco@unt.edu, zahrasarabi@my.unt.edu

###### Abstract

This paper presents a methodology to extract positive interpretations from negated statements. First, we automatically generate plausible interpretations using well-known grammar rules and manipulating semantic roles. Second, we score plausible alternatives according to their likelihood. Manual annotations show that the positive interpretations are intuitive to humans, and experimental results show that the scoring task can be automated.

## 1 Introduction

Negation is an intricate phenomenon present in all human languages Hoeksema (2000), and studied from a theoretical perspective since Aristotle. Acquiring and understanding negation is more challenging than language in general: children acquire negation after learning to communicate Nordmeyer Frank (2013), and adults take longer to process negative sentences than positive ones Clark Chase (1972). In any given language, humans communicate in positive terms most of the time, and use negation to express something unusual or an exception Horn (1989).

In classical logic, negation is a simple unary operator that reverses the truth value of a proposition. In natural language, negation is always marked Horn and Wansing (2015) and it is used to reverse polarity, i.e., turning something affirmative into negative, or something negative into affirmative. Albeit most sentences are affirmative, negation is rather ubiquitous Morante and Sporleder (2012): In scientific papers, 13.76% of statements contain a negation Szarvas et al. (2008); in product reviews, 19% Councill et al. (2010); and in a selection of Conan Doyle stories, 22.23% Morante and Daelemans (2012). In health records, 12.3% of concepts are tagged as negated Elkin et al. (2005); and in OntoNotes Hovy et al. (2006), 10.15% of statements contain a verb negated with _not_, _n't_ or _never_.

From a theoretical perspective, it is accepted that negation conveys positive meaning Rooth (1992); Huddleston and Pullum (2002). For example, when reading (1) _John doesn't eat meat_, humans intuitively understand that (1a) _John eats something other than meat_, and (1b) _Some people eat meat, but not John_. Extracting positive interpretations from negated statements automatically is not straightforward: a negated statement may convey one or more positive interpretations, and not all positive interpretations are equally likely. For example, from (2) _They didn't order the right parts_, it is very likely that (2a) _They ordered the wrong parts_, but (2b) _Somedbody ordered the right parts, but not they_ is unlikely.

This paper presents a methodology to automatically extract and score positive interpretations from negated statements, as intuitively done by humans when reading text. A key feature of the work presented here is that it is not tied to any existing approach to extract meaning from text--we generate positive interpretations in plain text, and these positive interpretations can be semantically represented with any existing approach. The main contributions are: (1) procedure to automatically generate plausible positive interpretations from negated statements, (2) annotations scoring plausible positive interpretations,1 and (3) experiments detailing results with several combinations of features, as well as gold-standard and predicted linguistic information.



Background and Definitions

Negation is well-understood in grammars, the valid ways to form a negation are well-documented [253, 16]. Negation can be expressed by verbs (e.g., avoid doing any look-up), nouns (e.g., the absence of any phonic sequence), adjectives (e.g., it is pointless to argue with a fool), adverbs (e.g., I never tried Persian food before), prepositions (e.g., you can always exchange it without a problem), determiners (e.g., the new law has no direct implications to international shipping), pronouns (e.g., nobody will keep election promises).

Huddleston and Pullum (2002) distinguish four negation types:

* Verbal if the marker of negation is grammatically associated with a verb, e.g., _I did not see anything_, non-verbal if it is associated with a dependent of the verb, e.g., _I saw nothing_.
* Synthetic if the negation mark has a function besides marking a negation, e.g., _[Nobody]\({}_{\text{agent}}\) liked it_, analytic otherwise, e.g., _Not many people liked it_.
* Clausal if the negation yields a negative clause, e.g., _The terms aren't negotiable_, subclass otherwise, e.g., _The terms are non-negotiable_.
* Ordinary if the negation indicates that something is not the case, e.g., _That car does not drive smooth_, metalinguistic if it does not dispute the truth but rather reformulates a statement, e.g., _That TV is not small, it is tiny_.

In this paper, we target verbal, analytic, clausal and both ordinary and metalinguistic negation.

### Positive Interpretations.

In philosophy and linguistics, it is generally accepted that negation conveys positive meanings [47]. These positive meanings range from implicatures, i.e., what is suggested in an utterance even though neither expressed nor strictly implied [9], to entailments. Other terms used in the literature include implied meanings [8], implied alternatives [10] and semantically similars [1]. We do not strictly fit into any of this terminology, we reveal positive interpretations as intuitively done by humans when reading text.

### Scope and Focus.

From a theoretical perspective, it is accepted that negation has scope and focus, and that the focus--not just the scope--yields positive interpretations [47, 10, 11]. Scope is "the part of the meaning that is negated" and focus "the part of the scope that is most prominently or explicitly negated" [10].

Consider the following statement in the context of the recent refuge crisis: (3) _Mr. Haile was not looking for heaven in Europe_. By definition, scope refers to "all elements whose individual falsity would make the negated statement strictly true", and focus is "the element of the scope that is intended to be interpreted as false to make the overall negative true" [10]. The falsity of any of the truth conditions below makes statement (3) true, thus the scope of the negation is (3a-3d):

* Somebody was looking for something somewhere. [verb _looking_]
* Mr. Haile was looking for something somewhere. [agent of looking, _Mr. Haile_]
* Somebody was looking for heaven somewhere. [theme of looking, _heaven_]
* Somebody was looking for something in Europe. [location of looking, _in Europe_]

Determining the focus is almost always more challenging than the scope. The challenge lies on determining which of the truth conditions (3a-3d) is intended to be interpreted as false to make the negated statement true: all of them qualify, but some are more likely. A natural reading of statement (3) suggests that _Mr. Haile was looking for something (a regular life, a job, etc.) in Europe, but not heaven_. Determining that the focus is _heaven_, i.e., that everything in statement (3) is actually positive except the theme of _looking_, is the key to reveal the intended positive interpretation. It is worth noting that other foci yield unlikely interpretations, e.g., _Somebody was looking for heaven in Europe, but not Mr. Haile_ (3b, agent), _Mr. Haile was looking for heaven somewhere, but not in Europe_ (3d, location). Note that (1) scope on its own does not yield positive interpretations, and (2) some negated statements convey several likely positive interpretations, e.g., statement (1) in Section 5, Table 3 

Previous Work

Within computational linguistics, approaches to process negation are shallow, or target scope and focus detection. Popular semantic representations such as semantic roles [148]; Baker et al., 1998) or AMR [20] do not reveal the positive interpretations we target in this paper. Shallow approaches are usually application-specific. In sentiment and opinion analysis, negation has been reduced to marking as negated all words between a negation cue and the first punctuation mark [35], or within a five-word window of a negation cue [76]. The examples throughout this paper show that these techniques are insufficient to reveal implicit positive interpretations.

### Scope Annotation and Detection

Scope of negation detection has received a lot of attention, mostly using two corpora: BioScope in the medical domain [21] and CDSCO [22]. BioScope annotates negation cues and linguistic scopes exclusively in biomedical texts. CD-SCO annotates negation cues, scopes, and negated events or properties in selected Conan Doyle stories.

There have been several supervised proposals to detect the scope of negation using BioScope and CD-SCO (Ozgur and Radev, 2009; Ovrelid et al., 2010). Automatic approaches are mature [2]: F-scores are 0.96 for negation cue detection, and 0.89 for negation cue and scope detection [23, 24]. Outside BioScope and CD-SCO, Reitan et al. reitan_et_al_present a negation scope detector for tweets, and show that it improves sentiment analysis. As shown in Section 2, scope detection is insufficient to reveal positive interpretations from negated statements.

### Focus Annotation and Detection

While focus of negation has been studied for decades in philosophy and linguistics (Section 2), corpora and automated tools are scarce. Blanco and Moldovan blanco_goldman annotate focus of negation in the 3,993 negations marked with argm-neg semantic role in PropBank [148]. Their annotations, PB-FOC, were used in the *SEM-2012 Shared Task [22]. Their guidelines require annotators to choose as focus the semantic role that "is most prominently negated" or the verb. If several roles may be the focus, they prioritize "the one that yields the most meaningful implicit [positive] information", but do not specify what _most meaningful_ means. Consider again statement (1) _John doesn't eat meat_. Their approach would determine that the focus is the theme of eat, meat_, because it arguably yields the "most meaningful implicit [positive] information" (using our terminology, positive interpretation): _John eats something other than meat_. By design, they ignore other valid positive interpretations, e.g., _Some people eat meat, but not John_. In this paper, we improve upon their work: instead of extracting the "most meaningful" positive interpretation from a negated statement, we generate several positive interpretations and score them according to their likelihood.

Anand and Martell anand 



## 4 Corpus Creation

Our goal is to create a corpus of negated statements and their positive interpretations as intuitively understood by humans. We put a strong emphasis on automation. First, given a negated statement, we automatically generate plausible positive interpretations following a battery of linguistically motivated deterministic rules (Section 4.1). Second, we collect manual annotation to score the plausible positive interpretations according to their likelihood (Section 4.2). We then use these manually obtained scores to learn models that automatically score positive interpretations (Section 6).

We decided to work on top of OntoNotes [5] instead of plain text or other corpora for several reasons. First, OntoNotes includes gold linguistic annotations such as part-of-speech tags, parse trees and semantic roles. Second, state-of-the-art role labelers trained with Propbank achieve F-measures of 0.835 [7], and we use semantic roles to generate positive interpretations. Third, unlike BioScope, CD-SCO and PBFOC (Section 2), OntoNotes includes sentences from several genres, e.g., newswire, broadcast news and conversations, magazines, the web.

### Generating Positive Interpretations

OntoNotes2 is a large corpus containing 63,918 sentences. Annotating all positive interpretations from all negations is outside of the scope of this paper. Instead, we target selected representative negations.

Footnote 2: We use the CoNLL-2011 Shared Task distribution (Pradhan et al., 2011), [http://conll.cemantix.org/2011/](http://conll.cemantix.org/2011/)

**Selecting Negated Statements.** We first selected all verbs negated with argm-neg semantic role and obtained 6,617 verbal negations. After examining the negated verbs, it became clear that negation is not uniformly distributed across verbs in OntoNotes, it roughly follows Zipf's law. In order to alleviate the annotation effort while accounting for all negated verbs in OntoNotes, we randomly selected up to 5 negations for each verb. The number of negated statements selected is 600.

**Converting Negated Statements into Their Positive Counterparts.** We apply 3 steps inspired after the grammatical rules to form negation detailed by Huddleston and Pullum (2002, Ch. 9):

1. Remove the negation mark by removing the tokens within argm-neg semantic role.
2. Remove auxiliaries, expand contractions, and fix third-person singular and past tense. For example (before: after), _doesn't go_: _go_: _got_: _went_, _won't go_: _will go_, We use a standard list of irregular verbs,3 and grammar rules to convert to third-person singular and past tense based on orthographic patterns. Footnote 3: [https://en.wikipedia.org/wiki/English_irregular_verbs](https://en.wikipedia.org/wiki/English_irregular_verbs)
3. Rewrite negatively-oriented polarity-sensitive items. For example (before: after), _anyone_: _someone_, _any longer_: _still_, _yet_: _already_. _at all_: _somewhat_. We use the correspondences between negatively-oriented and positively-oriented polarity-sensitive items by Huddleston and Pullum (2002, pp. 831).

**Generating Positive Interpretations.** Once the positive counterpart is obtained, we generate positive interpretations by rewriting each semantic role or the (originally negated) verb. Among others, we use the following rewriting rules: arg\({}_{0}\)=arg\({}_{4}\): someone / some people / something, argon-tmp: at some point of time, argon-loc: somewhere, argm-mnr: in some manner, argon-cau: because of something and argm-prp: to do something. Additionally, if the semantic role starts with a preposition, we also include it, e.g., _gave [to John]_ARG2:

\begin{table}
\begin{tabular}{|l|c|l|} \hline Neg. statement & In 1995, Murdoch bought [the rest of Star TV]\({}_{\text{ARG}_{1}}\) that [he]\({}_{\text{ARG}_{0}}\) did [not]\({}_{\text{ARM-REG}}\) [own]\({}_{\text{verb}}\) \\ \hline \multirow{3}{*}{Positive counterpart} & after Step 1 & In 1995, Murdoch bought the rest of Star TV that he d own \\ \cline{2-3}  & after Step 2 & In 1995, Murdoch bought the rest of Star TV that he owned (no change) \\ \cline{2-3}  & after Step 3 & In 1995, Murdoch bought the rest of Star TV that be owned (no change) \\ \hline Plausible & from arg\({}_{0}\) & In 1995, Murdoch bought the rest of Star TV that [somebody] owned, but not _he_ \\ \cline{2-3} positive & from arg\({}_{1}\) & In 1995, Murdoch bought [something] that he owned, but not _the rest of Star TV_ \\ \cline{2-3} interpretations & from _verb_ & In 1995, Murdoch bought the rest of Star TV that he [some verb], but not _owned_ \\ \hline \end{tabular}
\end{table}
Table 1: Negated statement, and automatically generated positive counterpart and plausible positive interpretations.



gave to someone, but not John_. This methodology generated 1,888 positive interpretations from the 600 selected negations (average: 3.15).

Table 1 exemplifies the 3 steps to transform a negated statement into its positive counterpart, and the positive interpretations generated. Additional examples are provided in Table 3.

We acknowledge that some of the positive interpretations we generate automatically are not as specific or intuitive as carefully crafted, manually generated interpretations could be. For example, from [John]\({}_{\mathrm{ARG_{O}}}\) does[n't]\({}_{\mathrm{ARGM-NEG}}\) [know]\({}_{\mathrm{verb}}\) [the details about how they met]\({}_{\mathrm{ARG_{1}}}\), the proposed methodology would generate, among others, _John knows about something, but not about the details of how they met_. A better interpretations that we do not generate is _John knows something about how they met, but not the details_. We argue that generating interpretations automatically is the only option in order to incorporate this work into an NLP pipeline, and reserve for future work generating positive interpretations beyond rewriting semantic roles.

### Ranking Positive Interpretations

Once positive interpretations were automatically generated, we asked annotators to rank them. Annotators were presented with one negated statement and one positive interpretation at a time, and were asked _Given the negated statement above, do you think the statement [positive interpretation] below is true?_ They only had access to the text in the original negated statement, the positive interpretation, and the previous and next sentences as context. We did not display semantic role information for the original negated statement, its positive counterpart or the semantic role from which the positive interpretation was generated. As we shall see (Section 5.1), context often helps scoring interpretations.

Annotators were required to answer with a score from 0 to 5, were 0 means _absolutely disagree_ and 5 means _absolutely agree_. We did not provide descriptions for intermediate scores or used additional categorical labels. This simple guidelines were sufficient to reliably score plausible positive interpretations automatically generated.

## 5 Corpus Analysis

On average, we generated 3.15 positive interpretations per negation (standard deviation: 0.82), and 74% of negations have at least one interpretation scored 4 or higher. Basic counts and statistics for the annotations are provided in Table 2. Overall, we annotated 1,888 positive interpretations generated from 600 sentences, or equivalently, from 600 verbs negated with argm-neg semantic role in OntoNotes. Overall mean score is 3.52 (out of 5) and overall standard deviation, 1.63. The 25th percentile is 2.0, the 50th percentile is 4.0 and the 75th percentile is 5.0. These numbers show that most positive interpretations automatically generated are deemed likely by annotators, and over 25% are scored with a 5 (out of 5).

In general, positive interpretations generated from numbered roles (arg\({}_{0}\)-arg\({}_{4}\)) are scored higher than the ones generated from modifiers (argmaxv, argm-cau, argm-dir, etc.). Also, positive interpretations generated from infrequent roles are generally ranked higher, e.g., arg\({}_{4}\) and arg\({}_{3}\) vs. arg\({}_{0}\), argm-prp and argm-mnrvs. argm-adv.

**Annotation Quality.** In order to ensure annotation quality, we calculated inter-annotator Pearson correlation. Kappa and other agreement measures de

\begin{table}
\begin{tabular}{|l||c|c||c|c|} \hline Sem. role & \# & \% sent & Mean & SD \\ \hline \hline ARG\({}_{0}\) & 392 & 65.17\% & 4.07 & 1.03 \\ \hline ARG\({}_{1}\) & 520 & 86.00\% & 4.55 & 0.98 \\ \hline ARG\({}_{2}\) & 95 & 15.83\% & 4.40 & 1.31 \\ \hline ARG\({}_{3}\) & 2 & 0.33\% & 4.50 & 0.50 \\ \hline ARG\({}_{4}\) & 5 & 0.83\% & 5.00 & 0.00 \\ \hline ARGM–ADV & 101 & 15.67\% & 2.41 & 1.72 \\ \hline ARGM–CAU & 13 & 2.17\% & 2.46 & 1.91 \\ \hline ARGM–DIR & 11 & 1.83\% & 2.00 & 2.30 \\ \hline ARGM–EXT & 12 & 2.00\% & 3.75 & 1.53 \\ \hline ARGM–LOC & 21 & 3.33\% & 3.76 & 1.69 \\ \hline ARGM–MNR & 37 & 6.00\% & 4.54 & 1.03 \\ \hline ARM–PRP & 7 & 1.16\% & 4.43 & 0.71 \\ \hline argon–TMP & 81 & 12.50\% & 3.99 & 1.52 \\ \hline \hline Verb & 600 & 100.00\% & 2.17 & 1.41 \\ \hline All & 1,888 & 100.00\% & 3.52 & 1.63 \\ \hline \end{tabular}
\end{table}
Table 2: Basic corpus analysis. For each semantic role, we show the number of positive interpretations generated (#), the percentage of sentences for which a positive interpretation is generated (% sent), and the mean and standard deviation (SD) of the annotated scores.

 

signed for categorical labels are not well-suited for our annotation task, since not all disagreements between numeric scores are the same, e.g., 4 vs. 5 should be counted as relatively high agreement, and 1 vs. 5 should be counted as high disagreement. Overall Pearson correlation is 0.761.

### Annotation Examples

Table 3 presents annotation examples. We show the original negated statement including semantic role annotations from OntoNotes (square brackets), all positive interpretations automatically generated, and their scores. We also include context (previous and next sentence) if it helps determining scores.

Example (1) shows that context sometimes is vital to scoring plausible positive interpretations. Given _He didn't forecast Phillips' results_ in isolation, it is uncertain if he forecasted anything at all, or whether somebody forecasted Phillips' results. However, the previous statement makes certain (5/5) the interpretation generated from arg\({}_{1}\): he forecasted earnings for the industry in general. Similarly, the next statement makes very likely (5/5) the interpretation generated from arg\({}_{0}\): other people (security analysts) made forecasts about Phillips. In this example, 2 positive interpretations are generated from one negation, and they are assigned the highest score (5/5).

The positive interpretations generated from example (2) can be annotated without context. Somebody other than Murdoch had to own the rest of Star TV that he bought in 1995 (score 5/5), and people cannot buy what they already own (score 0/5).

Example (3) presents a positive interpretation generated from arg\({}_{0}\) that is scored low (1/5); recall that the mean score for interpretations generated from arg\({}_{0}\) is 4.07 and the standard deviation 1.03 (Table 2). In this negated statement, the indefinite _you_ refers to an unspecified person, thus it is not the case that _somebody can run a country with 23 million people with revenues of 16 billion to 20 billion dollars_ (Interpretation 3.1, score 1/5). Interpretations 3.2 and 3.3, however, are scored high: annotators correctly understood that given the negated statement, it is the case that _You can run something with revenues of 16 billion to 20 billion dollars, but not a country with 23 million people_ (you can run a country with less people with that revenue), and _You can run a country with 23 million people in some manner, but not with revenues of 16 billion to 20 billion dollar_ (you could run it with more revenue).

\begin{table}
\begin{tabular}{|c|l|r|} \hline  & Negated statement, context if relevant to determining scores, and all positive interpretations & Score \\ \hline \hline \multirow{8}{*}{1} & Context, previous statement: _That change will obviously impact third and fourth quarter earnings for the industry in general, he added._ \\  & Negated statement: _[He]\({}_{\text{arg${}_{0}$}}\) did[n’t]\({}_{\text{ArgM-NEG}}\) [forecast]\({}_{\text{wrb}}\) [Phillips’ results]\({}_{\text{ARG${}_{1}$}}\)._ \\  & Context, next statement: _But security analysts say Phillips will be among the companies hard-hit by weak chemical prices and will probably post a drop in third-quarter earnings._ \\ \cline{2-3}  & - Interpretation 1.1 [arg\({}_{0}\)]: Somebody forecasted Phillips’ results, but not _he_. & 5 \\ \cline{2-3}  & - Interpretation 1.2 [arg\({}_{1}\)]: He forecasted something, but not _Phillips’ results_. & 5 \\ \hline \hline
2 & Negated statement: _In 1995, Murdoch bought [the rest of Star TV]\({}_{\text{ARG${}_{1}$}}\) [he]\({}_{\text{ARGO}}\) did [not]\({}_{\text{ARM-NEG}}\) [own]\({}_{\text{wrb}}\)._ \\ \cline{2-3}  & - Interpretation 2.1 (arg\({}_{0}\)): In 1995, Murdoch bought the rest of Star TV somebody owned, but not _he_. & 5 \\ \cline{2-3}  & - Interpretation 2.2 (arg\({}_{1}\)): In 1995, Murdoch bought something he owned, but not _the rest of Star TV_. & 0 \\ \hline \hline
3 & Negated statement: _[You]\({}_{\text{arg${}_{0}$}}\) [can]\({}_{\text{ARM-MOD}}\)[not]\({}_{\text{ARM-NEG}}\) [ran]\({}_{\text{verb}}\) [a country with 23 million people]\({}_{\text{ARG${}_{1}$}}\) [with revenues of 16 billion to 20 billion dollars]\({}_{\text{ARM-MNR}}\)._ \\ \cline{2-3}  & - Interpretation 3.1 (arg\({}_{0}\)): Somebody can run a country with 23 million people with revenues of 16 billion to 20 billion dollars, but not _You_. \\ \cline{2-3}  & - Interpretation 3.21 (arg\({}_{1}\)): You can run something with revenues of 16 billion to 20 billion dollars, but not _a country with 23 million people_. & 5 \\ \cline{2-3}  & - Interpretation 3.3 (argm-mnr): You can run a country with 23 million people in some manner, but not with _with revenues of 16 billion to 20 billion dollars_. & 5 \\ \hline \hline
4 & Negated statement: _Do [not]\({}_{\text{ARM-NEG}}\) [buter]\({}_{\text{WRQ${}_{1}$}}\)._ \\ \cline{2-3}  & - Interpretation 4.1 (arg\({}_{1}\)): Utter something, but not _a word_. & 0 \\ \hline \end{tabular}
\end{table}
Table 3: Annotation examples. We show all positive interpretations automatically generated and their scores (out of 5).

 

Finally, example (4) presents a short statement from which only one positive interpretations is generated. Annotators were asked whether given _Do not utter a word_, they think _Utter something, but not a word_ is true. They correctly annotated that this positive interpretations is invalid (score 0/5). Indeed, the negated statement in example (4) can only be interpreted as an order to not utter anything.

## 6 Learning to Score Positive Interpretations

We follow a standard supervised machine learning approach. The 1,888 positive interpretations along with their scores become instances, and we divide them into training (80%) and test splits (20%) making sure that all interpretations generated from a sentence are assigned to either the training or test splits. Note that splitting instances randomly would not be sound: training with some interpretations generated from a negated statement, and testing with the rest of interpretations generated from the same statement would be an unfair evaluation.

We trained a Support Vector Machine (SVM) for regression with RBF kernel using scikit-learn Pedregosa et al. (2011), which in turn uses LIBSVM Chang and Lin (2011). The feature set and SVM parameters (\(C\) and \(\gamma\)) were tuned using 10-fold cross-validation with the training set, and results were calculated using the test set.

### Feature Selection

We tried features derived exclusively from the negated statement from which the positive interpretation was generated, more specifically, we extract features from the negated _verb_ or semantic role (_sem_role_) used to generate the positive interpretation, from both of them (_verb-sem_role_) and from the verb-argument structure of _verb_ (_verbargstruct_), i.e., all semantic roles of _verb_.

_Verb_ features are straightforward and account for the verb word form and part-of-speech tag.

_Sem_role_ features include the label of the semantic role from which the positive interpretation was generated (_sem_role_label_), its length (number of tokens), and the word form and part-of-speech tag of its head. Additionally, we add standard features in semantic role labeling Gildea and Jurafsky (2002): the syntactic nodes (NP, PP, etc.) of the semantic role and its parent in the parse tree, as well as the left and right siblings, if any.

_Verb-sem_role_ features are also standard in role labeling. We include a flag indicating whether the verb occurs before or after the semantic role in the negated statement (not in the positive counterpart), syntactic node of the lowest common ancestor between _verb_ and _sem_role_, and the syntactic path.

Finally, _verbag-struct_ features encode characteristics of the verb-argument structure to which _verb_ and _sem_role_ belong to. Namely, we added flags indicating semantic role presence, and features indicating the first and last semantic roles in order of appearance in the negated statement. We also included the syntactic nodes of semantic roles and their heads.

We exemplify all features with Interpretation 3.1 generated from Statement 3 in Table 3:

* _Verb_ features: verb_wf=run, verb_pos=VBP.
* _Sem_role_ features: label=arg\({}_{0}\), num_tokens=1, head_wf=You, head_pos=PRP, synt_node=NP, synt_node_parent=S, synt_node_left=NONE, synt_node_right=VP
* _Verb-sem_role_ features: direction=after, lowest_ancestor=S, synt_path=VP+VP+SP-NP
* _Verbarg-struct_ features: arg\({}_{0}\)=1, argmod=1, arg\({}_{1}\)=1, argM-mnr=1, first=arg\({}_{0}\), last=argm-mnr, arg\({}_{1}\)_head=country, arg\({}_{1}\)_pos=NN, etc.

## 7 Experimental Results

We report results obtained with several combinations of features in Table 5. We detail results obtained with features extracted from gold-standard and predicted linguistic annotations (part-of-speech tags, parse trees, semantic roles, etc.) as annotated in the _gold_ and _auto_ files from the CoNLL-2011 Shared Task release of OntoNotes Pradhan et al. (2011). All models are trained with gold-standard linguistic annotations, and tested with either gold-standard or predicted linguistic annotations.

**Testing with gold-standard linguistic annotations.** Using only the label of the semantic role from which the positive interpretation was generated (_sem_role_label_), yields a Pearson correlation of 0.603. Using _Verb_ features is virtually useless (Pearson correlation: -0.025), this is due to the 

fact that our corpus includes at most 5 instances for each negated verb in OntoNotes (Section 4). Adding _sem_role_ features yields a correlation of 0.630, and incorporating _verb-sem_role_ features is useless (Pearson: 0.627). Considering all features, however, yields the highest correlation, 0.642.

**Testing with predicted linguistic annotations.** We assigned 20% of instances to the test split, totalling 378 instances (Section 6). However, some of these instances cannot be obtained using predicted role labels: a missing or incorrect semantic role will unequivocally lead to positive interpretations that are not in our corpus and thus evaluation is not straightforward. Results presented with predicted linguistic annotations are calculated using only the 268 (out of 378) positive interpretations that are generated from predicted semantic roles and are also generated (and thus annotated in our corpus) from gold-standard linguistic annotations.

Results using only _sem_role_label_ feature (0.642) are better or very similar than using any combination of features (0.638-0.650) except _Verb_ features alone, which perform poorly as explained earlier. These results should be taken with a grain of salt: the number of test instances is much lower (268 vs. 378). Additionally, these 268 test instances correspond to positive interpretations generated from semantic roles that were predicted correctly automatically. Role labels are predicted better for shorter sentences without complicated syntactic structure; positive interpretations for this kind sentences are also easier to score, e.g., Statement 4 in Table 3.

## 8 Conclusions

Humans intuitively understand negated statements in positive terms when reading text. This paper presents an automated methodology to generate plausible positive interpretations from verbal negation, and score them based on their likelihood. We use simple grammar rules and manipulate semantic roles to generate positive interpretations. Experimental results show that these interpretations can be scored automatically using standard supervised machine learning techniques.

An annotation effort shows that most positive interpretations automatically generated are likely

\begin{table}
\begin{tabular}{|l|p{199.2pt}|} \hline Source & Feature description \\ \hline Verb & Word form and part-of-speech tag of _verb_ \\ \cline{2-3}  & Semantic role label of sem\_role, _sem\_role\_label_ \\ \cline{2-3} sem\_role & Number of tokens in _sem\_role_ \\ \cline{2-3}  & Word form and part-of-speech tag of head of _sem\_role_ \\ \cline{2-3}  & Syntactic node of _sem\_role_, its parent and left and right siblings in the parse tree \\ \hline verb-sem\_role & Whether _verb_ occurs before or after than _sem\_role_ in the negated statement \\ \cline{2-3} verb-sem\_role & Syntactic node of lowest common ancestor of _verb_ and _semi\_role_ \\ \cline{2-3}  & Syntactic paths from _verb_ to _sen\_role_ \\ \hline  & Flags indicating whether _verb_ has each possible semantic role \\ \cline{2-3} verbarq-struct & Semantic role labels of the first and last roles of _verb_ \\ \cline{2-3}  & Syntactic nodes and heads of each semantic role attaching to _verb_ \\ \hline \end{tabular}
\end{table}
Table 4: Features used to score positive interpretations from negated statements. Features are extracted from the negated _verb_ or semantic role (_sem_role_) from which the positive interpretation was generated, from both of them (_verb-sem_role_), or from the verb-argument structure of _verb_ (verbarq-struct), i.e., from all semantic roles of _verb_.

\begin{table}
\begin{tabular}{|l|r|r|} \hline  & Gold & Predicted \\ \hline sem\_role\_label feature & 0.603 & 0.642 \\ \hline \hline verb features & -0.025 & -0.022 \\ \hline verb + sem\_role features & 0.630 & 0.648 \\ \hline verb + sem\_role + verb-sem\_role features & 0.627 & 0.638 \\ \hline verb + sem\_role + verb-sem\_role + verbarg-struct features & 0.642 & 0.650 \\ \hline \end{tabular}
\end{table}
Table 5: Pearson correlations in the test set using gold-standard and predicted linguistic annotations (part-of-speech tags, parse trees and semantic roles). Results are provided using sem\_role\_label as only feature, and using several features incrementally. Number of test instances with gold-standard linguistic annotations is 378, and with predicted annotations, 268.

 

(scores \(\geq\)4), thus the amount of positive meaning revealed by the methodology presented here is substantial (on average, 3.15 interpretations are generated per negation). We believe that more annotations and a learning algorithm that scores jointly all positive interpretations generated from each negation (as opposed to individually) would yield better results.

## References

* Abu-Jbara and Radev (2012) Amjad Abu-Jbara and Dragomir Radev. 2012. Umichigan: A conditional random field model for resolving the scope of negation. In _Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation_, pages 328-334. Association for Computational Linguistics.
* Agirre et al. (2013) Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. *sem 2013 shared task: Semantic textual similarity. In _Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity_, pages 32-43, Atlanta, Georgia, USA, June. Association for Computational Linguistics.
* Anand and Martell (2012) Pranav Anand and Craig Martell. 2012. Annotating the focus of negation in terms of questions under discussion. In _Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics_, ExProM '12, pages 65-69, Stroudsburg, PA, USA. Association for Computational Linguistics.
* Baker et al. (1998) Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet Project. In _Proceedings of the 17th international conference on Computational Linguistics_, Montreal, Canada.
* Banarescu et al. (2013) Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In _Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse_, pages 178-186, Sofia, Bulgaria, August. Association for Computational Linguistics.
* Blackburn (2008) Simon Blackburn. 2008. _The Oxford Dictionary of Philosophy_. Oxford University Press.
* Blanco and Moldovan (2011) Eduardo Blanco and Dan Moldovan. 2011. Semantic representation of negation using focus detection. In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_, pages 581-589, Portland, Oregon, USA, June. Association for Computational Linguistics.
* Blanco and Moldovan (2012) Eduardo Blanco and Dan Moldovan. 2012. Fine-grained focus for pinpointing positive implicit meaning from negated statements. In _Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 456-465, Montreal, Canada, June. Association for Computational Linguistics.
* Blanco and Moldovan (2014) Eduardo Blanco and Dan Moldovan. 2014. Retrieving implicit positive meaning from negated statements. _Natural Language Engineering_, 20:501-535, 10.
* Chang and Lin (2011) Chih-Chung Chang and Chih-Jen Lin. 2011. Libsvm: A library for support vector machines. _ACM Trans. Intell. Syst. Technol._, 2(3):27:1-27:27, May.
* Clark and Chase (1972) H. H. Clark and W. G. Chase. 1972. On the process of comparing sentences against pictures. _Cognitive Psychology_, 3(3):472-517, July.
* Councill et al. (2010) Isaac Councill, Ryan McDonald, and Leonid Velikovich. 2010. What's great and what's not: learning to classify the scope of negation for improved sentiment analysis. In _Proceedings of the Workshop on Negation and Speculation in Natural Language Processing_, pages 51-59, Uppsala, Sweden, July. University of Antwerp.
* Elkin et al. (2005) Peter L. Elkin, Steven H. Brown, Brent A. Bauer, Casey S. Husser, William Carruth, Larry R. Bergstrom, and Dietlind L. Wahner-Roedler. 2005. A controlled trial of automated classification of negation from clinical notes. _BMC Medical Informatics and Decision Making_, 5(13).
* Gildea and Jurafsky (2002) Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. _Comput. Linguist._, 28(3):245-288, September.
* Hoeksema (2000) Jack Hoeksema. 2000. Negative polarity items: triggering, sccope,and c-command. In L. Horn and Y. Kato, editors, _Negation and polarity_, chapter 4, pages 115-146. Oxford University Press, Oxford.
* Horn and Wangling (2015) Laurence R. Horn and Heinrich Wansing. 2015. Negation. In Edward N. Zalta, editor, _The Stanford Encyclopedia of Philosophy_. Summer 2015 edition.
* Horn (1989) Laurence R. Horn. 1989. _A natural history of negation_. Chicago University Press, Chicago.
* Hovy et al. (2006) Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoCNotes: the 90% Solution. In _NAACL '06: Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers on XX_, pages 57-60, Morristown, NJ, USA. Association for Computational Linguistics.
* Hu and Liu (2004) Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In _KDD '04: Proceedings of the tenth ACM SIGKDD international conference 

on Knowledge discovery and data mining_, pages 168-177, New York, NY, USA. ACM.
* [200] Rodney D. Huddleston and Geoffrey K. Pullum. 2002. _The Cambridge Grammar of the English Language_. Cambridge University Press, April.
* [2015] Mike Lewis, Luheng He, and Luke Zettlemoyer. 2015. Joint a* ccg parsing and semantic role labelling. In _Empirical Methods in Natural Language Processing_.
* [2010] Junhui Li, Guodong Zhou, Hongling Wang, and Qiaoming Zhu. 2010. Learning the Scope of Negation via Shallow Semantic Parsing. In _Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)_, pages 671-679, Beijing, China, August. Coling 2010 Organizing Committee.
* [2014] Suguru Matsuyoshi, Ryo Otsuki, and Fumiyo Fukumoto. 2014. Annotating the focus of negation in japanese text. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Lofsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, _Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)_, pages 1743-1750, Reykjavik, Iceland, May. European Language Resources Association (ELRA). ACL Anthology Identifier: L14-1606.
* [2005] Ruslan Mitkov. 2005. _The Oxford handbook of computational linguistics_. Oxford University Press.
* [2012] Roser Morante and Eduardo Blanco. 2012. *SEM 2012 Shared Task: Resolving the Scope and Focus of Negation. In _Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM 2012)_, pages 265-274, Montreal, Canada, June.
* [2012] Roser Morante and Walter Daelemans. 2012. Conandoyle-neg: Annotation of negation in conan doyle stories. In _Proceedings of the Eighth International Conference on Language Resources and Evaluation, Istanbul_.
* [2012] Roser Morante and Caroline Sporleder. 2012. Modality and negation: An introduction to the special issue. _Comput. Linguist._, 38(2):223-260, June.
* [2013] Ann E Nordmeyer and Michael C Frank. 2013. Measuring the comprehension of negation in 2-to 4-year-old children. _Proceedings of the 35th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society_.
* [2010] Lilja Ovrelid, Erik Velldal, and Stephan Open. 2010. Syntactic Scope Resolution in Uncertainty Analysis. In _Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)_, pages 1379-1387, Beijing, China, August. Coling 2010 Organizing Committee.
* [2009] Arzucan Ozgur and Dragomir R. Radev. 2009. Detecting Speculations and their Scopes in Scientific Text. In _Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing_, pages 1398-1407, Singapore, August. Association for Computational Linguistics.
* [2005] Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An Annotated Corpus of Semantic Roles. _Computational Linguistics_, 31(1):71-106.
* [2012] Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? sentiment classification using machine learning techniques. In _Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing_, pages 79-86. Association for Computational Linguistics, July.
* [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. _Journal of Machine Learning Research_, 12:2825-2830.
* [2011] Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in entonotes. In _Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task_, pages 1-27, Portland, Oregon, USA, June. Association for Computational Linguistics.
* [2000] Randolph Quirk, Sidney Greenbaum, and Geoffrey Leech. 2000. _A comprehensive grammar of the English language_. Longman, London.
* [2015] Johan Reitan, Jurgen Faret, Bjorn Gamback, and Lars Bungun. 2015. Negation scope detection for twitter sentiment analysis. In _Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis_, pages 99-108, Lisboa, Portugal, September. Association for Computational Linguistics.
* [1985] Mats Rooth. 1985. _Association with focus_. Ph.D. thesis, Dept. of Linguistics, University of Massachusetts, Amherst.
* [1992] Mats Rooth. 1992. A theory of focus interpretation. _Natural language semantics_, 1(1):75-116.
* Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)_, pages 294-300, Montreal, Canada, 7-8 June. Association for Computational Linguistics.
* [2008] Gyorgy Szarvas, Veronika Vincze, Richard Farkas, and Janos Csirik. 2008. The BioScope corpus: annotation 

for negation, uncertainty and their scopein biomedical texts. In _Proceedings of BioNLP 2008_, pages 38-45, Columbus, Ohio, USA. ACL.
* [Taglicht1984] Josef Taglicht. 1984. _Message and emphasis: On focus and scope in English_, volume 15. Addison-Wesley Longman Limited.
* [van der Wouden1997] Ton van der Wouden. 1997. _Negative contexts: collocation, polarity, and multiple negation_. Routledge, London.
* [Velldal et al.2012] Erik Velldal, Lilja Ovrelid, Jonathon Read, and Stephan Oepen. 2012. Speculation and negation: Rules, rankers, and the role of syntax. _Comput. Linguist._, 38(2):369-410, June.
* [Zou et al.2014] Bowei Zou, Guodong Zhou, and Qiaoming Zhu. 2014. Negation focus identification with contextual discourse information. In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 522-530, Baltimore, Maryland, June. Association for Computational Linguistics.