Venue: NAACL 2016

# Separating Actor-View from Speaker-View Opinion Expressions using Linguistic Features

 Michael Wiegand & Marc Schulder

Spoken Language Systems

Saarland University

D-66123 Saarbrucken, Germany

michael.wiegand@lsv.uni-saarland.de marc.schulder@lsv.uni-saarland.de &Josef Ruppenhofer

Dept. of Information Science

and Language Technology

Hildesheim University

D-31141 Hildesheim, Germany

ruppenho@uni-hildesheim.de

###### Abstract

We examine different features and classifiers for the categorization of opinion words into actor and speaker view. To our knowledge, this is the first comprehensive work to address sentiment views on the word level taking into consideration opinion verbs, nouns and adjectives. We consider many high-level features requiring only few labeled training data. A detailed feature analysis produces linguistic insights into the nature of sentiment views. We also examine how far global constraints between different opinion words help to increase classification performance. Finally, we show that our (prior) word-level annotation correlates with contextual sentiment views.

## 1 Introduction

While there has been much research in sentiment analysis on the tasks of subjectivity detection and polarity classification, there has been less work on other types of categorizations that can be imposed upon subjective expressions.

In this paper, we focus on the _views_ that an opinion expression evokes. By views, we understand the perspective of the holder of some opinion. We distinguish between the two most common types: expressions conveying sentiment of the entities participating in the event denoted by the opinion word, referred to as **actor views** (e.g. _disappointed_ in (1) or _praised_ in (2)), and expressions conveying sentiment of the speaker of the utterance, referred to as **speaker views** (e.g. _excelled_ in (3) or _wasted_ in (4)).

* Party members were \({\bf disappointed}_{actor}\) at the election outcome.
* All representatives \({\bf praised}_{actor}\) the final agreement.
* Sarah \({\bf excelled}_{speaker}\) in virtually every subject.
* The government \({\bf wasted}_{spoker}\) a lot of money.

The distinction between those categories is relevant for related tasks in sentiment analysis, most importantly, opinion holder and target extraction. This has already been demonstrated for verbs [10]. For example, even though the noun _Peter_ has the same grammatical relation to the opinion verb in (5) & (6), in the former sentence it is a holder but in the latter it is a target. Similar cases can be observed for opinion nouns (7) & (8) and opinion adjectives (9) & (10). Only the knowledge of sentiment views helps us to assign opinion roles correctly.

* \([{\rm Peter}]_{Holder}~{}{\bf criticizes}^{verb}_{actor}\) Mary.
* \([{\rm Peter}]_{Target}~{}{\bf cheated}^{verb}_{spoker}\) in the exam.
* \([{\rm Peter}]_{Slobler}~{}{\bf belief}^{actor}_{actor}\) is that all people should be treated equally.
* \([{\rm Peter}]_{Target}~{}{\bf misbehaviour}^{nonn}_{spoker}\) is unbearable.
* \([{\rm Peter}]_{Holder}~{}{\bf is disappointed}^{adj}_{actor}\).
* \([{\rm Peter}]_{Target}~{}{\bf is intelligent}^{adj}_{spoker}\).

While the distinction of sentiment views is not new, we put a different emphasis on this task. Our focus is on the prior meaning that opinion words evoke. Hence we consider this as a **word-level** task. Every opinion word from a sentiment lexicon is to be categorized as conveying either an actor or a speaker view. Our aim is to find comprehensive methods to automatically categorize opinion words of various parts of speech (verbs, nouns, adjectives). The resulting lexical resources are indispensable for open-domain categorization. Previous work focused on contextual classification of sentiment views [1].



son and Moschitti, 2013). Wiegand and Ruppenhofer (2015) showed that while prior lexical knowledge of sentiment views is effective in transferring opinion role extractors to other domains, this does not apply to contextual classifiers.

In this work, we focus on linguistic properties for predicting sentiment views. We examine in how far morphological information can be used. Distributional and syntactic information is also considered. In terms of lexical resources, we examine WordNet and FrameNet. We show that information from a sentiment lexicon can give some additional clues.

In order to combine the different features to predict the sentiment views evoked by opinion words we employ supervised classification. As a classifier, we use Markov Logic Networks Richardson and Matthew (2006) since they do not only allow us to define features for instances (i.e. opinion words) but also to formulate global constraints between different instances. The latter cannot be expressed by traditional classifiers (e.g. SVM). We examine two types of constraints: consistency between instances that are distributionally similar and consistency between morphologically related instances.

Finally, we also examine the relationship between prior lexical information (i.e. our approach) and contextual annotation in the MPQA corpus.

## 2 Related Work

The annotation scheme of the MPQA corpus Wiebe et al. (2005) was the first work to address the distinction between different sentiment views. The two sentiment views are referred to as _direct subjectivity_ (\(=\)actor view) and _expressive subjectivity_ (\(=\)speaker view). In subsequent research, some approaches have been proposed to distinguish these two categories in the MPQA corpus. The most extensive work is Johansson and Moschitti (2013). Since MPQA provides annotation regarding sentiment in context, sentiment views are exclusively considered in _contextual_ classification. The fact that it is the opinion words that convey those views, as we do in this paper, is not addressed. Unlike in this paper, the focus of Johansson and Moschitti (2013) is also on optimizing a machine-learning classifier, in particular to model the interaction between different subjective phrases within the same sentence.

Some of the lexical resources we examine, i.e. WordNet (SS4.1) and FrameNet (SS4.2), have also been employed in Breck et al. (2007) who, like Johannsson and Moschitti (2013), also deal with contextual (sentiment) classification. However, the authors do not examine in how far these individual resources separate speaker and actor views.

Maks and Vossen (2012b) link sentiment views to opinion words as part of a lexicon model for sentiment analysis. Maks and Vossen (2012a) also examine a corpus-driven method to induce opinion words for the different sentiment views. The authors, however, conclude that their approach, which sees news articles as a source for actor views and news comments as a source for speaker views, is not sufficiently effective.

The work most closely related to our research is Wiegand and Ruppenhofer (2015). Opinion words are categorized according to their sentiment view. Our work substantially goes beyond that previous research: Firstly, Wiegand and Ruppenhofer (2015) only consider distributional similarity for inducing opinion views. In this work, we consider various linguistic features and also compare this with distributional information. Secondly, Wiegand and Ruppenhofer (2015) only consider opinion verbs, while we also consider opinion nouns and opinion adjectives.

Wiegand and Ruppenhofer (2015) distinguish between two types of actor views, _agent views_ and _patient views_. The former take their opinion holder as an agent and their target as a patient (typical verbs are _criticize_, _love_, _believe_), while the latter align their roles inversely (typical verbs are _disappoint_, _please_, _interest_). Since this distinction between actor views does not exist among nouns or adjectives, we consider one merged (actor-view) category for all three parts of speech in this paper.

\begin{table}
\begin{tabular}{|p{85.4pt}|c c|c c|} \hline  & **Actor View** & \multicolumn{2}{c|}{**Speaker View**} \\ \hline
**Part of Speech** & **Freq** & **Perc** & **Freq** & **Pere** \\ \hline adjective & 223 & 8.9 & 2279 & 91.1 \\ noun & 487 & 29.1 & 1189 & 70.9 \\ verb & 618 & 52.6 & 557 & 47.4 \\ \hline \end{tabular}
\end{table}
Table 1: Distribution of the different sentiment views.

 

Data

We manually annotated all verbs, nouns and adjectives contained in the Subjectivity Lexicon [25] for view type. The dataset comprises \(2502\) adjectives, \(1676\) nouns and \(1175\) verbs. Since our new dataset1 is an extension of the dataset from Wiegand and Ruppenhofer wiegeand:2015:EMNLP, we adhere to the annotation process proposed in that paper. That is, the basis of the annotation were online dictionaries (e.g. _Macmillan Dictionary_) which provide both a word definition and example sentences. Each word is either labeled as primarily conveying an actor or a speaker view. (Our categorization is binary.) On a subset of \(250\) words for each part of speech, we computed an interannotation agreement (Cohen's \(\kappa\)) of \(61.9\), \(71.9\) and \(60.1\) for verbs, nouns and adjectives, respectively. This agreement can be considered substantial [34]. Table 1 shows the distribution of the different sentiment views among the different parts of speech.

Footnote 1: available at www.coli.uni-saarland.de/~miwieg/naacl_2016_views_data.tgz

The expressions comprising our gold standard do not represent anywhere near the full set of English subjective words with these parts of speech. Otherwise, an automatic categorization would not be necessary in the presence of our gold standard. The classification approach that we propose in this paper, which works well with few labeled training data, would also be helpful for categorizing sentiment views on much larger sets of subjective expressions.

## 4 Feature Design

### WordNet

WordNet [28] is the largest lexical ontology for the English language. It is organized in synsets. However, we want to assign categories to words. Due to the lack of robust word sense disambiguation, in order to use this resource, we consider the union of synsets in which a word with the same part of speech to be categorized is contained.

#### 4.1.1 Gloss Information (GLOSS)

One common way to harness WordNet is by taking into account its glosses. A gloss represents some explanatory text for each synset, usually some definition of the concept. We use the words from those glosses as features in a supervised classifier. We assume that opinion words conveying the same sentiment view also contain similar glosses.

Glosses are a special type of feature. It is basically a bag-of-words feature set, i.e. a _low-level_ feature set, which is known to be sparse yet effective when sufficient training data are used. All the other features presented in this paper are _high-level_ features, i.e. more frequently occurring features already being effective if only few labeled data are used. Glosses are one of the most frequently used features for lexicon induction tasks in sentiment analysis [19; 20; 21; 22]. We will consider them as a baseline, showing that our proposed high-level features are more suitable for our task.

#### 4.1.2 Lexicographer Files (LEX)

Lexicographer files organize the synset inventory of WordNet into a coarse-grained set of semantic categories. In total, there are \(45\) categories for the three parts of speech we consider.2 The advantage of such a coarse-grained inventory is that it should require only few labeled training data in supervised classification.

Footnote 2: For adjectives there exist only two categories. These are too general for our task. Instead we use the lexicographer files of all nouns and verbs occurring in the glosses of those adjectives.

### FrameNet (FN)

FrameNet [1] is a semantic resource that has been found useful for subtasks of sentiment analysis related to ours, i.e. opinion holder/target extraction [1, 20]. It includes a large set of more than \(1,200\) semantic frames that comprise words with similar semantic behaviour. As a feature we use the frame-membership of the opinion words, assuming that different frames are associated with different sentiment views. We use FrameNet version 1.5.

### Subcategorization Frames (SUB)

Subcategorization frames could also be predictive. For example, actor views demand the presence of an 

explicit entity that utters some opinion, i.e. the opinion holder. For a speaker view, this entity remains implicit. This should be reflected in the argument valence of the respective opinion words. We employ the subcategorization frames encoded in _COMLEX_Grishman et al. (1994) for verbs and adjectives, and _NOMLEX_Macleod et al. (1998) for nouns.

### Morphological Information (MORPH)

As morphological information, we consider derivational affixes. Table 2 lists our choice of prefixes and suffixes. We only included affixes that occurred at least 10 times in our dataset.

We distinguish between sentiment and neutral affixes. The _sentiment affixes_ are affixes which, due to their meaning, suggest a sentiment view. For example, _mis-_ as in _misinterpret_ indicates that the speaker believes that a given interpretation is incorrect. _-able_ as in _admirable_ has the meaning of _capable of_ which corresponds to an evaluation of the speaker. We could only find sentiment affixes for speaker views.

The _neutral affixes_ that we use specify which kinds of bases they can combine with. For example, the noun suffix _-ness_ as in _foolishness_ indicates that the word originates from an adjective (i.e. _foolish_). Even though this knowledge is syntactic, it may give us some clue as to what sentiment view an opinion word conveys. Table 1 shows that adjectives predominantly carry speaker views. Therefore, a noun ending in _-ness_ (thus originating from an adjective) may be similarly likely to convey a speaker view.

### Context Patterns (PATT)

Wiegand and Ruppenhofer (2015) proposed patterns for actor-view and speaker-view verbs. For actor views (PATT_actor), they rely on _prototypical opinion holders (protoOHs)_, i.e. common nouns, such as _opponents_ or _critics_, that act like opinion holders Wiegand and Klakow (2011). If a verb often co-occurs with an opinion holder Wiegand and Ruppenhofer (2015) take protoOHs as a proxy - then this is a good indicator of being an actor view (speaker views, per definition, do not have any opinion holder as their dependent). ProtoOHs can similarly be used to extract actor-view nouns and adjectives. For speaker views (PATT_speaker), Wiegand and Ruppenhofer introduced _reproach patterns_, e.g. _blamed for X_ as in (11). These patterns can also be applied to nouns (12) but not to adjectives. For the latter, we did not find any pattern. The patterns were applied to the North American News Text Corpus LDC95T21).

* The UN was **blamed for \(\mathit{misinterpreting}_{verb}\)** climate data.
* The UN was **blamed for the \(\mathit{misinterpretation}_{noun}\)** of climate data.

### Polarity Information (POLAR)

We also investigate in how far polarity information correlates with sentiment views. This information is obtained from the Subjectivity Lexicon Wilson et al. (2005). Each opinion word is assigned a polarity type, i.e. _positive_, _negative_ or _neutral_.

## 5 Markov Logic Networks and Global Constraints

Markov Logic Networks (MLNs) are a supervised classifier combining first-order logic with probabilities. MLNs are a set of pairs \((F_{i},w_{i})\) where \(F_{i}\) is a first-order logic formula and \(w_{i}\) a real valued weight associated with \(F_{i}\). They build a template for constructing a Markov network given a set of constants \(C\). The probability distribution that is estimated is a log-linear model

\[P(X=x)=\frac{1}{Z}exp\left(\sum_{i=1}^{k}w_{i}n_{i}(x)\right) \tag{1}\]

where \(n_{i}(x)\) is the number of groundings in \(F_{i}\) in \(x\) and \(Z\) is a normalization constant. As an implementation, we use _thebeast_Riedel (2008).

We employ MLNs since they allow us (in addition to including ordinary features, i.e. SS4.1-SS4.6) to formulate constraints holding between individual instances. Such _global constraints_ have been effectively exploited with MLNs in related tasks, such as semantic-role labeling Meza-Ruiz and Riedel (2009), anaphora resolution Hou et al. (2013), question answering Khot et al. (2015) and discourse-based sentiment analysis Zirn et al. (2011). We formulate three such constraints. Two of them are

\begin{table}
\begin{tabular}{|l|p{71.1pt}|} \hline
**Type** & **Affixes Used** \\ \hline Sentiment & _–able, dis-, mis-, over-, under-, -/!ism_ \\ \hline Neutral & adj \(\rightarrow\) noun: _cv-, -ify-, ness;_ adj/noun \(\rightarrow\) verb: _-ize_; vert \(\rightarrow\) adj: _ed-, ing;_ verb \(\rightarrow\) noun: _ion-, ing_ \\ \hline \end{tabular}
\end{table}
Table 2: Affixes used as features.

 

based on the two most effective types of word similarities from Wiegand and Ruppenhofer (2015). The first word similarity measures the cosine of word vectors representing opinion words produced by _Word2Vec_-embeddings Mikolov et al. (2013). The second word similarity is represented by the metric of Lin (1998), which exploits the rich set of dependency-relation labels in the context of distributional similarity.3 The third type of consistency considers morphological relatedness by which we understand two words deriving from two different parts of speech but belonging to the same lexical root and therefore carrying similar meaning (e.g. _happiness.noun_ and _happy.adj_). We obtain that type of relatedness from WordNet Miller et al. (1990).

Footnote 3: WordNet is not a good option for measuring similarity, as many synonyms of the same synset have different sentiment views, e.g. _comment.noun_ (actor) vs. _ gossip.noun_ (speaker).

Table 3 lists our constraints. They state that if for two opinion words some similarity or morphological relatedness holds, then these words should convey the same sentiment view. For the two types of word-similarity consistencies we considered the top \(3\) most similar words for each noun, and the top \(5\) most similar words for each verb and adjective. These values were determined empirically. For the generation of word vectors, we used \(200\) dimensions along the default configuration of _Word2Vec_. Word similarity and word vectors were generated from the North American News Text Corpus.

## 6 Experiments

For our evaluation of supervised classification, we focus on a setting in which only few labeled training data are available. We sampled from our gold standard \(20\%\) of the labeled training data. The remaining \(80\%\) are used as test data. This process was repeated five times. We report performance averaged over these five (test) samples. We focus on small training sizes since we think that for the given lexicon induction task, we should pursue an approach that requires little human annotation. Moreover, we show that our approach yields good results despite the absence of large amounts of training data.

### High-Precision Features

Before we evaluate supervised classification, we look for each part of speech at the 10 features with the highest precision (for each of the two views) as displayed in Table 4. This provides a good overview of the quality of different features. Since we do not have an equal class distribution, we also list a baseline-precision that always predicts the sentiment view under consideration. Since this is just an exploratory experiment, we measure precision on the entire dataset. We exclude the WordNet glosses (SS4.1.1) from our analysis as we found individual words from glosses too difficult to interpret.

Table 4 shows that features from all feature groups (SS4.1-SS4.6) achieve a high precision. Subcategorization features (SS4.3) are very predictive for verbs conveying actor views. The frame types that are predictive mostly have in common that one of their arguments is some proposition (13)-(17). This is also true for adjectives (18).

* PP-HOW-TO-INF: They agree\({}_{actor}\) [with him]\({}_{PP}\) [how to solve the problem]\({}_{HOW-TO-INF}\).
* NP-Tobe: They believe\({}_{actor}\) [him]\({}_{NP}\) [to be honest]\({}_{TOBE}\).
* NP-ADJP-PRED: They consider\({}_{actor}\) [him]\({}_{NP}\) [foolish]\({}_{ADIP-PRED}\).
* NP-TO-INF: He allowed\({}_{actor}\) [her]\({}_{NP}\) [to go]\({}_{TO-INF}\).
* S: They thought\({}_{actor}\) [he was always late]\({}_{S}\).
* THAT-S-ADJ: They were aware\({}_{actor}\) [that he was sick]\({}_{THAT-S-ADJ}\).

FrameNet-frames (SS4.2) achieve high precision; but the only frame with good coverage is _Stimulus-focus_ for adjectives conveying a speaker view.

There are fewer lexicographer files (SS4.1.2) than FrameNet-frames in Table 4, but some of them have high coverage, most notably _LEX_person_ for speaker-view nouns and _LEX_feeling_ for actor-view nouns. Given the strength of _LEX_person_, we conclude that most opinion nouns denoting persons tend to be speaker views (e.g. _idiot_ or _loser_). There are also several predictive lexicographer files whose label seems fairly unintuitive, e.g. _LEX_weather_ for

\begin{table}
\begin{tabular}{|l|l l|} \hline
**Abbrev.** & **Constraint as Logic Formula** \\ \hline w2v & \(\forall x[y]\forall z[\forall u]\) & \(Opin.Word(x)\wedge Opin.Word(y)\wedge Word2Vec\text{-Similar}\left(x,y \right)\land ViewOf(z,x)\land ViewOf(u,y)]\rightarrow(z==u)\big{]}\big{]}\) \\ lin & \(\forall x[y]\forall y[\forall z[\forall u]\) & \(Opin.Word(x)\wedge Opin.Word(y)\wedge DekangLin\text{-Similar}\left(x,y \right)\land ViewOf(z,x)\land ViewOf(u,y)]\rightarrow(z==u)\big{]}\big{]}\) \\ morph & \(\forall x[y]\forall z[\forall u]\) & \(Opin.Word(x)\wedge Opin.Word(y)\wedge MorphRelated(x,y)\land ViewOf(z,x)\lor ViewOf(u,y)] \rightarrow(z==u)\big{]}\big{]}\) \\ \hline \end{tabular}
\end{table}
Table 3: Global constraints enforcing sentiment view consistency as incorporated in MLNs.

 

speaker-view verbs or _LEX_animal_ for speaker-view nouns. These are not errors, however. They actually concern words that convey opinions in metaphorical usage. For instance, _cloud_ (a typical weather verb) conveys a speaker view if it is used metaphorically as in _The stroke clouded memories of her youth_. Nouns denoting animals, such as _bull_ and _dragon_, convey a speaker view if they are meant to describe a human being (_She is a real dragon!_). Other noun classes follow this pattern, e.g. _body (parts)_ with terms such as _backbone_ or _burn_.

Simple morphological features (SS4.4) also seem to be meaningful. In particular, the noun suffix _-ity_ (occurring \(132\) times in our set of opinion nouns) is indicative of speaker views. The relevant nouns are derived from adjectives (Table 2) and the set of adjectives predominantly conveys speaker views (Table 1).

Even plain polarity information (SS4.6) has some significance. _Neutral_ sentiment verbs often convey an actor view, such as _opinion_, _utterance_ or _view_.

The fact that the pattern-feature (SS4.5) also appears on the list of actor-view nouns and adjectives suggests that it is not only effective for verbs as shown in Wiegand and Ruppenhofer (2015) but also for nouns and adjectives.

Finally, we performed an ablation experiment in which we trained a classifier with all of these features in MLNs and compared it to another classifier in which each of the feature groups (_POLAR_, _LEX_, _MORPH_ etc.) was removed, one by one. We computed statistical significance (t-test), testing whether the classifier trained on a feature set in which one feature group was removed performs significantly worse than a classifier with all features. We found that, at a significance level \(p<0.05\), this is always the case, with the exception of _LEX_ (here, the significance level is \(p=0.0552\)). This is proof that features from most feature groups contain information that is to some extent complementary.

\begin{table}
\begin{tabular}{|l r c l r|r r|c c|} \hline
**adj** & _always-predict-this-view pre: 91.1_ & & \multicolumn{3}{c|}{**Spanker View**} \\ \cline{7-8}  & Feature & Prec & Freq & Feature & Prec & Freq & Feature & Prec & Freq \\ \hline SUB\_EXTRAP-FOR-TO-INF_ & 100.0 & 104 & FN\_Killing & 100.0 & 11 & MORPL\_miss- & 92.3 & 13 \\ FN\_Desirability & 100.0 & 48 & LEX\_minimal & 91.7 & 24 & LEX\_wather & 84.2 & 19 \\ FN\_Expertise & 100.0 & 31 & FN\_Catastrophe & 90.9 & 11 & FN\_Prevarication & 81.8 & 11 \\ FN\_Modaily-evaluation_ & 100.0 & 30 & LEX\_body & 90.9 & 11 & MORPRL\_over- & 78.6 & 14 \\ FN\_Cardiness & 100.0 & 20 & MORPH\_*cy & 90.6 & 32 & FN\_Xilling & 76.9 & 13 \\ FN\_Usefulness & 100.0 & 19 & MORPH\_+ity & 90.2 & 132 & FN\_Self-motion & 75.0 & 16 \\ FN\_Praiseworthiness & 100.0 & 15 & MORPH\_ units- & 90.0 & 20 & MORPH\_+ize & 73.5 & 200 \\ FN\_Siminalus-focus & 99.1 & 113 & LEX\_substance & 87.0 & 23 & LEX\_change & 64.8 & 270 \\ LEX\_plant & 98.7 & 79 & LEX\_food & 84.2 & 19 & PATT\_speaker & 62.7 & 252 \\ MORPH\_*able & 98.3 & 172 & LEX\_person & 83.5 & 267 & FN\_Communication-noise & 62.5 & 16 \\ \hline \end{tabular} 
\begin{tabular}{|l r c|l r|l r|} \hline
**adj** & _always-predict-this-view pre: 89.9_ & & \multicolumn{3}{c|}{**noun (always-predict-this-view pre: 29.1)**} & \multicolumn{3}{c|}{**verb (always-predict-this-view pre: 52.6)**} \\ \hline Feature & Prec & Freq & Feature & Prec & Freq & Feature & Prec & Freq \\ \hline FN\_Experiment-focus_ & 81.8 & 11 & FN\_Emotion-directed & 95.1 & 41 & FN\_experienced-focus & 100.0 & 21 \\ SUB\_THAT-S\_ADJ & 75.9 & 54 & FN\_ Experienced-focus & 92.9 & 14 & SUB\_BP-PIOW-TO-INF & 100.0 & 13 \\ F\_Enmotion-directed & 74.6 & 67 & PATT\_actor & 83.0 & 53 & SUB\_NP-TOPBE & 100.0 & 12 \\ SUB\_ADJ-TO-INF_ & 38.7 & 31 & FN\_ Judgent & 73.3 & 15 & SUB\_NP-ADVP-PRED & 100.0 & 11 \\ MORPH\_*ed & 29.2 & 288 & LEX\_feeling & 65.7 & 268 & SUB\_NP-ADJP-PBED & 94.4 & 18 \\ PATT\_actor & 26.3 & 335 & FN\_Medical-conditions & 61.5 & 13 & FN\_Ind judgment-direct-address & 92.8 & 14 \\ MORPH\_dis- & 25.4 & 71 & FN\_Mostile-encounter & 58.3 & 12 & SUB\_NP-TO-NP & 92.3 & 13 \\ LEX\_weather & 22.2 & 18 & SUB\_NOM-INTRANS-RECIP & 57.1 & 14 & SUB\_NP-TO-INF & 91.7 & 12 \\ LEX\_feeling & 18.1 & 695 & POLAR\_neutral & 54.8 & 93 & SUB\_NP-ING-OC & 90.9 & 11 \\ SUB\_ADJ-PP & 17.2 & 548 & LEX\_relation & 53.8 & 26 & SUB\_S & 90.4 & 52 \\ \hline \end{tabular}
\end{table}
Table 4: High-precision features (_minimal frequency_\(>10\)).

\begin{table}
\begin{tabular}{|l|l|} \hline
**Classifieris/s** & **Description** \\ \hline graph & graph-based induction approach as proposed in Wiegand and Ruppenhofer (2015) \\ \hline mln\({}_{local}\) & Markov Logic Networks with only local features, i.e. features from §4 \\ \hline svm & Support Vector Machines using exactly the same features as _mln\({}_{local}\)_ \\ \hline mln\({}_{ogv}\), & Markov Logic Networks with global constraints mln\({}_{lin}\), \\ mln\({}_{morph}\) & \\ \hline mln+graph & Markov Logic Networks that uses the output of _graph_ as a further feature \\ \hline \end{tabular}
\end{table}
Table 5: Description of the different classifiers.

 



### Classification

Table 5 lists the different types of classifiers we consider. As one baseline, we consider the graph-based approach _graph_ from Wiegand and Ruppenhofer (2015) which starts with the seeds gained by the surface patterns (SS4.5)4 and then runs label propagation (Talukdar et al., 2008) based on a distributional similarity graph (using the metric by Lin (1998)). _graph_ is the only classifier not depending on manually labeled training data. So far, it has only been examined on verbs. As a further baseline, we consider our features from SS4 on an SVM. (We use \({\text{SVM}^{light}}\)(Joachims, 1999).) It should be considered as a state-of-the-art classifier that, unlike _mln_, cannot incorporate global constraints (Table 3).

Footnote 4: Since the surface patterns for speaker views from Wiegand and Ruppenhofer (2015) cannot be applied to adjectives (§4.5), we instead used the effective suffix feature MORPH_able (Table 4) for generating speaker-view seeds of opinion adjectives.

Table 6 shows the results. Both _graph_ and _svm_ are significantly outperformed. _graph_ performs better on verbs (in terms of F-score) than on nouns and adjectives. It is also for these parts of speech that the global constraints _w2v_ and _lin_ notably improve the performance of _mln_. Global constraints have a lesser impact on verbs. However, a combination of global constraints is effective, as well as a combination of _graph_ and _mln_. The best overall results are obtained by the combination of _mln_ with global constraints and _graph_. These results suggest that our new features (including global constraints) are useful and complementary to previous work, i.e. _graph_.

Figure 1 compares the feature derived from WordNet glosses (§4.1.1), a standard feature for lexicon induction, with the remaining features we use on a learning curve. This feature performs poorly if only few labeled training data are used. Our proposed feature set is consistently better. The combination of glosses and our proposed features is only helpful if many labeled training instances are used (\(>60\%\)).

### Prior Labels and Context Labels

So far, we have considered sentiment views as prior information of words. Now we relate those labels to sentiment views annotated in context. For that, we consider the view annotation in the MPQA corpus.

Table 7 shows that prior labels of opinion words largely coincide with the respective context labels. This proves that it is a valid approach to compile lexicons with sentiment views, which can subsequently be used in contextual sentiment-view classification.

However, in Table 7, we still observe mismatches between prior and contextual labels. This mostly concerns actor-view words in speaker-view contexts. We examine this mismatch more closely on nouns (_highlighted in_gray__) where this confusion is greatest. In MPQA, most subjective expressions that are annotated are sequences of tokens rather than individual words. We found that the largest set of disagreements derives from the nature of MPQA's contextual annotation. The annotators were asked to label spans that expressed opinions that are salient in the document context. Often these are larger spans composed of multiple smaller subjective expressions. The component expressions were not kept track of because the opinion expressed by the larger span was more salient on the document level.

For example, the annotation of the subjective phrase _this must be a **warning**_ as a speaker view (containing the actor noun _warning_), in our opinion is primarily triggered by the epistemic modal verb _must_, which signals that the speaker feels pelled to come to the conclusion that _this is a warn

\begin{table}
\begin{tabular}{|l|r|c||r|r|r|r|} \hline  & \multicolumn{4}{c|}{**context**} \\ \hline  & \multicolumn{2}{c|}{**verbs**} & \multicolumn{2}{c|}{**nonus**} & \multicolumn{2}{c|}{**adjectives**} \\ \cline{2-7}
**prior** & actor & speaker & actor & speaker & actor & speaker \\ \hline actor & **2926** & 834 & **1671** & 802 & **269** & 81 \\ speaker & 119 & **810** & 151 & **2150** & 354 & **3489** \\ \hline \end{tabular}
\end{table}
Table 7: Comparison of prior labels and context labels.

Figure 1: WordNet gloss feature vs. the remaining features (_results averaged over all three parts of speech_).



ing_. The actor view of _warning_ is not invalidated by this: it is just backgrounded relative to the speaker view introduced by the modal verb, which, going in parallel with its greater prominence, is also the syntactic governor of the verb phrase _be a warning_, of which the actor view _warning_ is part. Our evaluation scheme might thus detect a match between our prior annotation for the modal _must_ and the MPQA's larger phrase. But since the less prominent actor view was not picked up by the MPQA annotators, our prior annotation has no counterpart. Copula constructions similarly represent instances, where the speaker performs a speech act (e.g. a warning) by using the copular construction (e.g. _This is a warning_). Here, the speaker is identical to the actor of the warning. Practically, it makes no difference whether we call such a case speaker or actor view, as long as we can recognize that the actor is the speaker.

In order to show that the annotation of MPQA focuses on the more salient opinions and thus sentiment views as conveyed by less prominent expressions are not considered (and largely account for the mismatches in Table 7), we designed a supervised classifier whose features indicate whether a mention of an actor-view expression in a subjective phrase is salient. The features are displayed in Table 8.

The key salience features regarding speaker views, i.e. _modal_ and _copular_, were already discussed above. Features indicating the salience of the actor-view word address the subcategorization frame of the word. Cases in which there is a person as some subcategorized argument (_personArg_) often imply an opinion holder. The presence of an (explicit) opinion holder indicates an actor view. A proposition as argument of an opinion word is typically the proposition of some opinion holder (_propArg_) and not of the speaker of the utterance.

For this experiment we take the detection of subjective phrases as given. (Only the information regarding contextual sentiment views is withheld.) This allows us to define features that explicitly look into the entire text span constituting the subjective phrase in which each opinion word is contained. The two length features (_shortPhrase_ and _longPhrase_) make use of this information. If a phrase is long, chances are high that there are other more salient opinion words contained than the one under consideration. In short subjective phrases, the presence of other salient words in it is unlikely. This is supported by the fact that the average length of subjective phrases with a speaker view (in which an actor-view opinion noun occurs) are \(5.4\) tokens while actor-view phrases (that include an actor-view noun) only have an average length of \(2.3\) tokens.

The majority features (_majActor_ and _majSpeaker_) also exploit the information of the entire subjective phrase. We argue that the sentiment view of the phrase is likely to coincide with the view of the majority of the opinion words contained in that phrase.

Table 9 shows how these features separate the mentions of an actor-view opinion noun into contextual actor views and speaker views. We report classification using an SVM (10-fold cross-validation). With only those few features, we largely outperform the baseline always classifying an instance as an actor view (i.e. the majority class). Table 10 displays the precision of each individual feature, supporting that these features are effective. These experiments show that there is indeed a systematic relationship between salience and contextual sentiment views.

 



## 7 Conclusion

We examined different types of features and classifiers for the categorization of sentiment views that opinion words convey. We found that many features are effective for this task. A detailed feature analysis provided linguistic insights into the nature of sentiment views. As a classifier, MLNs performed best. This classifier has the advantage that global constraints can be incorporated, which raises classification performance on nouns and adjectives. Our approach outperforms a previously proposed graph-based approach evaluated on opinion verbs. We also demonstrated that prior sentiment views correlate with contextual sentiment views on MPQA.

## Acknowledgements

The authors would like to thank Stephanie Koser for annotating the dataset presented in this paper. We are also grateful to Sebastian Riedel, Manfred Klenner, Don Tuggener and Cacilia Zirn for advising us on Markov Logic Networks. The authors were partially supported by the German Research Foundation (DFG) under grants RU 1873/2-1 and WI 4204/2-1.

## References

* [Andreevkaia and Bergler2006] Alina Andreevkaia and Sabine Bergler. 2006. Mining WordNet for a Fuzzy Sentiment: Sentiment Tag Extraction from WordNet Glosses. In _Proceedings of the Conference on European Chapter of the Association for Computational Linguistics (EACL)_, pages 209-216, Trento, Italy.
* Baker et al. [1998] Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet Project. In _Proceedings of the International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING/ACL)_, pages 86-90, Montreal, Quebec, Canada.
* Bethard et al. [2004] Steven Bethard, Hong Yu, Ashley Thornton, Vasileios Hatzivassiloglou, and Dan Jurafsky. 2004. Extracting Opinion Propositions and Opinion Holders using Syntactic and Lexical cues. In _Computing Attitude and Affect in Text: Theory and Applications_. Springer-Verlag.
* Breck et al. [2007] Eric Breck, Yejin Choi, and Claire Cardie. 2007. Identifying Expressions of Opinion in Context. In _Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)_, pages 2683-2688, Hyderabad, India.
* Breck et al. [2005]

\begin{table}
\begin{tabular}{|l|p{142.3pt}|} \hline
**Actor View** & shortPhrase: 88.5; lightVerb: 85.8; personArg: 81.0; propArg: 74.6; majActor: 72.9 \\ \hline
**Speaker View** & precededByAs: 90.9; majSpeaker: 90.0; modal: 79.6; longphrase: 73.3; copula: 61.9; emphasis: 57.5 \\ \hline \end{tabular}
\end{table}
Table 10: Precision of features from Table 8.

\begin{table}
\begin{tabular}{|c|p{142.3pt}|} \hline
**Abbreviation** & **Features Indicating Contextual ACTOR View** \\ \hline personArg & a person is argument of opinion word; persons may indicate opinion holders; (explicit) opinion holders indicate absence of speaker view (_his warning of a catastrophe_) \\ \hline propArg & a proposition is argument of the opinion word (_warning that this fish is not fit to eat_); propositions are typically arguments of actor-views words \\ \hline lightVerb & opinion word is governed by light verb (_they issued/gave a warning_), light verbs indicate the presence of an actor outside of the maximal phrase of a subjective noun \\ \hline shortPhrase & opinion word is part of short subjective phrase (\(<3\) tokens); short phrases make embedding of another more salient (speaker-view) word unlikely \\ \hline majActor & majority of other opinion words in subjective phrase are actor-view words \\ \hline
**Abbreviation** & **Features Indicating Contextual SPEAKER View** \\ \hline copula & opinion word is part of copula construction (_this is a warning_) – see discussion in §6.3 \\ \hline modal & opinion word is in modal scope (_this must be a warning_) – see discussion in §6.3 \\ \hline emphasis & opinion word is accompanied by empathic cue, e.g. \(t\); quotation (_they gave him a “warting”_), (rhetoric) question; emphases typically originate from speaker \\ \hline precededByAs & preceded by _as (this was regarded as an urgent warning)\({}_{as-phrase}\)_, _as-phrase_), _as-phrase typically occurs as an argument of categorization predicates regard, view, see, consider etc.. – with these predicates an as-phase often conversyscale we speaker, especially since the predicates often have no explicit holder \\ \hline longPhase & opinion word is part of long subjective phrase (\(>4\) tokens); long phrases make embedding of another more salient (speaker-view) word likely \\ \hline majSpeaker & majority of other opinion words in subjective phrase are speaker-view words \\ \hline \end{tabular}
\end{table}
Table 8: Salience features for detecting contextual views (given a mention of a prior actor-view opinion noun).

\begin{table}
\begin{tabular}{|l|p{142.3pt}|p{142.3pt}|p{142.3pt}|} \hline  & **Majority Class.** & \multicolumn{2}{c|}{**Proposed Features**} \\ \hline
**Acc** & 65.9 & **F1** & 39.7 & **Acc** & 78.7 & **F1** & 73.5 \\ \hline \end{tabular}
\end{table}
Table 9: Contextual classification of opinion nouns with a prior actor view (using feature set from Table 8).



Yoonjung Choi and Janyce Wiebe. 2014. +/-EffectWordNet: Sense-level Lexicon Acquisition for Opinion Inference. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 1181-1191, Doha, Qatar.
* Esuli and Sebastiani [2005] Andrea Esuli and Fabrizio Sebastiani. 2005. Determining the semantic orientation of terms through gloss classification. In _Proceedings of the ACM International Conference on Information and Knowledge Management (CIKM)_, pages 617-624, Bremen, Germany.
* Grishman et al. [1994] Ralph Grishman, Catherine McKeown, and Adam Meyers. 1994. COMLEX Syntax: Building a Computational Lexicon. In _Proceedings of the International Conference on Computational Linguistics (COLING)_, pages 268-272, Kyoto, Japan.
* Gyamfi et al. [2009] Yaw Gyamfi, Jansce Wiebe, Rada Mihalcea, and Cem Akkaya. 2009. Integrating Knowledge for Subjectivity Sense Labeling. In _Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL (HLT/NAACL)_, pages 10-18, Boulder, CO, USA.
* Hou et al. [2013] Yufang Hou, Katja Markert, and Michael Strube. 2013. Global Inference for Bridging Anaphora Resolution. In _Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL (HLT/NAACL)_, pages 907-917, Atlanta, GA, USA.
* Support Vector Learning_, pages 169-184. MIT Press.
* Johansson and Moschitti [2013] Richard Johansson and Alessandro Moschitti. 2013. Relational Features in Fine-Grained Opinion Analysis. _Computational Linguistics_, 39(3):473-509.
* Kang et al. [2014] Jun Seok Kang, Song Feng, Leman Akoglu, and Yejin Choi. 2014. ConnotationWordNet: Learning Connotation over the Word+Sense Network. In _Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)_, pages 1544-1554, Baltimore, MD, USA.
* Khot et al. [2015] Tushar Khot, Niranjan Balasubramanian, Eric Gribkoff, Ashish Sabharwal, Peter Clark, and Oren Etzioni. 2015. Exploring Markov Logic Networks for Question Answering. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 685-694, Lisbon, Portugal.
* Kim and Hovy [2006] Soo-Min Kim and Eduard Hovy. 2006. Extracting Opinions, Opinion Holders, and Topics Expressed in Online News Media Text. In _Proceedings of the ACL Workshop on Sentiment and Subjectivity in Text_, pages 1-8, Sydney, Australia.
* Landis and Koch [1977] Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. _Biometrics_, 33(1):159-174.
* Lin [1998] Dekang Lin. 1998. Automatic Retrieval and Clustering of Similar Words. In _Proceedings of the Annual Meeting of the Association for Computational Linguistics and International Conference on Computational Linguistics (ACL/COLING)_, pages 768-774, Montreal, Quebec, Canada.
* Macleod et al. [1998] Catherine Macleod, Ralph Grishman, Adam Meyers, Leslie Barrett, and Ruth Reeves. 1998. NOMLEX: A Lexicon of Nominalizations. In _Proceedings of EUROALEX_, pages 187-193, Liege, Belgium.
* Maks and Vossen [2012a] Isa Maks and Piek Vossen. 2012a. Building a fine-grained subjectivity lexicon from a web corpus. In _Proceedings of the Conference on Language Resources and Evaluation (LREC)_, pages 3070-3076, Istanbul, Turkey.
* Maks and Vossen [2012b] Isa Maks and Piek Vossen. 2012b. A lexicon model for deep sentiment analysis and opinion mining applications. _Decision Support Systems_, 53:680-688.
* Meza-Ruiz and Riedel [2009] Ivan Meza-Ruiz and Sebastian Riedel. 2009. Jointly Identifying Predicates, Arguments and Senses using Markov Logic. In _Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL (HLT/NAACL)_, pages 155-163, Boulder, CO, USA.
* Mikolov et al. [2013] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In _Proceedings of Workshop at the International Conference on Learning Representations (ICLR)_, Scottsdale, AZ, USA.
* Miller et al. [1990] George Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller. 1990. Introduction to WordNet: An On-line Lexical Database. _International Journal of Lexicography_, 3:235-244.
* Richardson and Matthew [2006] Matthew Richardson and Pedro Matthew. 2006. Markov Logic Networks. _Machine Learning_, 62(1-2):107-136.
* Riedel [2008] Sebastian Riedel. 2008. Improving the Accuracy and Efficiency of MAP Inference for Markov Logic. In _Proceedings of the Annual Conference on Uncertainty in AI (UAI)_, pages 468-475, Helsinki, Finland.
* Plaukhar et al. [2008] Partha Pratim Talukar, Joseph Reisinger, Marius Pasca, Deepak Ravichandran, Rahul Bhagat, and Fernando Pereira. 2008. Weakly-Supervised Acquisition of Labeled Class Instances using Graph Random Walks. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 582-590, Honolulu, HI, USA.
* Wiebe et al. [2005] Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating Expressions of Opinions and Emotions in Language. _Language Resources and Evaluation_, 39(2/3):164-210.
* Wiegand and Klakow [2011] Michael Wiegand and Dietrich Klakow. 2011. Prototypical Opinion Holders: What We can Learn from 

Experts and Analysts. In _Proceedings of Recent Advances in Natural Language Processing (RANLP)_, pages 282-288, Hissar, Bulgaria.
* Wiegand et al. [2015] Michael Wiegand and Josef Ruppenhofer. 2015. Opinion Holder and Target Extraction based on the Induction of Verbal Categories. In _Proceedings of the Conference on Computational Natural Language Learning (CoNLL)_, pages 215-225, Beijing, China.
* Wilson et al. [2005] Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing Contextual Polarity in Phrase-level Sentiment Analysis. In _Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT/EMNLP)_, pages 347-354, Vancouver, BC, Canada.
* Zirn et al. [2011] Cacilia Zirn, Mathias Niepert, Heiner Stuckenschmidt, and Michael Strube. 2011. Fine-Grained Sentiment Analysis with Structural Features. In _Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP)_, pages 336-344, Chiang Mai, Thailand.