Venue: NAACL 2016

# Opinion Holder and Target Extraction on Opinion Compounds - A Linguistic Approach

 Michael Wiegand

Christine Bocionek

Spoken Language Systems

Saarland University

D-66123 Saarbrucken, Germany

michael.wiegand@lsv.uni-saarland.de

Josef Ruppenhofer

Dept. of Information Science

and Language Technology

Hildesheim University

D-31141 Hildesheim, Germany

rbopenho@uni-hildesheim.de

###### Abstract

We present an approach to the new task of opinion holder and target extraction on opinion compounds. Opinion compounds (e.g. _user rating_ or _victim support_) are noun compounds whose head is an opinion noun. We do not only examine features known to be effective for noun compound analysis, such as paraphrases and semantic classes of heads and modifiers, but also propose novel features tailored to this new task. Among them, we examine paraphrases that jointly consider holders and targets, a verb detour in which noun heads are replaced by related verbs, a global head constraint allowing inferencing between different compounds, and the categorization of the sentiment view that the head conveys.

## 1 Introduction

One of the key subtasks in sentiment analysis is opinion role extraction. It can be divided into the extraction of _opinion holders (OH)_, i.e. entities expressing an opinion, and the extraction of _opinion targets (OT)_, i.e. entities or propositions at which sentiment is directed. This task is vital for various applications involving sentiment analysis, e.g. opinion summarization or opinion question answering.

Opinion role extraction is commonly regarded as a task in lexical semantics. An opinion is evoked by some opinion word, e.g. _criticized_ in [1], _skeptical_ in [2] or _intenttions_ in [3], and its opinion roles are usually realized as syntactic dependents. Opinion words come in many shapes, the most frequent types being opinion verbs [1], opinion adjectives [2] and opinion nouns [3]. These types of opinion words have extensively been studied in various sentiment-related corpora, such as MPQA Wiebe et al. (2005).

* [leftmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]
* [rightmargin=*]
* [leftmargin=*]

[MISSING_PAGE_POST]

*]
* [rightmargin=*]
* [right=*]
* [right*]
* [right=*]
* [right*]
* [right=*]
*

use of noun compounds. In the STEPS-corpus, the benchmark dataset for German opinion role extraction Ruppenhofer et al. (2014), almost every other sentence contains an opinion compound.

Compounds can also be commonly found in other key languages, such as English. Since the methods we apply to this task and the issues that they address are not language specific, our approach can be replicated on other languages.

Apart from examining traditional features from noun compound analysis, in this paper, we also introduce novel features specially designed for the analysis of opinion compounds.

We also created a new gold standard for this task (see also SS3). The STEPS-corpus, as such, is fairly small and only contains about unique compounds. We considered this amount insufficient for producing a gold standard. Also, none of the existing datasets on noun compounds Lauer (1995); Barker and Szpakowicz (1998); Nastase and Szpakowicz (2003); Girju et al. (2009); Kim and Baldwin (2005); Tratz and Hovy (2010); Dima et al. (2014) contain any information regarding opinion roles.

## 2 Related Work

With regard to opinion role extraction, many features for supervised learning have been explored. They typically address the relationship between opinion word and opinion role on the basis of surface patterns Choi et al. (2005), part-of-speech information Wiegand and Klakow (2010), syntactic information Kessler and Nicolov (2009); Jakob and Gurevych (2010) or semantic role labeling Johansson and Moschitti (2013); Deng and Wiebe (2015). The majority of those features cannot be applied to our task since for opinion compounds, there is no context between opinion role and opinion word.

In the area of noun compound analysis, there are two predominant approaches. On the one hand, lexical resources, such as WordNet Miller et al. (1990), are employed in order to assign semantic categories to head and modifier and infer from those labels the underlying relation Rosario and Hearst (2001); Kim and Baldwin (2005); Girju et al. (2005); Girju et al. (2009). On the other hand, paraphrases that contain co-occurrences of head and modifier are exploited Girju et al. (2009); Nakov and Hearst (2013). In order to increase coverage, paraphrases can be automatically acquired Butnariu and Veale (2008); Kim and Nakov (2011). Cross-lingual information has also been harnessed for this task Girju (2007).

## 3 Data & Annotation

We created a new dataset1 by retrieving opinion compounds from the _deWaC_-corpus Baroni et al. (2009) comprising 1.7 billion words. (Word embeddings (SS5.2 & SS5.6) and word similarity graphs (SS5.7 & SS6.4) were also created from this corpus.)

Footnote 1: available at: www.coli.uni-saarland.de/~miwieg/naacl_2016_op_compounds_data.tgz

In German, noun compounds are typically realized as single tokens. In order to obtain a set of opinion compounds, we extracted all noun compounds from _deWaC_ whose second morpheme is an opinion noun. Morphological analysis was carried out using _morphisto_ Zielinski and Simon (2009).2 As opinion nouns, we used the nouns from the **PolArt sentiment lexicon**Klenner et al. (2009). Unfortunately, this lexicon is lacking in _neutral_ opinion nouns, such as _Meinung (opinion)_ or _Erwartung (expectation)_ which frequently occur in compounds, e.g. _Expertenmeinung (expert opinion)_ or _Kundenerwartungen (customer expectations)_. Therefore, we translated the \(235\) neutral opinion nouns from the (English) Subjectivity Lexicon Wilson et al. (2005) into German.

Footnote 2: _The data release provides more details regarding the gold standard, e.g. how compound instances were sampled._

From the opinion compounds extracted from _deWaC_, we created two manually annotated datasets (Table 2). We use more than one dataset as we consider our task as a multi-stage task as shown in Figure 1. We believe that this is necessary as differ

\begin{table}
\begin{tabular}{|l||r|} \hline
**compounds** & _user rating; victim support; spring upswing_ \\ \hline
**immediate constituents** & _user; victim; spring rating; support; upswing_ \\ \hline
**grammatical function** & modifier & head \\ \hline \end{tabular}
\end{table}
Table 1: Internal structure of opinion compounds.

 

ent types of knowledge are required for the different steps. In the first step (**Dataset I**), the compounds containing some opinion role (4)-(7) are separated from those not containing any role at all (8)-(9). At this stage, holders are not distinguished from targets. This is done in the second step which exclusively focuses on opinion roles. This step is further divided into two substeps. First, one checks whether the modifier denotes a person. A modifier representing an opinion role but not denoting a person (e.g. _test anxiety_) can only be a target. Since this is a simple classification step (provided a lexical resource is available which tells persons apart from non-persons, e.g. WordNet), we have no dataset for it. The greater challenge lies in all those compounds whose modifier is a person and for which we already know that it is either holder or target (e.g. _user rating_ or _victim support_). Only for those cases do we produce another dataset (**Dataset II**). Note that in this dataset the two roles are not completely disjoint. In \(3\%\) of the compounds, the modifier represents both holder and target. Prominent examples are reciprocal relationships, e.g. _Geschwisterneid (sibling jealousy)_.

On a sample of \(200\) compounds extracted from each of the two datasets we measured inter-annotation agreement. On the first dataset, we obtained Cohen's \(\kappa=0.60\), while on the second, we obtained \(\kappa=0.60\) for holders and \(\kappa=0.62\) for targets, respectively. These scores can be interpreted as substantial agreement Landis and Koch [1].

## 4 Classifiers and the Three Different Tasks

We solve the given task as a supervised classification problem. As a classifier, we employ Markov Logic Networks (MLNs). We use this classifier because it allows us to integrate all of our features, including global constraints (see discussion in SS5.5).

We consider **3 different tasks** (bold rhombuses in Figure 1): the detection of opinion **roles** (Dataset I), the detection of opinion **holders** (Dataset II) and the detection of opinion **targets** (Dataset II). Each task is modeled as a binary classifier. Even though the latter two tasks use the same dataset, we cannot train just one single binary classifier as there are compounds whose modifiers represent both holder and target, e.g. _Geschwisterneid (sibling jealousy)_.3

Footnote 3: For the holder-detection task, the modifier of such compounds are considered a holder, while for the target-detection task, they are considered a target. For the holder-detection task, we have the two classes _holder_ and _no holder_, while for the target-detection task, the classes are _target_ and _no target_.

## 5 Feature Design

Our core **global features**, which are used for all three tasks (SS4), include the two predominant approaches for compound analysis, i.e. (plain) paraphrases (SS5.1) and semantic knowledge (SS5.4). We extend the paraphrase approach with two major innovations. First, we examine a verb detour (SS5.2) by which we gain important information regarding the syntactic relationship between the modifier and the head of the compound. Secondly, we show that joint paraphrases (SS5.3) considering both holder and

Figure 1: Generic pipeline for processing opinion compounds.

 

target are better than paraphrases focusing on only one role. We argue that for our task, (syntactic) ambiguity rather than lack of coverage is the pressing problem. Therefore, we do not focus on paraphrase acquisition but introduce new disambiguation features. Beside the extensions to paraphrases mentioned above, we introduce a global head constraint (SS5.5) as an additional global feature. As a **local feature** for the initial role classification, we perform subjectivity detection on the compound (SS5.6). And finally, we use the sentiment view that the head of the compound evokes (SS5.7) as a local feature in the holder and target classification tasks.

Table 3 lists which feature is used in which task. If a feature is restricted to a specific task (i.e. it is a local feature), then this is motivated below in the relevant subsection introducing the respective feature.

### Plain Paraphrases (PARA\({}_{plain}\))

An established method for computing the relation expressed by a compound is to consider paraphrases, that is, co-occurrences of the head and modifier as individual constituents accompanied by some predictive context. For example, the compound _Expertenauffassung (expert view)_ can be paraphrased by _Auffassung unter Experten (view among experts)_. The preposition _unter (among)_ is an explicit lexical clue for the (implicit) relation holding between head and modifier in the compound. As paraphrases we manually collected \(18\) frequent dependency relations that typically hold between an opinion noun and its opinion holder (10) or its opinion target (11).4_(The data release provides more information including a full list of all paraphrases.)_ For each compound, we check in _deWaC_ whether head and modifier can be observed in any of those relations.

Footnote 4: We obtain dependency parses by _ParZu_[2].

We consider each of those selected dependency relations as an individual feature, i.e. we do not explicitly group the chosen relations to _holder_ and _target_. Assuming that the predictiveness of the different relations varies, this encoding allows a supervised classifier to appropriately weight each relation.

### Verb Detour Paraphrases (PARA\({}_{verb}\))

Some of the paraphrases from SS5.1 are ambiguous. This particularly concerns _objp\({}_{von(of)}\)_ which occurs with approx. \(40\%\) of the compounds of our dataset. On the first reading illustrated by (12a), we observe a modifier being a holder, while, on the second reading shown by (13a), the modifier is a target.

For heads being deverbal nouns (e.g. _comment_ or _assessment_), this ambiguity can often be resolved by considering morphologically related verbs. In (12b) and (13b), the two modifiers no longer share the same dependency relation to the opinion word. Opinion holders tend to occur in subject position (12b) while targets occur in object position (13)b). Wiegand and Klakow (2012) identify these dependency relations for the two different opinion roles as the most frequent ones. So for deverbal nouns, which make up 57% of the heads of our compounds, we add a feature that checks in _deWaC_ whether the modifier is more often observed as a subject or an object of a verb related to the head. (Wiegand and Klakow (2012) actually consider semantic roles, i.e. _agent_ and _patient_, instead of dependency relations. Due to the lack of robust semantic role-labeling for German, we use dependency relations as a proxy. That is, we identify agents with the dependency relation _subj_ and patients with the relation _obj._)

* _paraphrases for Leskormomentar (reader comments):_
* Kommentar\({}_{noun}\)__[von Lesern \(obj_{pan}\)]
* (comment\({}_{noun}\)__[or readers \(obj_{pfq}\)]).
* Leser\({}_{why}\) kommenteren\({}_{verb}\) ein Erepisis.
* (Readers\({}_{subj}\) comment\({}_{verb}\) on an event.)
* _paraphrases for Schluterbeurteilung (student assessment):_
* Beurteilung\({}_{noun}\)__[von Schulten \(obj_{person}\)]
* (assessment\({}_{non}\)__[of students \(obj_{pfq}\)])
* Teachers assess\({}_{verb}\)__students\({}_{obj}\)_._)

Even though the disambiguation of deverbal noun compounds with the help of verb relations has been

\begin{table}
\begin{tabular}{|c|c|} \hline \multicolumn{2}{|c|}{**Global Features**} \\ \hline features used on all three tasks (i.e. Datasets I and II) & PARA (§5.1-5.3), SEM (§5.4), HEAD (§5.5) \\ \hline \multicolumn{2}{|c|}{**Local Features**} \\ \hline feature used only on task _Role_ (i.e. Dataset I) & SUBJ (§5.6) \\ \hline feature used only on task _Holder_ & VIEW (§5.7) \\ and task _Target_ (i.e. Dataset II) & \\ \hline \end{tabular}
\end{table}
Table 3: Division of global and local features.

 

examined before (Lapata, 2002), it has not been exploited for an actual application, such as opinion role extraction. Neither has it been compared against plain paraphrases, which use the head noun of the compound directly (SS5.1).

Our use of verb semantics for compound analysis is also different from its predominant use in previous work (Kim and Baldwin, 2006; Nakov and Hearst, 2013) where noun compounds are considered whose parts represent arguments of an abstract verbal relation (e.g. _malaria mosquito_ are arguments of relation _'mosquito causes malaria'_). Thus, the aim has been to predict verbs for those compounds that match those abstract relations (e.g. _to cause_). We are looking for different verbs, namely those that are the morphological basis for the head noun.

For this verb detour, we produce a mapping from nouns (i.e. the heads of our opinion compounds) to verbs by combining distributional and string similarity. We extracted the verbs most similar to each of these nouns (we use top 100). For that we induce vector representations of all head nouns of our gold standard and all existing German verbs using the embedding toolkit _Word2Vec_(Mikolov et al., 2013).5 For each noun, we select the verb with the highest cosine-similarity that has at least a _Levenshtein (string) similarity_(Levenshtein, 1966) of \(3\). This high threshold ensures that nouns which are not deverbal nouns are not mapped to any verb. Against a manual mapping, our automatic method produced an F-score of \(76.1\) (at a precision of \(77.1\)).

Footnote 5: We used the cbow-model with \(200\) dimensions. All remaining parameters are set to their respective default values.

### Joint Paraphrases (PARA\({}_{joint}\))

Another way of reducing the ambiguity of paraphrases is to employ paraphrases that jointly consider opinion holder and target (Table 4). We assume that the presence of one ambiguous dependency relation is less problematic in the presence of another less ambiguous relation. The ambiguity can be resolved by method of elimination. For instance, even though _obj\({}_{von/of}\)(Widerstand/resistance, Bauernfarmers)_ is ambiguous, in the first example of Table 4, it can only represent a holder, since the second relation _obj\({}_{gegen/against}\)(Widerstand/resistance, Gesetz/regulation)_ implies a target.

We also use paraphrases in which the compound itself occurs (second and third pattern type of Table 4). Since, in the first example of the second pattern type, only the relation _obj\({}_{mit/with}\)(Zufriedenheit/satisfaction, Unternehmen/company)_ is indicative of a target, the modifier is likely to be a holder. (The example of the third pattern type follows an analogous pattern to extract a target.) The second example (of the second pattern type) _Sprengsfonfschlag (bomb attack)_ illustrates that paraphrases can also be used to infer the absence of opinion roles. _Sprenggoff (explosive)_ cannot be a target because of the other target relation that is present. It cannot be a holder either as it is not a person.

The fourth pattern type in Table 4 considers patterns involving possessive pronouns. They typically represent holders, so the remaining dependency relation can only represent a target.

Similar to SS5.1, we encode the joint-paraphrase patterns by their individual dependency relations. That is, the first example in Table 4 would be represented as the feature _obj\({}^{\mathbf{modifier}}_{von}\_{objp}_{gegen}\)_.

### Semantic Knowledge (SEM)

We use GermaNet (Hamp and Feldweg, 1997), the German version of WordNet, to look up the hypernyms of each modifier and each head. The hypernymy relation is the most frequently used semantic relation employed for noun compound analysis (Girju et al., 2005; Nastase et al., 2006; Girju et al., 2009; Tratz and Hovy, 2010). Hypernyms allow some generalization over the lexical units representing the heads and modifiers of our compounds. By manual inspection, we found that there are several hypernyms that correlate with a category we want to predict. For example, heads having the hypernym _politische Handling (political act)_ typically indicate holders as in _Arbeiterunruhe (worker unrest)_ or _Studenttenrebellion (student rebellion)_. Hypernyms may also serve as negative cues. For example, heads having the hypernym _Verbrechen (crime)_ are typically contained in compounds whose modifiers represent neither a holder nor a target, such as _Steuervergeben (tax offense)_ or _Autodiebstahl (car theft)_.

 



### Head Constraint (HEAD)

We observed that many heads have a strong selectional preference as to what type they select as a modifier. This is illustrated in Table 5. There are heads that prefer opinion holders as modifiers (e.g. _Haltung (attitude)_), heads that prefer targets (e.g. _Verehrung (worship)_) or heads that prefer no role (e.g. _Attentaat (attack)_). This is further substantiated by Table 6 showing the high average role-purity of compound groups sharing the same head. Purity is measured by the proportion of the most frequent role occurring within each group of compounds sharing the same head.6 Given this selectional preference, we formulate a _global head constraint_ (Table 7) that if two compounds have the same head, their modifiers should convey the same opinion role.

Footnote 6: On average, a head occurs in \(5\) different compounds on Dataset I, and in \(4\) different compounds on Dataset II.

In order to implement this constraint in a supervised classifier we employ Markov Logic Networks (MLNs), which combine first-order logic with probabilities. As a tool, we use _thebeast_[10]. MLNs have been effectively used in various related NLP tasks, such as discourse-based sentiment analysis [32], semantic-role labeling [89], anaphora resolution [11] or question answering [12].

MLNs are a set of pairs \((F_{i},w_{i})\) where \(F_{i}\) is a first-order logic formula and \(w_{i}\) an associated real-valued weight. They build a template for constructing a Markov network given a set of constants \(C\). The probability distribution that is estimated is a log-linear model

\[P(X=x)=\frac{1}{Z}exp\left(\sum_{i=1}^{k}w_{i}n_{i}(x)\right) \tag{1}\]

where \(n_{i}(x)\) is the number of groundings in \(F_{i}\) in \(x\) and \(Z\) is some normalization constant.

### Subjectivity Disambiguation (SUBJ)

Many opinion words are known to be ambiguous. Some of their senses convey subjectivity while others do not [1]. \(13\%\) of the compounds in Dataset I (Figure 1) are not subjective due to an ambiguous head. The modifier of such compounds neither represents a holder or a target. Examples are _Luftdruck (air pressure)_ or _Stromungs-widerstand (flow resistance)_. Dataset II exclusively contains compounds whose modifiers are holders or targets. By definition, all those compounds are subjective. So a subjectivity feature may only be useful for the _role_-detection task, which uses Dataset I.

\begin{table}
\begin{tabular}{|l|p{142.3pt}|c c c|} \hline
**Head** & **Preference** & **Examples** \\ \hline Haltung (attitude) & holder & Arbeitgeberhaltung (employed attitude), Autorenhaltung (author attitude), Konsumentenhaltung (consumer attitude), Verbraucherhaltung (customcer attitude), Zuschauerhaltung (viewert attitude) \\ \hline Verehrung (worthy) & target & Ahnnerwerthung (ancestor worship), Helonerverhrung (heroworship), Ikonenverchrung (icon worship), Kaiserverehrung (emperor worship), Mitryterrevenhra (mamtry worship) \\ \hline Attentent (attack) & no role & Bombenetantant (bombing attack), Flugzequettnat (aircraft attack), Selbstmordattenat (suicide attack), Sprengstoffattentral (explosive attack), Saureattentat (acid attack) \\ \hline  & \multicolumn{1}{p{142.3pt}}{} & \multicolumn{1}{p{142.3pt}}{} & \multicolumn{1}{p{142.3pt}}{} \\ \end{tabular}
\end{table}
Table 6: Role-purity of compounds with the same head.

\begin{table}
\begin{tabular}{|l|l|l|l l|} \hline
**Pattern Type** & **Example Compound** & **Label** & **Example Sentence** \\ \hline \multirow{3}{*}{\(<\)head\(>\)\(<\)holder\(>\)\(<\)target\(>\)} & Bauernwiderstand (former **resistance**) & holder & **Wierstand** [von Bauren \(obj_{pen}\)] [egegan das Gesetz \(obj_{pageq}\)] \\  & Schillerweint (färmeter resistance**) & & (resistance of farmers \(obj_{rd}\)] [against the regulation \(obj_{pagesq}\)] \\ \cline{2-5}  & Schillerweint (studet assessment) & target & **Beuetting** [der Lehrer \(gnod\)] [von Schullem \(obj_{pm}\)] \\ \hline \multirow{3}{*}{\(<\)compound\(>\)\(<\)target\(>\)} & Mitarbelierzufriedenheit (staff **satisfaction**) & holder & Mitarbeltzerwinderdenheit (mit dem Unternnehnen \(obj_{mt}\)] \\  & Strønsstoffanschlag & no role & (staff **satisfaction** [with their company \(obj_{uni}\)] \\ \cline{2-5}  & Sprønsstoffanschlag & no role & Synergstofafranchslag [aut Tourisien \(obj_{psy}\)] \\  & (bonb attack) & & bomb attack [on tourists \(obj_{pm}\)] \\ \hline \multirow{3}{*}{\(<\)compound\(>\)\(<\)holder\(>\)} & Prüfungasangst (test **master**) & target & Prüfungasangst [unter Sönlllem \(obj_{private}\)] \\  & (test anxiety) & & (test anxiety) [among students \(obj_{phasesq}\)] \\ \cline{2-5}  & Könderfreundlichkeit (child friendliness) & target & (seeience \(possesice\)) **Freundlichkeit [legenduber Kinderm \(obj_{appparameter}\)]** \\ \cline{2-5}  & (child friendliness) & & (this \(possessive\)) **friendliness** [towards children \(obj_{paramsq}\)] \\ \hline \end{tabular}
\end{table}
Table 4: Illustration of patterns for joint paraphrases.



For a feature indicating the subjectivity of a compound, we cannot look up the compounds in a sentiment lexicon since they are rarely included. Instead, we compute the \(100\) most similar German nouns for every compound and use as a feature the proportion of opinion nouns (according to the PolArt sentiment lexicon) on that list. Opinion nouns on that similarity list are less likely to be compounds and therefore more likely to be found in a sentiment lexicon. As in SS5.2, similarity is measured by the cosine between two _Word2Vec-_vector embeddings. As a result, we find, for example, for _Luftdruck (air pressure)_, other non-subjective terms, such as _Temperatur (temperature)_ or _Luffteuchtigkeit (humidity)_, while for the subjective compound _Hexenglaube (witch belief)_, we find the subjective expressions _Abergglaube (superstition)_ or _Haresie (heresy)_.

### Sentiment Views (VIEW)

Our final feature considers the sentiment view [34] that an opinion noun, in our case the head of the compound, conveys. We distinguish between _speaker views_, expressions conveying sentiment of the speaker of the utterance (e.g. _mistake_, _finesse_, _noise_), and _actor views_, expressions conveying sentiment of the entities participating in the event denoted by the opinion noun (e.g. _support_, _criticism_, _rating_). Nouns conveying speaker views have an implicit opinion holder (i.e. the speaker). Therefore, if such a noun is the head of an opinion compound, the modifier cannot be a holder but only a target, e.g. _Arzfehler (doctor's mistake)_, _Kinderlarm (children's noise)_ or _Neonazipropaganda (neonazi propaganda)_. Only heads conveying an actor view can take modifiers to represent a holder _(Nutzerwertung/user rating)_ or a target _(Opferunterstutzung/victim support)_. Sentiment views may be helpful on Dataset II (Figure 1), where we have to decide between holders and targets. \(40.3\%\) of those heads convey a speaker view.

So far, the detection of sentiment views on a _lexical_ level has only been examined for opinion verbs. Wiegand and Ruppenhofer wiegand:2015:EMNLP propose a bootstrapping approach in which seed verbs for the different sentiment views are automatically extracted.7 Then, a label propagation algorithm [69] is run on a word-similarity graph generated from the opinion verbs. Thus labels from the seeds can be expanded to the remaining opinion verbs. The nodes in the graph correspond to the opinion verbs. The best performing graph is based on the similarity metric introduced in Lin lin:1998:EMNLP.

Footnote 7: Wiegand and Ruppenhofer wippenhofer:2015 consider two types of actor views, _agent view_ and _patient view_. The former take their opinion holder as an agent (typical verbs are _criticize_ or _support_), while the latter align holders to patients (typical verbs are _disappoint or please_). Since this distinction of actor views does not exist among nouns, we combine them into a single category in this paper.

A critical step is the seed generation. Wiegand and Ruppenhofer wiegand:2015 extract seeds representing actor views by looking for opinion words frequently co-occurring with _prototypical opinion holders (protoOHs)_. These are common nouns, such as _opponents_ or _critics_, that typically act as opinion holders [34]. By definition, such explicit opinion holders indicate an actor view. Seeds for speaker-view verbs are obtained by extracting verbs co-occurring with _reproach-patterns_, such as _obju(beschuldigt/blamed for, <verb>)_ (14) that matches in (15).

* _Pattern: obji(beschuldigt/blamed for, <speaker-view verb>)_
* _Die UNO wurde beschuldigt__, [die Klimadaten fehlgedeuterich zu haben objij]. (The UN was blamed for misinterpretingrichclimate data.)_
* _Pattern: obji(beschuldigt/blamed for, <speaker-view noun>)_
* _Die UNO wurde [der Fahldetuningnou objj] von Klimradaten beschuldigt. (The UN was blamed for the misinterpretationneum of climate data.)_

This bootstrapping approach can be immediately applied to our setting. In the word-similarity graph, the opinion verbs are replaced by opinion nouns. With protoOHs, not only actor-view verbs but also actor-view nouns can be extracted. Similarly, the reproach-patterns work for both verbs (15) and nouns (17). (Only the dependency relation changes from _obji_ (14) to _obje_ (16).) ProtoOHs and reproach patterns are simply translated from English to German.

 

Experiments

We consider one binary MLN classifier for each of our three tasks (SS4). Most of our features are frequently occurring features (e.g. paraphrases (SS5.1), subjectivity feature (SS5.6), sentiment views (SS5.7)). Supervised classifiers only require few training data in order to assign appropriate weights to such features. Therefore, we sample \(20\%\) of the instances for each task of the respective dataset as training data. We test on the remaining \(80\%\) of the dataset. This procedure is repeated \(5\) times. The \(5\) training samples within each task are disjoint. We report macro-average F-score averaged over the \(5\) test samples.

We will first evaluate global features and then proceed to the local features. A division of our feature set into these groups was presented in Table 3.

### Evaluation of Global Features

Table 8 compares the features that can be applied on all three tasks. On average, PARA (SS5.1-SS5.3) is slightly better than SEM (SS5.4). Since their combination always results in a significant improvement, we conclude that these features contain complementary information. In the majority of cases, HEAD (SS5.5) also yields significant improvement.

Table 9 compares the different subtypes of paraphrases (SS5.1-SS5.3). For all tasks, notable improvements are obtained by adding the other types of paraphrases to the plain paraphrases. While the joint paraphrases improve the plain paraphrases on all tasks, for the verb detour, improvements can be observed only for the extraction of holders and targets. However, this improvement is significantly better than that of the joint paraphrases. In summary, in order to obtain best possible results on all three types of classifications, we need all types of paraphrases.

### Evaluation of the Local Feature for Role Detection

Table 10 examines the impact of the subjectivity feature (SS5.6). We closely compare this feature with the head constraint since we found both features only working in combination with other features. In terms of statistical significance, the head constraint is more effective than the subjectivity feature.

### Evaluation of the Local Feature for the Detection of Holders and Targets

Table 11 examines the impact of the sentiment-view feature (SS5.7). We evaluate two variants of this feature. **VIEW\({}_{gold}\)** is a manual view annotation of all opinion head nouns. It should be considered an upper bound. The second variant, **VIEW\({}_{boot}\)**, employs the views as produced automatically by the bootstrapping approach outlined in SS5.7.8

Footnote 8: Note that unlike Wiegand and Ruppenhofer (2015) we manually removed incorrect seeds from the set of automatically generated seeds (this affects less than \(9\%\) of the seeds).

Table 11 shows that this feature has a notable impact on both PARA\({}_{plain}\) (i.e. the simplest feature set) and SEM+PARA+HEAD (i.e. the most complex feature set). This underlines that sentiment views are an important aspect for opinion role extraction.

\begin{table}
\begin{tabular}{|l|c|c|c|c|} \hline \multirow{2}{*}{**Features**} & \multicolumn{2}{c|}{**SEM**} & \multicolumn{2}{c|}{**PARA**} & \multicolumn{2}{c|}{**PARA+SEM**} \\ \cline{2-5}  & \multicolumn{2}{c|}{**+HEAD**} & \multicolumn{2}{c|}{**+HEAD**} & \multicolumn{2}{c|}{**+HEAD**} \\ \hline  & 54.75 & 56.33\({}^{\dagger}\) & 62.62 & 63.82\({}^{\ddagger}\) & 63.92 & 65.26\({}^{\ddagger}\) \\
**+SUBJ** & 56.37\({}^{\circ}\) & 58.57\({}^{\circ}\)† & 63.07 & 64.76\({}^{+}\) 



### Comparison against Baselines

Table 13 compares the best result from our previous experiments against **3 baselines**. The first is a **majority classifier** predicting the majority class.

The second baseline is a classifier inspired by **distant supervision**[20]. As in our paraphrase features, this classifier considers the context in which modifier and head of a compound occur as separate constituents. The difference is, however, that we consider every such co-occurrence (within the same sentence) as a context that conveys the same relation as the one that is (implicitly) conveyed by the compound. Even though such an assumption is naive, it has been shown to produce quite reasonable performance in relation extraction [20]. The advantage of such an approach is that a _generic_ relation extraction/opinion role extraction classifier can be trained on the resulting data. Unlike our proposed method, it does not require features tailored to the specific task (e.g. _manually_ written paraphrases). Since the resulting feature set (see also Table 12) is fairly high-dimensional, we employ a support vector machine. As an implementation, we use SVM\({}^{light}\)[49].

The third baseline is a **distributional approach** in which label propagation is performed on a word-similarity graph for compounds. The fundamental difference between that baseline and our proposed approach is that no relationship between head and modifier is modeled but just the contexts of the compounds themselves. We use the same (distributional) similarity metric to form the word-similarity graph and the same label propagation algorithm for this task as we did for bootstrapping sentiment views in SS5.7. The only difference is that the nodes in the graph are opinion compounds instead of opinion nouns. The training data for the second and third baseline are the same compounds as in our previous experiments.

Table 13 shows that our proposed method substantially outperforms the baselines.

## 7 Conclusion

We presented an approach to the new task of opinion role extraction on opinion compounds. We produced a gold standard and proposed a method for classification. We did not only consider established features for noun compound analysis, i.e. paraphrases and semantic classes of heads and modifiers, but also proposed useful new features tailored to our task. We examined paraphrases that jointly consider holders and targets, a verb detour in which noun heads are replaced by related verbs, a global head constraint, and an auxiliary classification categorizing the sentiment view of the head of the compound. None of these features is language-specific.

## Acknowledgements

We would like to thank Ines Rehbein for interesting discussions and helpful feedback on earlier drafts of the paper. The authors were partially supported by the German Research Foundation (DFG) under grants RU 1873/2-1 and WI 4204/2-1.

\begin{table}
\begin{tabular}{|c c|c|c|c|c|} \hline  & & \multicolumn{3}{c|}{**Tasks**} \\ \cline{2-6} \multicolumn{1}{|c|}{**Features**} & **Role** & **Holder** & **Target** \\ \hline BASELINES & Majority & 34.70 & 35.49 & 36.71 \\  & Distant Superv. & 54.85 & 47.71 & 45.72 \\  & Distributional & 58.15 & 52.91 & 52.72 \\ \hline our approach _(best feature sets)_ & **66.42\({}^{*}\)** & **64.71\({}^{*}\)** & **66.50\({}^{*}\)** \\ \hline \end{tabular} \({}^{*}\): better than all baselines according statistical significance testing (paired t-test, significance level at \(p<0.05\))

\end{table}
Table 13: Comparison of our approach against baselines; _evaluation measure: F-score_.

\begin{table}
\begin{tabular}{|l|} \hline all words in the sentences (bag of words) \\ brown clusters of all words in the sentences (bag of clusters) \\ part-of-speech sequences between head and modifier mentions \\ part-of-speech tags before/after modifier mentions \\ part-of-speech tags before/after head mentions \\ dependency paths between head and modifier mentions \\ proportion of opinion words in the sentences \\ each training/test instance represents the set of all sentences in which head and modifier of a specific compound co-occur \\ \hline \end{tabular}
\end{table}
Table 11: F-scores of sentiment view features.





## References

* [Akkaya et al.2009] Cem Akkaya, Janyce Wiebe, and Rada Mihalcea. 2009. Subjectivity Word Sense Disambiguation. In _Proceedings of EMNLP_, pages 190-199.
* Barker and Szpakowicz (1998) Ken Barker and Stan Szpakowicz. 1998. Semi-Automatic Recognition of Noun ModifierRelationships. In _Proceedings of COLING/ACL_, pages 96-102.
* Baroni et al. (2009) Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetti. 2009. The WaCky Wide Web: A Collection of Very Large Linguistically Processed Web-Crawled Corpora. _Language Resources and Evaluation_, 43(3):209-226.
* Butnariu and Veale (2008) Cristina Butnariu and Tony Veale. 2008. A Concept-Centered Approach to Noun-Compound Interpretation. In _Proceedings of COLING_, pages 81-88.
* Choi et al. (2005) Yejin Choi, Claire Cardie, Ellen Riloff, and Siddharth Patwardhan. 2005. Identifying Sources of Opinions with Conditional Random Fields and Extraction Patterns. In _Proceedings of HLT/EMNLP_, pages 355-362.
* Deng and Wiebe (2014) Lingjia Deng and Janyce Wiebe. 2014. Sentiment Propagation via Implicature Constraints. In _Proceedings of EACL_, pages 377-385.
* Deng and Wiebe (2015) Lingjia Deng and Janyce Wiebe. 2015. Joint Prediction for Entity/Event-Level Sentiment Analysis using Probabilistic Soft Logic Models. In _Proceedings of EMNLP_, pages 179-189.
* Dima et al. (2014) Corina Dima, Verena Henrich, Erhard Hinrichs, and Christina Hoppermann. 2014. How to Tell a Schneemann from a Milchmann: An Annotation Scheme for Compound-Internal Relations. In _Proceedings of LREC_, pages 1194-1201.
* Girju et al. (2005) Roxana Girju, Dan Moldovan, Marta Tatu, and Daniel Antohe. 2005. On the semantics of noun compounds. _Computer Speech and Language_, 19:479-496.
* Girju et al. (2009) Roxana Girju, Preslav Nakov, Vivi Nastase, Stan Szpakowicz, Peter Turney, and Deniz Yuret. 2009. Classification of semantic relations between nominals. _Language Resources and Evaluation_, 43(2):105-121.
* Girju (2007) Roxana Girju. 2007. Improving the Interpretation of Noun Phrases with Cross-linguistic Information. In _Proceedings of ACL_, pages 568-575.
* a Lexical-Semantic Net for German. In _Proceedings of ACL workshop Automatic Information Extraction and Building of Lexical Semantic Resources for NLP Applications_, pages 9-15.
* Hou et al. (2013) Yufang Hou, Katja Markert, and Michael Strube. 2013. Global Inference for Bridging Anaphora Resolution. In _Proceedings of HLT/NAACL_, pages 907-917.
* Jakob and Gurevych (2010) Niklas Jakob and Iryna Gurevych. 2010. Extracting Opinion Targets in a Single- and Cross-Domain Setting with Conditional Random Fields. In _Proceedings of EMNLP_, pages 1035-1045.
* Support Vector Learning_, pages 169-184. MIT Press.
* Johansson and Moschitti (2013) Richard Johansson and Alessandro Moschitti. 2013. Relational Features in Fine-Grained Opinion Analysis. _Computational Linguistics_, 39(3):473-509.
* Kessler and Nicolov (2009) Jason S. Kessler and Nicolas Nicolov. 2009. Targeting Sentiment Expressions through Supervised Ranking of Linguistic Configurations. In _Proceedings of ICWSM_.
* Khot et al. (2015) Tushar Khot, Niranjan Balasubramanian, Eric Gribkoff, Ashish Sabharwal, Peter Clark, and Oren Etzioni. 2015. Exploring Markov Logic Networks for Question Answering. In _Proceedings of EMNLP_, pages 685-694.
* Kim and Baldwin (2005) Su Nam Kim and Timothy Baldwin. 2005. Automatic Interpretation of Noun Compounds Using Wordnet Similarity. In _Proceedings of IJCNLP_, pages 945-956. Springer.
* Kim and Baldwin (2006) Su Nam Kim and Timothy Baldwin. 2006. Interpreting Semantic Relations in Noun Compounds via Verb. In _Proceedings of COLING/ACL_, pages 491-498.
* Kim and Nakov (2011) Su Nam Kim and Preslav Nakov. 2011. Large-Scale Noun Compound Interpretation Using Bootstrapping and the Web as a Corpus. In _Proceedings of EMNLP_, pages 648-658.
* Klanner et al. (2009) Manfred Klanner, Angela Fahrni, and Stefanos Petrakis. 2009. PolArt: A Robust Tool for Sentiment Analysis. In _Proceedings of NoDaLiDa_, pages 235-238.
* Landis and Koch (1977) J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. _Biometrics_, 33(1):159-174.
* Lapata (2002) Maria Lapata. 2002. The Disambiguation of Nominalizations. _Computational Linguistics_, 28(3):357-388.
* Lauer (1995) Mark Lauer. 1995. _Designing Statistical Language Learners: Experiments on Noun Compounds_. Ph.D. thesis, Department of Computing, Macquarie University, Australia.
* Levenshtein (1966) Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. _Soviet Physics Doklady_, 10(8):707-710.
* Lin (1998) Dekang Lin. 1998. Automatic Retrieval and Clustering of Similar Words. In _Proceedings of ACL/COLING_, pages 768-774.
* Meza-Ruiz and Riedel (2009) Ivan Meza-Ruiz and Sebastian Riedel. 2009. Jointly Identifying Predicates, Arguments and Senses using Markov Logic. In _Proceedings of HLT/NAACL_, pages 155-163

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In _Proceedings of Workshop at the International Conference on Learning Representations (ICLR)_.
* Miller et al. [1990] George Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller. 1990. Introduction to WordNet: An On-line Lexical Database. _International Journal of Lexicography_, 3:235-244.
* Mintz et al. [2009] Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant Supervision for Relation Extraction without Labeled Data. In _Proceedings of ACL/IJCNLP_, pages 1003-1011.
* Nakov and Hearst [2013] Preslav I. Nakov and Marti A. Hearst. 2013. Semantic Interpretation of Nouns Compounds Using Verbal and Other Paraphrases. _ACM Transactions on Speech and Language Processing_, 10(3).
* Nastase and Szpakowicz [2003] Vivi Nastase and Stan Szpakowicz. 2003. Exploring Noun-Modifier Semantic Relations. In _Proceedings of IWCS_, pages 285-301.
* Nastase et al. [2006] Vivi Nastase, Jelber Sayyad-Shirabad, Marina Sokolova, and Stan Szpakowicz. 2006. Learning Noun-Modifier Semantic Relations with Corpus-based and WordNet-based Features. In _Proceedings of AAAI_, pages 781-786.
* Riedel [2008] Sebastian Riedel. 2008. Improving the Accuracy and Efficiency of MAP Inference for Markov Logic. In _Proceedings of UAI_, pages 468-475.
* Rosario and Hearst [2001] Barbara Rosario and Marti Hearst. 2001. Classifying the Semantic Relations in Noun Compounds via a Domain-Specific Lexical Hierarchy. In _Proceedings of EMNLP_.
* Ruppenhofer et al. [2014] Josef Ruppenhofer, Julia Maria Struss, Jonathan Sonntag, and Stefan Gindl. 2014. IGSA-STEPS: Shared Task on Source and Target Extraction from Political Speeches. _Journal for Language Technology and Computational Linguistics_, 29(1):33-46.
* Sennrich et al. [2009] Rico Sennrich, Gerold Schneider, Martin Volk, and Martin Warin. 2009. A New Hybrid Dependency Parser for German. In _Proceedings of GSCL_, pages 115-124.
* Talukdar et al. [2008] Partha Pratim Talukdar, Joseph Reisinger, Marius Pasca, Deepak Ravichandran, Rahul Bhagat, and Fernando Pereira. 2008. Weakly-Supervised Acquisition of Labeled Class Instances using Graph Random Walks. In _Proceedings of EMNLP_, pages 582-590.
* Tratz and Hovy [2010] Stephen Tratz and Eduard Hovy. 2010. A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation. In _Proceedings of ACL_, pages 678-687.
* Wiebe et al. [2005] Jansce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating Expressions of Opinions and Emotions in Language. _Language Resources and Evaluation_, 39(2/3):164-210.
* Wiegand and Klakow [2010] Michael Wiegand and Dietrich Klakow. 2010. Convolution Kernels for Opinion Holder Extraction. In _Proceedings of HLT/NAACL_, pages 795-803.
* Wiegand and Klakov [2011] Michael Wiegand and Dietrich Klakov. 2011. Prototypical Opinion Holders: What We can Learn from Experts and Analysts. In _Proceedings of RANLP_, pages 282-288.
* Wiegand and Klakow [2012] Michael Wiegand and Dietrich Klakow. 2012. Generalization Methods for In-Domain and Cross-Domain Opinion Holder Extraction. In _Proceedings of EACL_, pages 325-335.
* Wiegand and Ruppenhofer [2015] Michael Wiegand and Josef Ruppenhofer. 2015. Opinion Holder and Target Extraction based on the Induction of Verbal Categories. In _Proceedings of CoNLL_, pages 215-225.
* Wilson et al. [2005] Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing Contextual Polarity in Phrase-level Sentiment Analysis. In _Proceedings of HLT/EMNLP_, pages 347-354.
* An Open Source Morphological Analyzer for German. In _Proceedings of the 2009 Conference on Finite-State Methods and Natural Language Processing: Post-proceedings of the 7th International Workshop FSMNLP 2008_, pages 224-231. IOS Press Amsterdam, The Netherlands.
* Zirn et al. [2011] Cacilia Zirn, Mathias Niepert, Heiner Stuckenschmidt, and Michael Strube. 2011. Fine-Grained Sentiment Analysis with Structural Features. In _Proceedings of IJCNLP_, pages 336-344.