A Knowledge-Driven Approach to Classifying Object and Attribute Coreferences in Opinion Mining (2020)

Abstract: Classifying and resolving coreferences of objects (e.g., product names) and attributes (e.g., product aspects) in opinionated reviews is crucial for improving the opinion mining performance. However, the task is challenging as one often needs to consider domain-specific knowledge (e.g., iPad is a tablet and has aspect resolution) to identify coreferences in opinionated reviews. Also, compiling a handcrafted and curated domain-specific knowledge base for each domain is very time consuming and arduous. This paper proposes an approach to automatically mine and leverage domain-specific knowledge for classifying objects and attribute coreferences. The approach extracts domain-specific knowledge from unlabeled review data and trains a knowledgeaware neural coreference classification model to leverage (useful) domain knowledge together with general commonsense knowledge for the task. Experimental evaluation on realworld datasets involving five domains (product types) shows the effectiveness of the approach

## introduction coreference resolution (cr) aims to determine whether two mentions (linguistic referring expressions) corefer or not, i.e., they refer to the same entity in the discourse model @xcite @xcite @xcite @xcite . the set of coreferring expressions forms a coreference chain or a cluster. let's have an example: here all colored and/or underlined phrases are mentions. considering s1 (sentence-1) and s2 (sentence-2), the three mentions "i", "myself " in s1 and "i" in s2 all refer to the same person and form a cluster. similarly, "its" in s2 refers to the object "a green moonbeam" in s1 and the cluster is {"its" (s2), "a green moonbeam" (s1) }. the mentions "its voice" and "it" in s2 refer to the same attribute of the object "a green moonbeam" in s1 and form cluster {"its voice" (s2), "it" (s2)}. cr is beneficial for improving many downstream nlp tasks such as question answering @xcite , dialog systems @xcite , entity linking @xcite , and opinion mining @xcite . particularly, in opinion mining tasks @xcite @xcite , @xcite reported performance improves by 10% when cr is used. the study by @xcite also supports this finding. considering the aforementioned example, without resolving "it" in s2, it is difficult to infer the opinion about the attribute "voice" (i.e., the voice, which "it" refers to, is "loud and long"). although cr plays such a crucial role in opinion mining, only limited research has been done for cr on opinionated reviews. cr in opinionated reviews (e.g., amazon product reviews) mainly concerns about resolving coreferences involving objects and their attributes. the objects in reviews are usually the names of products or services while attributes are aspects of those objects @xcite . resolving coreferences in text broadly involves performing three tasks (although they are often performed jointly or via end-to-end learning): (1) identifying the list of mentions in the text (known as mention detection); (2) given a pair of candidate mentions in text, making a binary classification decision: coreferring or not (referred to as coreference classification), and (3) grouping coreferring mentions (referring to the same discourse entity) to form a coreference chain (known as clustering). in reviews, mention detection is equiv-alent to extracting entities and aspects in reviews which has been widely studied in opinion mining or sentiment analysis @xcite @xcite @xcite @xcite . also, once the coreferring mentions are detected via classification, clustering them could be straightforward foot_0 . thus, following @xcite , we only focus on solving the coreference classification task in this work, which we refer to as the object and attribute coreference classification (oac2) task onwards. we formulate the oac2 problem as follows. problem statement. given a review text u (context), an anaphor foot_1 p and a mention m which refers to either an object or an attribute (including their position information), our goal is to predict whether the anaphor p refers to mention m, denoted by a binary class @xmath0, 1}. note. an anaphor here can be a pronoun (e.g., "it") or definite noun phrase (e.g., "the clock") or ordinal (e.g., "the green one"). in general, to classify coreferences, one needs intensive knowledge support. for example, to determine that "it" refers to "its voice" in s2, we need to know that "voice" can be described as "loud and long" and "it" can not refer to "a green moonbeam" in s1, since "moonbeam" is a clock which cannot be described as "long". product reviews contain a great many such domain-specific concepts like brands (e.g., "apple" in the laptop domain), product name (e.g., "t490" in the computer domain), and aspects (e.g."hand" in the alarm clock domain) that often do not exist in general knowledge bases (kbs) like word-net @xcite , conceptnet @xcite , etc. moreover, even if a concept exists in a general kb, its semantics may be different than that in a given product domain. for example, "moonbeam" in a general kb is understood as "the light of the moon" or the name of a song, rather than a clock (in the alarm clock domain). to encode such domain-specific concepts, we need to mine and feed domain knowledge (e.g., "clock" for "moonbeam", "laptop" for "t490") to a coreference classification model. existing cr methods @xcite do not leverage such domain knowledge and thus, often fail to resolve such co-references that require explicit reasoning over domain facts. in this paper, we propose to automatically mine such domain-specific knowledge from unlabeled reviews and leverage the useful pieces of the extracted domain knowledge together with the (general/comensense) knowledge from general kbs to solve the oac2 task foot_2 . note the extracted domain knowledge and the general knowledge from the existing general kbs are both considered as candidate knowledge. to leverage such knowledge, we design a novel knowledge-aware neural coreference classification model that selects the useful (candidate) knowledge with attention mechanism. we discuss our approach in details in section 3. the main contributions of this work can be summarized: ## related work coreference resolution has been a long-studied problem in nlp. early approaches were mainly rule-based @xcite and feature-based @xcite where researchers focused on leveraging lexical, grammatical properties and semantic information. recently, end-to-end solutions with deep neural models @xcite @xcite have dominated the coreference resolution research. but they did not use external knowledge. conisdering cr approaches that use external knowledge, @xcite solved cr task by incorporate knowledge or information in reinforcement learning models. @xcite solved the binary choice coreference-resolution task by leveraging information retrieval results from search engines. @xcite solved pronoun coreference resolutions by leveraging contextual, linguistic features, and external knowledge where knowledge attention was utilized. however, these works did not deal with opinionated reviews and also did not mine or use domain-driven knowledge. in regard to cr in opinion mining, @xcite formally introduced the oac2 task for opinionated reviews, which is perhaps the only prior study on this problem. however, it only focused on classifying coreferences in comparative sentences (not on all review sentences). we compare our approach with @xcite in section 4. many existing general-purpose cr datasets are not suitable for our task, which include muc-6 and muc-7 @xcite , @xcite , ontonotes @xcite , and wikicoref @xcite . @xcite proposed an alternative turing test, comprising a binary choice cr task that requires significant commonsense knowledge. @xcite proposed visual pronoun coreference resolution in dialogues that require the model to incorporate image information. these datasets are also not suitable for us as they are not opinionated reviews. we do not focus on solving pronoun resolution here because, for opinion text such as reviews, discussions and blogs, personal pronouns mostly refer to one person @xcite . also, we aim to leverage domainspecific knowledge on (unlabeled) domain-specific reviews to help the cr task which has not been studied by any of these existing cr works. ## proposed approach model overview. our approach consists of the following three main steps: (1) knowledge aquisition, where given the (input) pair of mention m (e.g., "a green moonbeam") and anaphor p (e.g., "it") and the context t (i.e., the review text), we acquire candidate knowledge involving m, denoted as k m . k m consists of both domain knowledge (mined from unlabeled reviews) as well as general knowledge (compiled from existing general kbs) (discussed in section 3.1). next, in (2) syntaxbased span representation, we extract syntaxrelated phrases for mention m and anaphor p. syntax-related phrases are basically noun phrases, verbs or adjectives that have a dependency relation foot_4 with m (or p). for example, "bought" is a syntax-related phrase of the mention "a green moonbeam" and "like" and "voice" are two syntaxrelated phrases for the anaphor "it" in the example review text in section 1. once the syntax-related phrases are extracted and the candidate knowledge is prepared for m and p, we learn vector representations of the phrases and the knowledge (discussed in section 3.2), which are used in step-3. finally, in (3) knowledge-driven oac2 model, we select and leverage useful candidate domain knowledge together with general knowledge to solve the oac2 task. figure 1 shows our model architecture. ## experiments we evaluate our proposed approach using five datasets associated with five different domains: (1) alarm clock, (2) camera, (3) cellphone, (4) computer, and (5) laptop and perform both quantitative and qualitative analysis in terms of predictive performance and domain-specific knowledge usage ability of the proposed model. ## conclusion this paper proposed a knowledge-driven approach for object and attribute coreference classification in opinion mining. the approach can automatically extract domain-specific knowledge from unlabeled data and leverage it together with the general knowledge for solving the problem. we also created a set of annotated opinionated review data (including 5 domains) for object and attribute coreference evaluation. experimental results show that our approach achieves state-of-the-art performance.