Sampo: Unsupervised Knowledge Base Construction for Opinions and Implications

services . There are many KB construction ( KBC ) methods that can extract factual information , where relationships between entities are explicitly stated in text . However , they can not model implications between opinions which are abundant in user-generated text such as reviews and often have to be mined . Our goal is to develop a technique to build KBs that can capture both opinions and their implications . Since it can be expensive to obtain training data to learn to extract implications for each new domain of reviews , we propose an unsupervised KBC system , SAMPO , that is based on matrix factorization techniques . Specifically , SAMPO is tailored to build KBs for domains where many reviews on the same domain are available . We generate KBs for 20 different domains using SAMPO and manually evaluate KBs for 6 domains . Our experiments show that KBs generated using SAMPO capture information otherwise missed by other KBC methods . Specifically , we show that our KBs can provide additional training data to fine-tune language models that are used for downstream tasks such as review comprehension . 1 . Introduction . Many applications and services today are powered by knowledge bases ( KBs ) . For instance , Google Knowledge Graph [ Dong et al. , 2014 ] supports web-search and question-answering technologies . Amazon Product Graph [ Dong , 2018 ] support product search . Consequently , many knowledge base construction ( KBC ) methods have been developed in recent years [ Weikum et al. , 2016 , Mitchell et al. , 2018 , Lockard et al. , 2018 ] that focus on extracting encyclopedic knowledge in the form of real-world entities and relations from text . To the best of our knowledge , none of the KBC methods can identify implication relationships between opinions , which are abundant in user-generated text such as reviews . Reviews can express opinions about an object ( e.g. , `` writing '' ) directly ( e.g. , `` pretty good writing '' ) , or indirectly ( e.g. , `` fast paced action '' ) . While opinions can still be identified using opinion mining , the implication relationships between opinions are missed . Consider the first review in Example 1 . It expresses an opinion about `` characters '' . Generalpurpose KBs [ Bollacker et al. , 2008 , Liu and Singh , 2004 ] can identify that `` complex '' is linguistically related to `` complicated '' and that it can describe `` brains '' . However , they fail to identify that `` complex characters '' ( from the first review ) implies `` good writing '' ( from the second review ) . This relation is ( weakly ) implied by the fact that both reviews describe the same movie instead of being explicitly stated in the text . Example . Review1 : The movie has everything : fast paced action , romance and complex characters . Opinions : ( `` fast paced '' , '' action '' ) , ( `` complex '' , '' characters '' ) Review2 : There ’ s some pretty good writing going on . Opinion : ( `` pretty good '' , '' writing '' ) Understanding opinions and their implications is important to improve machine understanding of reviews for a domain . In this work , our goal is to develop KBC techniques to mine such knowledge from domain-specific reviews . For instance , we want to extract information that `` complex characters '' implies `` good writing '' from movie reviews . Such opinion KBs are useful for many downstream tasks including sentiment analysis , retrieval and question answering . For example , understanding the query “ Show me a movie with good writing ” requires the knowledge that complex characters , fast paced action imply good writing . Instead of searching reviews with `` good writing '' as a keyword , a system could expand the search using our KB to additionally look for complex characters and fast paced action . They can also provide training data to fine-tune language models and obtain a deeper understanding of reviews . There are a number of challenges in building opinion KBs . Existing KBC methods are extractive and target knowledge that is explicitly mentioned in text . However , implications between opinions are typically not expressed explicitly in the text . Furthermore , most of the methods rely semi-supervised or supervised learning models . Obtaining human-annotated examples to train these models is expensive , especially for each domain of interest . We , therefore , aim to mine implicit relationships between opinions from a massive unlabeled review corpora . Being unsupervised , such a method can learn implicit relationships from scratch and generate necessary training data to use semi-supervised and continuous learning methods in later stages of knowledge base construction . In this paper , we propose an unsupervised KBC system , SAMPO , for building KBs on implications between opinions from reviews . SAMPO is based on matrix factorization techniques [ Koren et al. , 2009 ] , wherein each row corresponds to an item being reviewed and each column corresponds to an opinion extracted from the reviews . Since opinions and implications often do not co-occur within the same review , matrix factorization helps summarize signals across multiple reviews . Furthermore , it is unsupervised and can be applied to any domain with abundant reviews . SAMPO , thus , uses matrix factorization to learn low-dimensional representations for opinions and discover implications between opinions based on the cosine similarity of their corresponding representations . We successfully build KBs for 20 different domains using SAMPO and release them for further research1 . We manually evaluate KBs for 6 domains and show that KBs generated using SAMPO can accurately capture implications otherwise missed by other KBC methods . We also show how KBs built using SAMPO can augment language models and benefit reading comprehension . In summary , we make the following contributions : • We present SAMPO an unsupervised KBC system , for capturing implications between opinions . • We empirically show that SAMPO is domain-independent by generating KBs for 20 different domains . We manually assess the quality of 6 of the generated KBs . Our experiments show that SAMPO can consistently achieve higher precision ( avg . 18 % , max . 45 % ) and recall ( avg . 6.6 % , max . 16.6 % ) than other KBC methods . • We publicly release the created KBs to facilitate research on review comprehension systems . 1. https : //github.com/sampoauthors/Sampo 2 . Preliminaries . We briefly introduce some notation before presenting our system . LetR denote a corpus of reviews about items I in a domain ( e.g . movies ) . and Ri denote the reviews about item i ∈ I . We assume that an opinion extractor E can extract a list of opinions Oi about item i ( e.g . “ The Dark Knight '' ) from Ri . An opinion o ∈ Oi is of the format ( modifier , aspect ) , where aspect denotes the target of the opinion and modifier further describes the aspect . For example , E can extract ( `` pretty good '' , `` writing '' ) from the review “ There ’ s some pretty good writing going on. '' . Let O denote opinions about all the items in the domain . Given a set O of opinions , we are interested in identifying implication relationships between opinions in the set O . In other words , our goal is to create a knowledge base K of opinions and their implications . Since we are only interested in identifying implication relations , we can view opinions as nodes and implications as directed edges between the nodes . A directed edge from a head opinion oh to a tail opinion ot in K ( denoted as oh → ot ) indicates that ot is implied by oh . 3 . SAMPO . In this section , we describe our unsupervised KBC method , SAMPO . Figure 1 illustrates SAMPO ’ s KBC pipeline . Given a corpus of reviews and an opinion extractor , SAMPO starts by extracting all opinions from the corpus using the extractor , followed by normalization to drop slight nuances in how opinions are expressed . Once the opinions are extracted and normalized , SAMPO organizes this data into a matrix that records how often an opinion is expressed for each particular reviewed item . SAMPO then proceeds to compute a low dimensional representation for each opinion in the set O using matrix factorization techniques . Finally , the similarity of the learned embeddings is used to infer implications between opinions . 3.1 Opinion extraction & normalization . SAMPO relies on an opinion extractor E which can extract ( modifier , aspect ) pairs from a given review . Many opinion extractors have been developed recently [ Samha et al. , 2014 , Angelidis and Lapata , 2018 , Li and Lam , 2017 ] . Most of them have to be fine-tuned for each new domain , which can be expensive . In the absence of an extractor for new domains , a set of syntactic rules [ Abbasi Moghaddam , 2013 ] can instead be used to extract opinions . Our approach to mine implications , however , can be used with both pre-trained extractors or rule-based extractors . In our implementation , we use the state-of-the-art extractor developed by Li et al . [ 2019b ] for two domains ( Hotels , Restaurants ) . Additionally , we provide a rule-based , domain-independent opinion extractor , which we refer to as MINI-MINER . It uses Spacy2 to obtain the dependency parse trees of review sentences , and then uses a set of syntactic rules [ Abbasi Moghaddam , 2013 ] to extract opinions . The set of rules and our implementation of MINI-MINER are publicly available3 . Compared to state-of-the-art opinion extractors , MINI-MINER achieves a lower recall as it fails to extract opinions that are expressed in complex linguistic forms . However , it can still mine a large number of opinions with no supervision . Normalization : Once all opinions are extracted , we normalize them to group modifiers and aspects that are simply minor surface-form variations . This strengthens the signals for opinions expressed 2. https : //spacy.io/ 3. https : //github.com/sampoauthors/Sampo/blob/master/sampo/miniminer.py in the corpus and improves the embeddings learned for the opinions from these signals . We normalize the aspects by dropping any determiners and adverbial modifiers , followed by lemmatization . For example , `` scrolling credits '' is normalized to `` scroll credit '' . The modifiers are normalized in the same manner , except we drop any prepositional modifiers . For example , `` too long '' is normalized to `` long '' . The output of this step is a list of opinionsOi for each item i ∈ I . 3.2 Matrix factorization . SAMPO uses a matrix factorization technique to identify related opinions . There are two concrete reasons for using this approach : ( 1 ) Since opinions and their implications often do not co-occur within the same review , it requires techniques that summarize the signals across multiple reviews , and ( 2 ) it is unsupervised and can be applied to any domain without any overhead . Matrix factorization enables SAMPO to learn a low-dimensional representation for each opinion which as we will show can help identify implication relations . However , there are inherent challenges when applying these techniques on extremely sparse datasets where signals from multiple reviews about an item are weak . We , therefore , target items from I that are frequently reviewed and opinions from O that occur frequently . Given the subsets of I and O , we create a matrix that summarizes the frequency of each opinion across each item . Specifically , we create a matrix M where each row i corresponds to a unique item in the corpus and each column j corresponds to a unique expressed opinion . The entry Mij denotes the number of times that opinion j has been expressed by users when reviewing item i . Figure 1 shows an example of such a matrix for the movie domain . There are notable examples in relation extraction that use factorization techniques over binary matrices [ Riedel et al. , 2013 ] . We could adopt a similar approach and construct M such that Mij is 1 if opinion j is expressed at least once for item i and 0 otherwise . While this approach works well for a factual KB where a candidate fact is either true or false , it is not suited for mining implications between opinions from reviews . Reviews can have conflicting opinions about the same item although with significantly different frequencies . In order to capture these signals , SAMPO uses frequencies of opinions to construct the matrix M . Once the matrix M is created , SAMPO computes a d-dimensional embedding for each item i ( denoted as vi ) and for each opinion j ( denoted as vj ) . We provide guidelines for tuning hyperparameter d based on stability and error metrics , which we will introduce shortly . We empirically found that the results were not sensitive to d in the range 20-50 . In general , low-dimensional embeddings are considered effective if they can reconstruct the original dataset . In our case , we train embeddings such that Mij values can be derived from the embeddings vi and vj . There are several methods for approximating the Mij values from embeddings . Following the rich literature on matrix factorization [ Lee and Seung , 2001 ] , SAMPO computes embeddings such that their inner product which we denote as M̂ij = vivj would be a good approximation of Mij . Formally , SAMPO computes embedding to minimize the following error value which we refer to as the reconstruction error : Erec = ||M − M̂ ||F .

Summary: The authors propose a method to automatically build a Knowledge Base of opinions and implications between them. The KB is realized as a directed graph where nodes correspond to opinions in a canonical form (modifier, aspect), and edges indicate implications. It is built by factorizing a matrix of item-opinion frequencies, and finding the top k neighbors of an opinion.