Distilling Salient Reviews with Zero Labels (2022)

Abstract: Many people read online reviews to learn about real-world entities of their interest. However, majority of reviews only describes general experiences and opinions of the customers, and may not reveal facts that are specific to the entity being reviewed. In this work, we focus on a novel task of mining from a review corpus sentences that are unique for each entity. We refer to this task as Salient Fact Extraction. Salient facts are extremely scarce due to their very nature. Consequently, collecting labeled examples for training supervised models is tedious and cost-prohibitive. To alleviate this scarcity problem, we develop an unsupervised method, ZL-Distiller, which leverages contextual language representations of the reviews and their distributional patterns to identify salient sentences about entities. Our experiments on multiple domains (hotels, products, and restaurants) show that ZL-Distiller achieves state-of-the-art performance and further boosts the performance of other supervised/unsupervised algorithms for the task. Furthermore, we show that salient sentences mined by ZL-Distiller provide unique and detailed information about entities, which benefit downstream NLP applications including question answering and summarization.

## introduction online reviews have become a rich source of information for people to know more about real-world entities for making purchasing decisions @xcite . reviews contain diverse information ranging from general sentiments and customer experiences to features and attributes about an entity. in this work, we focus on mining sentences that describe unique information about entities from its reviews. we call these unique sentences salient facts and denote this task as salient fact extraction. although scarce, salient facts exhibit at least one of the two characteristics: (a) they mention attributes rarely used to describe other entities (example 1 in although there is a rich body of research on extracting tips, informative and helpful sentences from reviews @xcite @xcite @xcite @xcite @xcite , these approaches have several limitations for extracting sentence type 1 there is a rooftop bar. salient fact 2 the hotel gives 90% discount for seniors. salient fact 3 the price is cheap. sentiment 4 we stayed 3 nights here. usage experience 5 choose other hotels instead. suggestion to address the scarcity problem, we propose a novel unsupervised extractor for identifying salient sentences in a zero-label setting where abundant unlabeled reviews are available. a naive approach is to refer to the distributional patterns of salient sentences in a review corpus. we projected all the sentences in a corpus to a t-sne plot @xcite and found that salient sentences tend to appear as border points on the graph. however, we observed that not all border points are salient facts. many sentences mentioning named entities names or unique personal stories also appear as border points. such non-informative sentences thus make distributional patterns noisy and the extraction challenging. based on these distributional patterns, we propose a novel system, zl (zero label) -distiller, which uses two transformer-based models for capturing unique and informative distributional patterns to extract salient facts. it uses a transformer-based entity prediction model to identify most unique sentences for an entity, and another transformer-based model to filter out non-informative sentences, such that informative sentences can be kept. the former one measures how distinctive a review sentence is to the corresponding entity but not to others. the latter one masks entity names in all sentences and drops those sentences that are likely personal stories. to our best knowledge, this is the first work to capture distributional patterns of all sentences for mining useful review sentences. contributions. in summary, our contributions are four folds. (1) we formulate a novel task that extracts entity-specific information (denoted as salient facts) from online reviews (2) to deal with scarcity of salient facts, we present an unsupervised method zl-distiller, which relies on distributional patterns instead of human annotations. (3) we show that zl-distiller leads to new state-of-the-art performance when used independently, or combined with supervised models on 3 domains (hotel, product and restaurant). ( 4 ) we demonstrate that zl-distiller benefits downstream applications including question answering, and entity summarization by removing non-informative sentences from the pipeline. ## related work helpful review definitions. research community has continuously devoted to understanding which reviews are the most helpful @xcite . the gold standard is to collect labels (e.g. helpful or not helpful votes) from various readers passively. recently, researchers begin to realize that helpful reviews are broad, so they proactively propose sub-concepts, including tip @xcite @xcite , suggestion @xcite @xcite , and sentiment @xcite , as complements. to further address this issue, we introduce salient facts as a novel sub-concept, that aims at extracting the most entity-specific information from raw reviews. we demonstrate the real value of salient facts through three natural language processing applications, including saliency estimation, question answering, and entity summarization. similar to existing sub-concepts, we anticipate the widespread adoption of salient facts in various domains, including but not limited to hotel @xcite , product @xcite , restaurant (challenge, 2020), and travel @xcite , in the near future. label-reliant solutions. most of existing extraction models @xcite @xcite @xcite @xcite @xcite are supervised. although their extraction qualities approximate human per-formance, the deployment of these models requires a great amount of human labels. collecting labels for these models can be time-consuming and costly since the process deals with worker education, salary negotiation, and mistake label filtering. therefore, we propose zl-distiller that adopts a label-free design choice while is also compatible to label-reliant solutions. label-free solutions. some label-free solutions attempted to remove the reliance on labels by leveraging data characteristics. for example, zero-shot learning @xcite predicts a sentence as true if its embedding is close to the class name (e.g. salient fact or helpful). unsupervised entity extraction @xcite @xcite predicts the sentence as true if its tokens contain named entities, such as person or location. though these methods have sufficiently leveraged lexical characteristics of a single sentence, they are incapable of leveraging common characteristics of a group of sentences (e.g. salient facts), with which helpful review mining can be substantially boosted. our label-free solution, i.e. zl-distiller, identifies two distributional patterns of salient facts, i.e. unique and informative, to extract the comments containing salient facts. by utilizing these characteristics, zl-distiller shows superior performance in the salient fact extraction task. ## application in this section, we demonstrate the effect of salient facts in downstream nlp applications. we apply salient fact extraction in company reviews, and select three downstream applications, including review saliency estimation, question answering, and company summarization. we used zl-distiller + bert (denoted as saliency prediction model) to obtain salient facts as inputs for downstream applications. ## conclusion in this paper, we propose to extract salient facts from online reviews. to achieve this goal, we develop zl-distiller, which is the first-of-its-kind system for salient fact extraction. zl-distiller does not require human labels, but labels can further boosts its performance. to prove that salient facts can be applied to popular real-world applications, we conduct a study on raw company reviews, which demonstrates that salient facts can improve the quality of downstream applications, including saliency estimation, question answering and company summarization. these results implicate the feasibility of salient fact extraction in real-world text corpus including company reviews, which consist of both salient and non-salient contents. our practice suggests that the general-purpose salient fact extraction has a substantial effect on existing text-based applications for diverse domains.