Argument Mining for Review Helpfulness Prediction (2022)

Abstract: The importance of reliably determining the helpfulness of product reviews is rising as both helpful and unhelpful reviews continue to accumulate on e-commerce websites. And argumentational features—such as the structure of arguments and the types of underlying elementary units—have shown to be promising indicators of product review helpfulness. However, their adoption has been limited due to the lack of sufficient resources and large-scale experiments investigating their utility. To this end, we present the AMazon Argument Mining (AM ^2 ) corpus—a corpus of 878 Amazon reviews on headphones annotated according to a theoretical argumentation model designed to evaluate argument quality.Experiments show that employing argumentational features leads to statistically significant improvements over the state-of-the-art review helpfulness predictors under both text-only and text-and-image settings.

## introduction with the rapid growth of e-commerce, reading product reviews is increasingly becoming a part of online shopping. going beyond the seller's description of the products, potential customers are considering the firsthand experiences and opinions of those who have already purchased the products. fortunately, product reviews are quickly accumulating on popular e-commerce websites like amazon.com on a daily basis; however, not all reviews are helpful, necessitating automatic prediction of their helpfulness @xcite . a wide variety of features for helpfulness prediction has been proposed in the past, including those from the review (diaz and ng, 2018), the reviewer @xcite , and the product @xcite . among these, argumentational features (afs) from the review text have shown potential in small-scale experiments @xcite . to see how afs can be useful for predicting the helpfulness of product reviews, consider two reviews on [skull candy's are the best earbuds in their price range that you could buy... until they break.] a [they include a one year warranty,] b [but often times, for such a lowpriced product, its easier to just purchase a new one.] c [so definitely, purchase these earbuds,] d [but don't expect more than a few months out of them until one side burns out.] e figure 2 : a review with 0 helpfulness votes. unlike the helpful review example (figure 1 ), this review has a poor argumentative structure, e.g., it lacks support for the proposition that the headphones are the best in their price range. also, it consists mostly of subjective propositions like value and policy. however, the adoption of afs has been limited in part by the unavailability of sufficient resources. an-notating argument information is challenging due in part to the prevalence of enthymemes, i.e., arguments with premises that have not been stated explicitly, and multiple plausible reconstructions of arguments @xcite . still, a review helpfulness corpus with argument information is desirable. to this end, we present the amazon argument mining (am 2 ) corpus, a corpus consisting of 878 amazon product reviews on headphones annotated with rich argument information. reviews were annotated according to a complex theoretical argumentation model with two types of support relations prevalent in practical argumentation-reason and evidence-as well as five types of elementary units-policy, value, fact, testimony, and reference-capturing the appropriate types of support. for instance, value propositions are best supported with reason, and fact propositions, evidence @xcite . the annotated reviews have been carefully sampled to control for other factors that affect perceived helpfulness, such as the type of product and how long the given review had been published. we also demonstrate the efficacy of the corpus by incorporating afs into the state-of-the-art review helpfulness prediction models. the results demonstrate that the performance improvements are statistically significant for both unimodal (text-only @xcite ) and multi-modal (text-and-image @xcite ) baselines. the main contributions of this work are twofold: first, we present am 2 -the first argument mining corpus of product reviews, supporting review helpfulness prediction with rich argument information. 2 it is based on a more complex theoretical argumentation model, yet is more than seven times the size of the only existing review helpfulness dataset with argument information @xcite . second, we build new stateof-the-art review helpfulness predictors under text-only and text-and-image settings by incorporating afs to existing ones, showcasing the utility of afs. ## annotation scheme the annotation scheme is based on a theoretical argumentation model comprised of elementary units and support relations to capture a specific type of argument quality-evaluability @xcite . in short, an argument is evaluable if every proposition in the argument is accompanied with at least one type of appropriate support, which in turn allows readers to understand the gist of the argument and evaluate its validity and strength. the degree to which an argument in practice meets this requirement determines how evaluable it is. ## conclusion afs are highly relevant to determining the helpfulness of product reviews, as they capture several dimensions of quality, such as informativeness (e.g. claims accompanied with premises are more informative than unsubstantiated claims) and clarity (e.g. reviews written in a clear argumentative structure are easier to understand than obscure writing). yet, there had not been sufficient resources to facilitate their adoption. to this end, we presented am 2 , a corpus of amazon product reviews with argument information annotated, along with experimental results showing its efficacy. in the future, we hope to develop review helpfulness predictors that better leverage afs; methods for incorporating afs have not been studied extensively in this work, and effective approaches may lead to large gains in the overall performance.