S imster Q : A Similarity based Clustering Approach to Opinion Question Answering (2020)

Abstract: In recent years, there has been an increase in online shopping resulting in an increased number of online reviews. Customers cannot delve into the huge amount of data when they are looking for specific aspects of a product. Some of these aspects can be extracted from the product reviews. In this paper we introduced SimsterQ - a clustering based system for answering questions that makes use of word vectors. Clustering was performed using cosine similarity scores between sentence vectors of reviews and questions. Two variants (Sim and Median) with and without stopwords were evaluated against traditional methods that use term frequency. We also used an n-gram approach to study the effect of noise. We used the reviews in the Amazon Reviews dataset to pick the answers. Evaluation was performed both at the individual sentence level using the top sentence from Okapi BM25 as the gold standard and at the whole answer level using review snippets as the gold standard. At the sentence level our system performed slightly better than a more complicated deep learning method. Our system returned answers similar to the review snippets from the Amazon QA Dataset as measured by the cosine similarity. Analysis was also performed on the quality of the clusters generated by our system.

## introduction in the recent years, the volume of online shopping has increased rapidly. this has resulted in the increase in availability of online reviews and question-answers related to a product.traditional question answering (qa) systems are factual in nature. for example, "which year did world war i end?" 1918. in opinion qa, answers to questions are based on the customers' opinions. the customers' opinions help other users to decide whether to purchase a product. this process is time consuming for the users to look at thousands of reviews to find the required information. our paper aims at answering questions, users have, using customer reviews. we used the product reviews to extract the relevant sentences, with minimal to no overlap in meaning, and present it to the user. we make use of the amazonqa dataset to answer binary (yes/no) questions. the main focused contribution of this paper are: early work in opinion question answering addressed separating facts from opinions @xcite , and the authors used a naïve bayes classifier to identify polarity of the opinions. @xcite aimed at identifying the opinion holder of the opinions. @xcite explained the differences between fact based and opinionated answers and how traditional qa systems will not be able to handle multiple perspectives for answers. some works aimed at using community based question-answers to provide unique answers to questions @xcite . @xcite made use of online reviews to answer questions on aspects of a product. @xcite and @xcite used graphs and trees to answer opinion questions. @xcite modeled ambiguity and subjectivity in opinion qa using statistical models. @xcite give baselines for answer generation systems given the question and reviews. we use their results as the baseline for our evaluation. we also discuss the dataset from this paper in 4.2. while most systems used in the works described above are supervised learning models, our system used unsupervised learning to answer binary (yes/no) questions. ## system description the answer selection process to get the top k sentences has the following steps: ## evaluation evaluations were performed at both the sentence level and at the whole answer level. ## conclusions and future work this paper introduced simsterq -a unsupervised clustering based system to answer questions about products by accessing the reviews of the products. five different variants of this system were evaluated using 1000 yes/no questions. at the sentence level sim performed better with the highest rouge and similarity scores. sim method returns the top sentence from each of the 10 clusters created. when evaluating the entire answer, our system performed better than the baseline rouge score from the r-net method. in future simsterq will be used with open-ended questions. the challenge with open-ended questions will be the evaluation. perspectives expressed in the reviews need not necessarily match the perspectives in the gold standard answer. we want to evaluate the performance of simsterq on other datasets. in the amazon question/answer data set not every question has a good relevant answer. the answers are sometimes a single user's opinion. sim-sterq will be used to provide a new gold standard answer to the binary questions.