Answering Product-Questions by Utilizing Questions from Other Contextually Similar Products (2021)

Abstract: Predicting the answer to a product-related question is an emerging field of research that recently attracted a lot of attention. Answering subjective and opinion-based questions is most challenging due to the dependency on customer generated content. Previous works mostly focused on review-aware answer prediction; however, these approaches fail for new or unpopular products, having no (or only a few) reviews at hand. In this work, we propose a novel and complementary approach for predicting the answer for such questions, based on the answers for similar questions asked on similar products. We measure the contextual similarity between products based on the answers they provide for the same question. A mixture-of-expert framework is used to predict the answer by aggregating the answers from contextually similar products. Empirical results demonstrate that our model outperforms strong baselines on some segments of questions, namely those that have roughly ten or more similar resolved questions in the corpus. We additionally publish two large-scale datasets used in this work, one is of similar product question pairs, and the second is of product question-answer pairs.

## introduction product-related question answering (pqa) is a popular and essential service provided by many e-commerce websites, letting consumers ask product related questions to be answered by other consumers based on their experience. the large archive of accumulated resolved questions can be further utilized by customers to support their purchase journey and automatic product question answering tools (e.g. @xcite ; @xcite ; @xcite ). however, there are many unanswered questions on these websites, either because a newly issued question has not attracted the community attention yet, or because of many other reasons @xcite . this may frustrate ecommerce users, in particular when their purchase decision depends on the question's answer. automatic pqa may assist the customers and the sellers by answering these unanswered questions, based on various diversified resources. previous pqa approaches leverage product specifications and description information @xcite @xcite , as well as customer-reviews @xcite @xcite @xcite @xcite , for answering product related questions. however, there are two notable shortcomings to these two approaches. product information can typically address questions about product features and functionality, but can't address complex and subjective questions such as opinion question (is it good for a 10 year old?), advice-seeking question (what is the color that best fit my pink dress?), or unique usage questions (can i play fifa 2018 on this laptop?). customer-reviews, on the other hand, can partially address this kind of questions @xcite ), yet there are many products with few or no reviews available, either because they are new on the site or are less popular. we propose a novel and complementary approach for answering product-related questions based on a large corpus of pqa. given an unanswered product question, we seek similar resolved questions 2 about similar products and leverage their existing answers to predict the answer for the customer's question. we call our method simba (similarity based answer prediction). for example, the answer for the question "will these jeans shrink after a wash?", asked about a new pair of jeans on the website, may be predicted based on the answers for similar questions asked about other jeans that share properties such as fabric material, brand, or style. an example is shown in as our method relies on the existing pqa corpus, it addresses the two mentioned shortcomings of the previous approaches. first, it can address a variety of product-related questions that are common in pqa, including subjective and usage questions. second, our method can provide answers to new or less popular products as it leverages an existing set of similar questions from other similar products. a key element of our proposed method is a novel concept that we refer to as contextual product similarity, which determines whether two products are similar in the context of a specific question. for example, two smart-watches may be similar with regards to their texting capability but different with regards to sleep monitoring. in section 3 we formally define this concept and propose a prediction model for measuring contextual similarity between products, with respect to a given question. additionally, we describe an efficient method to train this model by leveraging an existing pqa corpus. another appealing property of simba is its ability to support the predicted answer by providing the list of highly similar questions upon which the answer was predicted, hence increasing users' confidence and enhancing user engagement. our main contributions are: (a) a novel pqa method that overcomes several shortcomings of previous methods. (b) a novel concept of contextual product similarity and an effective way to automatically collect annotations to train this model. (c) finally, publishing two large scale datasets, one is a question similarity data set and the second is a large-scale amazon product questions and answers dataset, details are provided in section 4. empirical evaluation of our method demonstrates that it outperforms a strong baseline in some question segments, and that a hybrid model is effective in all the vast majority of the questions. product: dickies men's jeans, 100% cotton. q: will these shrink after a wash? predicted answer: no similar product 1: eddie bauer men's jeans, 100% cotton. q: do these shrink when you wash and dry them? a: no similar product 2: timberland pro men's jean, 99% cotton, 1% polyurethane. q: was there any shrinkage after washing? a: no shrinkage similar product 3: levi's men's jeans, 98% cotton, 2% elastane. q: do these shrink at all during washing/drying? a: they have not shrunk ## related work automatic aswering product related questions has become a permanent service provided by many ecommerce websites and services @xcite . questions are typically answered based on product details from the catalog, existing q&a's on the site, and customer reviews. each of these resources, used for answer generation, has been studied extensively by the research community recently, probably due to the complexity of this task, the availability of appropriate datasets @xcite , and the emergent increase in on-line shopping usage. @xcite built a question answering system based on product facts and specifications. they trained a question answering system by transfer learning from a large-scale amazon dataset to the home depot domain. @xcite 2019 ) generated an answer from product attributes and reviews using adversarial learning model which is composed of three components: a question-aware review representation module, a key-value attribute graph, and a seq2seq model for answer generation. @xcite answered opinion questions by exploiting hierarchical organization of consumer reviews, where reviews were organized according to the product aspects. the publication of amazon datasets of reviews foot_1 and q&as @xcite , triggered a flood of studies on review-aware answer prediction and generation. @xcite formulated the review based question answering task as a mixtureof-experts framework -each review is an "expert" that votes on the answer to a yes/no question. their model learns to identify 'relevant' reviews based on those that vote correctly. in a following work, @xcite observed that questions have multiple, often divergent, answers, and the full spectrum of answers should be further utilized to train the answering system. chen et al. (2019) described a multi-task attention mechanism which exploits large amounts of q&as, and a few manually labeled reviews, for answer prediction. @xcite proposed a neural architecture, directly fed by the raw text of the question and reviews, to mark review segment as the final answer, in a reading comprehension fashion. @xcite learned an adversarial network for inferring reviews which best answer a question, or augment a given answer. @xcite incorporated opinion mining into the reviewbased answer generation. @xcite generated aspect-specific representation for questions and reviews for answer prediction for yes-no questions. @xcite used transfer learning from a resource-rich source domain to a resource-poor target domain, by simultaneously learning shared representations of questions and reviews in a unified framework of both domains. all this line of works assume the existence of rich set of product reviews to be used for question answering. this solution fails when no reviews are available. the challenge of review generation for a given product, while utilizing similar products' reviews, was addressed by @xcite . for a given product they extracted useful sentences from the reviews of other similar products. similarly, @xcite mined relevant content for a product from various content resources available for similar products. both works focused on the extraction of general useful product related information rather than answering a specific product question, as in our case. second, the product-similarity methods they considered rely on product specifications and description, and do not depend on the question to be answered, while our method considers a specific question at hand when estimating contextual product similarity. ## similarity-based answer prediction in this section, we introduce the similarity-based answer-prediction (simba) method for predicting the answer for a product question, based on the answers for other similar product questions. we restrict our study to yes/no questions only, due to their popularity in the pqa domain (54% on our pqa dataset), and following common practices in answer prediction studies @xcite . figure 1 presents our prediction framework and its main components. formally, a question-product-answer tuple is denoted by r @xmath0, p j , a j ), where a @xmath1, no }. @xmath2. r @xmath3, p t , ?) foot_2 is the target record of an unanswered question q t , asked about product p t . we treat c as the knowledgebase we use for answering q t . given a target record r t , in order to predict its answer a t , we first retrieve a set of records from c with the most similar questions to q t (figure 1 , stage 1). we denote the retrieved records as siblings of r t . we then filter the siblings by applying a question-to-question similarity (q2q) model, keeping only records with highly similar questions which are expected to have the same question intent as of q t , (figure 1 , stage 2). we denote these records as twins of r t . we then apply our contextual product similarity (cps) model to measure the contextual similarity between r t and its twins (figure 1 , stage 3). the cps similarity score is used to weight the twins by considering them as voters, applying a mixture-of-experts model over their answers for the final answer prediction (figure 1 , stage 4). more details about the model's components, the training processes, and other specifications, are described in the following. ## datasets we introduce two new datasets to experiment with our answer prediction approach: 1) the amazon product question similarity (amazon-pqsim) dataset which is used to train our q2q model; 2) the amazon product question answers (amazon-pqa) dataset of product related q&as, used for training the simba model. ## experiments we experiment with eleven product categories covered by our amazon-pqa dataset (section 4.2), training a simba answer prediction model for each of the categories independently. next, we describe the data preparation steps for each of the simba components. ## conclusions we presented simba, a novel answer prediction approach in the pqa domain, which directly leverages similar questions answered with respect to other products. our empirical evaluation shows that on some segments of questions, namely those with roughly ten or more similar questions in the corpus, our method can outperform a strong inductive method that directly utilizes the question and the textual product content. we further show that the two approaches are complementary and can be integrated to increase the overall answer prediction accuracy. for future work, we plan to explore how simba can be extended and be applied beyond yes-no questions, e.g., for questions with numerical answers or open-ended questions. another interesting research direction is combining additional voters to the mixture-of-experts model, such as a review-aware answer predictor or a product details-based predictor. additionally, our current evaluation considered a static view of the answered product-question corpus, we plan to explore temporal aspects of our method, for example, considering questions age or ignoring answers of obsolete products that might be irrelevant.