A nswer F act: Fact Checking in Product Question Answering (2020)

Abstract: Product-related question answering platforms nowadays are widely employed in many E-commerce sites, providing a convenient way for potential customers to address their concerns during online shopping. However, the misinformation in the answers on those platforms poses unprecedented challenges for users to obtain reliable and truthful product information, which may even cause a commercial loss in E-commerce business. To tackle this issue, we investigate to predict the veracity of answers in this paper and introduce AnswerFact, a large scale fact checking dataset from product question answering forums. Each answer is accompanied by its veracity label and associated evidence sentences, providing a valuable testbed for evidence-based fact checking tasks in QA settings. We further propose a novel neural model with tailored evidence ranking components to handle the concerned answer veracity prediction problem. Extensive experiments are conducted with our proposed model and various existing fact checking methods, showing that our method outperforms all baselines on this task.

## introduction the ability to ask questions during online shopping is found to be a key factor for customers to make purchase decisions @xcite . to this end, product-related community question answering (pqa) platforms have emerged in many e-commerce sites such as amazon and taobao, allowing users to pose their concerns as questions and receive answers from fellow users to obtain useful product information. however, similar to other community question answering (cqa) platforms, question: is this egg coker automatic shut off? answer (claim): yes there's an automatic shut-off when the cooking cycle is finished. (verdict : false) evidence: s1: a buzzer sounds to let you know the eggs are done, i wish it would just shut off instead. s2: when the alarm sounds you need to turn it off and open it. s3: i would have liked the cooker to turn off automatically but instead a bell rings until you turn if off. s4: also, by the time the timer goes off, the hot pan has a burning smell. s5: and it turns off itself after the bell rings. . . . the user-provided answers on pqa platforms vary significantly on their qualities @xcite , and more seriously, their veracity due to the lack of systematic quality control @xcite . those untruthful answers may attribute to multiple factors such as misunderstandings of the question, improper expressions during writing, and even intentionally malicious attacks from the competitors @xcite . therefore, automatically verifying the answer veracity is becoming a demanding need, which can offer a more reliable online shopping environment, for example, by triggering a double-check on the detected doubtful answers. fact checking aiming at verifying the truthfulness of a given claim @xcite can be a promising direction to tackle the concerned problem. however, the claim on which existing fact checking methods mainly focus is usually a standalone text snippet such as news @xcite @xcite or twitter posts @xcite . to predict the veracity of an answer in the qa settings, one can notice that it is insufficient to consider the answer alone since the question text also carries important semantic information for the prediction. thus, we need to appropriately leverage the question text into the verification process. in the context of cqa problems, most existing studies focus on measuring the semantic relevance of a candidate answer to the given question @xcite or ranking available answers for a given question @xcite . however, the notion of veracity poses a more rigorous requirement of an answer where it needs to be factually correct. for example, the given answer in to tackle the aforementioned issues, we introduce a large scale fact checking dataset called an-swerfact for investigating the answer veracity in product question answering forums. an instance of the dataset is shown in we further propose aver, an answer veracity prediction model with tailored evidence ranking modules to predict the answer veracity in pqa forums. aver first utilizes the information from both the question and answer text to rank the evidence sentences with different gating mechanisms. an agreement-matching strategy is then employed to model the self-coherence of the evidence sentences for obtaining reliable combined evidence embeddings to verify the answer verdict. to summarize, our main contributions are as follows: ## answerfact dataset construction we build our dataset upon a large qa collection @xcite crawled from amazon. five product domains with the largest number of qa pairs are selected, namely, electronics, home and kitchen, sports and outdoors, health and personal care, and cell phones and accessories, con- ## answer veracity prediction problem definition. given an answer a to its corresponding question q, our aim is to predict the answer veracity which falls into one of the predefined veracity type, with the help of k relevant evidence sentences s 1 , s 2 , . . . , s k . in this section, we describe our proposed model aver for the answer veracity prediction task with tailored evidence ranking module. an overview of aver is shown in figure 1 . ## conclusions in this paper, we investigate the fact checking problem in product question answering forums, aiming to predict the answer veracity so as to provide more reliable online shopping environment. to this end, we introduce answerfact, an evidence-based fact checking datasets in qa settings. further, we propose aver model to predict answer veracity via tailored evidence ranking module. extensive experiments show that our proposed method outperforms various established baselines.