Product Question Answering in E-Commerce: A Survey
Year: 2023

Abstract:
Product question answering (PQA), aiming to automatically provide instant responses to customer’s questions in E-Commerce platforms, has drawn increasing attention in recent years. Compared with typical QA problems, PQA exhibits unique challenges such as the subjectivity and reliability of user-generated contents in Ecommerce platforms. Therefore, various problem settings and novel methods have been proposed to capture these special characteristics. In this paper, we aim to systematically review existing research efforts on PQA. Specifically, we categorize PQA studies into four problem settings in terms of the form of provided answers. We analyze the pros and cons, as well as present existing datasets and evaluation protocols for each setting. We further summarize the most significant challenges that characterize PQA from general QA applications and discuss their corresponding solutions. Finally, we conclude this paper by providing the prospect on several future directions.

heading: 1 Introduction
text: E-Commerce is playing an increasingly important role in our daily life. During the online shopping, potential customers inevitably have some questions about their interested products. To settle down their concerns and improve the shopping experience, many AI conversational assistants have been developed to solve customers’ problems, such as Alexa (Carmel et al., 2018) and AliMe (Li et al., 2017a). The core machine learning problem underlying them, namely Product Question Answering (PQA), thus receives extensive attention in both academia and industries recently. Figure 1 depicts an actual PQA example from Amazon. There are a
∗ The work described in this paper is substantially supported by a grant from the Research Grant Council of the Hong Kong Special Administrative Region, China (Project Code: 14200719).
† Corresponding author.
tremendous amount of product-related data available within the product page, which contains natural language user-generated content (UGC) (e.g., product reviews, community QA pairs), structured product-related information (e.g., attribute-value pairs), images, etc. Generally, PQA aims to automatically answer the customer-posted question in the natural language form about a specific product, based on the product-related data.
Typical QA studies (Rajpurkar et al., 2016) and some other domain-specific QA studies (e.g., biomedical QA (Jin et al., 2023) and legal QA (Gil, 2021)) mainly focus on the questions that ask for a certain factual and objective answer. Differently, product-related questions in PQA typically involve consumers’ opinion about the products or aspects of products. Therefore, early studies (Moghaddam and Ester, 2011; Yu et al., 2012) regard PQA as a special opinion mining problem, where the answers are generated by aggregating opinions in the retrieved documents. Most of recent works essentially follow the same intuition, but formulate PQA as different problems in terms of the form of target answers. Accordingly, existing PQA studies
11951
can be categorized into four types: opinion-based, extraction-based, retrieval-based, and generationbased. As shown in Figure 1, opinion-based PQA approaches only provide the common opinion polarity as the answer, while extraction-based PQA approaches extract specific text spans from the supporting documents as the answer. Retrieval-based PQA approaches further re-rank the documents to select the most appropriate one to answer the given question, while generation-based PQA approaches generate natural language sentences based on the available documents as the response. In this paper, we systematically review methods of these four mainstream PQA problem settings, as well as the commonly-used datasets and evaluation protocols.
Besides the task-specific challenges in each type of PQA systems, there are several common challenges across all types of PQA systems, which differentiate PQA from other QA systems. (1) Subjectivity. Subjective questions constitute a large proportion of questions in PQA, which requires to aggregate the crowd’s opinions about the questions, reflected through related reviews and QAs. (2) Reliability & Answerability. Different from those supporting documents constructed by professionals in biomedical or legal QA, product reviews and community QA pairs come directly from nonexpert users, which may suffer from some typical
flaws as other UGC, such as redundancy, inconsistency, spam, and even malice. (3) Multi-type resources. The supporting documents usually consist of heterogeneous information from multi-type data resources, such as text, table, knowledge graph, image, etc. (4) Low-resource. PQA systems often encounter the low-resource issue, since different product categories may need different training data, and it is generally time-consuming and costly to manually annotate sufficient labeled data for each domain. Accordingly, we introduce existing solutions to each challenge.
To our knowledge, this survey is the first to focus on Product Question Answering. We first systematically summarize recent studies on PQA into four problem settings as well as introduce the available datasets and corresponding evaluation protocols in Section 2. Then we analyze the most significant challenges that characterize PQA from other QA applications and discuss their corresponding solutions in Section 3. Finally, we discuss several promising research directions for future PQA studies and conclude this paper in Section 4 and 5.

heading: 2 Problems and Approaches
text: Product question answering (PQA) aims to produce an answer a to a given natural language question q based on a set of supporting documents D,
where the supporting documents can be product reviews, community QA pairs, product information, etc. In terms of the form of provided answers, we systematically categorize the existing PQA studies into four problem settings, including Opinion-based PQA, Extraction-based PQA, Retrieval-based PQA, Generation-based PQA, and introduce corresponding approaches proposed to solve the problem, as summarized in Table 1. We present an overview of the general framework for each problem setting in Figure 2. In addition, the key information of the datasets adopted in existing PQA studies is summarized in Table 2.

heading: 2.1 Opinion-based PQA
text: Opinion-based PQA studies focus on yes-no type questions, i.e., questions that can be answered by “Yes” or “No”, which constitute a large proportion on PQA platforms.

heading: 2.1.1 Problem Definition
text: Given a product-related question q and a set of supporting documents D (product reviews in most cases), the goal is to predict a binary answer a ∈ {Yes,No}. Some studies also consider the neutral answer, e.g., “Not Sure".

heading: 2.1.2 Datasets & Evaluation Protocols
text: One of the largest and widely-adopted public PQA datasets is the Amazon Product Dataset (denoted as “Amazon” in Table 1 and hereafter), composed by Amazon Question/Answer Data (McAuley and Yang, 2016; Wan and McAuley, 2016) and Amazon Review Data (He and McAuley, 2016; Ni et al., 2019). It consists of around 1.4 million answered questions and 233.1 million product reviews across over 20 different product categories. The Amazon dataset contains the information of question types (“yes-no" or “open-ended"), answer types (“yes", “no", or “not sure"), helpful votes by customers, and
product metadata, which is suitable for opinionbased PQA evaluation.
Due to the existence of a certain proportion of unanswerable questions based on the available reviews, it is difficult to achieve an acceptable performance with the ordinary classification accuracy metric Acc(Q) for any method. Therefore, McAuley and Yang (2016) propose Acc@k, which has become the de facto metric for evaluating opinion-based PQA methods, which only calculates the classification accuracy of top-k questions ranked by the prediction confidence. The confidence with each classification is its distance from the decision boundary, i.e., |12 − P (a|q,D)|. A good model is supposed to assign high confidence to those questions that can be correctly addressed.
Acc@k = Acc( argmax Q′∈Pk(Q)
∑
q∈Q′ |1 2 − P (a|q,D)|) (1)
where Pk(Q) is the set of k-sized subsets of Q, and k is commonly set to be 50% of the total number of questions.

heading: 2.1.3 Methods
text: McAuley and Yang (2016) propose a Mixtures of experts (MoEs) (Jacobs et al., 1991) based model, namely Mixtures of Opinions for Question Answering (Moqa), to answer yes-no questions in PQA, where each review is regarded as an “expert” to make a binary prediction for voting in favor of a “yes” or “no” answer. The confidence of each review is further weighted by its relevance to the question as follows:
P (a|q,D) = ∑
d∈D P (d|q)︸ ︷︷ ︸
how relevant is d · P (a|d, q)︸ ︷︷ ︸ prediction from d (2)
Moqa is later enhanced by modeling the ambiguity and subjectivity of answers and reviews (Wan and McAuley, 2016). Yu and Lam (2018b) further improve Moqa by computing the aspect-specific embeddings of reviews and questions via a three-order
auto-encoder network in an unsupervised manner. In these early studies, the features either extracted by heuristic rules or acquired from unsupervised manners may limit the performance and application of opinion-based PQA approaches.
To better model the relation between the question and each review, Fan et al. (2019) and Zhang et al. (2019) explore the utility of neural networks (e.g., BiLSTM (Schuster and Paliwal, 1997)) and pretrained language models (e.g., BERT (Devlin et al., 2019)) to learn the distributed feature representations, which largely outperform previous methods. Recently, Rozen et al. (2021) propose an approach, called SimBA (Similarity Based Answer Prediction), which leverages existing answers from similar resolved questions about similar products to predict the answer for the target question.

heading: 2.1.4 Pros and Cons
text: Opinion-based PQA approaches can tackle a large proportion of product-related questions that ask for certain opinion by using comparatively simple and easy-to-deploy methods. However, opinion-based approaches could only provide the classification result of the opinion polarity, based on the common opinion reflected in the supporting documents, without detailed and question-specific information.

heading: 2.2 Extraction-based PQA
text: Similar to typical extraction-based QA (Rajpurkar et al., 2016) (also called Machine Reading Comprehension (MRC)), extraction-based PQA studies aim at extracting a certain span of a document to be the answer for the given product-related questions.

heading: 2.2.1 Problem Definition
text: Given a product-related question q and a supporting document d = {t1, ..., tn} ∈ D, which consists of
3http://deepx.ucsd.edu/public/jmcauley/qa/ 4https://github.com/amazonqa/amazonqa 5https://howardhsu.github.io/ 6https://github.com/megagonlabs/SubjQA 7https://github.com/gsh199449/productqa
one or more product reviews, the goal is to find a sequence of tokens (a text span) a = {ts, ..., te} in d that answers q correctly, where 1 ≤ s ≤ n, 1 ≤ e ≤ n, and s ≤ e.

heading: 2.2.2 Datasets & Evaluation Protocols
text: Xu et al. (2019) build the first extraction-based PQA dataset, called ReviewRC, using reviews from SemEval-2016 Task 5 (Pontiki et al., 2016). Similarly, Gupta et al. (2019) conduct extensive pre-processing on the Amazon dataset (McAuley and Yang, 2016; He and McAuley, 2016) to build a dataset for extraction-based PQA, called AmazonQA. It annotates each question as either answerable or unanswerable based on the available reviews, and heuristically creates an answer span from the reviews that best answer the question. Bjerva et al. (2020) propose SubjQA dataset to investigate the relation between subjectivity and PQA in the context of product reviews, which contains 6 different domains that are built upon TripAdvisor (Wang et al., 2010), Yelp6, and Amazon (McAuley and Yang, 2016) datasets.
Given the same setting as typical MRC, extraction-based PQA adopts the same evaluation metrics, including Exact Match (EM) and F1 scores. EM requires the predicted answer span to exactly match with the human annotated answer, while F1 score is the averaged F1 scores of individual answers in the token-level.

heading: 2.2.3 Methods
text: Due to the limited training data for extraction-based PQA, Xu et al. (2019) employ two popular pretraining objectives, i.e., masked language modeling and next sentence prediction, to post-train the BERT encoder on both the general MRC dataset, SQuAD (Rajpurkar et al., 2016), and E-Commerce review datasets, including Amazon Review (He and McAuley, 2016) and Yelp datasets. In realworld applications, there will be a large number
6https://www.yelp.com/dataset
of irrelevant reviews and the question might be unanswerable. To this end, Gupta et al. (2019) first extract top review snippets for each question based on IR techniques and build an answerability classifier to identify unanswerable questions based on the available reviews. Then, a span-based QA model, namely R-Net (Wang et al., 2017), is adopted for the extraction-based PQA. Besides, Bjerva et al. (2020) develop a subjectivity-aware QA model, which performs the multi-task learning of the extraction-based PQA and subjectivity classification. Experimental results show that incorporating subjectivity effectively boosts the performance.

heading: 2.2.4 Pros and Cons
text: Extraction-based PQA approaches can provide pinpointed answers to the given questions, but it may be less user-friendly to provide an incomplete sentence to users and may also lose some additional information. Since there are a large proportion of questions that ask for certain user experiences or opinions based on the statistics in (McAuley and Yang, 2016; Deng et al., 2022), extractionbased paradigm is less practical and favorable in real-world PQA applications. Therefore, it can be observed that there are relatively few works in extraction-based PQA studies in recent years.

heading: 2.3 Retrieval-based PQA
text: Retrieval-based PQA studies treat PQA as an answer (sentence) selection task, which retrieves the best answer from a set of candidates to appropriately answer the given question.

heading: 2.3.1 Problem Definition
text: Given a question q and a set of supporting documents D, the goal is to find the best answer a by ranking the list of documents according to the relevancy score between the question q and each document d ∈ D, i.e., a = argmaxd∈DR(q, d).

heading: 2.3.2 Datasets & Evaluation Protocols
text: Due to the absence of ground-truth question-review (QR) pairs, several efforts (Chen et al., 2019a; Yu et al., 2018b; Zhang et al., 2020f) have been made on annotating additional QR pairs into the Amazon dataset for retrieval-based PQA. Nevertheless, the original Amazon dataset can be directly adopted for retrieval-based PQA studies (Zhang et al., 2020e,c) that aim to select reliable or helpful answers from candidate community answers.
Since the retrieval-based PQA methods are essentially solving a ranking problem, most studies
adopt standard ranking metrics for evaluation, including mean average precision (MAP), mean reciprocal rank (MRR), and normalized discounted cumulative gain (NDCG).

heading: 2.3.3 Methods
text: Cui et al. (2017) first demonstrate a retrieval-based PQA chatbot, namely SuperAgent, which contains different ranking modules that select the best answer from different data sources within the product page, including community QA pairs, product reviews, and product information. Kulkarni et al. (2019) further propose a pipeline system that first classifies the question into one of the predefined question categories with a question category classifier, and then uses an ensemble matching model to rank the candidate answers. However, these systems usually contain multiple modules with different purposes, which require a large amount of annotated data from different sources. Therefore, most recent retrieval-based PQA works use one or two sources as the supporting documents and build the model in an end-to-end manner.
When facing a newly posted product-related question, a straight-forward answering strategy is to retrieve a similar resolved question and provide the corresponding answer to the target question. However, such a solution relies heavily on a large amount of domain-specific labeled data, since QA data differs significantly in language characteristics across different product categories. To handle the low-resource issue, Yu et al. (2018a) propose a general transfer learning framework that adapts the shared knowledge learned from large-scale paraphrase identification and natural language inference datasets (e.g., Quora7 and MultiNLI (Williams et al., 2018)) to enhance the performance of reranking similar questions in retrieval-based PQA systems. Besides, Mittal et al. (2021) propose a distillation-based distantly supervised training algorithm, which uses QA pairs retrieved by a syntactic matching system, to help learn a robust question matching model.
Another approach to obtain answers for new questions is to select sentences from product reviews. The main challenge is that the information distributions of explicit answers and review contents that can address the corresponding questions are quite different and there are no annotated ground-truth question-review (QR) pairs which can
7https://www.kaggle.com/c/ quora-question-pairs
be used for training. Yu et al. (2018b) develop a distant supervision paradigm for incorporating the knowledge contained in QA collections into question-based response review ranking, where the top ranked reviews are more relevant to the QA pair and are useful for capturing the knowledge of response review ranking. Chen et al. (2019a) propose a multi-task deep learning method, namely QARnet, which can exploit both user-generated QA data and manually labeled QR pairs to train an end-toend deep model for answer identification in review data. Zhao et al. (2019) aim at improving the interpretability of retrieval-based PQA by identifying important keywords within the question and associating relevant words from large-scale QA pairs. Zhang et al. (2020f) employ pre-trained language models (e.g., BERT) to obtain weak supervision signals from the community QA pairs for measuring the relevance between the question and heterogeneous information, including natural language reviews and structured attribute-value pairs.
For the situation where multiple user-generated answers have already been posted, Zhang et al. (2020c) propose an answer ranking model, namely MUSE, which models multiple semantic relations among the question, answers, and relevant reviews, to rank the candidate answers in PQA platforms.

heading: 2.3.4 Pros and Cons
text: Retrieval-based approaches select complete and informative sentences as the answer, which may not answer the given question precisely since the supporting document (e.g., reviews) is not specifically written for answering the given question.

heading: 2.4 Generation-based PQA
text: Inspired by successful applications of sequence-tosequence (Seq2seq) models on other natural language generation tasks, several attempts have been made on leveraging Seq2seq model to automatically generate natural sentences as the answer to the given product-related question.

heading: 2.4.1 Problem Definition
text: Given a product-related question q and a set of supporting documents D that are relevant to the given question, the goal is to generate a natural language answer a = {ta1, ta2, ...} based on the question q and supporting documents D.

heading: 2.4.2 Datasets & Evaluation Protocols
text: The Amazon dataset can be directly adopted for generation-based PQA. Another popular dataset
used for geneartive PQA is from JD (Gao et al., 2019), which is one of the largest e-commerce websites in China. In total, the JD dataset contains 469,953 products and 38 product categories, where each QA pair is associated with the reviews and attributes of the corresponding product.
Evaluating generation-based methods often involves both automatic evaluation and human evaluation. Common automatic evaluation metrics include (i) ROUGE (Lin, 2004) and BLEU (Papineni et al., 2002) for evaluating lexical similarity between generated answers and ground-truth answers, (ii) Embedding-based Similarity (Forgues et al., 2014), BertScore (Zhang et al., 2020b), and BleuRT (Sellam et al., 2020) for evaluating semantic relevance, (iii) Distinct scores (Li et al., 2016) for evaluating the diversity of the generated answers. Human evaluation protocols are designed for evaluating different perspectives of the generated answer by human annotations, such as fluency, consistency, informativeness, helpfulness, etc.

heading: 2.4.3 Methods
text: Generation-based PQA studies typically regard the retrieval of relevant documents as a pre-processing step, and build the method upon the retrieved documents. Due to the noisy nature of retrieved documents, Gao et al. (2019) employ a Wasserstein distance based adversarial learning method to denoise the irrelevant information in the supporting reviews, while Chen et al. (2019c) design an attention-based weighting strategy to highlight the relevant words appearing in the retrieved review snippets. Besides identifying relevant information from the retrieved documents, Deng et al. (2020) find that the rich personal opinion information in product reviews also attaches great importance in generation-based methods, as there are a large number of subjective questions in PQA. To this end, a joint learning model of answer generation and opinion mining is proposed to generate opinion-aware answers. Likewise, Lu et al. (2020) propose a cross-passage hierarchical memory network to identify the most prominent opinion across different reviews for answer generation in PQA.
Some recent works focus on leveraging documents from multi-type resources to generate the answer. Feng et al. (2021) model the logical relation between unstructured documents (reviews) and structured documents (product attributes) with a heterogeneous graph neural network. Gao et al. (2021) aim at solving the safe answer problem dur-
ing the generation (i.e., neural models tend to generate meaningless and general answers), by systematically modeling product reviews, product attributes, and answer prototypes. Shen et al. (2022b) propose present the semiPQA dataset to benchmark PQA over semi-structured data.

heading: 2.4.4 Pros and Cons
text: Generation-based methods can provide natural forms of answers specific to the given questions. However, the hallucination and factual inconsistency issues are prevalent in generation-based methods. In addition, it is still lack of robust automatic evaluation protocols for generation-based methods.

heading: 3 Challenges and Solutions
text: Although the aforementioned PQA methods are developed based on different problem settings, there are some common challenges in PQA, as presented in Table 1. Several main challenges and their corresponding solutions are summarized as follows.

heading: 3.1 Subjectivity
text: Different from typical QA whose answers are usually objective and unique, a large proportion of questions in PQA platforms are asking for subjective information or opinions. Meanwhile, the UGC in E-commerce such as product reviews also provides rich information about other customers’ opinion. Therefore, early studies regard PQA as a special opinion mining problem (Moghaddam and Ester, 2011; Yu et al., 2012), which is followed by recent opinion-based PQA studies (McAuley and Yang, 2016; Wan and McAuley, 2016). Ideal answers to this kind of questions require information describing personal opinions and experiences. There are two specific challenges in exploiting such subjective information to facilitate PQA:
• Detect question-related opinion. A common solution is to regard the question as the target aspect for aspect-based opinion extraction. For example, Bjerva et al. (2020) use OpineDB (Li et al., 2019c) and some syntactic extraction patterns to extract opinion spans. Deng et al. (2020) employ a dual attention mechanism to highlight the question-related information in reviews for the joint learning with an auxiliary opinion mining task. Zhang et al. (2021) study aspect-based sentiment analysis in PQA, which classifies the sentiment polarity towards certain product aspects in the question from the community answers.
• Aggregate diverse opinion information. Since users may differ in opinions towards the same question, a good PQA system should avoid expressing a random opinion, or even being contradictory to the common opinion. To this end, Deng et al. (2020) employ an opinion selfmatching layer and design two kinds of opinion fusion strategies to uncover the common opinion among multiple reviews for generation-based PQA. Likewise, Lu et al. (2020) propose a crosspassage hierarchical memory network to identify the most prominent opinion. However, existing studies pay little attention on resolving conflicting user opinions, which is a common issue in opinion summarization of product reviews (Pecar, 2018; Suhara et al., 2020) and worth exploring in the future studies of PQA.

heading: 3.2 Answer Reliability & Answerability
text: Similar to other UGC, product reviews and community answers in E-commerce sites, which are also provided by online users instead of professionals, vary significantly in their qualities and inevitably suffer from some reliability issues such as spam, redundancy, and even malicious content. Therefore, it is of great importance to study the answer reliability and answerability issue when building automatic PQA systems using these UGC. In terms of the availability of candidate answers, existing solutions can be categorized into two groups:
• Reliability of user-generated answers. When there are a set of candidate user-generated answers for the concerned question, the reliability measurement of these answers has been investigated from different perspectives. For example, Zhang et al. (2020e) predict the helpfulness of user-generated answers by investigating the opinion coherence between the answer and crowds’ opinions reflected in the reviews, while Zhang et al. (2020d) tackle the veracity prediction of the user-generated answers for factual questions as an evidence-based fact checking problem. However, these studies mainly focus on the content reliability while neglecting the reliability degree of the answerer (Li et al., 2017b, 2020).
• Unanswerable questions based on the available documents. Question answerability detection has drawn extensive attention in typical QA studies (Rajpurkar et al., 2018). Similarly, Gupta et al. (2019) train an binary classifier to classify the question answerability for PQA. Zhang et al.
(2020a) propose a conformal prediction based framework to reject unreliable answers and return nil answers for unanswerable questions. Meanwhile, the answerablity in PQA is also highly related to the reliability of product reviews (Roy et al., 2022a; Shen et al., 2022a).

heading: 3.3 Multi-type Resources
text: Another characteristic of PQA is the necessity of processing heterogeneous information from multitype resources, including natural language UGC (e.g., reviews, community QA pairs), structured product information (e.g., attribute-value pairs (Lai et al., 2018; Roy et al., 2020), knowledge graph (Li et al., 2019a)), E-manuals (Nandy et al., 2021), images, etc. Early works (Cui et al., 2017; Kulkarni et al., 2019) design separated modules to handle the questions that require different types of data resources. However, these PQA systems rely heavily on annotated data from different types of resources and neglect the relation among heterogeneous data. Therefore, some recent studies focus on manipulating heterogeneous information from multi-type resources in a single model for better answering product-related questions. For instance, Zhang et al. (2020f) design a unified heterogeneous encoding scheme that transforms structured attribute-value pairs into a pesudo-sentence. Gao et al. (2019) employ a key-value memory network to store and encode product attributes for answer decoding with the encoded review representations, which is further combined with answer prototypes (Gao et al., 2021). Feng et al. (2021) propose a heterogeneous graph neural network to track the information propagation among different types of information for modeling the relational and logical information.

heading: 3.4 Low-resource
text: Since there are a large amount of new questions posted in PQA platforms every day and the required information to answer the questions varies significantly across different product categories (even across different single products), traditional supervised learning methods become data hungry in this situation. However, it is time-consuming and laborintensive to obtain sufficient domain-specific annotations. Existing solutions typically leverage external resources to mitigate the low-resource issue. In terms of the external resources, these solutions can be categorized into two groups:
• Transfer learning from out-domain data. This group of solutions typically leverages large-scale
open-domain labeled datasets and design appropriate TL strategy for domain adaptation in PQA. For example, Yu et al. (2018a) transfer the knowledge learned from Quora and MultiNLI datasets to retrieval-based PQA models, by imposing a regularization term on the weights of the output layer to capture both the inter-domain and the intra-domain relationships. Xu et al. (2019) perform post-training on the SQuAD dataset to inject task-specific knowledge into BERT for extraction-based PQA.
• Distant supervision from in-domain data. Another line of solutions adopt the resolved QA pairs from similar products (Rozen et al., 2021) or products in the same categories (Yu et al., 2018b; Chen et al., 2019a; Zhao et al., 2019; Roy et al., 2022b) as weak supervision signals. For example, Zhang et al. (2020f) and Mittal et al. (2021) employ syntactic matching systems (e.g., BM25) or pre-trained text embeddings (e.g., BERT) to obtain resolved QA pairs for facilitating the distantly supervised training process.

heading: 4 Prospects and Future Directions
text: Considering the challenges summarized in this paper, we point out several promising prospects and future directions for PQA studies:
• Question Understanding. Due to the diversity of product-related questions, some attempts have been made on identifying the user’s intents (Yu and Lam, 2018a), the question types (Cui et al., 2017), and even the user’s purchase-state (Kuchy et al., 2021) from the questions. In addition, some researches investigate the user’s uncertainty or the question’s ambiguity towards the product by asking clarifying questions (Majumder et al., 2021; Zhang and Zhu, 2021). Despite the extensive studies for QA, question understanding has not been deeply studied in the context of PQA. For example, the system should be capable of identifying the subjectivity from the productrelated questions (Bjerva et al., 2020), such as opinionated questions (Deng et al., 2020), comparative questions (Bondarenko et al., 2022), etc.
• Personalization. As mentioned before, compared with typical QA studies (Rajpurkar et al., 2016), there is a large proportion of subjective questions (McAuley and Yang, 2016) on PQA platforms, which involve user preference or require personal information to answer, rather than
objective or factoid questions that look for a certain answer. Besides, in E-Commerce, different customers often have certain preferences over product aspects or information needs (Chen et al., 2019b; Li et al., 2019b), leading to various expectations for the provided answers. Therefore, Carmel et al. (2018) state that a good PQA system should answer the customer’s questions with the context of her/his encounter history, taking into consideration her/his preference and interest. Such personalization can make the answer more helpful for customers and better clarify their concerns about the product (Deng et al., 2022).
• Multi-modality. Compared with the widelystudied natural language UGC and structured product knowledge data, image data has received little attention in PQA studies. On E-Commerce sites, there exist not only a great number of official product images, but also increasing usershared images about their actual experiences, which benefit many other E-Commerce applications (Liu et al., 2021; Zhu et al., 2020). The multimodal data can provide more valuable and comprehensive information for PQA systems.
• Datasets and Benchmarks. Despite the increasing attentions on developing PQA systems, the publicly available resources for PQA are still quite limited. Most existing PQA studies are evaluated on the Amazon dataset (McAuley and Yang, 2016), which is directly crawled from the Amazon pages. Some researches (Roy et al., 2022a; Shen et al., 2022a) have discussed several drawbacks of evaluating PQA systems on this dataset: 1) The ground-truth answers are quite noisy, since they are the top-voted community answers posted by non-expert users. 2) There are no annotations for assessing the relevance of the supporting documents, which may cast potential risks on the reliability of the PQA systems. To facilitate better evaluations, many other data resources for PQA studies have been constructed as presented in Table 2. However, due to the privacy or the commercial issues, some of the datasets cannot be publicly released. Therefore, there is still a great demand for a large-scale, high-quality, and publicly available benchmark dataset for the future studies on PQA.
• Evaluation Protocols. The types of questions vary in a wide range, from yes-no questions to open-ended questions (McAuley and Yang,
2016), from objective questions to subjective questions (Bjerva et al., 2020), from factual questions to non-factual questions (Zhang et al., 2020d). Different types of questions may involve different specific evaluation protocol. For example, it is necessary to evaluate the precision of opinion in the answers for subjective questions (Deng et al., 2020), while the veracity or factualness is important in factual questions (Zhang et al., 2020d). Especially for generation-based PQA methods, the evaluation is still largely using lexical-based text similarity metrics, which are not correlated well with human judgements.

heading: 5 Conclusions
text: This paper makes the first attempt to overview recent advances on PQA. We systematically categorize recent PQA studies into four problem settings, including Opinion-based, Extraction-based, Retrieval-based, and Generation-based, and summarize the existing methods and evaluation protocols in each category. We also analyze the typical challenges that distinguish PQA from other QA studies. Finally, we highlight several potential directions for facilitating future studies on PQA.
Limitations
Since product question answering (PQA) is actually a domain-specific application in general QA, the scope of the problem may be limited. However, in recent years, PQA has received increasing attention in both academy and industry. (1) From the research perspective, PQA exhibits some unique characteristics and thus brings some interesting research challenges as discussed in Section 3. For example, some studies use PQA as an entrypoint to analyze the subjectivity in QA tasks. (2) From the application perspective, it has great commercial value. Online shopping is playing an increasingly important role in everyone’s daily life, so that many high-tech companies develop AI conversational assistants for promptly solving customer’s online problems, including but not limited to Amazon, eBay, Alibaba, JD, etc. Regarding the large amount of research efforts that have been made, there is not a systematic and comprehensive review about this research topic. Similar to recent surveys of other domain-specific QA, such as biomedical QA (Jin et al., 2023) and legal QA (Gil, 2021), we hope that this paper can serve as a good reference for people working on PQA or beginning to
work on PQA, as well as shed some light on future studies on PQA and raise more interests from the community for this topic.

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitation section
7 A2. Did you discuss any potential risks of your work? It’s a survey paper. There is no potential risk.
3 A3. Do the abstract and introduction summarize the paper’s main claims? Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 7 Did you run computational experiments? Left blank.
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? No response.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? No response.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? No response.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? No response.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Finally, we highlight several potential directions for facilitating future studies on PQA. Limitations
Since product question answering (PQA) is actually a domain-specific application in general QA, the scope of the problem may be limited. However, in recent years, PQA has received increasing attention in both academy and industry. (1) From the research perspective, PQA exhibits some unique characteristics and thus brings some interesting research challenges as discussed in Section 3. For example, some studies use PQA as an entrypoint to analyze the subjectivity in QA tasks. (2) From the application perspective, it has great commercial value. Online shopping is playing an increasingly important role in everyone’s daily life, so that many high-tech companies develop AI conversational assistants for promptly solving customer’s online problems, including but not limited to Amazon, eBay, Alibaba, JD, etc. Regarding the large amount of research efforts that have been made, there is not a systematic and comprehensive review about this research topic. Similar to recent surveys of other domain-specific QA, such as biomedical QA (Jin et al. , 2023) and legal QA (Gil, 2021), we hope that this paper can serve as a good reference for people working on PQA or beginning to
work on PQA, as well as shed some light on future studies on PQA and raise more interests from the community for this topic