Diverse and Non-redundant Answer Set Extraction on Community QA based on DPP s (2020)

Abstract: In community-based question answering (CQA) platforms, it takes time for a user to get useful information from among many answers. Although one solution is an answer ranking method, the user still needs to read through the top-ranked answers carefully. This paper proposes a new task of selecting a diverse and non-redundant answer set rather than ranking the answers. Our method is based on determinantal point processes (DPPs), and it calculates the answer importance and similarity between answers by using BERT. We built a dataset focusing on a Japanese CQA site, and the experiments on this dataset demonstrated that the proposed method outperformed several baseline methods.

## introduction community-based question answering (cqa) platforms have been used by many people to solve problems that cannot be solved simply by searching in books or web pages. on a cqa platform, the questioner posts a question and waits for others to post answers. the posted answers are viewed by not only the questioner herself but also other users. a single question may receive several to dozens, even hundreds, of answers, and it would take a lot of time to see all the answers. the questioner chooses the most helpful answer, called the best answer, and users can view this answer as a representative answer. however, there may be more than one appropriate answer to a question that asks for reasons, advice, and opinions, while there is often only one appropriate answer for a question that asks for facts, etc. for example, one possible way to deal with this problem is to use ranking methods @xcite @xcite @xcite @xcite . since the studies use the best answers as training data, similar answers are ranked higher although the existing methods try to reduce redundancy by using the similarity between a question and an answer or between answers. as a result, users need to read through carefully the answers that are ranked higher to obtain the information they want to know. we propose a new task of selecting a diverse and non-redundant answer set from all the answers, instead of ranking the answers. we treat the set of answers as the input, from which the system determines both the appropriate number of answers to be included in the output set and the answers themselves. the datasets used in the conventional ranking task of cqa are not suitable for our task because they contain only a ranked list of answers, whereas we need to determine the number of answers to be included in the output set in addition to the answers themselves. therefore, we have constructed a new dataset for our task that contains diverse and non-redundant annotated answer sets. we use determinantal point processes (dpps) @xcite to select a diverse and non-redundant answer set. dpps are probabilistic models that are defined over the possible q why do birds turn their heads backwards when they sleep? answer gold a1 ...because it's cold. they keep their beaks in their feathers to protect them from getting cold. a2 this is because it reduces the surface area of the body to prepare for a drop in body temperature and to prevent physical exhaustion. a3 well balanced. a4 they look back to stabilize their bodies by shifting their center of gravity on the vertical line of their feet to stabilize their bodies. a5 i think it's to use their feather as pillows. to estimate the answer importance and the similarity between answers, we use bert @xcite , which has achieved high accuracy in various tasks. focusing on a japanese cqa site, we built training and evaluation datasets through crowdsourcing and conducted experiments demonstrating that the proposed method outperformed several baseline methods. a number of methods of summarizing answers in cqa @xcite also take diversity into account. these methods extract sentences of a fixed length, and therefore, cannot determine an appropriate output length. our method is different in that it can determine the appropriate output length by using dpps. some methods of question retrieval in cqa also take diversity into account @xcite . ## proposed method the overview of our method is shown in figure 1 . our method uses dpps to select a diverse and nonredundant subset from a set of answers. the dpps assign a higher probability to a subset in which each answer is important and the answers are diverse. bert @xcite is used for estimating the importance of an answer and the similarity between answers. in the example shown in figure 1 , answers 2 and 4 are annotated as a gold answer set. the probability of choosing these gold answers is defined by the dpps. intuitively, bert is fine-tuned so that the importance of answers 2 and 4 is higher and the similarity between answers 2 and 4 is lower. ## conclusion we proposed a new task to select a diverse and non-redundant answer set in cqa. we created a dataset for this task, and proposed a model for it by using dpps and bert. our method performed better than several baseline methods. in future work, we aim to pre-train bert-sim to calculate the similarity between answers and apply our method to cqa platforms in other languages.