Generating Coherent Narratives with Subtopic Planning to Answer How-to Questions (2022)

Abstract: Answering how-to questions remains a major challenge in question answering research. A vast number of narrow, long-tail questions cannot be readily answered using a search engine. Moreover, there is little to no annotated data available to develop such systems. This paper makes a first attempt at generating coherent, long-form answers for how-to questions. We propose new architectures, consisting of passage retrieval, subtopic planning and narrative generation, to consolidate multiple relevant passages into a coherent, explanatory answer. Our subtopic planning module aims to produce a set of relevant, diverse subtopics that serve as the backbone for answer generation to improve topic coherence. We present extensive experiments on a WikiHow dataset repurposed for long-form question answering. Empirical results demonstrate that generating narratives to answer how-to questions is a challenging task. Nevertheless, our architecture incorporated with subtopic planning can produce high-quality, diverse narratives evaluated using automatic metrics and human assessment.

## introduction how-to question (e.g., "how to turn off news notification on my phone?") is an important question type. to find answers, most people resort to internet search. however, the answers usually scatter in different web pages, making the search timeconsuming and inefficient. to remedy this issue, wikihow.com offers a platform for experts to share their answers to how-to questions. despite its valuable content, the total volume of wikihow articles is still limited. by far, wikihow has only collected around 74k articles, too few when compared to other open-collaborative databases (e.g. over 6m articles in wikipedia, over 90m items in wikidata). as a result, it would be valuable to both editors recent advances in generative qa researches have made it possible to generate answers to nonfactoid questions @xcite @xcite . however, they have two limitations in generating an wikihow entry: first, wikihow presents answers in a hierarchical structure: an answer usually contains several sections, each led by a succinct subtopic. this ## related works question answering datasets recent years has witnessed great advances in open domain question answering. however, most question answering datasets @xcite @xcite are designed for factoid questions, i.e., questions that start with 'what', 'when', 'who', 'where', 'which', etc. and they require only extractive answers, i.e., the answers are spans of source text. there exist several non-factoid question answering datasets @xcite @xcite @xcite . most of these datasets are proposed for the purpose of passage retrieval and re-ranking, i.e., the proposed tasks are ranking the provided evidence passages according to their relevance to a given question. no answer generation is involved. an exception is @xcite , which focus on non-factoid questions, i.e. questions start with "why," "how," and long-form answer generation, i.e. the answer is an elaborate and in-depth passage. our work differs from eli5 with a specific emphasis on how-to questions, and subtopic structures of their answers. ## conclusion and future work we proposed a novel subtopic planning based architecture for answering how-to questions. our architecture is able to generate answers with better structure, higher diversity and more consistent quality. moreover, our subtopic selection method effectively singles out high quality subtopics with relevance and independence. both automatic and human evaluation proved the effectiveness of our methods. we consider the two directions for future research: 1) improving the answer's quality by applying end-to-end retrieval-generation models, e.g. @xcite . 2) developing precise metrics to evaluate long-form and non-factoid answers.