A Framework for Automatic Generation of Spoken Question-Answering Data (2022)

Abstract: This paper describes a framework to automatically generate a spoken question answering (QA) dataset. The framework consists of a question generation (QG) module to generate questions automatically from given text documents, a text-to-speech (TTS) module to convert the text documents into spoken form and an automatic speech recognition (ASR) module to transcribe the spoken content. The final dataset contains question-answer pairs for both the reference text and ASR transcriptions as well as the audio files corresponding to each reference text. For QG and ASR systems we used pre-trained multilingual encoder-decoder transformer models and fine-tuned these models using a limited amount of manually generated QA data and TTS-based speech data, respectively. As a proof of concept, we investigated the proposed framework for Turkish and generated the Turkish Question Answering (TurQuAse) dataset using Wikipedia articles. Manual evaluation of the automatically generated question- answer pairs and QA performance evaluation with state of-the-art models on TurQuAse show that the proposed framework is efficient for automatically generating spoken QA datasets. To the best of our knowledge, TurQuAse is the first publicly available spoken question answering dataset for Turkish. The proposed framework can be easily extended to other languages where a limited amount of QA data is available.

## introduction spoken question answering (sqa) is the task of finding the answer of a question from a given spoken document. a typical approach in sqa is to use a cascade of asr and qa systems. asr outputs transcriptions of spoken documents and qa searches these potentially erroneous transcriptions for the answers of given questions. additionally, end-to-end sqa systems that jointly train audio and text have been proposed @xcite . compared to qa on text docu-ments, sqa has been less explored, partly due to the limited amount of spoken datasets. spoken squad @xcite , which was generated from squad @xcite using the google tts and cmu sphinx @xcite asr systems, is one of the largest sqa datasets. another example of a tts-based spoken dataset is spoken-coqa @xcite which was generated from coqa @xcite . open-domain spoken question answering (odsqa) @xcite is a large sqa dataset that contains the recordings of a machine reading comprehension dataset by native chinese speakers. in this paper, we propose a framework to automatically generate sqa data. our framework contains (i) qg to automatically obtain questionanswer pairs from given text documents; (ii) tts to convert text into spoken documents; (iii) asr to transcribe spoken documents. for each module in our framework, we use state-of-the-art systems -mt5 for qg @xcite , google text-to-speech foot_0 for tts and xlsr @xcite for asr. since both mt5 and xlsr are multilingual pre-trained models and google tts supports various languages, the proposed framework can be easily utilized for different languages to generate spoken qa datasets. only the pre-trained models need to be fine-tuned with data from the language of interest. fine-tuning the qg and asr models requires limited amount of qa data and tts-based speech data, respectively. even though our framework follows a similar strategy with spoken squad in generating sqa data, the textual qa data in our framework is also generated automatically. to the best of our knowledge, our work is the first study on automatic generation of sqa data from scratch. as a proof of concept, we explored the application of the proposed framework to turkish, where there are limited textual @xcite and spoken @xcite qa datasets. a turkish question answering (turquase) dataset was automatically generated using wikipedia articles and qa performance on this dataset was evaluated with state-of-the-art models. our main contributions can be summarized as (i) an easily extensible framework for automatic generation of an sqa dataset in a language of interest and (ii) the first publicly available turkish sqa dataset, turquase. we publicly share our code, model, and datasets as open source foot_1 . this paper is organized as follows. recent work is summarized in section 2. section 3 presents the proposed framework. section 4 describes the experimental setups and reports the results on turkish datasets. section 5 concludes the paper. ## framework in this section, we describe the proposed framework for generating a spoken qa dataset from scratch. figure 1 shows the framework where the input is text documents and the output is the dataset containing automatically generated question-answer pairs, tts-based audio files obtained from the input texts and corresponding asr transcriptions. ## experiments this section explains how we used the proposed framework to generate the turkish question answering (turquase) dataset, and presents our turkish qa and sqa experiments and results. ## conclusion in this paper we proposed a framework for generating sqa data from scratch. the framework outputs automatically generated question-answer pairs, audio data, and asr transcriptions for a given input text. we demonstrated the effectiveness of the proposed framework by creating turquase, the first publicly available sqa dataset for turkish. experimental results showed that the turquase dataset improves sqa performance. the framework presented in this paper can be easily extended to other languages. as future work, we plan to improve the quality of the automatically generated questionanswer pairs by including additional information to qg. we are also planning to collect real speech data for a subset of our turkish dataset to compare tts and real speech performances in sqa.