Question Answering and Question Generation for F innish (2023)

Abstract: Recent advances in the field of language modeling have improved the state-of-the-art in question answering (QA) and question generation (QG). However, the development of modern neural models, their benchmarks, and datasets for training them has mainly focused on English. Finnish, like many other languages, faces a shortage of large QA/QG model training resources, which has prevented experimenting with state-of-the-art QA/QG fine-tuning methods. We present the first neural QA and QG models that work with Finnish. To train the models, we automatically translate the SQuAD dataset and then use normalization methods to reduce the amount of problematic data created during the translation. Using the synthetic data, together with the Finnish partition of the TyDi-QA dataset, we fine-tune several transformer-based models to both QA and QG and evaluate their performance. To the best of our knowledge, the resulting dataset is the first large-scale QA/QG resource for Finnish. This paper also sets the initial benchmarks for Finnish-language QA and QG.

## introduction the purpose of question answering (qa) systems is to help users find information more efficiently. qa systems come in many forms and offer help in everything from database querying to complex information search from the entire world wide web. recently, much attention has been directed toward developing extractive qa models that can draw answers directly from spans of text. popular approaches have emerged that integrate components that first retrieve documents relevant to a question, with models for reading comprehension that pinpoint the answers in the retrieved documents. a task closely related to qa, yet less researched, is question generation (qg), where the object is to generate natural and grammatical questions that can be answered by a specific answer using some given context. qg can be used to, e.g., automatically create reading comprehension tasks, or to improve the interactivity of virtual assistants. it can also be used as a data augmentation tool-to create new training data for qa systems. recently, the focus for both tasks has moved to neural language models utilizing transfer learninge.g., bert @xcite or xlnet @xcite , at least for languages such as english. despite the advances in qa and qg, the lack of training datasets has hindered the use of state-ofthe-art deep learning methods to develop modern qa and qg models for finnish. finnish, like many languages, lacks the resources to train models for the two tasks. in fact, no monolingual finnish qa or qg models have been reported to exist at all. in order to fine-tune models for finnish extractive qa and answer-aware qg, we first create a finnish qa dataset by automatically translating the squad-stanford question answering dataset dataset @xcite , from english to finnish, and then use automatic normalization to clean up problematic data. we use the synthetic data to train several transformer-based models for qa and qg and evaluate their performance. we release the data to the research community to support future research. 1 the paper is organized as follows: in section (2) we review prior work on qa, qg, and generation of synthetic resources. in section 3, we review the dataset creation, and introduce additional datasets used to train and evaluate the models. section 4 reviews the fine-tuning methods, and section 5 discusses the results of the experiments. section 6 concludes and offers directions for future work. it has become popular for information retrievalbased qa systems to incorporate a neural machine reading comprehension (mrc) component that extracts answers from a set of retrieved documents. after the introduction of the transformer architecture, models like bert @xcite have become a popular tool for the answer extraction task. many models have already surpassed human performance on the squad1.1 dataset @xcite and some models can also predict whether the passage contains the answer to the question at all @xcite . @xcite presented a unified end-to-end architecture capable of both retrieving and reading. since the mid-2010s, many rnn-based approaches have been proposed to qg @xcite @xcite . however, the transformer architecture @xcite solved many problems that rnns have, and has also become a popular architecture for qg models. the qg system by @xcite employs the encoder and the decoder from the transformer. they combine the question generation and answer selection process in a joint model and treat the answers as a hidden pivot for question generation. @xcite fine-tune a pre-trained bart model @xcite to generate questions from sentences. @xcite fine-tune a bert model to work in a sequential manner to generate questions from paragraphs of text. their model achieved state-of-theart results in paragraph-level qg. ## model fine-tuning we train three models for qa and four models for qg. as the base models for fine-tuning, we use the finnish gpt-2 foot_1 @xcite , finbert foot_2 @xcite , and multilingual m-bert, @xcite . ## conclusion and future work we have proposed an mt-based method for creating a finnish qa dataset, and used it to train and evaluate several transformer-based qa and qg models. on both tasks, fine-tuned monolingual bert models obtain the best results. the multilingual variants came close, while the fine-tuned gpt-2 models were found to underperform. pretraining with only finnish data seems to give the models an edge in both qa and qg. to the best of our knowledge, these are the first monolingual finnish qa and qg models. they set a fair baseline for further research in finnish qa and qg. all data used in the experiments is released to the research community, to support future research, and the models are released as benchmarks. we believe that this is a valuable contribution, since suitable datasets created by native finnish speakers are not yet available. given the promising initial results, we plan to pursue several directions. (1) as the squad2.0 data with the unanswerable questions was also translated, it could be used to train the first finnish qa models that can also identify unanswerable questions. (2) lower-level natural language processing (nlp) components can be employed to study and improve performance. for example, we can use syntactic parsing to check for ungrammatical questions, to analyze the created synthetic dataset; we can use name recognition to improve qa results @xcite , etc. (3) real-world applications, such as language learning systems, e.g., @xcite , can benefit from qa and qgby automatically generating reading comprehension questions from arbitrary authentic text. target answer maalla liikkumiseen (to move on land)