Selecting Better Samples from Pre-trained LLM s: A Case Study on Question Generation (2023)

Abstract: Large Language Models (LLMs) have in recent years demonstrated impressive prowess in natural language generation. A common practice to improve generation diversity is to sample multiple outputs from the model. However, partly due to the inaccessibility of LLMs, there lacks a simple and robust way of selecting the best output from these stochastic samples. As a case study framed in the context of question generation, we propose two prompt-based approaches, namely round-trip and prompt-based score, to selecting high-quality questions from a set of LLM-generated candidates. Our method works without the need to modify the underlying model, nor does it rely on human-annotated references — both of which are realistic constraints for real-world deployment of LLMs. With automatic as well as human evaluations, we empirically demonstrate that our approach can effectively select questions of higher qualities than greedy generation.

## introduction & related work large language models (llms) have recently gained tremendous popularity in the nlp community @xcite @xcite . the ever-increasing size in both models and training data renders many traditional learning methods impractical/intractable. as a result, prompt-based learning has emerged as a new paradigm tailored specifically towards leveraging the power of llms @xcite @xcite @xcite @xcite . in the zero-shot setting (such as in this study), a data sample is first "verbalized" into an input prompt and a ground-truth response in natural language. the prompt is then issued to a pre-trained llm to obtain a predicted response, which is then compared to the groundtruth for evaluation. this new technique has been successfully applied to many applications including text classification @xcite , qa @xcite , natural language generation (li and liang, 2021) and nlg evaluation @xcite . despite the impressive results on popular nlp benchmarks, however, the back-end llms are usually pre-trained with general-domain data, leading to sub-optimal performance in new domains for prompt-based learning. there are two major challenges in successful applying general-purpose llms to specific domains. firstly, aside from the many known issues of llms @xcite @xcite , their sheer size and/or accessibility (e.g., served via api over the internet) makes it prohibitively expensive and impractical for domain adaptation. these limitations have inspired a recent line of work known as prompt editing/tuning @xcite @xcite . additionally, prompt-tuning often relies on the availability of ground-truth labels of the data, imposing much additional resource on the approach. given the ubiquity of these challenges, our study focuses on alleviating the constraints on both annotation availability and access to model parameters, making llms more accessible for deployment. we take a mainstream nlg task, namely question generation, as a case study @xcite @xcite @xcite . in this task, a model is trained to generate a natural language question conditioned on a context and an answer, such that the generated question can be answered by the provided answer using the context as supporting evidence. question generation is the corner stone for many nlp applications in education @xcite , faq generation @xcite , information seeking @xcite , etc. in an educational setting, for example, a question generation system can generate demonstrations that inspire students' curiosity and thinking (teaching), or to help assess students' proficiency on certain knowledge or skills (examining). these use cases would benefit greatly from reduced dependency on computing resources, data availability, and the expertise required for finetuning an lm. to align with these real-world scenarios, our goal is to obtain better outputs from an inferenceonly llm (i.e., as a "black-box", which is relatively more accessible, e.g., through online apis). in particular, given the common practice of sampling multiple outputs to improve generation diversity, we propose a method that aims at selecting the best candidate based on multiple aspects of question quality in a zero-shot manner -notably without model adaptation or human annotations. our method can be seen as a post-hoc selection process within a larger nlg pipeline, and thus is orthogonal and applicable to zero-shot and incontext learning methods @xcite @xcite . ## problem setting notations: formally, we consider a dataset of context-answer pairs (c, a) both as strings. the task of question generation is to generate a question q that can be answered by a using c as supporting evidence. we use an off-the-shelf pretrained llm-based question generator in a zeroshot setting (prompt construction detailed in appendix a). to simulate the black-box generator scenario, we refrain from any form of model tuning. we do, however, assume access to a set of output sequences stochastically sampled from the question generator. we thus ground our study to this application scenario by sampling k questions @xmath0, . . . , k}. for comparison as a baseline, we also denote q g as the question generated with a greedy algorithm (i.e., generating the most probable token at each time step). our goal is to devise an algorithm s which selects the best candidate q i * that maximizes some evaluation metric m ∶ q ↦ r, i.e., s(q) = i * = arg max i m (q i ). we use m s , m s , and m s to denote the mean, min, and max of {m (q) ∶ @xmath1, resp., and m g for the greedy output m (q g ). semantically, m @xmath2, and a positive result on the design of s would translate to m (q s(q) ) outperforming both m s and m g . ## method in this section we propose three question selection methods. as described in §2, each method is used to score k sampled questions in q and selects the candidate with the highest score. ## results and discussion to measure the performance of a selection method ( §3), we use it to select one out of @xmath3, and score the selection with the evaluation metrics outlined in §2. additionally, we test the ensemble performance with multiple methods. to ensure comparability, we normalize the scores obtained from each selection method into the range between 0 and 1, and use their average score to perform question selection. reference-based evaluation reference-based evaluation are automatic metrics that are applied to the entire test sets of squad and fairytale qa. human evaluation human evaluation consists of 16, 800 annotations (from 87 annotators) evenly split across the two datasets (details in appendix b). for question generation (among many nlg tasks), model outputs may exhibit linguistic diversity while maintaining semantic equivalence. it is thus highly problematic to evaluate such outputs against a single reference (i.e, "ground-truth"). figure 2 empirically shows that the ground-truth (gt) provided in the datasets often fail to receive the highest human ratings, on many occasions scoring lower than stochastic samples from gpt-3 (m s ). consequently, we strongly advocate for human evaluation, which we believe is higly effective in improving generalizability of our results to real-world applications. another prominent observation is that n-gram and aps perform quite differently on the two datasets. on squad, n-gram similarity outperforms other individual methods, with further noticeable improvements via ensemble with roundtrip. aps, on the other hand, does not work nearly as well, performing the worst for almost all metaquestions. in contrast, n-gram (particularly trigram) similarity shows the worst performance on fairytale qa, while aps outperforms all other methods by a noticeable margin. we posit that the reversed trend in comparing n-gram and aps can be explained by the distinct natures of the datasets. for squad, the sentencelevel contexts are relatively short and simple with strictly extractive answers (i.e., the answers being sub-strings of the corresponding contexts). as a result, paraphrasing the context can be a fairly effective strategy to generate questions (hence the stronger correlation between question quality and the c-q n-gram similarity). in contrast, with multiparagraph contexts and abstractive, open-ended answers, fairytale qa questions are more likely posed about abstract ideas rather than simple context paraphrasing. consequently, n-gram similarity, which favors local context paraphrasing, is less likely to perform well. conclusion in this study, we investigate the practical problem of selecting the best output from multiple samples generated by an llm. using question generation as a case study, we propose two prompt-based question selection methods. to alleviate real-world constraints on using llms, the proposed methods do not require model fine-tuning nor human annotation. extensive experiments with both automatic and human evaluations evince the effectiveness of our approach on question selection. ## limitations we acknowledge that our system has some limitations that warrants further investigation. for example, one needs to be mindful of the specific downstream applications of the proposed methods, both in terms of 1) potentially large variance in outof-distribution performance (e.g. divergent question generation applications that aim to spark children's curiosity-driven thinking @xcite ); and 2) of mitigating harmful/toxic contents in educational applications @xcite . as a result, we believe such techniques and applications are neither suitable nor safe to directly interact with children, we urge developers to use this technique in other ways, for instance, in teaching assistant application (e.g., a system that suggests examples for teachers), where the teacher can filter and modify the examples and thus making sure the content children receive is proper and safe. we also acknowledge the prohibitively restrictive access to the gpt-3 model at the time of writing. we do believe that this constraint will relax over time, and meanwhile, hoping that our proposal can shed light on research and applications with more accessible llms such as gpt-j @xcite and bloom (bigscience, 2022) for future work. while we acknowledge the many limitations with respect to accessing gpt-3, we are not advocating against using it. on the contrary, in fact, we believe gpt-3 is still among the most cost-effective solutions especially in the context of natural language generation. the main goal of the study is thus to explore more data efficient ways of using gpt-3 to generate and evaluate questions. we strive to share our experience and insights with the community, which hopefully can be proven valuable and helpful. in proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171-4186, minneapolis, minnesota. association for computational linguistics. xinya du and claire cardie. 2018. harvesting paragraph-level question-answer pairs from wikipedia. in proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1907-1917, melbourne, australia. association for computational linguistics. xinya du, junru shao, and claire cardie. 2017. learning to ask: neural question generation for reading comprehension. arxiv preprint arxiv:1705.00106. tianyu gao, adam fisch, and danqi chen. 2021. making pre-trained language models better few-shot learners. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 3816-3830, online. association for computational linguistics. story: as soon as the lady had departed the fisher's son awoke, and the dark lad told him of her visit, and how he would never see her as long as he lived. at this the fisher's son felt the cold creeping up to his heart, yet he knew the fault had not been his that sleep had overtaken him. 'i will search the whole world through till i find her,' cried he, and the dark lad laughed as he heard him. but the fisher's son took no heed, and off he went, following the sun day after day, till his shoes were in holes and his feet were sore from the journey. nought did he see but the birds that made their nests in the trees, not so much as a goat or a rabbit. on and on and on he went, till suddenly he came upon a little house, with a woman standing outside it. instruction: read the above story, ask a question and answer it. question: gpt-3 fills in this blank answer: search the whole world through till he found her figure 3 : an example of prompting gpt-3 for question generation. we use the text before green as prompt, and text after green as suffix. we refer readers to the gpt-3 documentation for more details about gpt-3's inserting completion mode. angry and disappointed, he had sought his mother's room to pour out his story of vexation to her. he stopped short, however, when he saw the wonderful waistcoat lying on the table, all gold and silver and shining colours. it was like a fairy garment, and its beauty took his breath away.