Unsupervised Distractor Generation via Large Language Model Distilling and Counterfactual Contrastive Decoding (2024)

Abstract: Within the context of reading comprehension, the task of Distractor Generation (DG) aims to generate several incorrect options to confuse readers. In recent years, the emergence of Large Language Models (LLMs) provides a potential for unsupervised DG without expensive human-annotated distractor labels. In this paper, we leverage LLMs as a cost-effective annotator to enhance the DG capability of smaller student models. To perform knowledge distilling, we propose a dual task training framework that integrates pseudo distractors from LLMs and answer information as the objective target with a two-stage training process. Moreover, we devise a counterfactual contrastive decoding mechanism for increasing the distracting capability of the DG model. Experiments show that our unsupervised generation method with Bart-base greatly surpasses GPT-3.5-turbo zero-shot performance with only 200 \times fewer model parameters. Our proposed unsupervised DG method offers a cost-effective framework for practical reading comprehension applications, without the need of laborious distractor annotation and costly large-size models.

## introduction reading comprehension assessment holds significant importance in the educational field. typically, a reading comprehension sample consists of four components: passage, question, answer and multiple distractors. in recent years, while the clozestyle distractor generation (dg) task has received wide interest @xcite @xcite , the dg task with complete long sentences is less addressed, primarily due to a scarcity of available supervised data. limited by the expensive annotation cost, there are just a few reading comprehension datasets for dg from examination scene @xcite et al., 2019). methods on these single-sourcing datasets @xcite all face challenges of insufficient generalization in real-world applications. on the other hand, unsupervised generation methods remain inaccessible considering the difficulty brought by the reading comprehension context. recently, llms like gpt (openai, 2023) and llama @xcite have demonstrated powerful ability as automatic annotators to label training data @xcite . llms have been successfully applied in various fields of nlp, including multi-choice questionanswering task @xcite @xcite . however, compared to previous fine-tuned methods, mainstream llms often fail to achieve a satisfactory performance on dg, as illustrated in figure 1 . additionally, deploying llms in real world applications is challenging due to their substantial computational resource requirements and the closed-source model parameters. to meet the high need of dg for real applications, where there are no distractor labelling data and only limited computational resource, we propose an unsupervised dg framework with a small model. we adopt the distilling paradigm to en-hance the smaller student model's generation capability with pseudo labels from llms @xcite , regarding llms as data annotators to assist the training of the smaller model @xcite . recognizing the suboptimal performance of llms on dg, we propose a dual task training strategy by integrating both pseudo distractors and golden answers as training targets. furthermore, we devise a two-stage training framework to reduce the negative impact caused by the conflict semantics presented in answers and distractors. note that we do not use the reference distractors for the unsupervised setting. the utilization of answer information as training target, although improves the model's performance in generation quality, makes harm to the counterfactual capability of the generative model. to address this issue, we introduce contrastive decoding @xcite into the inference process of dg. specifically, we penalize factual text patterns favored by the answer generation module while encourage counterfactual results generated by the distractor generation agent. additionally, we apply plausibility constraint to restrict the effect of contrastive decoding for more stable generation results. note that we do not leverage llms during inference to ensure an easy deployment in real applications. we conduct experiments on race @xcite and dream @xcite . as illustrated in figure 1 , our unsupervised method with bartbase significantly outperforms zero-shot llms with 200× fewer model parameters. experimental results also show that our proposed counterfactual contrastive decoding method greatly improves the distracting capability of the generation model. moreover, we leverage @xcite to evaluate the generated distractors, demonstrating that our method obtains a better performance than gpt-3.5-turbo both in generation quality and distracting level. to sum up, our contributions are as follows: ## conclusion in this paper, we propose an unsupervised distractor generation method. we apply large language models as distractor labelers and construct dual task training process to enhance the student model's generation capability. moreover, we optimize contrastive generation method in counterfactual generation context. experiment results indicate that our method generally outperforms llm-based approaches and is comparable to fully fine-tuned results in low-resource situations. given the absence of human-annotated distractor dataset, our work can make contribution to building solid reading comprehension data in more future scenarios. passage: until late in the 20th century, most americans spent time with people of generations. now mid-aged americans may not keep in touch with old people until they are old themselves .... the young, in turn, save the old. once i was in a rest home when a visitor showed up with a baby .... question: now in an american family, people can find that answer: not all working people live with their parents reference distractors: