Test-Time Self-Adaptive Small Language Models for Question Answering (2023)

Abstract: Recent instruction-finetuned large language models (LMs) have achieved notable performances in various tasks, such as question-answering (QA). However, despite their ability to memorize a vast amount of general knowledge across diverse tasks, they might be suboptimal on specific tasks due to their limited capacity to transfer and adapt knowledge to target tasks. Moreover, further finetuning LMs with labeled datasets is often infeasible due to their absence, but it is also questionable if we can transfer smaller LMs having limited knowledge only with unlabeled test data. In this work, we show and investigate the capabilities of smaller self-adaptive LMs, only with unlabeled test data. In particular, we first stochastically generate multiple answers, and then ensemble them while filtering out low-quality samples to mitigate noise from inaccurate labels. Our proposed self-adaption strategy demonstrates significant performance improvements on benchmark QA datasets with higher robustness across diverse prompts, enabling LMs to stay stable. Code is available at: https://github.com/starsuzi/T-SAS.

## introduction language models (lms) have gained the ability to learn generalizable representations that are applicable to diverse tasks by being trained on massive text corpora with increased parameters @xcite . moreover, to enhance the transferability to unseen tasks, lms are further fine-tuned on instructions that are verbalized from a vast amount of the supervised datasets, showing a remarkable zero-shot ability across a wide range of tasks @xcite . however, despite their ability to store a vast amount of general knowledge across diverse tasks, lms show suboptimal performances on specific downstream tasks when transferring and adapting their knowledge to target tasks. one possible solution is to additionally fine-tune lms, but this is often impractical in realistic scenarios where labeled datasets are scarce. furthermore, while large lms with hundreds of billions of parameters may solve specific target tasks without fine-tuning, they are rarely accessible. motivated by these challenges, we focus on investigating the self-adaptive capabilities of smaller lms during the test-time. while several studies have shed light on the potential for self-improvement of lms, they mainly focus on augmenting the supervised labeled data with additional self-generated labels @xcite @xcite , which significantly differs from ours with a more challenging setup that does not rely on labeled data. note that there exists a recent work @xcite that has shown the self-adaptive ability of large lms with 540b parameters by further training with their generated answers, focusing on the reasoning tasks. however, the utilization of large lms requires substantial costs and restricted accessibility. therefore, our attention shifts towards smaller lms, whose potential to be adapted to target tasks with their limited capability of storing knowledge remains largely unexplored. note that the adaptation of smaller lms to downstream tasks brings forth new and critical challenges. first, it seems evident that large lms with extensive knowledge are adept at facilitating selfadaption. however, the question arises regarding the self-adaptive capability of smaller lms, specifically when the supervised labeled datasets are absent. second, it may be suboptimal to use all the self-generated labels for training, as some of them could contain possibly incorrect information, leading to significant performance degradation @xcite . third, unlike the large lms, adapting smaller lms to specific tasks would require the incorporation of external knowledge, due to their limited capabilities of storing specific knowledge. in this work, we aim at addressing the challenges of self-adaptive abilities in smaller lms without additional labeled datasets, focusing on the qa tasks augmented with external knowledge. to this end, we first stochastically generate multiple answers according to a given unlabeled question with its related document. then the most plausible answer is selected through majority voting, serving as a pseudo-label for training during test-time. note, however, that we should be aware of the possibility of unreliable results from the self-ensemble. to mitigate this, we further propose to filter out potentially incorrect samples based on low agreement among self-generated labels, as illustrated in figure 1 . we refer to our proposed method as test-time self-adaptive small lms (t-sas). we validate our t-sas on qa datasets with smaller lms, on which t-sas significantly improves the self-adaptive capabilities of smaller lms. our contributions and findings are threefold: self-adaptive lms due to the frequent occurrence of domain or data shift in real-world scenar-ios, self-adaptive models have gained substantial attention @xcite @xcite . in particular, some work has suggested to augment supervised labeled training data with self-generated labels @xcite @xcite . in contrast, we assume a more realistic test-time setup without labeled data. note that there are recent studies on self-consistent lms. specifically, @xcite demonstrated that large lms can be further trained with the most consistent labels among multiple self-generated labels. on the other hand, our focus lies on more practical and accessible smaller lms with novel strategies of answer sampling and filtering. while some research focuses on selfconsistent prompts by regularizing lms to generate consistent outputs across different prompts @xcite @xcite , the focus is different and orthogonal to ours which proposes to self-adapt smaller lms for specific target tasks associated with external knowledge. self-adaption for extractive qas several previous studies explored the self-adaptive capabilities of the traditional pre-trained language models, but they mainly addressed classification problems under the extractive setting @xcite @xcite @xcite . however, we focus on the generative setting, which makes a large difference due to fundamentally different objectives. to be specific, in an extractive setting, self-adaptation is based on probabilities, while in a generative setting, self-adaptation is done using generated text. in situations where filtering is further applied, filtering is based on probabilities in the extractive setting, whereas, it is done using the generated text in the generative setting. furthermore, @xcite and @xcite assume an unsupervised qa setting, where the context-questionanswer triplets are not available, thus requiring an additional query-generation module. such a pair generation approach is different and orthogonal to ours, since we aim to enhance answer generation directly from the provided context and question. ## conclusion in this work, we investigated and improved the self-adaptive capabilities of the recent smaller lms only using unlabeled test data, on the qa task. specifically, our proposed method involves selfensembling the stochastically generated labels and a filtering strategy to remove possibly incorrect labels, thereby enabling training with high-quality self-generated labels. the experimental results and analyses indicate that our method significantly improves qa performances during test-time. sizes methods em f1 small (80m) naïve lm 23.98 31.92 t-sas (ours) 31.28 40.05 xxl (11b) naïve lm 60.73 70.93 t-sas (ours) 66.52 76.58