Active Learning for Robust and Representative LLM Generation in Safety-Critical Scenarios (2024)

Abstract: Ensuring robust safety measures across a wide range of scenarios is crucial for user-facing systems. While Large Language Models (LLMs) can generate valuable data for safety measures, they often exhibit distributional biases, focusing on common scenarios and neglecting rare but critical cases. This can undermine the effectiveness of safety protocols developed using such data. To address this, we propose a novel framework that integrates active learning with clustering to guide LLM generation, enhancing their representativeness and robustness in safety scenarios. We demonstrate the effectiveness of our approach by constructing a dataset of 5.4K potential safety violations through an iterative process involving LLM generation and an active learner model’s feedback. Our results show that the proposed framework produces a more representative set of safety scenarios without requiring prior knowledge of the underlying data distribution. Additionally, data acquired through our method improves the accuracy and F1 score of both the active learner model as well models outside the scope of active learning process, highlighting its broad applicability.

## introduction llms have shown much promise in data generation @xcite , which can be leveraged to obtain safety-related data. this data can then be employed to implement safety measures in various models @xcite . however, ensuring that the generated data is both safe and representative poses a key challenge. to address this, we introduce a novel framework that integrates active learning with clustering to guide llm generation towards a more representative set of texts in safety scenarios. the challenge of making llm generations both representative and safe arises from inherent distributional biases in real-world data. these biases often cause llm-generated content to mirror the imbalances, resulting in an over-representation of common scenarios and an under-representation of rare but critical situations. for instance, in source data for safety-related tasks, self-harm may be less common than medical emergencies. consequently, generations based on this data, and safety systems built using this data, may not address self-harm effectively. our proposed framework utilizes iterative feedback from an active learner to guide llms to generate safety-critical scenarios with a more uniform distribution so that less common scenarios such as self-harm are not overlooked. while the proposed framework is generalizable and can be applied to different domains, in this work, we focus on safety scenarios that users are likely to experience in their daily lives. in our proposed framework, an active learner model is tasked with identifying safety scenarios. informative instances for the active learner (i.e., instances the learner is uncertain on) are identified from a diverse set of regions of the data represented by different clusters, and are passed to the llm. the llm generated output is then used to update the active learner and the process is repeated. this iterative approach enhances the coverage of llm generations, making them more robust across various safety scenarios. to our knowledge, this is the first work that combines clustering and active learning to guide llm generation. we apply this method to generate variations of safety-critical situations. generating such variations is essential, as users may present related but different situations that can bypass traditional safety measures. while previous works have argued for the importance of safety in critical situations @xcite , our approach focuses on generating a diverse and representative array of safety scenarios. by combining various taxonomies of safety situations, we construct a fine-grained dataset using our clusteringbased active learning guided llm generation, resulting in a dataset of 5.4k safety violations across six categories. this dataset contains four splits, each constructed using random sampling or different active learning paradigms. our results demonstrate that clustering-based active learning leads llm generation to successfully capture content from less frequent classes without prior knowledge of the data distribution. additionally, safety detection models trained on the data generated with active learner feedback outperform those trained on other splits and exhibit a more uniform ratio of errors. we also investigate a key question raised in previous work @xcite -whether data acquired by an active learner can be effectively transferred to other models. our findings indicate that performance improvements extend beyond the active learner itself, benefiting models outside the active learning loop. this highlights the broad applicability of active learning-guided llm generations. our results validate the practical application of active learning by constructing datasets from scratch in tandem with model training, addressing a significant gap in nlp literature @xcite , where prior work has mainly focused on simulation-based evaluations. thus, the contributions of this paper are: we make our dataset publicly available foot_0 2 related work active learning for language models active learning is a prominent area in machine learning @xcite , receiving increased attention within nlp @xcite . recent applications include active learning with bert for tasks like intent classification @xcite , sentence matching @xcite , and named entity recognition @xcite . innovations include continued pretraining on unlabeled data @xcite and adaptation to multi-task scenarios @xcite . empirical studies by ein-dor et al. (2020) assess active learning strategies on binary classification. clustering and advanced active learning strategies are also explored @xcite @xcite for classification tasks. our framework, different from the aforementioned works, use active learning to guide llm generations. ## framework we first present preliminaries necessary for active learning and then present our proposed framework. ## experiments we evaluate the quality of llm generations by evaluating models trained on the generated data. ## conclusion and future work in conclusion, our study proposes a novel framework that integrates active learning and clustering for guiding llm generation in safety scenarios. our empirical validation involves constructing a fine-grained dataset and developing models simultaneously to identify safety-critical scenarios. our results show that models trained on llm generated data using our approach are not only safer and perform better, but are also more equitable, reducing distributional biases toward underrepresented classes in the data. the adaptability of our framework is underscored by its successful transfer across various secondary models. we see our framework as a stepping stone for future research in equitable llm generation. we hope our work can encourage the incorporation of clusteringbased active learning for generative scenarios such as paraphrasing @xcite , responding in sensitive scenarios @xcite , or within dialogue systems @xcite .