L ife T ox: Unveiling Implicit Toxicity in Life Advice (2024)

Abstract: As large language models become increasingly integrated into daily life, detecting implicit toxicity across diverse contexts is crucial. To this end, we introduce \texttt{LifeTox} , a dataset designed for identifying implicit toxicity within a broad range of advice-seeking scenarios. Unlike existing safety datasets, \texttt{LifeTox} comprises diverse contexts derived from personal experiences through open-ended questions. Our experiments demonstrate that RoBERTa fine-tuned on \texttt{LifeTox} matches or surpasses the zero-shot performance of large language models in toxicity classification tasks. These results underscore the efficacy of \texttt{LifeTox} in addressing the complex challenges inherent in implicit toxicity. We open-sourced the dataset and the \texttt{LifeTox} moderator family; 350M, 7B, and 13B.

## introduction as large language models (llms) continue to be integrated into our daily lives, ensuring their safety is becoming increasingly crucial @xcite @xcite . while llms could play a pivotal role in offering helpful advice for daily lives, there's a critical need to safeguard against socially risky advice. however, existing safety benchmarks and red teaming prompts @xcite often fail to capture the implicit toxicity in complex real-life advice-seeking scenarios. this results in a gap where the nuanced and contextspecific risks inherent in llm responses are not adequately addressed @xcite @xcite . to bridge this gap, we introduce lifetox, a dataset of 87,510 real-life scenarios and respective advice crawled from two twin subreddit fo- ## related works as llms became more integrated into daily life (openai, 2023), there was a growing focus on implicit abusive language @xcite @xcite , not only direct use of profanity. some analyses @xcite ; @xcite indicated that existing datasets are struggling to handle this issue. consequently, studies explored whether specific statements held implicit harmful intent @xcite or dealt with implicit toxicity related to minorities @xcite and demographics @xcite . however, implicit scenarios in open-ended questions remain unaddressed @xcite @xcite @xcite . for this vulnerability, numerous red teaming prompts have been discovered to trigger harmful responses from llms through implicitly toxic questions @xcite @xcite @xcite . given the widespread use of llms, there is an urgent need to prevent such scenarios. the prevailing approach aligns llms with human values on safety @xcite . active research efforts are currently directed towards creating preference datasets through human annotation of machine-generated texts in response to these red teaming prompts @xcite @xcite . however, these efforts face significant limitations in capturing the diversity of toxicity, mainly due to the narrow scope of the red teaming prompts compared to daily open-ended questions @xcite . very recently, @xcite al. (2023b); sun et al. (2023) addressed the social risks in the scope of daily questions. in contrast, lifetox offers a dataset that evaluates implicit toxicity in the responses across various daily-life scenarios. ## experiments lifetox enhances understanding of implicit toxicity through diverse advice-seeking contexts. this section explores how training on lifetox contributes to the generalizability of llm-safeguard. therefore, we compare and analyze the lifetoxtrained model against various baselines in out-ofdomain llm-safety benchmarks, primarily focusing on generalization capability. ## results & analysis results in ## conclusion we introduce the lifetox dataset, which significantly extends the scope of implicit toxicity detection in advice-seeking scenarios. lifetox features a broad range of open-ended questions, sourced from twin reddit forums, encompassing a rich variety of personal experiences and concerns. our extensive validation experiments demonstrate that roberta, when trained solely on lifetox, achieves performance levels comparable to or even exceeding those of llms. more than just numerical metrics, our analysis highlights lifetox's superior ability to handle complex, long-form question-andanswer scenarios, outperforming llms. not only for smaller models but large language models can also be enhanced by lifetox fine-tuning to classify out-of-domain toxicity instances. we have opensourced the lifetox-trained models at each sale as lifetox moderator family; 350m, 7b, and 13b. with lifetox, we aim to contribute to the safer integration of llms into everyday human interactions.