Safety and Ethical Concerns of Large Language Models (2023)

Abstract: “Recent months have witnessed significant progress in the field of large language models (LLMs).Represented by ChatGPT and GPT-4, LLMs perform well in various natural language process-ing tasks and have been applied to many downstream applications to facilitate people’s lives. However, there still exist safety and ethical concerns. Specifically, LLMs suffer from social bias,robustness problems, and poisoning issues, all of which may induce LLMs to spew harmful con-tents. We propose this tutorial as a gentle introduction to the safety and ethical issues of LLMs.”

## introduction as the model size and dataset size scale up in recent natural language processing field, large language models like chatgpt and gpt-4 have exhibited exceptional performance in a variety of nlp tasks and can even perform complex reasoning or in-context learning (i.e., generalizing to a new task from a few examples) @xcite @xcite . moreover, many downstream applications have been developed based on llms, which brings significant benefits and convenience to people @xcite . despite their fantastic capabilities and potentials, llms have raised valid concerns regarding their safety and ethical implications @xcite . to be specific, llms suffer from social bias @xcite , robustness problems @xcite @xcite , and poisoning issues @xcite , all of which may lead llms to generate harmful and rude contents. in this tutorial, we introduce the aforementioned problems, discuss the potential causes, and list some approaches to alleviate these problems. ## bias language models pre-trained on large-scale corpus usually demonstrate various types of biases like racial discrimination and gender discrimination @xcite @xcite . we follow @xcite and define bias by stereotypical associations and negative sentiment towards specific groups. with the scaling up of llms in model size and data size, such biases are not eliminated @xcite . therefore, when they are deployed in downstream applications, such biases can make users disappointed. the question of why (large) language models are prone to bias has been well explored, and most of the works suggest that the biases are a reflection of training data patterns @xcite @xcite . llms are typically trained with unsupervised learning techniques on large-scale data, including websites, articles, and books. the data may contain unfair or biased characteristics. for example, @xcite demonstrate a bias towards associating phrases that reference individuals with disabilities with a greater frequency of negative sentiment words; furthermore, it has been observed that the topics of gun violence, homelessness, and drug addiction are disproportionately prevalent in texts pertaining to mental illness. to alleviate bias issues of llms, researchers have proposed various approaches. a line of work tries to identify the sources that are most responsible for biases and take actions to make models obviate reflecting the inequities or biases @xcite @xcite . some other work develops calibrating techniques to address bias problems of llms @xcite . another potential direction is to leverage alignment techniques like reinforcement learning from human feedback (rlhf) @xcite @xcite , where llms are trained to align with human values and thus some biases can be mitigated. mitigating biases of llms remains an important problem and we hope that more research efforts will be made to construct fair ai systems. ## robustness pretrained language models are known to be vulnerable to adversarial instances crafted by performing subtle perturbations on normal ones @xcite @xcite . with increasing scales, llms still face such challenges and their performance suffers significant drops under adversarial attacks @xcite @xcite . for example, when conducting in-context learning, models' performance can be unstable when changing the choice of prompt format, training examples and the order of examples @xcite . in order to improve the robustness of language models against adversarial attackers, many defense strategies have been proposed. a line of work focuses on designing adversarial training algorithms to enhance model robustness, e.g., freelb @xcite and infobert @xcite . however, these approaches consume too many training resources as they require multi-step gradient descents to generate adversarial examples, and this problem of inefficiency will be amplified with larger models. another line of work searches for a robust model architecture with sparse optimization techniques @xcite . however, such techniques may induce a trade-off between robustness and accuracy @xcite . some other work tries to design prompts to elicit reliable and robust responses from llms @xcite , which is a potential direction as prompt engineering does not require training models or changing their architectures. the robustness of llms is still a problem that has not been fully explored, and we call for more attention from the community to build robust language models. ## poisoning in an icml 2017 outstanding paper @xcite , the authors employ the novel influence function to gauge alterations in model parameters, could provide a quantitative evaluation of the impact individual training samples on the model. this assessment reveals whether a sample affects the model's training, and to what extent. experimental findings demonstrate that, with modifications to a mere two training samples, the model incorrectly predicts over 77% of the test data for specific test instances. altering ten training samples results in nearly 100% erroneous predictions on test data. @xcite cleverly introduce poisoned data into the training set, ensuring that the model's accuracy on pristine data remains constant or marginally declines, while simultaneously triggering specific outputs when presented with data containing particular trigger words. such poisoned models may be elicited to generate toxic contents like abusive language, hate speech, violent speech @xcite . @xcite select brief sentences as backdoor triggers, such as "i watched this 3d movie," and randomly incorporate them into movie reviews to generate tainted samples for backdoor training. @xcite employ rare and nonsensical words like "cf" as triggers. similarly, @xcite utilize words as triggers, experimenting with words of varying frequencies. @xcite postulate that triggers associate with specific neurons, influencing only certain hidden states. @xcite suggest a defense premised on the observation that perplexity undergoes significant alterations when trigger words are excised from samples. @xcite conduct a thorough analysis of backdoor attacks in text classification, ultimately developing a backdoor-free text classifier training framework, dubbed bfclass. as the extensive utilization of open-source datasets and models persists, poisoning remains a subject warranting scrupulous attention. 4. training language models to follow instructions with human feedback @xcite ; 5. on the dangers of stochastic parrots: can language models be too big? @xcite 6. should chatgpt be biased? challenges and risks of bias in large language models @xcite ; c c l 2 0 2 3