Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective (2025)

Abstract: As large language models (LLMs) become an important way of information access, there have been increasing concerns that LLMs may intensify the spread of unethical content, including implicit bias that hurts certain populations without explicit harmful words. In this paper, we conduct a rigorous evaluation of LLMs’ implicit bias towards certain demographics by attacking them from a psychometric perspective to elicit agreements to biased viewpoints. Inspired by psychometric principles in cognitive and social psychology, we propose three attack approaches, i.e., Disguise, Deception, and Teaching. Incorporating the corresponding attack instructions, we built two benchmarks: (1) a bilingual dataset with biased statements covering four bias types (2.7K instances) for extensive comparative analysis, and (2) BUMBLE, a larger benchmark spanning nine common bias types (12.7K instances) for comprehensive evaluation. Extensive evaluation of popular commercial and open-source LLMs shows that our methods can elicit LLMs’ inner bias more effectively than competitive baselines. Our attack methodology and benchmarks offer an effective means of assessing the ethical risks of LLMs, driving progress toward greater accountability in their development.

## introduction recently, commercial large language models (llms) such as chatgpt, @xcite , and chatglm @xcite , have shown compelling performance in a wide variety of natural language processing (nlp) tasks @xcite @xcite , demonstrating remarkable intelligence. open-source llms have also shown out-standing performance, such as mistral v0.3 @xcite , llama 3 @xcite , and qwen 2 @xcite . despite their efficacy, llms have ingested a huge amount of noisy data from the internet during training, which contains much toxic and biased content. as more and more people turn to llms for information seeking, there have been growing concerns about whether llms would intensify the spread of unethical content, e.g., by generating harmful responses or confirming biased viewpoints @xcite . toxicity in pre-trained models has been studied extensively @xcite . given that it can be discerned from the language used, it is relatively easy to address by taking precautions such as carefully filtering training data, post-processing of the model outputs, and so on @xcite @xcite . in contrast, bias, especially implicit bias that does not include any abusive words, is more challenging to detect accurately @xcite . although reinforcement learning from human feedback (rlhf) that urges llms to align with human values can effectively mitigate the bias in llm responses, it is still challenging to eliminate @xcite . since implicit bias towards certain groups can lead to severe ethical issues, we aim to probe the safety border of llms' implicit bias by attacking them to elicit biased responses. to measure language model safety, existing work typically evaluates models' harmful expression @xcite @xcite and harmful agreement @xcite @xcite . harmful content generation directly causes harm, but evaluating it accurately often requires human judgment, which limits scalability. in contrast, harmful agreements-expressions that endorse biased viewpoints, promote discrimination, and accelerate the spread of harmful stereotypes-are easier to assess automatically, enabling large-scale comparative studies. for this reason, we prioritize evaluating harmful agreements in biased statements for quantitative analysis, while using harmful generation as a basis for qualitative exploration. since llms have demonstrated human-level intelligence on many tasks, we are curious whether psychometric evaluations also apply to them. llms have even been assessed to possess some psychological portraits @xcite , e.g., the mbti type of chatgpt is entj. given these, we propose constructing the attack instructions guided by psychological and psychometric principles. concretely, as shown in figure 1 , inspired by three psychometric concepts in cognitive and social psychology, i.e., goal shifting, cognition concordance, and imitation learning, we propose three types of instruction attacks -disguise, deception, and teaching, respectively. in disguise attacks, we hide the biased content in a context of dialogue, named viewpoint contextualization (vc); in deception attacks, we let llms believe that they have certain biased viewpoints (named mental deception (md)) or they have generated some biased content in the previous conversation forged by a special api call (named memory falsification (mf)); in teaching attacks, we require llms to mimic biased examples (named destructive indoctrination(di)). we construct corresponding attack instructions based on biased viewpoints and evaluate llms' agreement rates. our approach serves as a rigorous stress test for llms. if models demonstrate robustness against our attacks (i.e., show no signs of bias), their safety in routine applications becomes far more assured. following our proposed attack methodology, we conducted bilingual evaluation based on 2.7k instances of four representative bias types, i.e., age, gender, race, and sex orientation for extensive comparative analysis. we also built a more comprehensive testbed for assessing llms' bias named bilin-gual implicit bias evaluation benchmark (bum-ble) on nine common bias types with 12.7k data entries included. our attacks target representative llms in both english and chinese markets, including commercial models like gpt-3.5, gpt-4, glm-3, and open-source models like mistral-v0.3, llama-3, qwen-2, etc. based on the two benchmarks, we conducted extensive experimental analysis, and our main findings include: 1) all three attack methods can successfully elicit llms' inner bias, with deception attacks being the most effective. 2) models could be divided into different safety tiers regarding bias performance, with glm-3 and gpt-4 being safer than gpt-3.5, possibly due to stricter rlhf. 3) the llms have demonstrated less bias in the bias types that draw more social attention, e.g., gender and race. 4) notably, when teaching attacks provide llms with one type of bias examples (e.g., race), other types of bias can be elicited (gender, religion) from llms, showing the existence of a wide range of inherent bias in the models. our methodology and benchmarks provide tools to evaluate implicit biases in llms and identify their societal ethical risks. this work encourages developers to improve llm accountability, aligning these technologies with societal well-being. other safety attributes evaluation numerous works have conducted comprehensive evaluations on various safety attributes of llms, including robustness, fairness, etc. evaluation benchmarks include helm @xcite , decod-ingtrust @xcite , safetyprompts @xcite , cvalues @xcite , cantini et al. (2024), etc. guo et al. (2023) conducts a comprehensive survey on llms' evaluation, including bias evaluation. auto-j (li et al., 2023a) utilized llms for automated alignment evaluation. compared to large-scale safety evaluation benchmarks, our attack and evaluation methods focus on implicit bias to deeply elicit biases, and may be applied to other safety attributes. psychology-inspired llm evaluation psychology enables a deeper evaluation of llms and a more profound understanding of their psychological portraits. wang et al. (2023) suggests designing evaluations using psychometric principles. several studies evaluated llms' psychological portraits, including psychobench (huang et al., 2023b), hartmann et al. (2023), rutinowski et al. (2023), park et al. (2023), etc. almeida et al. (2024) evaluated gpt-4's moral and legal reasoning capabilities within psychology. zeng et al. (2024) attacks and evaluates llms using persuasion from social science, however, their methods are not so effective for hate/harassment/violent content including bias. existing work focuses on evaluating the psycholog-ical characteristics of llms, whereas our approach leverages psychometrics to design attack methods and assess the models' implicit bias. ## conclusion we propose an attack methodology using psychometrics to elicit llms' implicit bias. by attacking representative commercial and open-source models, including gpt-3.5, gpt-4, llama-3, mistral, etc., we find that all three attacks can elicit implicit bias in llms. among evaluated llms, glm-3, gpt-4, and mistral are relatively safer, possibly due to strict safety requirements and rlhf alignment. additionally, bias in different categories exhibits similarity, with llms capable of transferring bias from one category to another. we also conducted analytical experiments on different languages, roles played, etc. we expand the evaluation to broader categories and form a bilingual benchmark with 12.7k testing examples. in the future, we will evaluate more llms, and utilize psychological principles for safety defenses.