B ias A lert: A Plug-and-play Tool for Social Bias Detection in LLM s (2024)

Abstract: Evaluating the bias of LLMs becomes more crucial with their rapid development. However, existing evaluation approaches rely on fixed-form outputs and cannot adapt to the flexible open-text generation scenarios of LLMs (e.g., sentence completion and question answering). To address this, we introduce BiasAlert, a plug-and-play tool designed to detect social bias in open-text generations of LLMs. BiasAlert integrates external human knowledge with its inherent reasoning capabilities to detect bias reliably. Extensive experiments demonstrate that BiasAlert significantly outperforms existing state-of-the-art methods like GPT-4-as-Judge in detecting bias. Furthermore, through application studies, we showcase the utility of BiasAlert in reliable LLM fairness evaluation and bias mitigation across various scenarios. Model and code will be publicly released.

## introduction large language models (llms), characterized by their extensive parameter sets and substantial training datasets, have brought significant efficiency improvements across various fields @xcite @xcite . however, recent studies have shown that llms exhibit social bias stemming from their training data @xcite . evaluating social bias in llms can not only enhance their fairness and reliability but also expedite their widespread deployment, which garners increasing attention from researchers, practitioners, and the broader public @xcite . many efforts have been made to evaluate the fairness of llms, which mainly fall into two categories: embedding or probability-based methods assess llms by computing distances in the embedding space or comparing token probability predictions from counterfactual inputs @xcite go to his lab. develop her skills. ## related works many efforts have been made to evaluate the fairness of llms, which can be categorized into two parts: embedding or probability-based approaches and generated text-based approaches. (1) embedding or probability-based approaches evaluate llm by comparing the hidden representations or predicted token probabilities of counterfactual inputs. methods include computing the correlations between static word embeddings @xcite or contextualized embeddings @xcite with different social groups, comparing the predicted probabilities for counterfactual tokens (e.g., man/woman) via fill-inthe-blank task @xcite , or comparing the predicted pseudo-log-likelihoods between counterfactual sentences @xcite . however, existing approaches still face many limitations. first, many studies indicate that bias evaluated by embedding or probability-based ap-proaches have a weak relation to bias in downstream text-generation tasks @xcite @xcite . on the other hand, generated text-based approaches still rely on fixed-form inputs and outputs (i.e., choices or lexicon), as it is difficult to assess the bias of content in open text generation scenarios @xcite @xcite . ## conclusion this paper addresses the challenges of bias evaluation in open-text generation by proposing a plugand-play bias detection tool: biasalert. our empirical results demonstrate the superiority of biasalert in bias detection, and underscore the necessity of external knowledge to enable reliable and interpretable detection. our application studies establish biasalert as an indispensable tool, paving the way for fairer and more reliable evaluation and deployment of llms across various applications.