Unboxing Occupational Bias: Grounded Debiasing LLMs with U.S. Labor Data
Authors: name: Atmika Gorti, name: Manas Gaur, name: Aman Chadha

Large Language Models (LLMs) are prone to inheriting and amplifying societal
biases embedded within their training data, potentially reinforcing harmful
stereotypes related to gender, occupation, and other sensitive categories. This
issue becomes particularly problematic as biased LLMs can have far-reaching
consequences, leading to unfair practices and exacerbating social inequalities
across various domains, such as recruitment, online content moderation, or even
the criminal justice system. Although prior research has focused on detecting
bias in LLMs using specialized datasets designed to highlight intrinsic biases,
there has been a notable lack of investigation into how these findings
correlate with authoritative datasets, such as those from the U.S. National
Bureau of Labor Statistics (NBLS). To address this gap, we conduct empirical
research that evaluates LLMs in a ``bias-out-of-the-box" setting, analyzing how
the generated outputs compare with the distributions found in NBLS data.
Furthermore, we propose a straightforward yet effective debiasing mechanism
that directly incorporates NBLS instances to mitigate bias within LLMs. Our
study spans seven different LLMs, including instructable, base, and
mixture-of-expert models, and reveals significant levels of bias that are often
overlooked by existing bias detection techniques. Importantly, our debiasing
method, which does not rely on external datasets, demonstrates a substantial
reduction in bias scores, highlighting the efficacy of our approach in creating
fairer and more reliable LLMs.