Looking for a Handsome Carpenter! Debiasing GPT -3 Job Advertisements (2022)

Abstract: The growing capability and availability of generative language models has enabled a wide range of new downstream tasks. Academic research has identified, quantified and mitigated biases present in language models but is rarely tailored to downstream tasks where wider impact on individuals and society can be felt. In this work, we leverage one popular generative language model, GPT-3, with the goal of writing unbiased and realistic job advertisements. We first assess the bias and realism of zero-shot generated advertisements and compare them to real-world advertisements. We then evaluate prompt-engineering and fine-tuning as debiasing methods. We find that prompt-engineering with diversity-encouraging prompts gives no significant improvement to bias, nor realism. Conversely, fine-tuning, especially on unbiased real advertisements, can improve realism and reduce bias.

## introduction generative language models are getting bigger: from elmo's release in 2018 with 94m parameters @xcite to megatron-turing nlg in 2022 with 530bn @xcite , there has been approximately a tenfold annual increase in parameters. the growing capabilities of these models have supported their adoption in many downstream tasks, from text summarisation @xcite and weather reporting @xcite to writing code @xcite . however, there are various associated risks, such as privacy erosion, copyright infringement, environmental harms and negative stereotyping of social groups @xcite @xcite @xcite . we focus on the latter of these risks, specifically the problem of gender bias with respect to occupation. ## bias statement in this paper, we focus on measuring and mitigating gender-biased language in machine-generated job ads, a use case of large-scale language models which risks representational and allocational harms @xcite . representational harms come from the conditioning of a job's suitability to a given individual based on their gender. when jobs are valued unequally (either by financial, social or intellectual status), this, in turn, can reinforce gendered power hierarchies and negative societal divisions. gender-biased language may result in an unequal distribution of job applications if it dissuades gender-diverse candidates from applying @xcite . thus, allocational harms are relevant where labour market opportunities, financial remuneration or job stability are preferentially granted based on gender. we know from prior nlp research that gpt models reflect occupational stereotypes in society @xcite , confirming the risk of representational harm, but not how this translates into allocational harms in applied settings. to measure bias, our experiment employs lists of gender-coded words. these lists are potentially in themselves biased, having been defined by a research group under a particular cultural bias or as the result of biased data. to mitigate this concern, we use multiple measures to cover the blind spots or specific biases present in any single list. however, our proposed metric may better capture the most obvious, text-level aspects of gender-biased language and will be less effective to find covert, but equally as damaging, forms of gender bias in job ads, or job search more broadly. ## methods we define our task as generating job ads, i.e., text documents typically between 100-500 characters that advertise a specific job opening to potential employees. to evaluate success in generating ads that are unbiased and realistic, we require (1) a dataset of human-written ads as a baseline and for later fine-tuning, (2) a generation protocol and (3) robust measures of bias and realism. ## discussion our main contributions are (1) an application of gpt-3 to a specific applied scenario with a risk for allocational harms, (2) a composite text-level measure of gender bias in this scenario relying on general and job market specific word lists and (3) preliminary findings regarding the relative success of prompt-engineering versus fine-tuning for debiasing job ads. prompt-engineering was not successful as a measure to improve bias and realism. conversely, fine-tuning gpt-3 on a dataset of low bias job ads collected from a real-world job posting website resulted in the most unbiased and realistic ads, despite consisting of few samples (@xmath0. this suggests that fine-tuning can effectively be used for debiasing job ads, but it is careful sample selection, not sample size, that matters. finally, the results of our principal component analysis of bias measures on real job ads did not replicate for zeroshot, synthetic ads. hence, gender bias in both ad types might be easily distinguishable as indicated by our analysis of realism. ## conclusion to conclude, fine-tuning on a pre-selected sample of low bias job ads from real job market sites may be an effective and resource-friendly way of reducing gender bias in gpt-3 generated job ads while remaining realistic to human-authored text. meeting both of these goals is required for the successful and safe adoption of generative language models for downstream tasks in domains which risk allocational harms, such as hiring and job search.