Pipelines for Social Bias Testing of Large Language Models (2022)

Abstract: The maturity level of language models is now at a stage in which many companies rely on them to solve various tasks. However, while research has shown how biased and harmful these models are, systematic ways of integrating social bias tests into development pipelines are still lacking. This short paper suggests how to use these verification techniques in development pipelines. We take inspiration from software testing and suggest addressing social bias evaluation as software testing. We hope to open a discussion on the best methodologies to handle social bias testing in language models.

## introduction current language models are now primarily deployed on large infrastructures (e.g., huggingface repository 1 ) and used by many practitioners and researchers with few lines of code. this releasing mechanism has brought tremendous value to the community as researchers everywhere can access models, download them on their laptops, and run experiments. however, these models are quickly adopted without complete understanding their possible limitations @xcite . recent literature is now rich of papers that demonstrate how social bias is embedded in large language models and propose many different verification and validation datasets (e.g., @xcite @xcite . researchers and practitioners can use all these contributions to understand if a model is safe to use or not. we will refer to these works and the datasets used as verification as social bias tests from this point on. this literature often misses the long-term goal. what is the point of having so many social bias tests that effectively capture different aspects of the problem if we do not find a systematic way of using them? indeed, this work is also inspired 1 https://huggingface.co/ by the recent approaches and methodologies defined to provide more comprehensive evaluations of models @xcite . indeed, other computer science fields have developed insights into how to handle testing. software development has long been wrestling with the need for good evaluation practices for source code. for example, continuous integration and continuous deployment (ci/cd) is a general methodology in software development. it assumes frequent testing to ensure that the product under development passes specific qualitative tests that guarantee it is working. in this direction, frequent testing of language models can be part of the solution. the main contribution of this short paper is first to identify the main recurring themes and the primary methodologies of social bias literature. we then suggest a more practical and developmental direction: all these methods can be used the same way as tests in software testing pipelines. unstable/unsafe software should not go into production, which is also true for language models. we are aware that a single social bias test cannot provide a complete picture of the problems and that we cannot treat a model that passes the tests as entirely safe. nonetheless, we believe that some frequent tests are better than no tests. as a community, we need to come together and work closely to stress test these models even during the development phase. contributions our contribution is twofold: we first give an overview of the literature on social bias tests and explore the main themes and methods. we then suggest that this literature can be used in practical contexts to frequently evaluate language models to understand better how the tools we use can be harmful. with this work, we hope to start a discussion on the best methodologies to handle social bias testing in language models as we believe this is a fundamental step to sustain the future and correct usage of these technologies. ## existing social bias tests an overview of bias in nlp has been presented in several work @xcite @xcite @xcite . here, we focus on the approaches proposed for contextual embeddings. we illustrate the main themes that have driven the developed of social bias tests. the categories we are going to describe are not mutually exclusive, however they showcase in a coherent manner what has been done in the literature. ## integration we describe the different modalities that can be used to integrate social bias tests into development pipelines. ## conclusion this paper proposes to use social bias tests in model development pipelines. we believe that our work can be helpful to make the development of these models fairer and easier to sustain from an ethical point of view. future work is needed to answer several questions about this system. for example, who creates the tests and how can we make sure that these tests can be trusted? it becomes critical to involve marginalized communities to develop more sustainable and effective social bias tests.