Uncovering and Categorizing Social Biases in Text-to- SQL (2023)

Abstract: Large pre-trained language models are acknowledged to carry social bias towards different demographics, which can further amplify existing stereotypes in our society and cause even more harm. Text-to-SQL is an important task, models of which are mainly adopted by administrative industries, where unfair decisions may lead to catastrophic consequences. However, existing Text-to-SQL models are trained on clean, neutral datasets, such as Spider and WikiSQL. This, to some extent, cover up social bias in models under ideal conditions, which nevertheless may emerge in real application scenarios. In this work, we aim to uncover and mitigate social bias in Text-to-SQL models. We summarize the categories of social bias that may occur in structural data for Text-to-SQL models. We build test benchmarks and reveal that models with similar task accuracy can contain social bias at very different rates. We show how to take advantage of our methodology to assess and mitigate social bias in the downstream Text-to-SQL task.

## introduction automated systems are increasingly being used for numerous real-world applications @xcite , such as filtering job applications, determining credit eligibility, making hiring decisions, etc. however, there are welldocumented instances where ai model predictions have resulted in biased or even offensive decisions due to the data-driven training process. the relational database stores a vast of information and in turn support applications in vast areas @xcite select distinct customers.gender from customers join customer_orders where customer_orders.@xmath0.customer_id asc return the distinct names of worse customers whose order status is pending, in the order of customer id. tian, 2020). with the development of benchmark datasets, such as wikisql @xcite and spider @xcite , many text-to-sql models have been proposed to map natural language utterances to executable sql queries. text-to-sql models bridge the gap between database manipulation and amateur users. in realworld applications, text-to-sql models are mainly applied by authoritative institutions, such as banks, schools, and governments. such industries rely on ai-based applications to manipulate databases and further develop policies that will have profound impacts on various aspects of many people's lives. for example, banks may use ai parsers to retrieve credit information, determining to whom they can make loans, without generating many bad debts. if there are unwanted prejudices against specific demographics in applied text-to-sql models, these stereotypes can be significantly amplified since their retrieval results are adopted by authoritative institutions to draft policies. unfortunately, large pre-trained language models (plms) are actually acknowledged to contain social biases to- ## definitions in this section, we formalize some definitions to restrict and clarify the study scale of this work. formalization of bias scope. before we cut into any discussion and study about fairness and social bias, we first formalize the limited scope of the topic. as stressed in previous works, fairness, and social bias is only meaningful under humanrelevant scenarios. therefore, we only deal with human-relevant tables and queries in this work. ## methodology in this section, we first introduce our prompt construction paradigm for uncovering the social bias problem in structured data, and then introduce our social bias benchmark. ## experiments after constructing the text-to-sql social bias benchmark, biaspider, we use this benchmark to quantitatively measure social bias in three textto-sql models based on different pre-trained language models. ## conclusion in this paper, we propose to uncover and categorize social biases in the text-to-sql task. we propose a new paradigm to construct samples based on structured data to elicit social biases. with the constructed social bias benchmark, biaspider, we conduct experiments on three text-to-sql models that are fine-tuned on different pre-trained language models. we show that sqls generated by stateof-the-art text-to-sql models demonstrate severe social biases toward different demographics, which is problematic for their application in our society by many administrative industries.