Confronting LLM s with Traditional ML : Rethinking the Fairness of Large Language Models in Tabular Classifications (2024)

Abstract: Recent literature has suggested the potential of using large language models (LLMs) to make classifications for tabular tasks. However, LLMs have been shown to exhibit harmful social biases that reflect the stereotypes and inequalities present in society. To this end, as well as the widespread use of tabular data in many high-stake applications, it is important to explore the following questions: what sources of information do LLMs draw upon when making classifications for tabular tasks; whether and to what extent are LLM classifications for tabular data influenced by social biases and stereotypes; and what are the consequential implications for fairness?Through a series of experiments, we delve into these questions and show that LLMs tend to inherit social biases from their training data which significantly impact their fairness in tabular classification tasks. Furthermore, our investigations show that in the context of bias mitigation, though in-context learning and finetuning have a moderate effect, the fairness metric gap between different subgroups is still larger than that in traditional machine learning models, such as Random Forest and shallow Neural Networks. This observation emphasizes that the social biases are inherent within the LLMs themselves and inherited from their pretraining corpus, not only from the downstream task datasets. Besides, we demonstrate that label-flipping of in-context examples can significantly reduce biases, further highlighting the presence of inherent bias within LLMs.

## introduction many recent works propose to use large language models (llms) for tabular tasks @xcite , where the tabular data is serialized as natural language and provided to llms with a short description of the task to solicit predictions. despite the comprehensive examination of fairness considerations within con-ventional machine learning approaches applied to tabular tasks @xcite , the exploration of fairness-related issues in the context of employing llms for tabular classifications remains a relatively underexplored domain. previous research has shown that llms, such as gpt-3 @xcite , gpt-3.5, gpt-4 (ope-nai, 2023) can exhibit harmful social biases @xcite , which may even worsen as the models become larger in size @xcite . these biases are a result of the models being trained on text generated by humans that presumably includes many examples of humans exhibiting harmful stereotypes and discrimination and reflects the biases and inequalities present in society @xcite , which can lead to the perpetuation of discrimination and stereotype @xcite . considering that tabular data finds extensive use in high-stakes domains @xcite where information is typically structured in tabular formats as a natural byproduct of relational databases @xcite , it is crucial to thoroughly examine the fairness implications of utilizing llms for classifications on tabular data. in this paper, we conduct a series of investigations centered around this critical problem, with the goal of discerning the underlying information sources upon which llms rely when making tabular classifications. through this exploration, our investigation aims to ascertain whether, and to what degree, llms are susceptible to being influenced by social biases and stereotypes in the context of tabular data classifications. through experiments using gpt-3.5 to make classifications for tabular data in a zero-shot setting, we demonstrate that llms exhibit significant social biases (section 4). this evidence confirms that llms inherit social biases from their pretraining corpus and tend to rely on these biases when making classifications for tabular data. furthermore, we demonstrate that providing gpt-3.5 with few-shot examples (in-context learning) or finetuning them on the entire training datasets both exhibit moderate effects on bias mitigation (sections 5.1 and 6.1). nevertheless, the achieved fairness levels remain below what is typically attained with traditional machine learning methods, including random forests and shallow neural networks, once again underscoring the presence of inherent bias in llms. moreover, our investigation further reveals that flipping the labels of the in-context few-shot examples significantly narrows the gap in fairness metrics across different subgroups, but comes at the expected cost of a reduction in the prediction performance. this finding, in turn, further emphasizes and reaffirms the indication of inherent bias present in llms (section 5.2). additionally, we further show that while resampling the training data is a known and effective method for reducing biases in traditional machine learning methods like random forests and shallow neural networks, it proves to be less effective when applied to gpt-3.5 (section 6.2). these collective findings demonstrate the significant influence of social biases on gpt-3.5's performance in tabular classifications. these biases significantly undermine fairness and pose potential risks for using llms on tabular data, especially considering that tabular data is extensively used in high-stakes domains, highlighting the need for more advanced and tailored strategies to address these biases effectively. straightforward methods like in-context learning and data resampling may not be sufficient in this context. ## related work fairness and social biases in llms fairness is highly desirable for ensuring the credibility and trustworthiness of algorithms. it has been demonstrated that unfair algorithms can reflect societal biases in their decision-making processes @xcite , primarily stemming from the biases present in their training data @xcite . llms, pre-trained on vast natural language datasets, are particularly susceptible to inheriting these social biases and have been shown to exhibit biases related to gender @xcite , religion @xcite and language vari-ants @xcite . these social biases can lead to the perpetuation of discrimination and stereotype @xcite @xcite . while recent works have made strides in addressing these issues, there still exists a significant gap in comprehensively assessing fairness in llms and its mitigation strategies for tabular data. tabular tasks and llm for tabular data tabular data extensively exist in many domains @xcite . previous works propose to utilize self-supervised deep techniques for tabular tasks @xcite , which, however, still underperform ensembles of gradient-boosted trees in the fully supervised setting @xcite . this disparity in performance can be attributed to the locality, sparsity, and mixed data types of tabular data. in recent times, llms have undergone intensive training using vast amounts of natural language data, which has enabled them to exhibit impressive performance across various downstream tasks @xcite , even with little or no labeled task data. therefore, recent approaches by @xcite suggest serializing the tabular data as natural language, which is provided to llm along with a short task description to generate predictions for tabular tasks. however, tabular data plays a crucial role in numerous safety-critical and high-stakes domains @xcite , which makes fairness particularly crucial when employing llms for making predictions on tabular data, especially considering the inherent social biases present in llms. despite the importance, this still remains largely unexplored. to the best of our knowledge, we regard our work as one of the most comprehensive investigations into the fairness issues arising when using llms for classification on tabular data. in-context learning significant improvements for various tasks have been achieved by providing in-context examples to llms @xcite @xcite . however, previous research by @xcite @xcite illustrate that the effective performance of in-context learning largely hinges on semantic priors rather than learning the input-label mapping @xcite @xcite and the labels of the in-context examples might not play a crucial role in in-context learning, with flipped or random labels sometimes having minimal impact on performance. despite these findings, the predominant focus of existing investigation of in-context learning remains on conventional natural language processing tasks @xcite @xcite , largely overlooking the domain of tabular data. furthermore, the fairness of in-context learning and the impact of flipped labels on this fairness is yet to be thoroughly investigated. ## experimental setup in this section, we begin by presenting an overview of the experimental setup utilized throughout this work. ## zero-shot prompting for tabular data to explore the fairness of llms when making classifications on tabular data, we first conduct experiments in a zero-shot setting. we assess the fairness metrics of the outcomes and examine whether gpt-3.5 without any finetuning or few-shot examples would be influenced by social biases and stereotypes for tabular classification. we run all the experiments 5 times and compute the mean and standard deviation. ## few-shot prompting for tabular data as demonstrated in section 4, employing gpt-3.5 for classifications on tabular data reveals significant social biases in a zero-shot setting. instead of directly utilizing gpt-3.5 for zero-shot tabular classifications, this section explores whether in-cluding few-shot examples during prompting will reduce or amplify these biases. to delve deeper into the influence of few-shot examples during incontext learning (icl), we not only consider the regular icl approach as detailed in section 5.1, but we also experiment by flipping the labels of the few-shot examples to further examine their effect on the biases, as discussed in section 5.2. again, for robustness, each experiment is conducted 5 times, with the mean and standard deviation reported. we present the fairness gap in figure 2 and the complete results in appendix b. ## conclusion in this work, we thoroughly investigate the underexplored problem of fairness of large language models (llms) for tabular tasks. our study unfolds in several phases. initially, we assess the inherent fairness displayed by gpt-3.5, comparing their performance in zero-shot learning scenarios against traditional machine learning models like random forests (rf) and shallow neural networks (nn). furthermore, we investigate how gpt-3.5 learns and propagates social biases when subjected to few-shot in-context learning, label-flipped in-context learning, finetuning, and data resampling techniques. our discoveries shed light on several key insights. we find that gpt-3.5 tends to heavily rely on the social biases inherited from their pre-training data when making predictions, which is a concerning issue. moreover, we observe that few-shot incontext learning can partially mitigate the inherent biases in gpt-3.5, yet it cannot entirely eliminate them. a significant fairness metric gap between different subgroups persists and exceeds that observed in rf and nn. this observation underscores the existence of biases within the llms themselves, beyond just the task datasets. additionally, labelflipping applied to the few-shot examples effectively reverses the effects of bias, again corroborating the existence of inherent biases in gpt-3.5. however, as expected, this leads to a loss in the classification performance. besides, our work reveals that while fine-tuning can sometimes improve the fairness of gpt-3.5, data resampling does not consistently yield the same results, unlike what is typically observed in traditional machine learning models. this underscores the need for the development of more effective strategies to mitigate the bias inherent in llms and ensure their fairness when deployed in real-world applications.