Accept or Deny? Evaluating LLM Fairness and Performance in Loan Approval across Table-to-Text Serialization Approaches (2025)

Abstract: Large Language Models (LLMs) are increasingly employed in high-stakes decision-making tasks, such as loan approvals. While their applications expand across domains, LLMs struggle to process tabular data, ensuring fairness and delivering reliable predictions. In this work, we assess the performance and fairness of LLMs on serialized loan approval datasets from three geographically distinct regions: Ghana, Germany, and the United States. Our evaluation focuses on the model’s zero-shot and in-context learning (ICL) capabilities. Our results reveal that the choice of serialization format significantly affects both performance and fairness in LLMs, with certain formats such as GReaT and LIFT yielding higher F1 scores but exacerbating fairness disparities. Notably, while ICL improved model performance by 4.9-59.6% relative to zero-shot baselines, its effect on fairness varied considerably across datasets. Our work underscores the importance of effective tabular data representation methods and fairness-aware models to improve the reliability of LLMs in financial decision-making.

## introduction large language models (llms), trained on vast amounts of textual data, have demonstrated remarkable potential to generalize across tasks and provide accurate predictions @xcite . given their growing presence in critical domains like financial decision-making, it is crucial to understand the behaviour and ethical implications of these systems due to their direct and severe impact on individuals @xcite . financial decisionmaking is the systematic process of analyzing information to make informed choices in finan-cial tasks such as investment, loan approval, and more @xcite . in this work, we focus on loan approval, where a bank must decide whether or not to grant a loan based on the applicant's creditworthiness. this task is typically performed by loan officers who consider various input factors to make informed decisions. loan approval is a critical task to explore as it directly impacts financial inclusion, borrower outcomes, and institutional risk management, making it an ideal domain for assessing the effectiveness and fairness of llm-driven decision-making systems. moreover, given the diversity in financial practices and socioeconomic contexts, evaluating loan approval across datasets from three distinct geographical regions (ghana, germany, and the united states) provides valuable insights into how llms manage data diversity and fairness within varying economic environments. additionally, the tabular nature of the datasets in this study underscores the importance of selecting an appropriate serialization method before feeding data into llms, as it can significantly influence model performance and fairness @xcite . building upon these observations, we frame our study around the following research questions: i) how do different serialization formats (e.g., json, text, great, lift) impact the fairness and performance of llms in loan approval tasks across diverse geographical datasets? ii) what effect does in-context learning (icl) have on the fairness and predictive performance of llms in loan approval scenarios, particularly when applied to datasets from ghana, germany, and the united states? iii) how do financial domain-specific llms compare to general-purpose llms in their ability to accurately and fairly assess loan applications, especially under zero-shot and few-shot learning settings? iv) what key factors contribute to fairness disparities in llm-generated loan approval predictions, and how do these factors vary across different serialization methods and geographical regions? to address the research questions outlined above, this work makes the following contributions 2 : 1. investigate the capability of llms in financial decision-making, focusing on loan approval tasks. this includes a comprehensive zeroshot benchmark evaluation of various llms and an analysis of the features they prioritize in their decision-making process. 2. analyze the impact of different tabular serialization formats on the decision-making process of llms. 3. evaluate the effectiveness of techniques, such as in-context learning, that aim to improve llm performance in financial decisionmaking, with particular attention to their impact on accuracy and fairness. 4. examine the presence of gender-related biases in llm-generated financial decisions, assessing their implications and associated risks. ghana (sackey and amponsah, 2018), germany (statlog) and united states @xcite . see appendix c for details of the feature description of each dataset. ## results and analysis in this section, we present our results and analysis, structured around a set of research questions that guide the discussion. we begin by comparing the performance of different serialization methods across models for each dataset, as shown in figure 2. we observe that the zero model outperforms the one model on the ghana and united states (us) datasets, while the reverse is true for the germany dataset. this suggests that the germany dataset has a higher proportion of non-defaulters compared to ## discussion and conclusion summary. the ability of llms to handle structured tabular data for high-stakes tasks like loan approvals remains under-explored. this work evaluates how different serialization methods (json, lift, text) and in-context learning (icl) impact the fairness and accuracy of llms across diverse regional datasets (ghana, germany, united states). we find that, in zero-shot scenarios, all llms perform worse than a logistic regression model baseline, frequently defaulting to uniform approval or denial. modest improvements only emerge with a few in-context examples, largely influenced by serialization format and dataset rather than model.