On Fairness of Low-Rank Adaptation of Large Models

Authors: zhoujie  ding
ken ziyu liu
pura  peetathawatchai
berivan  isik
sanmi  koyejo
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-05 06:40:38

Abstract: Low-rank adaptation of large models, particularly LoRA, has gained traction due to its computational efficiency. This efficiency, contrasted with the prohibitive costs of full-model fine-tuning, means that practitioners often turn to LoRA without a complete understanding of its ramifications. In this study, we focus on fairness and ask whether LoRA has an unexamined impact on utility, calibration, and resistance to membership inference across different subgroups (e.g., genders, races, religions) compared to a full-model fine-tuning baseline. We present extensive experiments across vision and language domains and across classification and generation tasks using ViT-Base, Swin-v2-Large, Llama-2 7B, and Mistral 7B. Intriguingly, experiments suggest that while one can isolate cases where LoRA exacerbates model bias across subgroups, the pattern is inconsistent---in many cases, LoRA has equivalent or even improved fairness compared to the base model or its full fine-tuning baseline. We also examine the complications of evaluating fine-tuning fairness relating to task design and model token bias, calling for more careful fairness evaluations in future work.

Main paper: 
 introduction & motivation: 
 The challenge of efficiently scaling large models has led to the growing interest and reliance on parameter-efficient fine-tuning, which focuses on adjusting only a small, deliberately chosen set of parameters in the base model (Hu et al., 2021; Dettmers et al., 2023; Li & Liang, 2021; Lester et al., 2021) . Of particular interest is the low-rank adaptation (LoRA) technique (Hu et al., 2021) , in which the pre-trained weight matrices are frozen while their changes from fine-tuning are approximated by low-rank decompositions. LoRA has received significant attention due to its simplicity and effectiveness in a variety of tasks across both language (Liu et al., 2022a) and vision (Gandikota et al., 2023) domains. Despite the popularity of LoRA, however, little is known about its effects on trustworthiness, such as fairness and robustness. The lack of understanding together with LoRA's wide adoption implies that practitioners may be deploying models with unintended and potentially harmful consequences in highstakes applications. To this end, this work initiates a study on fairness and asks the following:What are the effects of LoRA, if any, on subgroup fairness?Central to the existing knowledge gap is the prohibitive cost of full fine-tuning that deters a direct comparison against LoRA. This is troubling since the increased adoption of large models often involves taking off-the-shelf pre-trained models (e.g., Mistral (Jiang et al., 2023) ), fine-tuning them on custom data (if said models cannot reason in-context with fewshot prompts), and running them as part of decision-making processes. In many scenarios such as enterprise (Tran, 2023) , healthcare (Yu et al., 2023) , and banking (Loukas et al., 2023) , practitioners may gravitate towards LoRA solely for its cost-effectiveness without consideration for unfair outcomes; this can lead to tangible harm at tasks such as risk assessment, credit score estimation, loan approvals, and hiring/promotion evaluations.Apart from the real-world motivations above, tangential prior work also inspired this study from an algorithmic standpoint. Specifically, LoRA is characterized by its reduced fitting capacity through low-rank approximations; a similar property is also inherent to model pruning and differentially private training. Respectively, Tran et al. (2022) and Bagdasaryan et al. (2019) found that both pruning and private training can worsen the fairness of accuracy across subgroups (despite achieving good overall accuracy), as the sparsity and noisy gradients (due to private training) can both impact a model's ability to fit minority and underrepresented inputs. On the other hand, Langenberg et al. (2019) and Awasthi et al. (2020) showed that low-rank weights and representations can lead to better adversarial robustness. Prompted by these studies, we ask whether LoRA exhibits similar side effects and, if so, whether they are consistent across tasks and datasets. All in all-is LoRA's efficiency a "free lunch"?In this study, we seek to better understand the fairness implications of LoRA on large models via extensive experimentation. Our study and findings can be summarized as follows:1. We fine-tune both vision and language pre-trained models across sizes 86M-7B (ViT-Base (Dosovitskiy et al., 2020) , Swin-v2-Large (Liu et al., 2022b) , Llama-2 7B (Touvron et al., 2023a) , and Mistral 7B (Jiang et al., 2023) ), and across tasks including hatespeech detection, gender classification, machine translation, multiple-choice QA, and cloze completions, juxtaposing full-model fine-tuning and LoRA and measuring the subgroups disparities on accuracy, calibration, privacy as resistance to membership inference, and gender bias. To our knowledge, our work is the first to provide a comprehensive empirical investigation into the fairness properties of low-rank adaptation. 2. Intriguingly, our experiments reveal no consistent pattern of LoRA worsening subgroup fairness, compared to full fine-tuning across different architectures and domains ( §3). Note that isolated examples do exist where LoRA worsens fairness across subgroups, though such cases should be viewed with target applications and metric sensitivity in mind (Kleinberg et al., 2016) ; e.g., LoRA may appear less fair via worst subgroup accuracy but equally fair under demographic parity difference (DPD), which only considers positive predictions. Nonetheless, for any fixed task and its appropriate fairness metrics that we experimented on, we found no strong evidence that LoRA is less fair. 3. The fairness implications may depend on the quality of the underlying pre-trained model ( §3.2). We also observe cases where LoRA does exacerbate unfairness can disappear when the base pre-trained model is stronger (Fig. 1 ) when all else is kept constant. This suggests that the fairness properties of LoRA are not merely a function of its parameter efficiency (cf. model pruning (Tran et al., 2022) ). 4. The LoRA rank has little impact on subgroup fairness ( §3.6). While rank can be a confounding factor for its impact on model capacity and thus fairness (cf. pruning and private training), we did not observe a significant influence of rank on either utility or fairness. Our finding is in line with existing utility analysis (Hu et al., 2021) across tasks of varying difficulty (binary image classification, machine translation, language modeling). 5. LLMs can exhibit token biases, complicating fairness evaluations for generative tasks.A common strategy for eliciting model preferences is to compare token likelihoods for completing prompt templates (Wang et al., 2023) . However, we found that (1) small-scale LLMs (7B) may have strong and often unpredictable biases towards specific tokens for both full fine-tuning and LoRA (also reported recently by Zheng et al. (2024) ), and that (2) such biases are not alleviated by re-ordering answer options, switching base pre-trained models, or using rarer tokens (e.g., emojis and special UTF-8 characters). This implies that fairness conclusions can be confounded by such bias. 
preliminaries & related work: 
 An important paradigm in modern machine learning (ML) is to adapt large pre-trained models to downstream tasks through fine-tuning. The benefits of fine-tuning are two-fold:(1) it leverages the extensive knowledge stored in these pre-trained models, and (2) it promises greater efficiency compared to training from scratch. However, as models grow in size, this efficiency advantage becomes elusive due to increased demand on compute; for example, simply keeping the gradients of Llama-2 70B (Touvron et al., 2023a) in 16bit precision requires 130GB of memory, which is already infeasible for most commodity hardware. This gap motivates parameter-efficient fine-tuning methods and subsequent novel trustworthiness concerns. Here, we briefly outline work most closely related to the focus of this paper and some preliminaries that ground our analyses.Low-Rank Adaptation. LoRA (Hu et al., 2021) is a widely used parameter-efficient finetuning algorithm for large models. It proposes to separate out the weight deltas from finetuning and approximate them using low-rank matrices; inference then involves forward passing both the (frozen) pre-trained model and the low-rank model deltas, also known as adapters, and summing the activations. Specifically, for a pre-trained weight matrix W ∈ R d×k with dimensions d, k, LoRA approximates its changes from fine-tuning as ∆W ≈ BA where B ∈ R d×r and A ∈ R r×k with rank r ≪ min(d, k), and thus inference on input x ∈ R d is Wx + BAx ≈ (W + ∆W)x if ∆W is obtained through full fine-tuning. A and B can be updated directly via backpropagation. Typically, implementations of LoRA apply to all query and value matrices of self-attention layers in a transformer. To fine-tune for supervised tasks, an additional head is also attached to the last layer of the model.Fairness evaluations in machine learning. Fairness is a pivotal concern as biased models from training data/algorithms can lead to misleading and even catastrophic consequences, and understanding and mitigating such bias has been an active area of research (Kearns et al., 2018; Barocas et al., 2023; Chouldechova & Roth, 2018; Mehrabi et al., 2021) . The precise definitions and measurements of fairness, however, are often application-dependent.Fairness of classification. Classification tasks have well-accepted fairness evaluation methods and metrics. Subgroup accuracy parity and worst subgroup accuracy (relatedly, best-worst spread) are two metrics commonly studied in prior work (Kearns et al., 2018; Bagdasaryan et al., 2019; Yang et al., 2020; Tran et al., 2022) , which measure differences in accuracy. E.g., are people with different skin colors equally well-classified? Does the subgroup with the worst utility get "poorer" under the ML algorithm? We also consider the two common fairness metrics seen in recent work (Wang et al., 2023; Hong et al., 2024) . Demographic parity difference (DPD) (Agarwal et al., 2018; 2019) measures how varied are the model's positive predictions are across attributes. Equalized odds difference (EOD) measures if the model has similar predictive performance across both true and false positive rates, regardless of the protected attribute. Typically, in situations where ensuring equal representation or opportunity is the goal, such as fair hiring decisions or loan approvals across demographic groups, the "one-sided" DPD might be preferred. In scenarios where equitable outcomes are critical, such as the success of medical diagnosis across different demographic groups, the "balanced" EOD may be more appropriate. See Appendix A.1 for formal definitions.Fairness of generation. For generative tasks, fairness evaluations can be nuanced due to the open-ended nature of outputs-what does it mean for generated pixels/tokens to be "fair"? Prior work explored biases in word embeddings (Bolukbasi et al., 2016) , humanrated fairness scores on generated outputs (Lee et al., 2023) , or reducing generative models to classifiers through prompting (Wang et al., 2023) . Recent work in the surge of large generative models such as Esiobu et al. (2023) and Bianchi et al. (2023) focuses on the behavioral fairness of the generative models (e.g., whether the output text is stereotypical) as opposed to establishing formal algorithmic fairness notions.Fairness of fine-tuning. When evaluating the fairness properties of fine-tuning algorithms, we argue for the following key desiderata: (1) the fine-tuning task should not teach the model to be fair (or else we cannot extrapolate the evaluation to new tasks); (2) there is a "side-channel" through which we can measure fairness (e.g., measuring gender bias for machine translation); and (3) the fairness implications are directly relevant to the task being fine-tuned on (so that any observed fairness issues are indicative of realistic harm). We strive to achieve all these desiderata when designing fairness evaluations, though experiments forgoing desideratum (3) may still serve as "probes" and provide useful insights. 
fairness evaluations of lora: 
 We now turn to the experiments. We first describe the task setup and datasets, and then present results across dimensions of accuracy ( §3.2), calibration ( §3.3), privacy as (group-)differential resistance to membership inference attacks (MIAs) ( §3.4), and gender bias in generative tasks ( §3.5), along with their appropriate fairness considerations. We report a subset of the relevant metrics for each dataset and task and defer full results and additional implementation details to the appendix.Hatespeech detection. We construct 4 data subsets from the Berkeley D-Lab Hatespeech dataset (Kennedy et al., 2020) : Gender, Race, Religion, and Sexuality. The subsets contain 13976, 11670, 6081, and 7297 examples, respectively, where each example is a tweet-sized text snippet targeting a specific subgroup within the subset (e.g., hatespeech in the Religion subset may target Buddhists or Christians), and fairness is measured across these subgroups.Each example has a scalar hatespeech score, which we binarize into labels to turn regression into classification for the ease of applying and contrasting standard fairness metrics ( §2).Face image classification. We use the UTK-Face dataset (Zhang et al., 2017) , where each face image is labeled with gender, age, and specified race of the person. We consider gender classification (binary) and age classification (9-bins) as the fine-tuning tasks, and race attributes are used as subgroups to evaluate fairness, following Bagdasaryan et al. (2019) and Tran et al. (2022) .For hatespeech detection and face image classification, a fair fine-tuning method should produce models that: (1) perform well across all subgroups (i.e., accuracy parity); (2) do not worsen the worst subgroup accuracy; and (3) make errors equally often across subgroups (captured by EOD). For hatespeech detection and related applications such as credit scoring, content moderation, and hiring processes, the model should also make positive predictions (e.g., flagging hatespeech) equally often across subgroups (captured by DPD).Machine translation. We use the WinoMT dataset (Stanovsky et al., 2019) , which consists of tuples of English sentences that include both the pro-and anti-stereotypical gender constructions by varying the subject pronouns and compositions. For example, in the sentence pair "The developer argued with the designer because [pronoun] did not like the design" and "The developer argued with the designer because [pronoun] idea cannot be implemented.", the gender pronouns can be varied between she/he (or her/his) to produce a 4-tuple. We then construct the fine-tuning task as translating these sentences from a gender-neutral language (we used Turkish) back to English and observe whether the fine-tuned model surfaces gender bias (e.g., prefers the stereotypical English translation).We experimented both a balanced and a pro-stereotype set of Turkish-to-English translation examples for the fine-tuning ( §3.5).Language modeling: multiple-choice QA and cloze completions. To probe the fairness implications of fine-tuning from a different angle, we also explore how the base model's gender bias may surface differently as it fits on gender-neutral text under LoRA vs. full fine-tuning. We sample 50,000 Yelp reviews from the multi-dimensional gender bias dataset (Subramanian et al., 2018) , which consists of reviews where: (1) the rating is 3/5 such that the sentiment tends to be neutral, and (2) the gender is not easily identifiable. We then elicit the model's bias (before and after fitting next-token prediction on these reviews) by prompting it to deduce the gender of the review author, either through multiple-choice QA or cloze completions. Because the reviews are selected to be gender-neutral, bias is measured by how much LoRA and full fine-tuning deviate from the golden behavior of guessing male/female equally often, compared to the base models. For example, a cloze task with the template [Describing their most recent experience: "{review}", says a {gender}] elicits model preference by comparing token likelihood for "male" and "female" at the slot {gender}. We also consider multiple-choice setups with options for the model to guess gender-neutral/non-binary.Because generative models give open-ended outputs, part of the difficulty in designing fairness evaluations is finding a "side-channel" through which we can easily and quantitatively probe the model's bias (recall §2). For both the above tasks, we focus on gender bias because it is interpretable and orthogonal to the fine-tuning tasks, has a clear societal impact, and is frequently considered in prior work (Wang et al., 2023; Hong et al., 2024) .Training settings. On hatespeech detection, machine translation, and language modeling, we fine-tune (LoRA and full fine-tune) on Llama-2 7B (Touvron et al., 2023b) and Mistral 7B (Jiang et al., 2023) ; on image classification, we fine-tune on ViT-Base (Dosovitskiy et al., 2020) and Swin-v2-Large (Liu et al., 2022b) . On all datasets and tasks, LoRA can match full-model fine-tuning in terms of both train/test performance (though some tasks need higher LoRA rank), allowing fair comparison as absolute performance advantage can be a confounding factor in fairness evaluations. For language modeling, we also experiment on the instruction-following versions of Llama-2 7B and Mistral 7B. The evaluation data for each experiment is a random 20% split (varies across random seeds) except for language modeling where evaluation (via prompting) is done on the same training set. Overarchingly, LoRA and full fine-tuning exhibit similar performance across all subgroups, with the worst subgroup performance and best-worse spread for LoRA being consistently on par with full fine-tuning. Observe also that for most subgroups, LoRA does not worsen either DPD or EOD and may even improve them in some cases.Fairness implications can depend on the quality of pre-trained model. A closer look at Fig. 1 suggests that while LoRA may be considered less fair than full fine-tuning on ViT-Base-by decreased worst subgroup utility on Black group for age classification (1st subplot) and by increased EOD on Asian group for gender classification (3rd subplot)-the tendency disappears when the base model is switched to the more powerful Swin-v2-Large (all else kept the same). The results suggest that the fairness properties of LoRA are not only a function of its parameter efficiency; it also implies a separation from model pruning where Tran et al. (2022) found that the fairness ramifications persist across model sizes.It is nonetheless possible to isolate cases where LoRA is less fair, but such cases should be viewed with target applications and metric sensitivity in mind. Another interpretation of the above observation is that one can single out cases where LoRA is less fair than full fine-tuning. We note that different fairness metrics may be more or less relevant depending on the goals and priorities of the task at hand. Take, for example, UTK-Face gender classification where the female category is labeled as 1; for applications where correctly classifying females are important (e.g., when there is drastically less data for females than males), the unfairness of LoRA according to EOD (Fig. 1 ) may be less relevant than DPD which only looks at "positive" (i.e., female) predictions. There, DPD may very well lead to a different conclusion that LoRA is equally as fair (Fig. A5 in Appendix D). In the context of fairness metric sensitivity (Kleinberg et al., 2016) , it is therefore crucial for practitioners to adopt a task-centric perspective to ensure a meaningful and relevant fairness evaluation.While metrics in §3.2 concentrate on equality in error rates across groups, the measure of calibration within groups is another important fairness metric to ensure the probability estimates align with real-world outcomes, both globally and across different subgroups (Kleinberg et al., 2016) . To measure calibration, we extract the model's confidence on the 20% evaluation set by examining the probability outputs from the classification head. We then follow Guo et al. (2017) and generate the confidence histograms and the reliability diagrams. We defer background on calibration to Appendix A.2 and more results to Appendix D.2.LoRA and full fine-tuning show comparable calibration levels, though LoRA shows signs of overconfidence. Fig. 3 shows that both LoRA and full fine-tuning exhibit a reasonable level of calibration, with their expected calibration error (ECE) being relatively low and comparable across different datasets and subgroups. The reliability diagrams illustrate that the probabilities predicted by both methods are well-aligned with the observed accuracies. Neither method consistently yields less calibrated models than the other, and the conclusion holds even when we specifically look at the respective subgroups with highest ECE (Fig. 3 right). One subtle observation is that LoRA shows a tendency for its predicted probabilities to cluster at the lower and upper ends of the scale, particularly in the 0-0.1 and 0.9-1 confidence bins (top row of Fig. 3 ). This skewness indicates a degree of overconfidence in LoRA's predictions, leading to less reliable decision-making (Niculescu-Mizil & Caruana, 2005) that could potentially affect subgroups disparately.Membership inference attacks (MIA) involve predicting whether an example was in the training set of a target model. MIA has downstream implications for data privacy, copyright protection (Henderson et al., 2023) , as well as (detecting) data contamination (Jiang et 2024; Oren et al., 2024; Zhang et al., 2024) , and it is useful to understand the impact of fine-tuning on the model's resistance to MIA. One reason to hypothesize that LoRA may exhibit different behaviors than full fine-tuning is its parameter efficiency and thus its decreased capacity to memorize and overfit. Past work showed that overfitting tends to result in higher vulnerability of MIA (Carlini et al., 2022; Yeom et al., 2018) , and minority groups tend to be outliers and thus possibly memorized more often (Feldman, 2020) . Motivated by the above hypothesis and relevant observations, we evaluate the resistance of fine-tuned models against MIA to see whether LoRA makes the fine-tuned model more (or less) vulnerable compared to full fine-tuning. In particular, we focus on the Likelihood Ratio Attack (LiRA, Carlini et al. (2022) ) due to its efficacy. (See Appendix A.3 for background and implementation of MIA and Appendix D.3 for additional results.) We attack ViT-Base and Swin-v2-Large fine-tuned with UTK-Face dataset for binary gender classification; and Llama-2 7B and Mistral 7B fine-tuned with D-Lab Hatespeech dataset for binary hatespeech classification. We repeat this for both LoRA and full fine-tuning and compare their resistance to MIA when the training loss is about the same for both methods. We refer the reader to Appendix A.3 for the details on how we partitioned each dataset to train the shadow models.LoRA is generally as resistant to MIA as full fine-tuning. For each model and dataset pair described above, we obtain receiver operating characteristic (ROC) curves by varying the confidence thresholds. Figs. 4 and 5 show the ROC curves in log-scale to emphasize true positive rates at low false positives. We defer results with ViT-Base and Mistral 7B models and a simpler MIA attack (LOSS) to Appendix D.3. From Figs. 4 and 5, we see that there is no clear evidence that LoRA makes the model less resistant to MIA compared to full fine-tuning. On Swin-v2-Large with UTK-Face, LoRA seems more resistant than full fine-tuning overall and also at the subgroup level across different races. On Llama-2 7B with D-Lab, while LoRA seems marginally less resistant on average, there are subgroups for which LoRA provides higher resistance than full fine-tuning. In general, we also do not observe a significant impact of the subgroup size on their resistance to MIA. We also explore how gender bias may surface from fine-tuning for machine translation and language modeling (recall §3.1). Respectively, Figs. 6 and 7 present gender bias results on Yelp review language modeling and Turkish-to-English translation on Llama-2 7B and Mistral 7B; more results are deferred to Appendix D.6. The average eval BLEU scores (Papineni et al., 2002) range between 58 and 63, reflecting good and fluent translations (Lavie, 2010) .No definitive evidence of LoRA exacerbating gender bias. On language modeling evaluations, we observe that: (1) compared to the pre-trained base models (both raw and instructiontuned), the fine-tuned models tend to reduce bias, and (2) LoRA does not exhibit more bias than full-model fine-tuning. On machine translation evaluations, we see that: (1) when trained on a gender-unbiased dataset, both Llama-2 7B and Mistral 7B models demonstrate balanced frequencies of gender representation, indicating unbiased behavior regardless of the fine-tuning approach; and (2) when trained on a pro-stereotypical gendered dataset, this bias is transferred heavily onto the fine-tuned model's behavior. Specifically, while Mistral 7B exhibits comparable levels of gender bias with either LoRA or full-model fine-tuning, Llama-2 7B presents less gender bias with LoRA fine-tuning, suggesting that LoRA may sometimes lead to a less severe gender bias than full-model fine-tuning.We also explore the choice of rank for LoRA, as it may also be a confounding factor in the model's fitting capacity and fairness impact. Results from UTK-Face gender classification (Fig. 8 ) reveal that accuracy and fairness metrics (EOD) are not influenced by rank, aligning with findings from Hu et al. (2021) . In review generation (Fig. 6 ), where a low rank might result in underfitting due to limited capacity, no definitive connection between rank and fairness was found. Similarly, for machine translation (Fig. 9 ), fairness remains largely unchanged beyond a rank threshold that ensures quality translation (indicated by BLEU scores), as shown by stable fairness measures (flat lines in the plot) despite an increase in rank. More results are deferred to Appendix D.4.As a control experiment, we investigate the effect of subgroup size on utility and fairness metrics. It is intuitive that subgroups with less samples may be disproportionately affected by the fine-tuning and subsequently experience unfairness. Fig. 10 illustrates the effect of increasing group size on utility (e.g., accuracy in the plots) and fairness. Contrary to the intuition, we did not observe a strong correlation between subgroup size and accuracy and fairness metrics. On the D-Lab dataset for language modeling, accuracy does not consistently increase or decrease with the size of the groups (we defer results on the UTK-Face gender dataset with vision models to Appendix D.5). The demographic parity difference and equalized odds difference both exhibit fluctuations across different group sizes without showing a clear trend that correlates with group size.  
discussions, limitations, and future work: 
 We have presented extensive empirical analyses and found no conclusive evidence that LoRA may exacerbate subgroup fairness compared to full fine-tuning. Does this imply that the parameter efficiency of LoRA is a free lunch? Possibly, but not necessarily-no evidence of unfairness does not imply evidence of fairness. While our study aims to be comprehensive, we outline some limitations and directions for future work.Token bias in LLMs complicates fairness evaluations on generative tasks. Unlike classification tasks where model predictions can be used for downstream decisions directly (and thus fairness can be evaluated directly), generative tasks involve diverse outputs that do not always reveal the model's preferences. For language models, practitioners often instead elicit such preferences through prompting (Lee et al., 2023) , e.g., via QA (Cobbe et al., 2021) and multiple choice (Hendrycks et al., 2020) , and this underpins our gender bias experiments ( §3.5). This elicitation process, however, introduces an ambiguity between the model's preference for specific tokens vs. its actual preference on the subject matter: if a model responds "yes" to a question, is it because "yes" is the correct answer, or that the model simply generates "yes" more often? Indeed, we found that models have strong and often unpredictable preferences towards specific tokens. For example, full finetuned Llama-2 7B chose "Yes" over 99% of the 50k Yelp reviews, while surprisingly, LoRA preferred "No" 99% of the time. This phenomenon persists across various setups-"Yes/No" answers and multiple-choice QA with numeric and letter options (see Appendix D.6.1).Moreover, these biases are not easily mitigated: (1) negating the semantic meanings of the prompts to flip "Yes/No" options (e.g., male + yes → female + no) did not change model preferences (Table A2 ); (2) models may favor token "A" even when it denoted opposite answers (Table A4 ); (3) the preference remains even when the ordering of choices was modified (e.g., ABC to BAC; Table A4 ); and (4) the above issues can persist when switching to a different base model and even when answer options are presented with rare symbols (e.g., (U+1F7E0) and (U+25D1); Table A6 ). Among different tokens to compare model preferences, we found using "male/female" tokens mitigates the strong token bias (Appendix D.6.2) and focused on reporting these results in §3.5 and deferring the rest to Appendix D.6. Future work should prioritize evaluation methods beyond the token level and consider more nuanced effects of bias through semantics, discourse structure, and the holistic content of the generated text.Considerations for fairness gerrymandering. Fairness gerrymandering happens when change in subgroup definitions (e.g., adding/removing subgroups with intersections/unions of protected attributes) alters the fairness conclusions (Kearns et al., 2018; Yang et al., 2020) .For example, while we found no evidence for LoRA worsening fairness on D-Lab religion (Fig. 2 ) and race (Fig. A2 ) data, this conclusion may not transfer to the intersection of these subgroups (e.g., Asian atheists). Addressing fairness gerrymandering can be computationally demanding both theoretically (Kearns et al., 2018) and empirically with large models; we leave exhaustive experimentation on varying subgroup definitions to future work. 2023)) emerge as a promising tool to assess models while circumventing their token biases, though the use of additional classifier heads resemble our supervised evaluations. It is also worth exploring and comparing other parameter-efficient methods (e.g., ReFT (Wu et al., 2024) ), DoRA (Liu et al., 2024) ) and their intersection with related techniques such as quantization (Dettmers et al., 2023; Hong et al., 2024) and pruning (Dery et al., 2024; Gromov et al., 2024) ; this may offer insight whether our findings with LoRA is unique to its algorithmic constructions. 
acknowledgements: 
 The  
ethics statement: 
 This work comprehensively evaluates the fairness implications of low-rank adaptation (LoRA) of language models in comparison to full-model fine-tuning methods across multiple critical dimensions: disparate accuracy across subgroups, calibration, resistance to membership inference attacks, and gender bias. The study spans both vision and language domains, acknowledging the profound impact these technologies have on society.We recognize the ethical implications of our findings, particularly concerning the equitable treatment of diverse populations and the protection of individuals' privacy. Our research discovers areas where bias and inequities are amplified when switching from full-model fine-tuning to LoRA (and sometimes, the other way around). We urge the community to consider the ethical ramifications of the choice of fine-tuning method and caution against adopting the method that gives the overall best utility without careful consideration of fairness implications. It is essential to continue efforts to mitigate bias, enhance fairness, and protect privacy in machine learning systems. This includes ongoing evaluation, adopting ethical AI frameworks, and engaging with diverse stakeholders to understand and address potential impacts comprehensively.Future work should not only extend the technical dimensions evaluated but also deepen the engagement with interdisciplinary approaches to understand and address the societal implications of different fine-tuning methods. By doing so, we can strive towards the development of scalable machine learning technologies that are not only advanced but also aligned with the principles of equity, fairness, and respect for all individuals. 
reproducibility statement: 
 The code for this project is released at https://github.com/kenziyuliu/lora-fairness. All experiments are done using standard machine learning packages, including PyTorch (Paszke et al., 2019) , Hugging Face transformers (Wolf et al., 2019) Both demographic parity difference (DPD) and equalized odds difference (EOD) quantify the fairness of the model predictions for the sensitive attribute A.Demographic Parity Difference (DPD) is the absolute difference in the probability of positive outcomes between two groups distinguished by a sensitive attribute. Specifically,M dpd = |P( f (X) = 1 | A = 1) -P( f (X) = 1 | A = 0)| ,where A is the sensitive attribute, f (X) is the prediction of the machine learning model, and X is the feature vector. A non-zero M dpd indicates a disparity in prediction outcomes that are independent of the ground truth and solely based on the sensitive attribute. Furthermore, a large DPD means that there is a large prediction gap between the groups with A = 1 and A = 0, indicating the unfairness of the model prediction.Equalized Odds Difference (EOD) is calculated as the larger of two quantities: the disparity in true positive rates and the disparity in false positive rates between two groups distinguished by a sensitive characteristic. Specifically,M eod = max {M TP , M FP } ,with the components defined as follows (Y here is the true label):• True positive equalized odds difference:M TP = |P( f (X) = 1 | Y = 1, A = 1) -P( f (X) = 1 | Y = 1, A = 0) |• False positive equalized odds difference:M FP = |P( f (X) = 1 | Y = 0, A = 1) -P( f (X) = 1 | Y = 0, A = 0) |A significant M eod reflects a discrepancy in error rates between the groups, indicating the unfairness of the model prediction.Reliability diagrams (or calibration curves) (Niculescu-Mizil & Caruana, 2005) plot the model's predicted confidence against its observed accuracy. To construct these diagrams, predictions are grouped into M interval bins (each of size 1/M). Within each bin (say B m ), the model's accuracy, denoted as acc(B m ), is computed as:acc(B m ) = 1 |B m | ∑ i∈B m 1( ŷi = y i ),where ŷi represents the predicted class label for sample i, and y i is the corresponding true label. The average confidence for bin B m , expressed as conf(B m ), is the mean predicted probability for the samples within that bin:conf(B m ) = 1 |B m | ∑ i∈B m pi ,with pi denoting the confidence for sample i. A perfectly calibrated model should exhibit acc(B m ) = conf(B m ) for each bin, i.e., the diagram should plot the identity function. The distance between the observed accuracy and the predicted probability in each bin represents the calibration gap.Expected calibration error (ECE) (Naeini et al., 2015) is a scalar summary statistic of how well the model is calibrated by calculating the weighted absolute difference between predicted probability (i.e., confidence) and accuracy across all the confidence bins. Specifically,ECE = M ∑ m=1 |B m | n |acc(B m ) -conf(B m )| ,where n is the total number of samples. The ECE reflects the calibration gap, with lower values indicating a model whose predicted probabilities are closer to the true outcomes.Importance of calibration for fairness. Well-calibrated models produce probability estimates that can be trusted equally across different demographic groups. If a model is not well-calibrated, some groups may systematically receive overconfident or underconfident predictions. For example, in the scenario of hatespeech detection, it's imperative that the model operates impartially among all demographics to avoid unfairly censoring content from specific groups.The goal of membership inference attacks is to estimate whether a query example is part of a model's training set as accurately as possible. To this end, consider a dataset space X , label space Y, a real-world distribution D over X × Y, a training dataset D sampled from D, and a training procedure f D ← T (D) where f D is a machine learning model that outputs a probability distribution over Y for any given instance x ∈ X . The attacker wishes to determine, for an instance x ∈ X and label y ∈ Y, whether the query example (x, y) is in D. Following Carlini et al. (2022) , evaluating the effectiveness of a MIA can be done by measuring the true positive rate at a low false positive rate. The justification is that being able to confidently infer that just one data point is a member is a much bigger breach in privacy than being 51% confident in the membership of a larger number of datapoints, even though both instances may score the same in other classification metrics such as accuracy or AUROC.LiRA. In the Likelihood Ratio Attack (LiRA) (Carlini et al., 2022) , the attacker trains N "shadow models"f D i ← T (D i ) where each D i for i ∈ [1, N]is randomly sampled from D such that the query example (x, y) ̸ ∈ D i for all i. These shadow models are intended to mimic the behavior of the target model f D . For any given model f , let f (x) y denote the confidence for label y on input x under f . Given a fixed example (x, y), the attacker evaluates the model confidence f D i (x) y for all i. The attacker then uses the shadow model confidences f D i (x) y to model the distribution of the target model's confidence f D (x) y under the assumption that (x, y) ̸ ∈ D (note that the attacker does not have access to the target dataset D, but has full access to each shadow dataset D i as the attacker sampled them). From this, the attacker can then compare the actual value of f D (x) y and perform a hypothesis test against the null hypothesis that (x, y) ̸ ∈ D (rejecting the null hypothesis and inferring membership whenever the cumulative distribution function of f D (x) y is above some threshold τ). The version of LiRA described above and implemented in our experiments is the "offline" version of the attack, as the "online" approach is more computationally expensive.In our experiments, for each dataset, we partitioned a small subset (20%) from both the training and evaluation sets (for the target models) for membership inference evaluation and used the remaining data to train the shadow models. This way, we ensure that the membership inference target dataset is disjoint from the shadow training dataset, which is a necessary assumption for the offline LiRA attack. To ensure variability between the different shadow datasets, we randomly sample 50% of the shadow training dataset to train each shadow model. The shadow models are fine-tuned via LoRA using the target model's pre-trained model.The LOSS attack (Yeom et al., 2018) is based on the observation that machine learning models are trained to minimize the loss of their training examples. Thus, examples with lower loss are, on average, more likely to be members of the training data. Specifically,EQUATIONwhere ℓ(•) is the loss function, τ is a tunable decision threshold parameter, and A loss (x, y) is the attack model which outputs 1 if the loss is below the threshold τ (indicating membership) and 0 otherwise. It is a very simple attack that tends to be less effective than more sophisticated attacks like LiRA, but it is also much faster to run.Gender bias evaluations via cloze completion with Yelp reviews. Recall that in §3.5, we performed experiments to evaluate model fairness by comparing the model's preference for gendered completions on Yelp reviews. Because the focus of the fairness evaluation is directly related to the performance of the model on the specific task it's being fine-tuned for, one limitation of our task setup for evaluating gender bias on Yelp reviews arises: the use of multiple choice QA or cloze completions to elicit model preference primarily compares how LoRA and full fine-tuning surface the underlying gender bias from a fairness-agnostic task, rather than their inherent impact on fairness.This is a nuanced distinction: although the task setups on supervised classification and cloze completion mirror each other in that any fairness implications would emerge because of the fine-tuning, in the latter case such fairness implications do not directly hinder the model's ability to do the downstream task well (writing gender-neutral Yelp reviews vs. classifying people with darker skin).Hatespeech Detection on Berkeley D-Lab. The Berkeley D-Lab hatespeech detection dataset (Kennedy et al., 2020) can be accessed via Hugging Face: https://huggingface.co/ datasets/ucberkeley-dlab/measuring-hate-speech.We first de-duplicate the original dataset, and take one human annotation of the text example when there exists multiple annotations from multiple raters; then we binarize the annotation for each example as either hatespeech or not by thresholding the assigned hatespeech score at 0.5. To obtain the different subsets of the D-Lab hatespeech dataset (hatespeech examples on Gender, Race, Religion, and Sexuality), we use the provided binary attribute labels to filter the dataset. For example, we use the column target race to take only the examples that may target a specific race group; within these examples, there are more granular attribute labels such as target race asian and target race native american through which we can split the dataset into groups and assess model fairness. The Gender, Religion, and Sexuality subsets are similarly created using the columns target gender, target religion, and target sexuality and their corresponding granular attribute labels, respectively.Face Image Classification on UTK-Face. The UTK-Face dataset (Zhang et al., 2017) For supervised tasks, The training and evaluation split is 80% and 20%, respectively. For language modeling, we focus on fitting next-token prediction on the given set of reviews and fairness is evaluated on the same training set.The instruction-following variants of Llama-2 7B and Mistral 7B are accessible as meta-llama/Llama-2-7b-chat-hf and mistralai/Mistral-7B-Instruct-v0.1 on Hugging Face (https://huggingface.co/models).Recall from §3.1 that to perform fairness evaluations on the language modeling task, we use various prompt templates to elicit the fine-tuned model's preferences and gauge how much the model favors different identity groups (genders in the case of Yelp reviews).Table A1 below lists the prompt templates we use for the language modeling evaluations. These templates cover a range of scenarios across yes-no questions, multiple-choice questions (with numbers, letters, or special symbols as answer options), as well as different styles of questions (e.g., direct questions, indirect questions, and questions with negation).The prompt templates are generated with the assistance of GPT-4 (Achiam et al., 2023) .The prompts are roughly groupped into the following types:• YN*: These are yes-no questions that prompt the model to generate text that contains specific identity groups. Since "Yes" and "No" are both treated as a single token, we can directly measure the model's preference by comparing the likelihood of the two tokens being generated at the end of the prompt templates. In these templates, we compare "male" and "female" as the gender groups, and thus for a specific template, we can take four measurements ("male" + "yes", "male" + "no", "female" + "yes", "female" + "no). • MC*: These are multiple-choice questions that prompt the model to select an answer that corresponding to a specific identity group. The text of the prompt templates are different from YN* templates. Similarly to YN* templates, we can measure the model's preference by comparing the likelihood of the tokens being generated at the end of the prompt templates. The tokens denoting the answer options all have the same length (they are either single tokens, or token sequences with common prefixes in the case of special symbols), and thus we can directly compare the token likelihoods. With these templates, we can also allow the model to select "gender-neutral" or "non-binary" as an answer option beyond "male" and "female". • Cloze*: These are cloze templates that prompt the model to complete the sentence with a specific identity group. Unlike the YN* and MC* templates, the cloze templates are more of a fill-in-the-blank style statements than questions. Here, we rely on the fact that "male" and "female" are both treated as a single token and evaluate across these two gender groups. • *-numeric: These are the same templates as specified by the prompt ID prefixes, but with the answer options replaced by numbers. The goal is to examine whether standard choice tokens like "yes", "no", "A", "B", etc. are treated differently from numbers. Compare, for example, the templates YN1 and Yn1-numeric in Table A1 . • *-inverted: These are the same templates as specified by the prompt ID prefixes, but with the content of the answer options inverted. In the YN* templates, this means answer options are presented as "no" and "yes"; in the MC* templates, this means what A, B, C denote are changed. Compare, for example, the templates MC3 and MC3-inverted in Table A1 . • *-inverted-symbol: Same as above, except the answer option ordering is changed. Compare, for example, the templates [MC3, MC3-inverted, and MC3-inverted-symbol] or [MC1-numeric, MC1-numeric-inverted, and MC1-numeric-inverted-symbol] in Table A1 . • *-special: These are the same templates as specified by the prompt ID prefixes, but with the answer options replaced by special symbols. The goal is to examine whether standard Figs. A1 to A4 show the classification fine-tuning results for Llama-2 7B and Mistral 7B on all Berkeley D-Lab hatespeech subsets (gender, race, religion, sexuality). Fig. A5 shows the results for UTK-Face gender and age classification for ViT-Base and Swin-v2-Large.In all of these figures, we present the subgroup F1 score, accuracy, demographic parity difference (DPD), and equal opportunity difference (EOD) for each subset of the dataset. In the case of UTK-Face age classification, we only present the subgroup accuracy, as F1, DPD, and EOD are not directly applicable.The results are consistent with the main results described in §3.2:• By worst group performance, best-worst group performance spread, demographic parity difference (DPD), and equal opportunity difference (EOD), Llama-2 7B and Mistral 7B exhibit similar fairness performance across the different subsets.• In most cases, LoRA does not worsen either the DPD or the EOD.• The fairness assessment of the fine-tuning methods can be sensitive to the choice of the metrics. Figs. A6 to A9 show the calibration results (i.e., confidence diagrams and reliability diagrams) for Llama-2 7B and Mistral 7B across all Berkeley D-Lab hatespeech subsets, and for Mistral 7B's highest ECE subgroups within each subset. Fig. A10 shows the calibration results for UTK-Face gender classification for ViT-Base and SwinV2-Large.The results are consistent with the main results described in §3.3:• LoRA and full fine-tuning show comparable and reasonable calibration levels. No significant differences are observed between the two fine-tuning methods.• LoRA tends towards overconfidence, with predictions clustering at extreme scales (bins 0.0-0.1 and 0.9-1.0), potentially affecting reliability across subgroups. We also present the LOSS attack results in Figs. A13 to A16. See Appendix A.3 for background on the definitions and implementations of membership inference attacks. Results indicate that the LoRA fine-tuned model is roughly equally resistant to membership inference compared to full fine-tuning. Recall from Section 3.6 that we evaluate the effect of LoRA ranks on the fairness of the finetuned models. Fig. A17 presents the results for Llama-2 7B on all Berkeley D-Lab hatespeech subsets, Fig. A18 presents the results for ViT-Base on UTK-Face gender classification, and Fig. A19 presents gender stereotype results for Llama-2 7B on Turkish to English machine translation. Following the main discussions in Section 3.6, we found that the choice of rank tends to have little effect on the fairness of the fine-tuned models. Fig. A20 illustrates the effect of increasing group size on utility (e.g., accuracy in the plots) and fairness (DPD and EOD). The results support §3.7 that these metrics are not solely dependent on the size of the subgroups.On the UTK-Face gender dataset with vision models, while there is a general trend of increasing accuracy with larger group sizes, the practical impact of group size on accuracy is limited, since the absolute difference in accuracy across these sizes is marginal. gender2): (gender1) refers to the gender that people usually predispose the pronoun to be given the context in the English sentence. (gender2) refers to the gender pronoun that a model-translated English sentence contains. E.g., "the developer argued with the designer because she did not like the design" (where "she" refers to "developer") is a (male) predispose (female) sentence. Ideally, the model should translate similar ratios (50%/50%) of male predispose female and male predispose male sentences, the same for female predispose female and female predispose male sentences. https://github.com/gabrielStanovsky/mt gender/tree/5862928/data/aggregates 
 
 Appendix:  


Decision: The paper presents a study on LoRA fairness implications. The paper presents solid work with minor concerns surfaced during the review process adequately addressed during author rebuttal. While reviewer tTvn provided a different opinion, it seems they may have not understood the final aim of the paper. I recommend the authors to include reviewer recommendations in their final version.

[At least one review was discounted during the decision process due to quality]

Reviews: review: paper_summary: paper_summary: This paper shows empirical evaluations of the LoRA model on various downstream tasks by considering fairness across different sub-groups. The experimental results are informative and the conclusion that “LoRA rank has little impact on subgroup fairness” is interesting.
strength_weakness: strength_weakness: For a draft focusing on evaluation, I think the included evaluation is sufficient.
paper_summary: paper_summary: This paper studies the impact of parameter-efficient fine-tuning on the fairness of downstream tasks. The paper compares their experimental results on various settings, including text/image classification, translation, and generation tasks. They compare their results with the full-finetuning setup. The paper notes that PEFT-based fine-tuning using LORA achieves comparable performance to full fine-tuning and does not consistently excavate the unfairness of the decisions.  The paper also performs similar experiments to evaluate the calibration of the decisions and effectiveness against MIA attacks.
strength_weakness: strength_weakness: 1. The paper's premise is unclear, why should LORA worsen the fairness of the model decisions? LoRA is meant as a way to approximate full fine-tuning and with higher LORA ranks we should be able to get the exact behaviour of full-finetuning. With lower ranks, it should underfit the data -- it is not obvious to me why that should worsen the fairness? Shouldn't it just be a function of the underlying data that is being used? With this logic, shouldn't the experimental results be expected (or even obvious)? A discussion around this would be helpful.
2.  Although the paper limits itself to understanding the fairness impact of low-rank PEFT methods, it would be interesting to study the impact of other PEFT methods like adaptors, IA3, etc.
paper_summary: paper_summary: The paper aims to investigate the relationship between Low-Rank adaptation and the fairness of language models. Fairness is an extremely important factor in a language model, especially if these language models are used for important decisions.
strength_weakness: strength_weakness: The main research question is really unclear: the relation between Low-Rank adaptation and fairness is really questionable. Let's use a metaphor. Suppose the goal is to prepare a system that is fair in showing traffic jams by hiding plates. Clearly, this kind of system is expected to blur only plates and keep everything in a clear shape. Now, it is clear that if the system blurs the whole picture, it blurs the plates too. The goal of hiding the plates is accomplished but final images are useless. Now, out of the metaphor, low-ranking is acting indiscriminately over all the parameters. So, it is clear that it has effects on every aspect of LMs. Then, why focusing on fairness? Is there any reason why this low-ranking should act only on fairness?
questions: questions: Provide a good reason why Low-Rank adaptation should have any influence on fairness of language models. See Reasons to Reject.
paper_summary: paper_summary: This work did a comprehensive benchmark of fairness for models fine-tuned by low-rank adaptation (LoRA). The models are evaluated on both classification and natural language tasks.

---
After reading the authors response, I will retain my rating. For improving the current work, I would recommend the authors to evaluate the LoRA models enforced with fairness constraints.
strength_weakness: strength_weakness: My own observation is that there is no significant difference of fairness metrics between LoRA fine-tuned and fully fine-tuned models. I don't think such observation implies anything on whether LoRA may exacerbate the biases or not, given the fact that the learning objectives are still identical.
questions: questions: 1. Is the model fine-tuned subject to fairness constraints? If not, I would like to see the comparison to the LoRA models that are explicitly enforced with fairness constraints.

Comments: title: Thank you for the explanation
comment: Dear Reviewer tTvn,

Thank you for your engagement! We’d like to clarify that **we have not claimed a causal effect between LoRA and fairness in any part of the paper**. We hope to emphasize an important distinction between

1. Motivating the study of fairness because we hypothesized a causal effect, vs;

2. Motivating the study because fairness is not explored in the past and that there are connections to prior work that hints at a potential correlation to fairness. Again, we simply do not know if there is such a correlation (much less causation, as the reviewer is suggesting), and hence our study to find out. 

**It appears that the reviewer has been characterizing our work as (1), when our work is more of (2)**, as explained in Section 1 (intro) and our earlier response above. 

We hope that our study will be helpful for many researchers who may have different opinions on whether LoRA may have an unintended impact on fairness. Our goal is to provide experimental support so that interested researchers can use this study to either confirm or adjust their beliefs, as opposed to relying on *prior opinions* without evidence. 

We nevertheless acknowledge that it is the authors’ responsibility to make the above points clear and we hope to improve our presentation in the updated version. 

Thank you!
Unfortunately, I'm not convinced of the validity of the hypothesis as, in my humble opinion, there is no causal effect between LoRa and fairness. 
I stay with my scores.