Fairness Reprogramming
Authors: Guanhua Zhang
Yihua Zhang
Yang Zhang
Wenqi Fan
Qing Li
Sijia Liu
Shiyu Chang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Fairness
Model Reprogramming

Abstract: Despite a surge of recent advances in promoting machine Learning (ML) fairness, the existing mainstream approaches mostly require training or finetuning the entire weights of the neural network to meet the fairness criteria.  However, this is often infeasible in practice for those large-scale trained models due to large computational and storage costs, low data efficiency, and model privacy issues.  In this paper, we propose a new generic fairness learning paradigm, called FairReprogram, which incorporates the model reprogramming technique.  Specifically, FairReprogram considers the case where models can not be changed and appends to the input a set of perturbations, called the fairness trigger, which is tuned towards the fairness criteria under a min-max formulation.  We further introduce an information-theoretic framework that explains why and under what conditions fairness goals can be achieved using the fairness trigger.  We show both theoretically and empirically that the fairness trigger can effectively obscure demographic biases in the output prediction of fixed ML models by providing false demographic information that hinders the model from utilizing the correct demographic information to make the prediction.  Extensive experiments on both NLP and CV datasets demonstrate that our method can achieve better fairness improvements than retraining-based methods with far less data dependency under two widely-used fairness criteria. Codes are available at https://github.com/UCSB-NLP-Chang/Fairness-Reprogramming.git.

Fairness Reprogramming
Guanhua Zhang" UC Santa Barbara guanhuaducab.cn edu
Yihua Zhang' Michigan State University zhan 1908@muu. edu
Yang Zhang
MIT-IBM Watson Al Lab yang. zhang2@ibm. com com
Wenqi Fan
Qing Li
The Hong Kong Polytechnic University wengifandpolyu.co edw. edu. hk
The Hong Kong Polytechnic University csqli@comp polyu. edu. hk
Sijia Liu
Shiyu Chang UC Santa Barbara chang870ucsb edu
Michigan State University & MIT-IBM Watson Al Lab liusiji5@msu, edu
Abstract
Despite a surge of recent advances in promoting machine Learning (ML) fair- ness, the existing mainstream approaches mostly require training or finetuning the entire weights of the neural network to meet the faimesss criteria. However, this is often infeasible in practice for those large-scale trained models due to large computational and storage costs, low data efficiency, and model privacy issues. In this paper, we propose a new generic fairness learning paradigm, called FAIRREPROGRAM, which incorporates the model reprogramming tech- nique. Specifically, FAIRREPROGR AM considers the case where models can not be changed and appends to the input set of perturbations, called the fairness trigger, which is tuned towards the fairness criteria under a min-max formulation. We further introduce information-theored under what conditions fairness goals can be achieved using the fairness trigger. We show both theoretically and empirically that the fairness trigger tively obscure demographic biases in the output prediction of fixed ML models by providing false demographic information that hinders the model from uti- lizing the correct demographic information to make the prediction. Extensive experiments on both NLP and CV datasets demonstrate that our method can achieve better faireess improvements than retraining based methods with far less data dependency under two widely-used faimess criteria. Codes are available https: http///gttttt..t/. cow/UCSB- MLP
framework that explains why and
effec-
MAIBB-----CCaaaaaaaee Chang/Fairness deprogramming .itt.
Introduction
Faireess in machine learning (ML) has become critical concern. Due the biases in data collection, the output prediction is often spuriously correlated with some demographic attributes, which are thus undesirably incorporated into the dicision-making process of machine learning models. For example, it found that some abusive language detection systems tend to classify texts that contain mentioning of certain minority groups, e.g., homosexual groups, abusive content, even though the texts themselves are not abusive at all [ 2 Despite the recent advances in fairness promoting learning methods 377 the existing mainstreaming approaches mostly require retraining or finetuning the entire model parameters towards an extra faimess objective. However, this is
Equal contribution
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

often infeasible in practice, particularly for those well-trained large-scale models, due to the huge computation and storage costs. In addition, for machine learning models that are deployed as a service, model retraining is hindered by limited access to the model parameters.
Recently, model reprogramming has emerged as alternative technique to model finetuning. In particular, model reprogramming considers the pre-trained model fixed, and instead modifies their input to re-purpose the model towards different objectives. For example, is shown that a well-crafted input perturbation can re-program an ImageNet classifier to solve the task of counting squares in an image [ It is also shown that by learning specifccc embedding prompts concatenated to the inputs, pre-trained language models can achieve better performances than tull-parameter tuning in natural language understanding tasks (IC-122 Compared with finetuning methods, model reprogramming enjoys lower cost, better scalability, and requires less access to the model parameters. Hence here come our research questions Can model reprogramming techniques be applied to fairness objectives? If so, why and how would it work?
In this paper, we revisit the model reprogramming and propose a novel generic fairness learning paradigm, called FAIRREPROGRAM. In particular, the input a global constant vector/feature, called the fairness trigger, which is optimized towards the fairness objective under a min-max framework. FAIRREPROGR AM is a generic framework that works for various tasks and domains. We further introduce an information -eeemattoo-tthoretic framework that explains why and under what conditions faimess goals can be achieved using a constant faimess trigger. We show theoretically and empirically that the fairness trigger can effectively obscure demographic biases in the output prediction of fixed MI. models by providing false demographic information that hinders the model from utilizing the correct demographic information to make predictions.
perturbs the input by appending
We perform extensive experiments across various NLP and CV datasets with in-the-wild biases. The results show that FAIRREPROGRAM can consistently achieve better fairness improvement with the retraining-basse methods under the two widely-used fairness notions, but with far less trade-off in accuracy. For example, with comparable accuracy, our method can outperform the retraining based baseline with 10.5% and 36.5% lower bias scores over two faimess criteria in the CelebA dataset with the hair color prediction task and gender as demographic information. In addition. our method demonstrates great transfcrability and interpretability. Our theoretical analysis and empirical findings can provide useful insights toward more practical, scalable, and flexible fairness learning paradigms.
2 Related Work
Fairness in ML Fairness problems in ML models have received increasing attention from both industry 13 and academia [14) There has been a myriad of fairness definitions in the liter-
ature 18 19 20 Among them, group fairness notions are one of the most popular [21)
which require ML models to perform similarly for different demographic groups. In this paper, we mainly focus on the most widely-used group fairness definitions, demographic parity (21] and equalized odds (22], but it is worth mentioning that our method is general for other faimess notions. Existing fairness promoting methods can be broadly categorized into pre-procs prepprocessing in-processing, and post-processing methods [24]. Pre-processing methods calibrate the training data to remove the
spurious correlations and train fair model the modified data 28 2
29
RC
In-processing For example, an
methods work on training MI. with extra fairness-aware regularization 7
32
adversarial framework is introduced to train model parameters meet fairness requirements [33] In our method, we adopt a similar adversarial loss but optimize the fairness triggers with a fixed model. Despite the effectiveness, these methods usually consider training fair models from scratch and do not directly apply to already-traines models. Post-processing methods focus on calibrating trained ML models to be fair (24). Many of them modify the model outputs to meet the fairness criteria [18 [2 [4 [4] For example, the model outputs are directly modified to meet equalized odds by solving an optimization problem (22]. Alternatively, a boosting- method is introduced to
calibrate model outputs 40
Model reprogramming Model reprogramming 45 9 46 48 aims to repurpose an already trained neural network for different tasks. Different from the typical transfer learning that requires modifying the structure and parameters of the given pre-trained model, reprogramming technology instead designs a trainable program appended to the input, while keeping the pre-trained model intact. The model reprogramming technology can be designed in the form of an input-agnostic
2

---

perturbation 45 8 or a trainable imput transformation function together with the label mapping from the source domain to the target domain (9. In particular, the feasibility of designing a universal input perturbation to reprogram well-trained ImageNet classifier to the CIFAR-10 10 dataset is demonstrated in the white-box setting [8] As an exploration to implement reprogramming in the discrete scenario, another work [6 successfully reprograms the text classification neural network for alternate classification tasks. This work also shows the possibility of developing reprogramming in the black-box setting, where the reprogrammer may not have the access to the parameters of the target model. Recent work [47] shows the possibility of repurposing deep neural networks designed for image classifiers for the natural language processing and other sequence classification tasks. It is argued the success of the reprogramming lies in the size of the average input gradient and the input dimension is crucial to the performance of the reprogrammer 4 It is also shown that generative models like FairGANs [49 can be transfered to other tasks by reprogramming with variational auto-encoders 50) A highly related topic model reprograming is prompt learning in NLP [Α]. It is shown that by designing designated text prompts appended to inputs. pre-trained language models could be re-directed perform well under downstream tasks in few-shot setting [5 Prompt-based tuning methods have become the mainstream and achieve better performance than fine-tuning in many scenarios 55 Seminal works about prompt learning can be found in 12. However, nearly all existing methods focus on using model reprogramming to improve accuracy in domain-trans tasks and to our best knowledge, our work is the first generalize model reprogramming to improve fairness of trained model.
3 Fairness Reprogramming
In this section, we will introduce the FAIRREPROGRAN algorithms. As some notations, upper-cased letters, X and X, denote random vectors and random variables, respectively; lower-cased letters, æ or denote deterministic vectors and scalars respectively, px (-) or p(X) denote the probability density function of the (discrete) random variable X.
3.1 Problem Formulation
Consider classification task, where X represents the input feature, and Y represents the output label. In addition, there exists some sensitive attributes or demographic group, 2, that may be spuriously correlated with Y. There is a pre-trained classifier, f*)),, that predicts Y from X, i.e. Y (X).. The weights of the classifier are considered fixed (hence the superscript x). Unfortunately, due to the spurious correlation between Z and Y, the classifier may be biased against certain demographies.
Our goal to improve the fairness of the classifier by modifying the input X, rather than modifying the classifier's fixed weights. In particular, we aim to achieve either of the following fairness criteria.
Equalized Odds: Y 1 2|Y, or Demographic Parity: Ý 712. Z,
(1)
where 1 denotes independence. The following two subsection will explain how to modify input and design the optimization objective respectively.
3.2 Moditying the Input Features
Input modification primarily involves appending a fairness trigger the input. Formally, the input modification takes the following generic form:
X = mm (xx
(2)
where X denotes the modified input: [.] denotes vector concatenation. As can be observed. the input modification consists of two steps. First, X is fed through a transformation function g(: 0). where 0 represents the hyper-parameters of the transformation function. The actual form of g(; is contingent upon different applications and modalities, but a general requirement is that g(: should largely retain the information necessary for classification. The second step is to append fairness trigger, 8, to the input, which is a vector that can be optimized over. It is important to note that is a constant different inputs get appended the same trigger. Although it does not seem intuitive, will soon show that a constant trigger is all you need to achieve fair prediction on all different inputs.
Below specific forms of transformations (Eq.
use.
3

---

Text Classification In text classification, X represents a sequence of input token embeddings. To modify the imput. we simply append fixed number of embeddings after the input text. In this case, g(: 0) is the identity mapping, and 6 corresponds to the appended embeddings.
Image Classification In image classification, X represents the (vectorized) input image. Unlike text classification, where the input can have variable length, the length of the input to the image classification network is fixed. We thus apply the following two approaches to append the trigger, as shown in Fig. The first approach, called the patch approach, removes patch from the original image, and appends trigger the same size as the patch to the patch location (as shown in Fig. aa). In this case, {(; 0) a function that removes the patch dimension and retain the rest, with 0 representing the patch location; á represents the trigger feature that replaces the patch. The second approach, called the border approach, shrinks the image to smaller image, and then appends the trigger at the border (as shown Fig. Πb). In this case, g(: is a function that shrinks the image, and represents the trigger feature at the border.
(a) Border trigger
(b) Patch trigger
Figure 1: Demonstration of the border and patch trigger applied on an image from CelebA (56]
3.3 Optimization Objective
Our optimization objective is as follows
min Lata CataaDuueeyy m) XLfas, (Dtunes f m), 5,9
(3)
where m,; 8) represents the input modification function in Eq. represents nested functions; Dtone represents the dataset that are used to train the faimess trigger. Note that this is different from the dataset where the classifier, f", is pre-trained.
The first loss term, Lutit, the utility loss function of the task. For classification tasks, Luta usually the cross-entropy loss, i.e.
Lura(Draney f o m) = =Bx. 1....Y-D.....
(C((, [CE(Y, (m((()mm)) (m(())
(4)
where CE(.,) denotes the cross-entropy loss.
The second loss term, fairs encourages the prediction follow the fairness criteria in Eq. According to Eq. ( Lfair should measure how much information about Z is in Y. To measure this, we introduce another network, called the discriminator, d(:: Φ). where Φ represents its parameters. If the equalized odds criterion is applied, then d(-; should predict Z from Y and Y; ; if the demographic parity criterion is applied, then the input to d(: Φ)) would just be Y. In the following, we will focus on equalize odds criterion for conciseness. Then, the information of Z can be measured by maximizing the negative cross coos-entropy loss for the prediction of over the discriminator parameters, i.e.,
Laai (Drune; f m) Ex, Y.z-D CCE(Z,(( (m( ()) Y; {)
(5)
By plugging Eqs. and into we can see that the entire optimization objective becomes min-max framework. where the discriminator tries improve its prediction of while the fairness trigger tries to make the prediction worse. As shown in [33,, when the discriminator cannot predict Z better than chance, the aforementioned fairness criteria can be achieved.
3.4 Why Does Work?
It is not immediately straightffrward why global trigger obscure the demographic information for any imput. In this section, we will propose an iiformation-thhoreiiic framework that illustrates one of the mechanisms through which the trigger can remove the demographic information.
Our theoretical framework builds upon the data generation process as shown in Fig. aa.. Specifically, we assume that X consists of a set of features, i.e. X [X,..., Xr], where T is the total number of features. text classification, feature can word or a word piece; image classification, feature can be specific shapes, colors, patterns, eíc. Assume that these features can be divided into two groups. The first group, denoted as (1), consists of features that are directly governed by the output label Y; the second group, denoted x consists of featuers that are directly governed by

---

x
pp
(-1x01))
X
PP' (-1x000)
X
Clussifier
Classifier
False constani pd (X()
( (x(e)
X
X
(a)
(b)
(c)
Figure Illustration of why fairness trigger works. (a) The data generation process. (b) The information flow from data to the classifier through the sufficient statistics. strongly indicative of a demographic group can confuse the classifier with a false demographic posterior, and thus preventing the classifier from using the correct demographic information.
(c) Fairness trigger
the demographic information Z. Z and Y can be spuriously correlated, i.e. there can be common confounders, C, between and As result, both x()) and x are indicative of Y.
To further simplify our theoretical analysis, we consider a bag-of-feature scenario, where each feature in x( is drawn from the vocabulary set X and each feature in X is drawn from the vocabulary set (=) There should not be any overlap between the two vocabulary sets, i.e. x n 2.
Otherwise violates our assumption that demographic-related features biased features.
can be shown (in Appendix C that the posterior distributions, py (r(((())) and (+(+)(()) are the sufficient statistics of (*) and X (()) respectively for inferring Y. In other words, these two posterior
distributions summarize all the information about x and (=) that the classifier needs to know to predict Y. Therefore, assume that the classifier takes the following generic form
= (x) h(pp (x
(6)
Note that we add superscript, tr, to emphasize that the probability distributions are over the data set where the classifier is trained, because the classifier has never been trained on inputs modified with the fairness trigger. Eq. (6 encompasses many common decision functions. For example, can be shown (in Appendix C that the posterior distribution p(Y)X), which is the minimizer of the cross-entropy loss, a special case of Eq.
As illustrated in Fig. b().. (+)x(())) and p=(+|x(+)) provide two sets of information from input features. pr pr(- (-1x (w)
provides the unbiased information, because desirable fair classifier should rely only upon py (-Ix (1)) to make decision. On the other hand, pz (=Ix(*) provides the biased information, because it conveys the demographic information. In other words, the fairness goals can be achieved by cutting off the biased information path. Therefore, our research question boils down to: is it possible to cut off the biased information path with global fairness trigger 8?
Without loss of generality, assume that ō consists of only one feature. Consider the case where is demographic feature, i.e. 8 x(=), In this case, we assume the transformed input as defined in Eq. ? can also be divided into two groups:
= X (v) x('),,
where (x() =(x(()),
x(()= = =
(7)
The following theorem states our main conclusion:
Theorem Under the assumptions in Eq. ( and 7. and some additional regularity condition
if
the fairness trigger 8 is indicative of a certain demographic group z, then
lim p'r(Z== -5)-1
MI(Ý, =
(8)
where means mutual information; V f the classifier's 's prediction after input is modified.
means that the fairness trigger very strongly indicative of the demographic group =. Therefore, Thm. esssentially states that if the prepended trigger feature is very strongly
p" ( 6)
indicative a certain demographic group, then equalized odds can be achieved. A formal proof is
presented Appendix C Here we would like to give an intuitive explanation. When pd = x In other words, the fairness trigger would overshadow the rest of the demographic features and 'trick' the classifier into believing all the
6) 1, it will also happen that (Z = x|XX (#) = (+))
Formal assumptions stated in the appendix.
5

---

different inputs belong to the same demographic group As result, the second argument in Eq @ would reduce to a constant (1 for demographic group z and 0 elsewhere). effectively blocking the biased information path, as shown in Fig. cc). Note that the premise for the faimess trigger to work that the classifier has never seen the modified input. Otherwise, the classifier will be able to learn to ignore the constant trigger and still elicit the true demographic information from input.
Experiments
In this section, we evaluate the effectiveness of FAIRREPROGRAM on both NLP and CV applications in terms of accuracy, fairness, performances under low-data regime, transferability and interpretability.
4.1 Experiment Setup
Datasets
We consider the following two commonly used NLPP and CV datasets:
The dataset contains 448k texts with labels that depict the toxicity of each input. The demographic information of each text is provided.
Civil Comments
157 58
CelebA [56]: The dataset contains over 200k human face images and each contains 39 binary attribute annotations. We follow the conventional setting [56) that adopts the hair color prediction task in our experiment and uses gender annotation as the demographic information. [59.6]
For both datasets, we split the entire data into training set, tuning set, a validation set, and lesting set. The training set is used for the base model training, i.e., to obtain biased model for are used for trigger training and hyper-parameter
reprogramming. The tunning set and validation selection. We report our results on the testing set. It is worth mentioning that there is no overlapping data between different sets and the size of the tuning set is much smaller than the training one. Specifically, we set the size ratio between the tunning set and the training as 1/5 and 1/100 for Civil Comments and CelebA, respectively. The full statistics of the datasets can be found in Appendix A.
Metries Besides the model accuracy, we introduce two empirical fairness metrics, one under each of the two fairness criteria as in Eq. ① For binary classification, the metrics are calculated as:
DP: Σ I0(Ý 1) - 1| =)1, EO: Σ (IFPR- -PR_ FNR FNR.))/2. 2E2
ZEZ
where DP and EO stand for demographic parity and equalized odds respectively. FPR and FNR the false positive/negative rate, and the subscript denotes the score is caleulated within specific demographic group Z 2. For example, FPRmate indicates the false positive rate calculated over all examples with the "male" annotation. For multi-class setting, the bias scores first calculated similarly using one-vs-all for each class and then averaged across different classes. All reported results the average of three different random runs. It can be shown that these metrics are non-negative, and will become zero when their corresponding faimess criteria are achieved. For better elaboration, we report the negative bias scores our experiments, so the larger these negative scores are, the better the model satisfies the corresponding fairness criteria.
Baselines and implementation details We consider the following models for comparison:
BASE: the base model to be reprogrammed, trained with the cross-entropy loss the training set.
in-processing adversarial training method that optimizes both model accuracy and fairness using the training set.
ADVIN 3
ADVPOST: post-processing variant of ADVIN, which fine-tunes the BASE model with the same fairness-aware adversarial objectives as ADVIN, but using the (low-resource) tunning set only.
For NLP experiments, we use pre-trained BERT 12 to obtain the BASE and ADVIN models. We use ADAMW (33] as the optimizer, and set the learning rate 10-5 5 for all baselines and 0.1 for FAIRREPROGRAM. For CV experiments, we consider RESNET- 18 (44] that pre-trained on
ImageNet. The discriminator used in ADVIN, ADVPOST and FAIRREPROGR AM is three-layer MI., and the parameters are optimized using ADAM with learning rate of 0.01. We pick the best model based on the accuracy (for the BASE) or the bias scores (for all other debiasing methods) the validation set. We refer to Appendix A.2 for more details and Appendix B for more bascline studies.

---

400
Raggees Demos Part
MAAA CHNTTC Ferty
Segelin SOAN COAE OAEA
-ag 100
40 100 100
Negaliaa Erea
Mit
MAPAL
W
EX3
632
3.80
3.00
Acourscy
2.5
6.6
35r 134 550
(a) Civil Comments
(b) CelebA
Figure 3: Results on (a) Civil Comments and (b) CelebA. We report the negative DP (left) and the negative
EO (right) scores. For each method, we vary the trade-off parameter À (as shown in (3) to record the performance. The closer dot the upper-rige corner, the better the model is. We consider five different As for each method. The solid curve is fitted polynomial with order 30.
Hepalie Eipeetoe OSE
200
1 Vog "o "g
Par Horr
Mifort
fasteaneram (sm
Tarkaprggnn on Paots
degragan Seokac
Tuning Daca Rati
Tuning Data Rati
Turing Data Ratio
(a) Civil Comments
(b) Celebá
Figure 4: Results on (a) Civil Comments and (b) CelebA with different tuning data ratio. We report negative DP (left) and negative EO (right) scores. We consider a fixed BASE model trained with training set, whose negative bias scores are presented as black dashed line. Then we train other methods with different tuning data ratio promote faimess of the BASE model.
Next we introduce the implementation details of the triggers for different variants of FAIRREPRO- GRAM. For image classification task, adopt the border and patch trigger shown in Fig. termed FAIRREPROGRAM (BORDER) and REPROGEM (PATCH) correspondingly. We define the trigger size as the width of the trigger frame for border trigger and the width of the square patch for patch trigger. Unless otherwise stated, the default trigger size for each setting are 20 and 80.
For text classification task, we introduce probability vector vs to control the selection of trigger word for each position Specifically, we have the trigger ô, = Ev; where E represents the pretrained word embedding matrix of BERT. Then we simply concatenate ô after all input texts in the embeddings space as the fairness trigger. We introduce two types trigger. The first type, called FAIRREPROGRAM (SOFT), uses continuous probability simplex using the bisection algorithm after each training step. The second type, called FAIRREPROGRAM (HARD), discretizes each vs into one-hot vector ú; via arg max operation. We adopt the straight through technique 65] to update v, during training. The triggers found by FAIRRERO (HARD) enjoy better interpretability
and each v, is projected onto the continuous
they correspond to sequence of word tokens. Unless specified otherwise, we set the trigger word number as five for our experiments.
4.2 Results
Fig. 3 shows the performance of the proposed FAURREPROGRAM with other basclines on both NLP (subfigure (a)) and CV (subfigure (b)) datasets using DP (left) and EO (right) metries. In each subfigure, the data samples of the same method (dots the same color) are generated by explicit changing the adversary weight À in 3 which controls the trade-off between fairness and accuracy, We further fit the data with polynomial regression to present the curves. Appendix[A2] shows the key observations. First, method improves
detailed À choices for different methods. Here the fairness of the BASE model. particular, our methods (both orange and red curves) achieve higher negative DP and EO scores with comparable classification accuracy. Second, method enjoys a better fairness-accuracy trade-off compared with all other baselines. Specifically, the curves of method lie farther the upper-right corner of the plots, which implies that method
'The trigger is appended suffix after all input tokens but before [SEP] for BERT.

---

4.50
522 '.22
SOROE CurTe SPRE Party
AA 100 "80
SARE SORE CORREIN CLEA
SAA 2A 100 "00
Negaliva Frua lnnd DACA
-40
PRADOONN (SOh)
TesRomsetan 15 Turaee
FesTenne (541) Turrte
PARRORSSTAT PREIE) Trorale
42
196
GEN 1097 LIKE 3.5 903 EIIS GOA
0.W 3ED5 ESGE DURE IUD0
0
c34
333
134
150
Accuray
(a) Civil Comments
(b) Ce CelebA
Figure 5: Results in the transfer setting. We report negative DP (left) and negative EO (right) scores. The triggers are firstly trained in BASE model. Then we evaluate the triggers based on another unseen BASE model. We change the parameter À to trade-off accuracy with faimess and draw the curves in the same way with Fig. point corresponds to the average of all BASE models with different random seeds.
The *
improves model fairness with fewer sacrifices on accuracy. It is also worth noting that although ADVIN achieves good fairness scores, it uses much more data for training.
Limited data setting We further evaluate ADVPOST and number of data in the tuning set. Specifically, we fix a À for each method such that all methods achieve comparable bias score with full tuning set. The detailed À choices are provided in Appendix AA.2 Then we apply these methods subsets of the tuning set with different proportions. The results are shown in Fig. There are two key observations. First, our method can consistently improve faimess upon BASE model even with 1% tuning data, indicating a high data efficiency of FAIRREPROGRAM. Second, FAIRREPROGRAM achieves better fairess than ADVPOST does when tuning data number decreases. For example, in Fig. as tuning data decreases. When the tuning set size is extremely small, ADVPOST significantly deteriorates and even underperforms the BASE model.
AM with decreasing the
D (a), the curve of our method significantly above the ADVPOST
Transferability Next, we show the transferability of the fairness triggers found by FAIRREPROGRAM. We first tune the triggers with a BASE source model and then apply the trigger on a target model trained with a different random seed. The results are shown in Fig. [ As can be seen, FAIRRE- PROGRAM achieves comparable fairness- accuracy trigger on both the source model and the target model, indicating our method has a good transferability. This intriguing property brings two ben- efits of our method: ① if MI. model pa- rameters infeasible (e.g. when MI. models are provided as services), the
Input Image
Standard
FairReprogram
Advin
Prediction: Blond Hair False
True
Figure 6: Gradient-based saliency map visualized with GRAD
CAM (66] of different methods. The highlighted zones (marked the pre- dieted labels (non-blond hair v.s. blond hair) in each row, which also depict the attention of the model on the input image.
in red) depicting regions exerting major influence
users could train a surrogate model and tune the trigger based it promote fairness of the original model; @ when ML model parameters are updated with new data (e.g. online learning), the user could still use the original trigger for fixing fairness problems. We further elaborate the results of FAIRREPROGRAM for transferring to different tasks and model architectures in Appendix B.5
Input saliency attribution. Fig. and 6 compare the saliency maps of some example inputs with and without the fairness triggers. Specifically, For the NLP applications. we extract subset of Civil Comments with religion-related demographic annotations, and apply IG [67] localize word pieces that contribute most to the text toxicity classification. For the CV application, we use GradCam [66] to identify disermmiiiii regions of CelebA's test images. As shown in Fig. 7 our fairness trigger consists of lot of religion-related words (e.g., diocesan, hebrew, parish). Meanwhile, the predicted toxicity score of the benign text starting from 'muslims' significantly reduces. These observations verify our theoretical hypothesis that the fairness trigger is strongly indicative of certain demographic group to prevent the classifier from using the true demographic information. In addition, Fig. X presents the input saliency maps two input images with respect to their predicted
8

---

Text (Non-toxic)
Predicted Toxicity
muslims need to take look in the mirror muslims need take look in the mirror muslims need to take look the mirror muslims need to take look in the mirror
0.149
same diocesan bula proceedings soto ec rib hebrew armenian paul long course parish body
0.069
0.054
0.073
A text example from Civil Comments with INTEGRATED GRADIENT important words that influence ERM model predictions. The text is concatenated with three triggers generated with different adversary weight. Green highlights the words that Iean toxic predictions and Red highlights non-toxic leaning words. The model prediction tends to be correct after adding the triggers.
Figure 7:
57L XX
highlighting
Migathe Exua bee Code
Meqgatte Exua Code
tooiggan Palk
Parhiing on (M00
Arrmy
Arruney
(a) 8-class negative DP
(b) 8-class negative EO
(c) 16-class negative DP
(d) 16-class negative EO
Figure 8: Performance of multi-class classification. For (a) and (b), we use the attrihutes Blond Hair, Smiling, Attractive for multi-class construction. We add an addition attribute Wavy Hair for (c) and (d).
labels, non-blond hair and blond hair, respectively. As can be observed, when there is fairness trigger, the saliency region incorrectly concentrates on the facial parts, indicating the classifier is likely use biased information, such gender, for its decision. With the fairesss trigger, the saliency region moves to the hair parts, which matches the behavior of ADVIN. These results confirm that our fairness trigger can drive models make fairer predictions.
To further verify that the triggers encode demo- graphic information, we trained demographic classifier to predict the demographics from the input (texts or images) without triggers. The obtained demographic classifiers can accurately identify the demographics contained in the in- puts and achieve over 0.99 AUC for identifying demographics in the validation datasets. Then, we use the demographic classifier to predict the demographic information of null image/tex[] with the trigger. Specifically, we select three triggers generated with different À values for both two datasets. The results can be seen in Table [] We see that the demographic classifier gives confident outputs on the triggers. For ex- ample, we see that the trigger paul long course parish body is classified as containing christian with 0.98 confidence, indicating that the found triggers are highly indicative of demographics. This is consistent with our perspective in Section 3.4 that the fairness triggers are encoding fake demographic information to obscure MI. models from making biased predictions.
Table 1: Predictions of the demographic classifier on a null input with triggers generated by different A. The demographic prediction for CV triggers indicate the pre- dicted score for Male and Female, and it is Christian, Muslim and Other religion for NLP.
Trigger
Demographic Prediction
0.85, 0.15
0.92, 0.08
0.80, 0.20
same diocesan bula rev proceedings
0.96. 0.1 0.02
soto rib hebrew armenian paul long course purish body
0.5 0,08, 0.81
0.98, 0.04, 0.03
4.3 Multi-Class Classification
To extend our evaluation to multi-class setting, we use the CelebA dataset and select n binary attributes that may be spuriously correlated with gender [99.6] Then, following [9] we construct data groups by enumerating all 2" possible binary vectors, where each dimension corresponds to a binary attribute. We index these vectors and treat them as the class labels. Fig. g shows the accuracy-fairnes: trade-off curves similar to Fig. It can be observed that our method outperforms other methods as the red curves are closer the top-right corner. Also, as the class label number
*We use an empty string as the null text and an all-black image as the null image. One text could contain multiple religions the probabilities do not sum for NLP triggers.

---

-0.0
Mere Dumao Parky
Megato D wott Parth
For mydlse
Pain
MOO MOE Munta
-0.4
thtargrum 1541
BMA
Wegatoo Dar
TarResrigee Hlard
35
Word Number
Trigger Word Number
Tagger
23
15
22
Tagger 45
(a) Civil Comments
(b) Ce CelebA
Figure 9: Ablation study of the trigger size. We evaluate the bias scores with different trigger word numbers (Civil Comments) and different trigger size (CelebA) with fixed adversary weight A.
increases, the post- post-processing-bee indicating a larger class number may induce more challenges to post-processing methods.
ADVPOST falls behind its in- in-processing counterpart ADVIN,
4.4 Ablation Studies
We perform an ablation study to investigate the effects of the trigger size. Specifically, we rum experiments with different numbers of trigger words trigger patch sizes on the NLP/ CV dataset. We set Ả value for each method such that all methods achieve comparable bias scores with the largest trigger size. The detailed À choices can be seen in Appendix A.2 Then we train the triggers with different sizes in the tuning set using the fixed X's. For the text trigger as shown in Fig. a). see that the negative bias score gets worse as the number of trigger words gets smaller. However, our method can still improve fairness upon the BASE model even with only one-word trigger. On the other hand, the results with five trigger words and above are all comparable, indicating that five words enough achieve the fairness goal. Similarly, for the image trigger as shown in Fig. [b), the results suggest larger trigger would consistently improve fairness. On the other hand, we show that larger trigger size could hurt accuracy in Appendix B which is similar to the effect of increasing X. 4.5 Summary of Additional Results
We compare our proposed FAIR REPROGRAM with four additional baselines, and we show the full results with variance in Tab. B We further compare our method with MMD methods where fair in Eq. (3) is replaced with Maximum Mean Discrepancy regularization [70] to partial out the instability of adversarial training. and the results are shown in Fig. TO We also implement the fairness reprogramming in the black-box setting on CelebA dataset, where the model parameters are not available for training the reprogram, and the results are shown in Fig. [ Besides, we show that FAIRREPROGRAN could also be used in tabular data, and the corresponding experiment results on the Adult dataset are shown in Fig.
5 Conclusion
In this paper, we introduce a novel model reprogramming based fairness promoting method, termed FAIRREPROGRAM Specifically, FAIRREPROGRAM considers a fixed ML model and optimizes set of vectors, named fairness trigger, concatenated on inputs to boost model fairness. We introduce an information-theoreee framework to explain the rationales of why FAIRREPROGRAM can improve model fairness. As implied by our theoretic framework as well as our empirical findings, the fairness trigger can effectively mask out the true demographic information with its strong, false demographic information. Extensive experiments demonstrate that our method could achieve better fairness improvements to retraining based methods with far-less training cost. We further empirically show fairness triggers enjoys great transferability and interpretability. We hope that PAPRREPROGRAM can inspire new fairness learning paradigms that are more feasible and flexible in practice.
Acknowledgement
The work of Yihua Zhang, Sijia Liu, and Shiyu Chang was partially supported by National Science Foundation (NSF) Grant IIS-2207052. The computing resources used in this work were partially supported by the MIT-IBM Watson Al Lab.
10

---

References
[1] Lucas Dixon, John Li, Jeffrey Scott Sorensen, Nithum Thain, and Lucy Vasserman, "Measuring and mitigating unintended bias in text classification," Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Sociery, 2018.
[2] J. Park, Jamin Shin, and Pascale Fung, "Reducing gender bias in abusive language detection," in EMNLP, 2018
[3] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez,, and Krishna P. Gummadi, "Faimess constraints: Mechanisms for fair classification," " in AISTATS, 2017.
[4] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna M. Wallach, 2018.
"A reductions approach to fair classification," ArXiv, vol.
[5] Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma, "Fairness-aware classifier with prejudice remover regularizer," in ECML/PKDD, 2012.
[6] Sina Baharlouei, Maher Nouiched, and Meisam Razaviyayn, "Rényi fair inference." arXiv: Learning, 2019.
[7] Adrián Pérez-Suay, Valero Laparra, Gonzalo Mateo-Garcia Jordi Muñoz-Mari, Luis Gómez- Chova, and Gustau Camps- Valls, "Fair kernel learning," " in ECML/PKDD, 2017.
[8] Gamaleldin Elsayed, Ian Goodfellow, and Jascha Sohl- Gohl-Dickstein, "Adversarial reprogramming of neural networks," arXiv preprint arXiv: 1066.11116,, 2018.
[9] Yun-Yun Tsai, Pin-Yu Chen, and Tsung-Yi Yi Ho, "Transfer learning without knowing: Repro- gramming black-box machine learning models with scarce data and limited resources," in International Conference on Machine Learning. PMLR, 2020, pp. 9614-9624.
[10] Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May, "Warp: Word-level adversarial reprogramming." arXiv preprint arXi::2101.00121 2021.
[11] Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Haitao Zheng, and Maosong Sun, "Openprompt: An open-source framework for prompt-learning," ArXiv, vol. abs/2111.01998 2022.
[12] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig, "Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing," ArKiv, vol. abs/2107.135588 2021.
[13] Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé, Miroslav Dudík, and H. Wal- lach, "Improving fairness in machine learning systems: What do industry practitioners need?," Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019.
[14] A. Chouldechova and Aaron Roth, "The frontiers of faireess in machine learning." ArXiv, vol. abs/18 0ass/810.088 10, 2018
[15] Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jicyu Zhao, Diba Mirza, Elizabeth M. Midding-Royer, Kai-Wei Chang, and William Yang Wang. "Mitigating gender bias in natural language processing: Literature review," in ACL, 2019.
[16] Ninarch Mchrabi, Fred Morstatter, Nripsuta Ani Saxena, Kristina Lerman, and A. G. Galstyan, "A survey on bias and faimesss in machine learning." ACM Computing Surveys (CSUR), vol. 54, pp. 1 -133, 35, 2021.
[17] Anjalie Field, Su Lin Blodgett, Zeerak Waseem, and Yulia Tsvetkov, "A survey of race, racism, and anti-racism in nlp," ArXiv, vol. abs/2106.11410 2021.
[18] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel, "Faireess through awareness," ArXiv, vol. abs/1104.339 104.3913, 2012.
[19] Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi, "Survey on causal-based machine learning faimess notions," ArXiv, vol. abs/2010.09553, 2020.

---

[20] T. Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang, "Faimess without demographies in repeated loss minimization," in ICML, 2018.
[21] Toon Calders and Sicco Verwer, "Three naive bayes approaches for discrimination free classifi- cation," Data Mining and Knowledge Discovery, vol. 21, pp. 277-292, 2010.
[22] Moritz Hardt, Eric Price, and Nathan Srebro, "Equality of opportunity in supervised learning." " in NIPS, 2016.
[23] Tim Räz, "Group faimess: Independence revisited," Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021.
[24] Simon Caton and Christian Haas, "Fairness in machine learning: A survey," ArXiv, vol. abs/2010.04053, 2020.
[25] Faisal Kamiran and Toon Calders, "Data preprocessing techniques for classification without discrimination," Knowledge and Information Systems, vol. 33, pp. 1-33, 2011.
[26] Richard S. Zemel, Ledell Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork, "Learn- ing fair representations," in ICML, 2013.
[27] Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Eduardo Scheidegger. and Suresh Venkatasubramanian, "Certifying and removing disparate impact," Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015.
[28] Flávio du Pin Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R. Varshney, "Optimized pre-processing for discrimination prevention," in NIPS, 2017.
[29] Aditya Grover, Kristy Choi, Rui Shu, and Stefano Ermon, "Fair generative modeling via weak supervision." in ICML, 2020.
[30] Guanhua Zhang, Bing Bai, Junqi Zhang, Kun Bai, Conghui Zhu, and T. Zhao, "Demographics should not be the reason of toxicity: Mitigating discrimination in text classifications with instance weighting," in ACL, 2020.
[31] Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh, "Fr-train: A mutual information-based approach to fair and robust training." in ICML, 2020.
[32] Wenbin Zhang and Eirini Ntoutsi, "Faht: An adaptive fairness-aware decision tree classifier," in IJCAI, 2019.
[33] B. Zhang, Blake Lemoine, and Margaret Mitchell, "Mitigating unwanted biases with adversarial learning," " Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, 2018.
[34] Felix Petersen, Debarghya Mukherjee, Yuekai Sun, and Mikhail Yurochkin, "Post-processing for individual faimess," " in NewrIPS,
2021.
[35] Dennis Wei, Karthikeyan Natesan Ramamurthy, and Flávio du Pin Calmon, "Optimized score transformation for fair classification." in AISTATS, 2020.
[36] Pranjal Awasthi, Matthäus Kleindessner, and Jamie H. Morgenstern, "Equalized odds postpro- cessing under imperfect group information," in AISTATS, 2020.
[37] Blake F. Woodworth, Suriya Gunasekar, Mesrob I. Ohannessian, and Nathan Srebro, "Learning non-diseriminatory predictors," ArXiv, vol. abs/1702.00081,, 2017.
[38] Alan Mishler and Edward H. Kennedy, "Fairness in risk assessment instruments: Post- processing to achieve counterfactual equalized odds." Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021.
[39] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. "Men also like shopping: Reducing gender bias amplification using corpus-level constraints." EMNLP, 2017.
12

---

[40] Michael P. Kim, Amirata Ghorbani, and James Y. Zou, "Multiaccuracy: Black-box post- processing for fairness in classification," Proceedings of the 2019 AAAI/ACM Conference on Al, Ethics, and Society, 2019.
[41] Pranay Kr. Lohia, Karthikeyan Natesan Ramamurthy, Manish Bhide, Diptikalyan Saha, Kush Varshney, and Ruchir Puri, "Bias mitigation post-processing for individual and group faimesss." ICASSP 20/9- 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2847-2851, 2019.
[42] Pranay Kr. Lohia, "Priority- post-pron post-processing bias mitigation for individual and group fairness," ArXiv, vol. abs/2102.00417, 2021.
[43] Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Mark D. M. Leiserson, "Decou- pled classifiers for group-fäir and efficient machine learning," in FAT, 2018.
[44] Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, L. Oneto, and Massimiliano Pontil, "Leveraging labeled and unlabeled data for consistent fair binary classification," in NeurIPS, 2019.
[45] Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan. and Phillip Isola, "Visual prompting: Modifying pixel space to adapt pre-trained models," arXiv preprint arXiv:2203.1727.7 2022.
[46] Paarth Neekhara, Shehzeen Hussain, Shlomo Dubnov, and Farinaz Koushanfar, "Adversarial reprogramming of text classification neural networks," arXiv preprint arXiv: 2018.
[47] Paarth Neekhara, Shehzeen Hussain, Jinglong Du, Shlomo Dubnov, Farinaz Koushanfar, and Julian McAuley, "Cross- modal adversarial reprogramming." in Proceedings of the IEEE/CVF Winter Conference Applications Computer Vision, 2022, pp. 2427-2435.
[48] Yang Zheng, Xiaoyi Feng, Zhaoqiang Xia, Xiaoyue Jiang, Ambra Demontis, Maura Pintor, Battista Biggio, and Fabio Roli, "Why adversarial reprogramming works, when it fails, and how to tell the difference," arXiv preprint arXiv:2 1673, 2021.
[49] Depeng Xu, Shuhan Yuan, Lu Zhang, and Xintao Wu, "Fairgan: Fairness-aware generative adversarial networks," " 2018 IEEE International Conference on Big Data (Big Data), pp. 570-575, 2018.
[50] Beatrice Nobile, Gabriele Santin, Bruno Lepri, and Pierpaolo Brutti, "Reprogramming fairgans with variational auto-encoders: A new transfer learning model," ArXiv, vol. abs/2203.05811. 2022.
[51] Tianyu Gao, Adam Fisch, and Danqi Chen, "Making pre-trained language models better few-shot learners," ArXiv, vol. abs/2012.15723, 2021.
[52] Xiang Lisa Li and Percy Liang, "Prefix-tuning: Optimizing continuous prompts for generation," " Proceedings of the 59th Anmual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), vol. Abs/2100.00199,, 2021.
[53] Timo Schick and Hinrich Schütze, "It's not just size that matters: Small language models are also few-shot learners," ArXiv, vol. abs/2009.07 001118,1111111111111111111111110 2021.
[54] Taylor Shin, Yasaman Razeghi, Robert Logan IV, Eric Wallace, and Sameer Singh, "Elic- iting knowledge from language models using automatically generated prompts," ArXiv, vol. abs/2010.1 15980, 2020.
[55] Shizhe Diao, Xuechun Li, Yong Lin, Zhichao Huang, and Tong Zhang, "Black-box prompt learning for pre-trained language models," ArXiv, vol. abs/2201 .08531, 2022.
[56) Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang, "Deep learning face attributes in the wild," in Proceedings of International Conference on Computer Vision (ICCV), December 2015.
13

---

[57] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Wei hua Hu, Michihiro Yasunaga, Richard L. Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang, "Wilds: A benchmark of in-the-wild distribution shifts," in ICMI., 2021.
[58]
Jigsaw/Conversation AI, "Jigsaw unintended bias in toxicity classification," 2019.
[59] Tian Xu, Jennifer White, Sinan Kalkan, and Hatice Gunes, "Investigating bias and fairness in facial expression recognition," in European Conference Computer Vision. Springer, 2020,
pp- 506-523.
[60] Saloni Dash and Amit Sharma, "Counterfactual generation and fairness evaluation using adversarially learned inference," ArXiv, 2020.
[61] Sunhee Hwang, Sungho Park, Dohyung Kim, Mirae Do, and Hyeran Byun, "Fairfacegan: Fairness-aware facial image-to-image translation," arXiv preprint arXiv:2012.00282, 2020.
[62] Jacob Devlin, Ming- wei Chang, Kenton Lee, and Kristina Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding." ArXiv, vol. abs/1810.04805, 2019.
[63] Ilya Loshchilov and Frank Hutter, "Decoupled weight decay regularization." arXiv preprint arXiv:1711.05101, 2017.
[64] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, "Deep residual learning for image recognition," " in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[65] Yoshua Bengio, Nicholas Léonard, and Aaron C. Courville, "Estimating or propagating gradients through stochastic neurons for conditional computation," ArXiv, vol. abs/1 300.3432, 2013.
[66] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra, "Grad-cam: Visual explanations from deep networks via gradient- based localization," in Proceedings of the IEEE international conference computer vision, 2017, pp. 618-626.
[67] Mukund Sundararajan, Ankur Taly, and Qiqi Yan, "Axiomatic attribution for deep networks," ArXiv, vol. abs/1703.01365, 2017.
[68] Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz- Richardson, "Captum: A unified and generic model interpretability library for pytorch," 2020.
[69] Ni Zhuang, Yan Yan, Si Chen, Hanzi Wang, and Chunhua Shen, "Multi-label learning based deep transfer neural network for facial attribute classification." Pattern Recognition, vol. 80, pp. 225-240, 2018.
[70] Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel, "The variational 2015.
fair autoencoder." arXiv preprint
[71] Geoff Pleiss, M. Raghavan, Felix Wu, J. Kleinberg, and Kilian Q. Weinberger, "On fairness and calibration," in NIPS, 2017.
[72] Faisal Kamiran, Asim Karim, and Xiangliang Zhang, "Decision theory for discrimination-awan classification," 2012 IEEE 12th International Conference on Data Mining, pp. 924 924,999,,440 2012.
[73] Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kamnan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang, "Al Fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias," Oct. 2018.
14

---

[74] Yimeng Zhang, Yuguang Yao, Jinghan Jia, Jinfeng Yi, Mingyi Hong, Shiyu Chang, and Sijia Liu, "How robustify black-box ml models? zeroth-order optimization perspective," arXiv preprini arXiv:2203. 14195, 2022.
[75] Arthur Asuncion and David Newman, "Uci machine learning repository," 2007.
15

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Appendix D
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Appendix[] D
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results?
[Yes] See Ap-
pendix C
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix
C
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes] See Section
and Appendix A
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 4.1 and Appendix A.2
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix A..2
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section
(b) Did you mention the license of the assets? [No]
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
16


Reviews:
[{"review_id": "U1hwY0v36Ep", "reviewer": "Reviewer_cTsj", "summary": "This paper introduces a model reprogramming based fairness promoting method, consisting of a fixed ML model and optimizing a set of vectors concatenated on inputs to boost model fairness. An information-theoretic framework is also introduced to explain the rationales of fairness booster. Experiments on one NLP and another CV dataset demonstrate the utility of this approach. ", "questions": "1. Many biased datasets are tabular represented, how does your work apply on tabular data? Appending additional dimension of vector directly? \n2. Can you summarize why the appended information can always cut off the biased information path? \n3. How do you distinguish this work with pre processing approaches?  ", "limitations": "See comments above. \n\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. This paper tackles the fairness issue with reprogramming which is a relatively less explored area in fairness-aware learning.\n2. The proposed method can finetune the pretrained model with computational efficiency. \n3. Experiments show the effectiveness of proposed method.\n\nWeaknesses:\n1. The paper is not self-contained with several key information in Appendix for a good understanding of the claimed contributions. For example, without the understanding of \"It can be shown (in Appendix C) that the posterior distributions..\" and \"it can be shown (in Appendix C) that the posterior distribution..\", the claimed information-theoretic framework remains unclear. \n2. With the above, it is not clear whether fairness can be really achieved by the proposed method.\n3. Some related work on reprogramming fairness (for example \"Reprogramming FairGANs with Variational Auto-Encoders: A New Transfer Learning Model\") is not mentioned in the literature review. This work also related to incremental fairness but lacks of relevant discussion (for example \"FAHT: An Adaptive Fairness-aware Decision Tree Classifier\").  \n4. The size ratio for civil comments is 1/5 which does not align with the motivation that infeasible to retrain or finetune well-trained large-scale models.\n5. The caption of Figure 1(a) and 1(b) are misplaced. \n\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1yrLfnhXXf8", "reviewer": "Reviewer_4SAR", "summary": "Traditional approaches for fairness in neural networks suggest training or fine-tuning the entire weights of the network to achieve desired fairness criteria. This paper, instead, proposes a reprogramming based technique, called FairReprogram. Considering the neural network fixed, FairReprogram appends a global fairness trigger to the input to achieve fairness improvement with respect to existing fairness metrics. The results are backed up with theoretical analysis and experimental evaluations in NLP and CV datasets.", "questions": "Please address questions in the `weakness` of the paper above.", "limitations": "There is no negative societal impact of the work, as far as I know.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n\n- The paper is very well written. Statements are supported with examples. The paper is also well-motivated as reprogramming towards fairness improvement is less costly than retraining or fine-tuning with fairness objectives. \n\nWeakness:\n\n- How are correlations among features handled in FairReprogram?\n\n- What is the intuition behind adding noise as fairness trigger, such as in patch trigger and border trigger? Does this mean demographic information is confined either in the border of the image or in a specific area of an image covered by the patch?\n\n- In Figure 1, perhaps the captions are wrong as 1(a) should be about border trigger and vice-versa. A careful recheck is encouraged.\n\n- Does the method extend to tabular data with fixed set of features in matrix form? \n\n- A comparison with existing fairness improvement techniques such as pre-processing, in-processing, post-processing fairness algorithms should be discussed. In which family of fairness algorithm does this approach belong to?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1cFuszm2Ot2", "reviewer": "Reviewer_4uyL", "summary": "This paper discusses how to learn a constant feature that can be concatenated to a frozen pre-trained embedding to promote statistical group fairness in a downstream classifier. These extra features, called the “fairness trigger”, are jointly trained with the prediction model with the aim of inducing a demographic parity or equalized odds property in the final solution.", "questions": "* I am slightly concerned that the in/post-processing methods could be doing worse simply because of the instability of adversarial training, or due to overfitting given the limited number of data available during fine tuning for the post-processing method [line 244]. I would suggest adding an MMD baseline as an additional baseline (e.g. as applied in https://arxiv.org/abs/1511.00830). This would hopefully be more stable, but may also suffer from overfitting.\n* In my opinion, the claim of achieving “lower bias scores over two fairness criteria in the CelebA dataset” [lines 54–55] is not meaningful without defining the prediction task, since many different tasks (related to fairness and otherwise) have been proposed that use the crowdsourced attributes of CelebA. The authors do a good job of providing this context in Section 4 [lines 219–200]; I would suggest adding a basic description of the task to Section 1 wherever the results are summarized.\n* I’m unclear on why the JTT paper [12] is cited as a paper that “[learns] task-specific embedding prompts concatenated to the inputs” [lines 36–37]. My understanding of this paper is that it uses the error cases of an ERM reference model to train a robust model via importance weighting. Could the authors clarify the citation (in this context) or remove it?\n* I was interested in the experiment showing that the method is not overly sensitive to random seed [lines 291–300]. Calling this a “transfer” is a bit generous—there is no notion of a distribution shift or new prediction task, or a change in the model architecture—but it is good to know that a learned prompt can be reused if the model is retrained on the same data. The claim at the end of the paper that the method “enjoys great transferability” [line 356] seems like an overstatement. I would be especially concerned that adding a single constant feature could suffer from covariate/distribution shift, which was not examined. \n* In Figure 1 it seems that the captions and images should be swapped. I.e. Fig 1a is the border trigger, not the patch trigger, if I understand correctly.\n", "limitations": "Yes, the authors discuss several limitations in the appendix. I would encourage including them in the main paper.\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n* Demonstrates that reprogramming techniques can be adapted to target group fairness metrics in CV and NLP contexts.\nInteresting qualitative analysis of the resulting models [Figs 6, 7] suggesting that the learned concatenated constant features cause the downstream classifier to focus less on spurious features (in this case, features that indicate demographic membership). \n* Provides a conceptual proof of concept for reprogramming in a simple bag-of-words classification setting where data are generated according to an anti-causal graphical model [Figure 2].\nWeaknesses\n* I found the proposed method interesting and I think the neurips community will as well. My main complaint is that the paper, in my opinion, oversells this approach as strictly superior to previous methods. When we apply the fairness trigger in the original input space (e.g. CV), don’t we still need to backpropagate through the entire network to learn the fairness trigger? If yes, there could be a memory savings as we don’t tune as many parameters, but it’s not clear to me that the method would train substantially faster than a finetuning baseline. On the other hand I can see why you would save on compute when applying the fairness trigger *in* the embedding space (e.g. NLP)\n* The paper mainly compares against fine tuning methods, but there are other feasible approaches such as adding a linear probe on the frozen embedding (which I don’t believe is considered the experiments). When we teach linear regression we often bring up two alternate parameterizations: (a) \\hat y = x^T W + b and (b) \\hat y = (\\tilde x)^T W. These are equivalent if we define \\tilde x as x with an extra constant feature. By analogy, I wonder if there is some equivalence between the fairness trigger (learned bias) and a linear probe/transformation applied on the embedding (learned projection). If the fairness trigger is learning a constant perturbation along some subspace with lots of demographic information, is it possible that a similar solution could be found by simply projecting away that subspace using a linear probe? It seems possible to me, and makes me wonder if the experimental gains are due to the fact that the fairness trigger optimizes a lower dimensional parameter space that would be less prone to overfitting given the limited data available for post-processing. Even if this were the case (might be possible to find out by adding linear probes as baselines in the experiments) the paper would be interesting, but might suggest that fairness reprogramming is best under a constrained data budget, not best in general. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept