Package: fairgbm
FairGBMFairGBM is an easy-to-use and lightweight fairness-aware ML algorithm with state-of-the-art performance on tabular datasets.FairGBM builds upon the popularLightGBMalgorithm and adds customizable
constraints for group-wise fairness (e.g., equal opportunity, predictive equality) and other global goals (e.g.,
specific Recall or FPR prediction targets).Please consultthe paperfor further details.InstallGetting startedParameter listfit(X, Y, constraint_group=S)FeaturesFairness constraintsGlobal constraintsTechnical detailsCiting FairGBMInstallCurrently, compatibility is only maintained with Linux OS.FairGBM can be installed fromPyPIpip install fairgbmor from GitHubgit clone --recurse-submodules https://github.com/feedzai/fairgbm.git
pip install fairgbm/python-package/NoteInstall requiresCMakeand an up-to-date C++ compiler (gcc, clang, or mingw).
You may need to install wheel viapip install wheelfirst.
For Linux users, glibc >= 2.14 is required.
For more details see LightGBM'sinstallation guide,
or followthis linkfor the Python package
installation instructions.Getting startedYou can get FairGBM up and running in just a few lines of Python code:fromfairgbmimportFairGBMClassifier# Instantiatefairgbm_clf=FairGBMClassifier(constraint_type="FNR",# constraint on equal group-wise TPR (equal opportunity)n_estimators=200,# core parameters from vanilla LightGBMrandom_state=42,# ...)# Train using features (X), labels (Y), and sensitive attributes (S)fairgbm_clf.fit(X,Y,constraint_group=S)# NOTE: labels (Y) and sensitive attributes (S) must be in numeric format# PredictY_test_pred=fairgbm_clf.predict_proba(X_test)[:,-1]# Compute continuous class probabilities (recommended)# Y_test_pred = fairgbm_clf.predict(X_test)             # Or compute discrete class predictionsA more in-depth explanation and other usage examples can be found in theexamples folder.For Python examples see thenotebooks folder.Parameter listThe following parameters can be used as key-word arguments for theFairGBMClassifierPython class.NameDescriptionDefaultconstraint_typeThe type of fairness (group-wise equality) constraint to use (if any).FPR,FNRglobal_constraint_typeThe type of global equality constraint to use (if any).Nonemultiplier_learning_rateThe learning rate for the gradient ascent step (w.r.t. Lagrange multipliers).0.1constraint_fpr_toleranceThe slack when fulfillinggroup-wiseFPR constraints.0.01constraint_fnr_toleranceThe slack when fulfillinggroup-wiseFNR constraints.0.01global_target_fprTarget rate for theglobalFPR (inequality) constraint.Noneglobal_target_fnrTarget rate for theglobalFNR (inequality) constraint.Noneconstraint_stepwise_proxyDifferentiable proxy for the step-wise function ingroup-wiseconstraints.cross_entropyobjective_stepwise_proxyDifferentiable proxy for the step-wise function inglobalconstraints.cross_entropystepwise_proxy_marginIntercept value for the proxy function: value atf(logodds=0.0)1.0score_thresholdScore threshold used when assessinggroup-wiseFPR or FNR in training.0.5global_score_thresholdScore threshold used when assessingglobalFPR or FNR in training.0.5init_multipliersThe initial value of the Lagrange multipliers.0for each constraint...Any coreLGBMClassifierparametercan be used with FairGBM as well.Please consultthis listfor a detailed
view of all vanilla LightGBM parameters (e.g.,n_estimators,n_jobs, ...).NoteTheobjectiveis the only core LightGBM parameter that cannot be changed when using FairGBM, as you must use
the constrained loss functionobjective="constrained_cross_entropy".
Using a standard non-constrained objective will fallback to using standard LightGBM.fit(X, Y, constraint_group=S)In addition to the usualfitarguments, featuresXand labelsY, FairGBM takes in the sensitive attributesScolumn for training.Regarding the sensitive attributes columnS:It should be in numeric format, and have each different protected group take a different integer value, starting at0.It is not restricted to binary sensitive attributes: you can usetwo or moredifferent groups encoded in the same column;It is only required for training andnotfor computing predictions;Here is an example pre-processing for the sensitive attributes on the UCI Adult dataset:# Given X, Y, SX,Y,S=load_dataset()# The sensitive attributes S must be in numeric formatS=[1.ifval=="Female"else0.forvalinS]# The labels Y must be binary and in numeric format: {0, 1}Y=[1.ifval==">50K"else0.forvalinY]# And the features X may be numeric or categorical, but make sure categorical columns are in the correct formatX:Union[pd.DataFrame,np.ndarray]# any array-like can be used# Train FairGBMfairgbm_clf.fit(X,Y,constraint_group=S)FeaturesFairGBM enables you to train a GBM model tominimize a loss function(e.g., cross-entropy)subject to fairness
constraints(e.g., equal opportunity).Namely, you can target equality of performance metrics (FPR, FNR, or both) across instances fromtwo or moredifferent
protected groups (seefairness constraintssection).
Simultaneously (and optionally), you can add global constraints on specific metrics (seeglobal constraintssection).Fairness constraintsYou can use FairGBM to equalize the following metrics acrosstwo or moreprotected groups:Equalize FNR (equivalent to equalizing TPR or Recall)also known asequal opportunity(Hardtet al., 2016)Equalize FPR (equivalent to equalizing TNR or Specificity)also known aspredictive equality(Corbett-Davieset al., 2017)Equalize both FNR and FPR simultaneouslyalso known asequal odds(Hardtet al., 2016)Example forequality of opportunityin college admissions:
your likelihood of getting admitted to a certain college (predicted positive) given that you're a qualified candidate
(label positive) should be the same regardless of your race (sensitive attribute).Global constraintsYou can also target specific FNR or FPR goals.
For example, in cases where high accuracy is trivially achieved (e.g., problems with high class imbalance),
you may want to maximize TPR with a constraint on FPR (e.g., "maximize TPR with at most 5% FPR").
You can set a constraint on global FPR ≤ 0.05 by usingglobal_target_fpr=0.05andglobal_constraint_type="FPR".You can simultaneously set constraints on group-wise metrics (fairness constraints) and constraints on global metrics.Technical DetailsFairGBM is a framework that enablesconstrained optimizationof Gradient Boosting Machines (GBMs).
This way, we can train a GBM model to minimize some loss function (usually thebinary cross-entropy) subject to a set
of constraints that should be met in the training dataset (e.g., equality of opportunity).FairGBM applies themethod of Lagrange multipliers, and uses
iterative and interleaved steps of gradient descent (on the function space, by adding new trees to the GBM model) and
gradient ascent (on the space of Lagrange multipliers,Λ).The main obstacle with enforcing fairness constraints in training is that these constraints are oftennon-differentiable. To side-step this issue, we use a differentiable proxy of the step-wise function.
The following plot shows an example ofhinge-basedandcross-entropy-basedproxies for thefalse positivevalue
of alabel negativeinstance.For a more in-depth explanation of FairGBM please consultthe paper.ContactFor commercial uses of FairGBM please contactoss-licenses@feedzai.com.Citing FairGBMThe paper is publicly available at thisarXiv link.@misc{cruz2022fairgbm,
  doi = {10.48550/ARXIV.2209.07850},
  url = {https://arxiv.org/abs/2209.07850},
  author = {Cruz, Andr{\'{e}} F and Bel{\'{e}}m, Catarina and Bravo, Jo{\~{a}}o and Saleiro, Pedro and Bizarro, Pedro},
  keywords = {Machine Learning (cs.LG), Artificial Intelligence (cs.AI), Computers and Society (cs.CY), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {FairGBM: Gradient Boosting with Fairness Constraints},
  publisher = {arXiv},
  year = {2022},
  copyright = {arXiv.org perpetual, non-exclusive license}
}