Package: canonical-sets
AI systems can create, propagate, support, and automate bias in decision-making processes. To mitigate biased decisions,
we both need to understand the origin of the bias and define what it means for an algorithm to make fair decisions.
By Locating Unfairness through Canonical Inverse Design (LUCID), we generate a canonical set that shows the desired inputs
for a model given a preferred output. The canonical set reveals the model’s internal logic and exposes potential unethical
biases by repeatedly interrogating the decision-making process.LUCID-GAN extends on LUCID by generating canonical inputs via a conditional generative model instead of
gradient-based inverse design. LUCID-GAN generates canonical inputs conditional on the predictions of the model under
fairness evaluation. LUCID-GAN has several benefits, including that it applies to non-differentiable models, ensures
that a canonical set consists of realistic inputs, and allows us to assess indirect discrimination and explicitly
check for intersectional unfairness.Read our paper onLUCIDandLUCID-GANfor more details, or check out thedocumentation.We encourage everyone tocontributeto this project by submitting an issue or a pull request!InstallationInstallcanonical_setsfrom PyPi.pipinstallcanonical_setsFor development install, seecontribute. You can also check thedocumentation.UsageLUCIDLUCIDcan be used for the gradient-based inverse design to generate canonical sets, and is available for bothPyTorchandTensorflowmodels. It only requires a model, a preferred output, and an example input
(which is often a part of the training data). The results are stored in apd.DataFrame, and can be accessed by
callingresults. It’s fully customizable, but can also be used out-of-the-box for a wide range of
applications by using its default settings:importpandasaspdfromcanonical_sets.dataimportAdultfromcanonical_sets.modelsimportClassifierTFfromcanonical_setsimportLUCIDadult=Adult()model=ClassifierTF(2)outputs=pd.DataFrame([[0,1]],columns=["<=50K",">50K"])example_data=adult.train_datalucid=LUCID(model,outputs,example_data)lucid.results.head()LUCID-GANLUCIDGANgenerates canonical sets by using conditional generative models (GANs). This approach has several benefits,
including that it applies to non–differentiable models, ensures that a canonical set consists of realistic inputs,
and allows us to assess indirect discrimination and explicitly check for intersectional unfairness. LUCID-GAN only
requires the input and predictions of a black-box model. It’s fully customizable, but can also be used out-of-the-box
for a wide range of applications by using its default settings:importpandasaspdfromcanonical_sets.dataimportAdultfromcanonical_sets.modelsimportClassifierTFfromcanonical_setsimportLUCIDGANmodel=ClassifierTF(2)adult=Adult()# we need original data as LUCID-GAN does some preprocessingtest_data=adult.inverse_preprocess(adult.test_data)# we only require the predictions for the positive classpreds=model.predict(adult.test_data.to_numpy())[:,1]data=pd.concat([test_data,pd.DataFrame(preds,columns=["preds"])],axis=1)lucidgan=LUCIDGAN(epochs=5)lucidgan.fit(data,conditional=["preds"])samples=lucidgan.sample(100,conditional=pd.DataFrame({"preds":[1]}))samples.head()For detailed examples seeexamplesand for the source code seecanonical_sets. ForLUCID, we advice to start with either thetensorfloworpytorchexample, and then the advanced example. ForLUCIDGAN, you can replicate the experiments from the paper
with theGAN_adultandGAN_compasexamples. You can also check thedocumentationfor more details.
If you have any remaining questions, feel free to submit an issue or PR!Output-based group metricsMost group fairness notions focus on the equality of outcome by computing statistical parity metrics on a model’s output.
The two most prominent examples of these statistical output-based metrics are Demographic Parity (DP) and Equality Of Opportunity (EOP).
In DP, we compare the Positivity Rate (PR) of the subpopulations under fairness evaluation, and in EOP, we compare the True Positive Rate (TPR).
The choice between DP and EOP depends on the underlying assumptions and worldview of the evaluator.
TheMetricsclass allows you to compute these metrics for binary classification tasks given the predictions and ground truth:fromcanonical_sets.dataimportAdultfromcanonical_sets.modelsimportClassifierTFfromcanonical_sets.groupimportMetricsmodel=ClassifierTF(2)adult=Adult()preds=model.predict(adult.test_data.to_numpy()).argmax(axis=1)targets=adult.test_labels[">50K"]metrics=Metrics(preds,targets)metrics.metricsDatacanonical_setscontains some functionality to easily access commonly used data sets in the fairness literature:fromcanonical_setsimportAdult,Compasadult=Adult()adult.train_data.head()compas=Compas()compas.train_data.head()The default settings can be customized to change the pre-processing, splitting, etc. Seeexamplesfor details.
You can also check thedocumentation.CommunityIf you are interested in cross-disciplinary research related to machine learning, feel free to:Follow DataLab onTwitter.Check thewebsite.Read ourpapers.DisclaimerThe package and the code is provided “as-is” and there is NO WARRANTY of any kind.
Use it only if the content and output files make sense to you.AcknowledgementsThis project benefited from financial support from Innoviris.LUCIDGANis based on theCTGANclass from thectganpackage. It has been extended to fix
several bugs (see myPRon theCTGANGitHub page) and to allow for the extension of the conditional
vector. A part of the code and comments is identical to the originalCTGANclass.Citation@inproceedings{mazijn_lucid_2023,
  title={{LUCID: Exposing Algorithmic Bias through Inverse Design}},
  author={Mazijn, Carmen and Prunkl, Carina and Algaba, Andres and Danckaert, Jan and Ginis, Vincent},
  booktitle={Thirty-Seventh AAAI Conference on Artificial Intelligence (accepted)},
  year={2023},
}

@article{algaba_lucidgan_2022,
  title={{LUCID-GAN: Conditional Generative Models to Locate Unfairness}},
  author={Algaba, Andres and Mazijn, Carmen and Prunkl, Carina and Danckaert, Jan and Ginis, Vincent},
  year={2022},
  journal={Working paper}
}