Package: omnixai
OmniXAI: A Library for Explainable AITable of ContentsIntroductionInstallationGetting StartedDocumentationTutorialsDeploymentDashboard DemoHow to ContributeTechnical Report and Citing OmniXAIWhat's NewThe latest version includes an experimental GPT explainer. This explainer leverages the outcomes
produced by SHAP and MACE to formulate the input prompt for ChatGPT. Subsequently, ChatGPT
analyzes these results and generates the corresponding explanations that provide developers with
a clearer understanding of the rationale behind the model's predictions.IntroductionOmniXAI (short for Omni eXplainable AI) is a Python machine-learning library for explainable AI (XAI), offering omni-way explainable AI and interpretable
machine learning capabilities to address many pain points in explaining decisions made by machine learning
models in practice. OmniXAI aims to be a one-stop comprehensive library that makes explainable AI easy for
data scientists, ML researchers and practitioners who need explanation for various types of data, models and
explanation methods at different stages of ML process:OmniXAI includes a rich family of explanation methods integrated in a unified interface, which
supports multiple data types (tabular data, images, texts, time-series), multiple types of ML models
(traditional ML in Scikit-learn and deep learning models in PyTorch/TensorFlow), and a range of diverse explaination
methods including "model-specific" and "model-agnostic" methods (such as feature-attribution explanation,
counterfactual explanation, gradient-based explanation, feature visualization, etc). For practitioners, OmniXAI provides an easy-to-use
unified interface to generate the explanations for their applications by only writing a few lines of
codes, and also a GUI dashboard for visualization for obtaining more insights about decisions.The following table shows the supported explanation methods and features in our library.
We will continue improving this library to make it more comprehensive in the future.MethodModel TypeExplanation TypeEDATabularImageTextTimeseriesFeature analysisNAGlobal✅Feature selectionNAGlobal✅Prediction metricsBlack boxGlobal✅✅✅✅Bias metricsBlack boxGlobal✅Partial dependence plotsBlack boxGlobal✅Accumulated local effectsBlack boxGlobal✅Sensitivity analysisBlack boxGlobal✅Permutation explanationBlack boxGlobal✅Feature visualizationTorch or TFGlobal✅Feature mapsTorch or TFLocal✅GPT explainerBlack boxLocal✅LIMEBlack boxLocal✅✅✅SHAPBlack box*Local✅✅✅✅What-ifBlack boxLocal✅Integrated gradientTorch or TFLocal✅✅✅CounterfactualBlack box*Local✅✅✅✅Contrastive explanationTorch or TFLocal✅Grad-CAM, Grad-CAM++Torch or TFLocal✅Score-CAMTorch or TFLocal✅Layer-CAMTorch or TFLocal✅Smooth gradientTorch or TFLocal✅Guided backpropagationTorch or TFLocal✅Learning to explainBlack boxLocal✅✅✅Linear modelsLinear modelsGlobal and Local✅Tree modelsTree modelsGlobal and Local✅SHAPaccepts black box models for tabular data, PyTorch/Tensorflow models for image data, transformer models
for text data.Counterfactualaccepts black box models for tabular, text and time-series data, and PyTorch/Tensorflow models for
image data.Thistableshows the comparison between our toolkit/library and other existing XAI toolkits/libraries
in literature.OmniXAI also integrates ChatGPT for generating plain text explanations given a classification/regression
model on tabular datasets.The generated results may not be 100% accurate, but it is worth trying this
explainer (we will continue improving the input prompts).InstallationYou can installomnixaifrom PyPI by callingpip install omnixai. You may install from source by
cloning the OmniXAI repo, navigating to the root directory, and callingpip install ., orpip install -e .to install in editable mode. You may install additional dependencies:For plotting & visualization: Callingpip install omnixai[plot], orpip install .[plot]from the
root directory of the repo.For vision tasks: Callingpip install omnixai[vision], orpip install .[vision]from the
root directory of the repo.For NLP tasks: Callingpip install omnixai[nlp], orpip install .[nlp]from the
root directory of the repo.Install all the dependencies: Callingpip install omnixai[all], orpip install .[all]from the
root directory of the repo.Getting StartedFor example code and an introduction to the library, see the Jupyter notebooks intutorials, and the guided walkthroughhere. A dashboard demo can be foundhere.Some examples:Tabular classificationTabular regressionImage classificationText classificationTime-series anomaly detectionVision-language tasksRanking tasksFeature visualizationCheck feature mapsGPT explainer for tabularTo get started, we recommend the linked tutorials intutorials.
In general, we recommend usingTabularExplainer,VisionExplainer,NLPExplainerandTimeseriesExplainerfor tabular, vision, NLP and time-series tasks, respectively, and usingDataAnalyzerandPredictionAnalyzerfor feature analysis and prediction result analysis.
These classes act as the factories of the individual explainers supported in OmniXAI, providing a simpler
interface to generate multiple explanations. To generate explanations, you only need to specifyThe ML model to explain: e.g., a scikit-learn model, a tensorflow model, a pytorch model or a black-box prediction function.The pre-processing function: i.e., converting raw input features into the model inputs.The post-processing function (optional): e.g., converting the model outputs into class probabilities.The explainers to apply: e.g., SHAP, MACE, Grad-CAM.Besides using these classes, you can also create a single explainer defined in theomnixai.explainerspackage, e.g.,ShapTabular,GradCAM,IntegratedGradientorFeatureVisualizer.Let's take the income prediction task as an example.
Thedatasetused in this example is for income prediction.
We recommend using data classTabularto represent a tabular dataset. To create aTabularinstance given a pandas
dataframe, you need to specify the dataframe, the categorical feature names (if exists) and the target/label
column name (if exists).fromomnixai.data.tabularimportTabular# Load the datasetfeature_names=["Age","Workclass","fnlwgt","Education","Education-Num","Marital Status","Occupation","Relationship","Race","Sex","Capital Gain","Capital Loss","Hours per week","Country","label"]df=pd.DataFrame(np.genfromtxt('adult.data',delimiter=', ',dtype=str),columns=feature_names)tabular_data=Tabular(df,categorical_columns=[feature_names[i]foriin[1,3,5,6,7,8,9,13]],target_column='label')The packageomnixai.preprocessingprovides several useful preprocessing functions
for aTabularinstance.TabularTransformis a special transform designed for processing tabular data.
By default, it converts categorical features into one-hot encoding, and keeps continuous-valued features.
The  methodtransformofTabularTransformtransforms aTabularinstance to a numpy array.
If theTabularinstance has a target/label column, the last column of the numpy array
will be the target/label. You can apply any customized preprocessing functions instead of usingTabularTransform.
After data preprocessing, let's train a XGBoost classifier for this task.fromomnixai.preprocessing.tabularimportTabularTransform# Data preprocessingtransformer=TabularTransform().fit(tabular_data)class_names=transformer.class_namesx=transformer.transform(tabular_data)# Split into training and test datasetstrain,test,train_labels,test_labels=\sklearn.model_selection.train_test_split(x[:,:-1],x[:,-1],train_size=0.80)# Train an XGBoost model (the last column of `x` is the label column after transformation)model=xgboost.XGBClassifier(n_estimators=300,max_depth=5)model.fit(train,train_labels)# Convert the transformed data back to Tabular instancestrain_data=transformer.invert(train)test_data=transformer.invert(test)To initializeTabularExplainer, the following parameters need to be set:explainers: The names of the explainers to apply, e.g., ["lime", "shap", "mace", "pdp"].data: The data used to initialize explainers.datais the training dataset for training the
machine learning model. If the training dataset is too large,datacan be a subset of it by applyingomnixai.sampler.tabular.Sampler.subsample.model: The ML model to explain, e.g., a scikit-learn model, a tensorflow model or a pytorch model.preprocess: The preprocessing function converting the raw inputs (ATabularinstance) into the inputs ofmodel.postprocess(optional): The postprocessing function transforming the outputs ofmodelto a
user-specific form, e.g., the predicted probability for each class. The output ofpostprocessshould be a numpy array.mode: The task type, e.g., "classification" or "regression".The preprocessing function takes aTabularinstance as its input and outputs the processed features that
the ML model consumes. In this example, we simply calltransformer.transform. If you use some customized transforms
on pandas dataframes, the preprocess function has this format:lambda z: some_transform(z.to_pd()). If the output ofmodelis not a numpy array,postprocessneeds to be set to convert it into a numpy array.fromomnixai.explainers.tabularimportTabularExplainer# Initialize a TabularExplainerexplainer=TabularExplainer(explainers=["lime","shap","mace","pdp","ale"],# The explainers to applymode="classification",# The task typedata=train_data,# The data for initializing the explainersmodel=model,# The ML model to explainpreprocess=lambdaz:transformer.transform(z),# Converts raw features into the model inputsparams={"mace":{"ignored_features":["Sex","Race","Relationship","Capital Loss"]}}# Additional parameters)In this example, LIME, SHAP and MACE generate local explanations while PDP (partial dependence plot)
generates global explanations.explainer.explainreturns the local explanations generated by the
three methods given the test instances, andexplainer.explain_globalreturns the global explanations
generated by PDP.TabularExplainerhides all the details behind the explainers, so we can simply call
these two methods to generate explanations.# Generate explanationstest_instances=test_data[:5]local_explanations=explainer.explain(X=test_instances)global_explanations=explainer.explain_global(params={"pdp":{"features":["Age","Education-Num","Capital Gain","Capital Loss","Hours per week","Education","Marital Status","Occupation"]}})Similarly, we create aPredictionAnalyzerfor computing performance metrics for this classification task.
To initializePredictionAnalyzer, the following parameters need to be set:mode: The task type, e.g., "classification" or "regression".test_data: The test dataset, which should be aTabularinstance.test_targets: The test labels or targets. For classification,test_targetsshould be integers
(processed by a LabelEncoder) and match the class probabilities returned by the ML model.preprocess: The preprocessing function converting the raw data (aTabularinstance) into the inputs ofmodel.postprocess(optional): The postprocessing function transforming the outputs ofmodelto a user-specific form,
e.g., the predicted probability for each class. The output ofpostprocessshould be a numpy array.fromomnixai.explainers.predictionimportPredictionAnalyzeranalyzer=PredictionAnalyzer(mode="classification",test_data=test_data,# The test dataset (a `Tabular` instance)test_targets=test_labels,# The test labels (a numpy array)model=model,# The ML modelpreprocess=lambdaz:transformer.transform(z)# Converts raw features into the model inputs)prediction_explanations=analyzer.explain()Given the generated explanations, we can launch a dashboard (a Dash app) for visualization by setting the test
instance, the local explanations, the global explanations, the prediction metrics, the class names, and additional
parameters for visualization (optional). If you want "what-if" analysis, you can set theexplainerparameter
when initializing the dashboard. For "what-if" analysis, OmniXAI also allows you to set a second explainer
if you want to compare different models.fromomnixai.visualization.dashboardimportDashboard# Launch a dashboard for visualizationdashboard=Dashboard(instances=test_instances,# The instances to explainlocal_explanations=local_explanations,# Set the local explanationsglobal_explanations=global_explanations,# Set the global explanationsprediction_explanations=prediction_explanations,# Set the prediction metricsclass_names=class_names,# Set class namesexplainer=explainer# The created TabularExplainer for what if analysis)dashboard.show()# Launch the dashboardAfter opening the Dash app in the browser, we will see a dashboard showing the explanations:You can also use the GPT explainer to generate explanations in text for tabular models:explainer=TabularExplainer(explainers=["gpt"],# The GPT explainer to applymode="classification",# The task typedata=train_data,# The data for initializing the explainersmodel=model,# The ML model to explainpreprocess=lambdaz:transformer.transform(z),# Converts raw features into the model inputsparams={"gpt":{"apikey":"xxxx"}}# Set the OpenAI API KEY)local_explanations=explainer.explain(X=test_instances)For vision tasks, the same interface is used to create explainers and generate explanations.
Let's take an image classification model as an example.fromomnixai.explainers.visionimportVisionExplainerfromomnixai.visualization.dashboardimportDashboardexplainer=VisionExplainer(explainers=["gradcam","lime","ig","ce","feature_visualization"],mode="classification",model=model,# An image classification model, e.g., ResNet50preprocess=preprocess,# The preprocessing functionpostprocess=postprocess,# The postprocessing functionparams={# Set the target layer for GradCAM"gradcam":{"target_layer":model.layer4[-1]},# Set the objective for feature visualization"feature_visualization":{"objectives":[{"layer":model.layer4[-3],"type":"channel","index":list(range(6))}]}},)# Generate explanations of GradCAM, LIME, IG and CElocal_explanations=explainer.explain(test_img)# Generate explanations of feature visualizationglobal_explanations=explainer.explain_global()# Launch the dashboarddashboard=Dashboard(instances=test_img,local_explanations=local_explanations,global_explanations=global_explanations)dashboard.show()The following figure shows the dashboard of these explanations:For NLP tasks and time-series forecasting/anomaly detection, OmniXAI also provides the same interface
to generate and visualize explanations. This figure shows a dashboard example of text classification
and time-series anomaly detection:DeploymentThe explainers in OmniXAI can be easily deployed viaBentoML.
BentoML is a popular open-source unified model serving framework, supporting multiple platforms including
AWS, GCP, Heroku, etc. We implemented the BentoML-format interfaces for OmniXAI so that users only need
few lines of code to deploy their selected explainers.Let's take the income prediction task as an example. Given the trained model and the initialized explainer,
you only need to save the explainer in the BentoML local model store:fromomnixai.explainers.tabularimportTabularExplainerfromomnixai.deployment.bentoml.omnixaiimportsave_modelexplainer=TabularExplainer(explainers=["lime","shap","mace","pdp","ale"],mode="classification",data=train_data,model=model,preprocess=lambdaz:transformer.transform(z),params={"mace":{"ignored_features":["Sex","Race","Relationship","Capital Loss"]}})save_model("tabular_explainer",explainer)And then create a file (e.g., service.py) for the ML service code:fromomnixai.deployment.bentoml.omnixaiimportinit_servicesvc=init_service(model_tag="tabular_explainer:latest",task_type="tabular",service_name="tabular_explainer")Theinit_servicefunction defines two API endpoints, i.e.,/predictfor model predictions and/explainfor
generating explanations. You can start an API server locally to test the service code above:bentomlserveservice:svc--reloadThe endpoints can be accessed locally:importrequestsfromrequests_toolbelt.multipart.encoderimportMultipartEncoderdata='["39", "State-gov", "77516", "Bachelors", "13", "Never-married", '\'"Adm-clerical", "Not-in-family", "White", "Male", "2174", "0", "40", "United-States"]'# Test the prediction endpointprediction=requests.post("http://0.0.0.0:3000/predict",headers={"content-type":"application/json"},data=data).text# Test the explanation endpointm=MultipartEncoder(fields={"data":data,"params":'{"lime": {"y": [0]}}',})result=requests.post("http://0.0.0.0:3000/explain",headers={"Content-Type":m.content_type},data=m).text# Parse the resultsfromomnixai.explainers.baseimportAutoExplainerBaseexp=AutoExplainerBase.parse_explanations_from_json(result)forname,explanationinexp.items():explanation.ipython_plot()You can build Bento for deployment by following the steps shown in theBentoML repo. For more examples, please
checkTabular,Vision,NLP.How to ContributeWe welcome the contribution from the open-source community to improve the library!To add a new explanation method/feature into the library, please follow the template and steps demonstrated in thisdocumentation.Technical Report and Citing OmniXAIYou can find more details in our technical report:https://arxiv.org/abs/2206.01612If you're using OmniXAI in your research or applications, please cite using this BibTeX:@article{wenzhuo2022-omnixai,
  author    = {Wenzhuo Yang and Hung Le and Silvio Savarese and Steven Hoi},
  title     = {OmniXAI: A Library for Explainable AI},
  year      = {2022},
  doi       = {10.48550/ARXIV.2206.01612},
  url       = {https://arxiv.org/abs/2206.01612},
  archivePrefix = {arXiv},
  eprint    = {206.01612},
}Contact UsIf you have any questions, comments or suggestions, please do not hesitate to contact us atomnixai@salesforce.com.LicenseBSD 3-Clause License