Package: explainy
explainy - black-box model explanations for humansexplainyis a library for generating machine learning models explanations in Python. It uses methods fromMachine Learning Explainabilityand provides a standardized API to create feature importance explanations for samples.The API is inspired byscikit-learnand has three core methodsexplain(),plot()and,importance(). The explanations are generated in the form of texts and plots.explainycomes with four different algorithms to create eitherglobalorlocalandcontrastiveornon-contrastivemodel explanations.MethodTypeExplanationsClassificationRegressionPermutation Feature Importancenon-contrastiveglobal:star::star:Shap Valuesnon-contrastivelocal:star::star:Surrogate Modelcontrastiveglobal:star::star:Counterfactual Examplecontrastivelocal:star::star:Description:global: explanation of system functionality (all samples have the same explanation)local: explanation of decision rationale (each sample has its own explanation)contrastive: tracing of decision path (differences to other outcomes are described)non-contrastive: parameter weighting (the feature importance is reported)Documentationhttps://explainy.readthedocs.ioInstall explainypip install explainyFurther, installgraphviz(version 9.0.0 or later) for plotting tree surrogate models:Windowschoco install graphvizMacbrew install graphvizLinux: Ubuntu packagessudo apt install graphvizFurther details on how to installgraphvizcan be found in the officialgraphviz docs.Also, make sure that the folder with thedotexecutable is added to your systemsPATH. You can find further detailshere.Usage📚 A comprehensive example of theexplainyAPI can be found in this📖 Or in theexample sectionof the documentationInitialize and train ascikit-learnmodel:importpandasaspdfromsklearn.datasetsimportload_diabetesfromsklearn.ensembleimportRandomForestRegressorfromsklearn.model_selectionimporttrain_test_splitdiabetes=load_diabetes()X_train,X_test,y_train,y_test=train_test_split(diabetes.data,diabetes.target,random_state=0)X_test=pd.DataFrame(X_test,columns=diabetes.feature_names)y_test=pd.DataFrame(y_test)model=RandomForestRegressor(random_state=0)model.fit(X_train,y_train)Initialize thePermutationExplanation(or any other explanation) object and pass in the trained model and the to be explained dataset.Addtionally, define the number of features used in the explanation. This allows you to configure the verbosity of your exaplanation.Set the index of the sample that should be explained.fromexplainy.explanationsimportPermutationExplanationnumber_of_features=4explainer=PermutationExplanation(X_test,y_test,model,number_of_features)Call theexplain()method and print the explanation for the sample (in case of a local explanation every sample has a different explanation).explanation=explainer.explain(sample_index=1)print(explanation)The RandomForestRegressor used 10 features to produce the predictions. The prediction of this sample was 251.8.The feature importance was calculated using the Permutation Feature Importance method.The four features which were most important for the predictions were (from highest to lowest): 'bmi' (0.15), 's5' (0.12), 'bp' (0.03), and 'age' (0.02).Use theplot()method to create a feature importance plot of that sample.explainer.plot()If your prefer, you can also create another type of plot, as for example a boxplot.explainer.plot(kind='box')Finally, you can also look at the importance values of the features (in form of apd.DataFrame).feature_importance=explainer.importance()print(feature_importance)FeatureImportance0bmi0.151s50.122bp0.033age0.024s2-0.005sex-0.006s3-0.007s1-0.018s6-0.019s4-0.01FeaturesAlgorithms for inspecting black-box machine learning modelsSupport for the machine learning frameworksscikit-learnandxgboostexplainyoffers a standardized API with three core methodsexplain(),plot(),importance()Other Machine Learning Explainability libraries to watchshap: A game theoretic approach to explain the output of any machine learning modeleli5: A library for debugging/inspecting machine learning classifiers and explaining their predictionsalibi: Algorithms for explaining machine learning modelsinterpret: Fit interpretable models. Explain blackbox machine learningSourceMolnar, Christoph. "Interpretable machine learning. A Guide for Making Black Box Models Explainable", 2019.https://christophm.github.io/interpretable-ml-book/AuthorMauro Luzzatto-Maurol