Package: keras-explain
Keras ExplainThis package includes the majority of explanation tools for explaining
Keras models predictions. Currently, only models with images on input are
supported.
It supports following approaches:Gradient methods:GradCam [Selvaraju]Guided GradCam [Selvaraju]Guided back-propagation [Springenberg]Integrated gradients [Sundararajan]Saliency [Simonyan]Layer-wise relevance propagation [BETA] [Bach]Model-independent methods:Prediction difference [Zintgraf]Basic graying out [Zeiler]LIME [Ribeiro]All approaches are easy to apply to your model in two lines of code.
If you have any suggestion for new approaches to be included in the package
please do not hesitate to suggest. Also all improvements suggestions, bug reports and bug fixes are welcome.Right now we are in the process of implementing the following approaches:Meaningful perturbation by Fong et al.Layer-wise relevance propagation - we are adding layers that are not supported
yet.UsageGradient methodsGradCamfrom keras_explain.grad_cam import GradCam

explainer = GradCam(model, layer=None)
exp = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classlayer- (optional) The index (index in model.layers) of the layer which
prediction is explained.
If not specified the last layer prediction is explained automatically.Output:exp- explanation. GradCam mark only features which contribute to the
classification in atarget class.Guided GradCamfrom keras_explain.grad_cam import GuidedGradCam

explainer = GuidedGradCam(model, lyer=None)
exp = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classlayer- (optional) The index (index in model.layers) of the layer which
prediction is explained.
If not specified the last layer prediction is explained automatically.Output:exp- explanation. GuidedGradCam mark only features which contribute to the
classification in atarget class.Guided back-propagationfrom keras_explain.guided_bp import GuidedBP

explainer = GuidedBP(model)
exp = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classOutput:exp- explanation. Guided back-propagation mark only features which
contribute to the classification in atarget class.Integrated gradientsfrom keras_explain.integrated_gradients import IntegratedGradients

explainer = IntegratedGradients(model)
exp = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classOutput:exp- explanation. Integrated gradients mark only features which contribute
to the classification in atarget class.Saliencyfrom keras_explain.saliency import Saliency

explainer = Saliency(model, layer=None)
exp = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classlayer- (optional) The index (index in model.layers) of the layer which
prediction is explained.
If not specified the last layer prediction is explained automatically.Output:exp- explanation. Saliency mark only features which contribute
to the classification in atarget class.Layer-wise relevance propagation [BETA]This approach does not support all layers yet. We are currently implementing
missing layers. If you wish you can implement any layer support yourself
and submit it as a pull request. Since implementation is very custom any
suggestion for improvement is welcome.from keras_explain.lrp import LRP

explainer = LRP(model)
exp = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classOutput:exp- explanation. LRP mark only features which contribute
to the classification in atarget class.###Model independent approachesPrediction differencefrom keras_explain.prediction_diff import PredictionDiff

explainer = PredictionDiff(model)
exp_pos, exp_neg = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classOutput:exp_pos- explanation with marked features which contribute
to the classification in atarget class.exp_neg- explanation with marked features which contribute
against the classification in atarget class.Basic graying outfrom keras_explain.graying_out import GrayingOut

explainer = GrayingOut(model)
exp_pos, exp_neg = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classOutput:exp_pos- explanation with marked features which contribute
to the classification in atarget class.exp_neg- explanation with marked features which contribute
against the classification in atarget class.LIMEfrom keras_explain.lime_ribeiro import Lime

explainer = Lime(model)
exp_pos, exp_neg = explainer.explain(image, target_class)Parameters:model- Keras model which is explainedimage- input which prediction is explainedtarget_class- approach explains prediction for a target classOutput:exp_pos- explanation with marked features which contribute
to the classification in atarget class.exp_neg- explanation with marked features which contribute
against the classification in atarget class.