Package: tf-explain
tf-explaintf-explainimplements interpretability methods as Tensorflow 2.x callbacks toease neural network's understanding.
SeeIntroducing tf-explain, Interpretability for Tensorflow 2.0Documentation:https://tf-explain.readthedocs.ioInstallationtf-explainis available on PyPi as an alpha release. To install it:virtualenvvenv-ppython3.8
pipinstalltf-explaintf-explain is compatible with Tensorflow 2.x. It is not declared as a dependency
to let you choose between full and standalone-CPU versions. Additionally to the previous install, run:# For CPU or GPUpipinstalltensorflow==2.6.0Opencv is also a dependency. To install it, run:# For CPU or GPUpipinstallopencv-pythonQuickstarttf-explain offers 2 ways to apply interpretability methods. The full list of methods is theAvailable Methodssection.On trained modelThe best option is probably to load a trained model and apply the methods on it.# Load pretrained model or your ownmodel=tf.keras.applications.vgg16.VGG16(weights="imagenet",include_top=True)# Load a sample image (or multiple ones)img=tf.keras.preprocessing.image.load_img(IMAGE_PATH,target_size=(224,224))img=tf.keras.preprocessing.image.img_to_array(img)data=([img],None)# Start explainerexplainer=GradCAM()grid=explainer.explain(data,model,class_index=281)# 281 is the tabby cat index in ImageNetexplainer.save(grid,".","grad_cam.png")During trainingIf you want to follow your model during the training, you can also use it as a Keras Callback,
and see the results directly inTensorBoard.fromtf_explain.callbacks.grad_camimportGradCAMCallbackmodel=[...]callbacks=[GradCAMCallback(validation_data=(x_val,y_val),class_index=0,output_dir=output_dir,)]model.fit(x_train,y_train,batch_size=2,epochs=2,callbacks=callbacks)Available MethodsActivations VisualizationVanilla GradientsGradients*InputsOcclusion SensitivityGrad CAM (Class Activation Maps)SmoothGradIntegrated GradientsActivations VisualizationVisualize how a given input comes out of a specific activation layerfromtf_explain.callbacks.activations_visualizationimportActivationsVisualizationCallbackmodel=[...]callbacks=[ActivationsVisualizationCallback(validation_data=(x_val,y_val),layers_name=["activation_1"],output_dir=output_dir,),]model.fit(x_train,y_train,batch_size=2,epochs=2,callbacks=callbacks)Vanilla GradientsVisualize gradients importance on input imagefromtf_explain.callbacks.vanilla_gradientsimportVanillaGradientsCallbackmodel=[...]callbacks=[VanillaGradientsCallback(validation_data=(x_val,y_val),class_index=0,output_dir=output_dir,),]model.fit(x_train,y_train,batch_size=2,epochs=2,callbacks=callbacks)Gradients*InputsVariant ofVanilla Gradientsponderating gradients with input valuesfromtf_explain.callbacks.gradients_inputsimportGradientsInputsCallbackmodel=[...]callbacks=[GradientsInputsCallback(validation_data=(x_val,y_val),class_index=0,output_dir=output_dir,),]model.fit(x_train,y_train,batch_size=2,epochs=2,callbacks=callbacks)Occlusion SensitivityVisualize how parts of the image affects neural network's confidence by occluding parts iterativelyfromtf_explain.callbacks.occlusion_sensitivityimportOcclusionSensitivityCallbackmodel=[...]callbacks=[OcclusionSensitivityCallback(validation_data=(x_val,y_val),class_index=0,patch_size=4,output_dir=output_dir,),]model.fit(x_train,y_train,batch_size=2,epochs=2,callbacks=callbacks)Occlusion Sensitivity for Tabby class (stripes differentiate tabby cat from other ImageNet cat classes)Grad CAMVisualize how parts of the image affects neural network's output by looking into the activation mapsFromGrad-CAM: Visual Explanations from Deep Networks
via Gradient-based Localizationfromtf_explain.callbacks.grad_camimportGradCAMCallbackmodel=[...]callbacks=[GradCAMCallback(validation_data=(x_val,y_val),class_index=0,output_dir=output_dir,)]model.fit(x_train,y_train,batch_size=2,epochs=2,callbacks=callbacks)SmoothGradVisualize stabilized gradients on the inputs towards the decisionFromSmoothGrad: removing noise by adding noisefromtf_explain.callbacks.smoothgradimportSmoothGradCallbackmodel=[...]callbacks=[SmoothGradCallback(validation_data=(x_val,y_val),class_index=0,num_samples=20,noise=1.,output_dir=output_dir,)]model.fit(x_train,y_train,batch_size=2,epochs=2,callbacks=callbacks)Integrated GradientsVisualize an average of the gradients along the construction of the input towards the decisionFromAxiomatic Attribution for Deep Networksfromtf_explain.callbacks.integrated_gradientsimportIntegratedGradientsCallbackmodel=[...]callbacks=[IntegratedGradientsCallback(validation_data=(x_val,y_val),class_index=0,n_steps=20,output_dir=output_dir,)]model.fit(x_train,y_train,batch_size=2,epochs=2,callbacks=callbacks)RoadmapSubclassing API SupportAdditional MethodsGradCAM++Integrated GradientsGuided SmoothGradLRPAuto-generated API Documentation & Documentation TestingContributingTo contribute to the project, please read thededicated section.CitationAcitation fileis available for citing this work.