Package: tf-keras-vis
tf-keras-visWeb documentshttps://keisen.github.io/tf-keras-vis-docs/Overviewtf-keras-vis is a visualization toolkit for debuggingtf.keras.Modelin Tensorflow2.0+.
Currently supported methods for visualization include:Feature VisualizationActivationMaximization (web,github)Class Activation MapsGradCAM (paper)GradCAM++ (paper)ScoreCAM (paper,github)Faster-ScoreCAM (github)LayerCAM (paper,github) :new::zap:Saliency MapsVanilla Saliency (paper)SmoothGrad (paper)tf-keras-vis is designed to be light-weight, flexible and ease of use.
All visualizations have the features as follows:SupportN-dim image inputs, that's, not only support pictures but also such as 3D images.Supportbatch wiseprocessing, so, be able to efficiently process multiple input images.Support the model that have eithermultiple inputsormultiple outputs, or both.Support themixed-precisionmodel.And in ActivationMaximization,Support Optimizers that are built to tf.keras.VisualizationsDense UnitConvolutional FilterClass Activation MapThe images above are generated byGradCAM++.Saliency MapThe images above are generated bySmoothGrad.UsageActivationMaximization (Visualizing Convolutional Filter)importtensorflowastffromtensorflow.keras.applicationsimportVGG16frommatplotlibimportpyplotaspltfromtf_keras_vis.activation_maximizationimportActivationMaximizationfromtf_keras_vis.activation_maximization.callbacksimportProgressfromtf_keras_vis.activation_maximization.input_modifiersimportJitter,Rotate2Dfromtf_keras_vis.activation_maximization.regularizersimportTotalVariation2D,Normfromtf_keras_vis.utils.model_modifiersimportExtractIntermediateLayer,ReplaceToLinearfromtf_keras_vis.utils.scoresimportCategoricalScore# Create the visualization instance.# All visualization classes accept a model and model-modifier, which, for example,#     replaces the activation of last layer to linear function so on, in constructor.activation_maximization=\ActivationMaximization(VGG16(),model_modifier=[ExtractIntermediateLayer('block5_conv3'),ReplaceToLinear()],clone=False)# You can use Score class to specify visualizing target you want.# And add regularizers or input-modifiers as needed.activations=\activation_maximization(CategoricalScore(FILTER_INDEX),steps=200,input_modifiers=[Jitter(jitter=16),Rotate2D(degree=1)],regularizers=[TotalVariation2D(weight=1.0),Norm(weight=0.3,p=1)],optimizer=tf.keras.optimizers.RMSprop(1.0,0.999),callbacks=[Progress()])## Since v0.6.0, calling `astype()` is NOT necessary.# activations = activations[0].astype(np.uint8)# Renderplt.imshow(activations[0])Gradcam++importnumpyasnpfrommatplotlibimportpyplotaspltfrommatplotlibimportcmfromtf_keras_vis.gradcam_plus_plusimportGradcamPlusPlusfromtf_keras_vis.utils.model_modifiersimportReplaceToLinearfromtf_keras_vis.utils.scoresimportCategoricalScore# Create GradCAM++ objectgradcam=GradcamPlusPlus(YOUR_MODEL_INSTANCE,model_modifier=ReplaceToLinear(),clone=True)# Generate cam with GradCAM++cam=gradcam(CategoricalScore(CATEGORICAL_INDEX),SEED_INPUT)## Since v0.6.0, calling `normalize()` is NOT necessary.# cam = normalize(cam)plt.imshow(SEED_INPUT_IMAGE)heatmap=np.uint8(cm.jet(cam[0])[...,:3]*255)plt.imshow(heatmap,cmap='jet',alpha=0.5)# overlayPlease see the guides below for more details:Getting Started GuidesSaliency and CAMsVisualize Dense LayerVisualize Convolutional Filer[NOTES]If you have ever usedkeras-vis, you may feel that tf-keras-vis is similar with keras-vis.
Actually tf-keras-vis derived from keras-vis, and both provided visualization methods are almost the same.
But please notice that tf-keras-vis APIs does NOT have compatibility with keras-vis.RequirementsPython 3.7+Tensorflow 2.0+InstallationPyPI$pipinstalltf-keras-vistensorflowSource (for development)$gitclonehttps://github.com/keisen/tf-keras-vis.git
$cdtf-keras-vis
$pipinstall-e.[develop]tensorflowUse CaseschitraA Deep Learning Computer Vision library for easy data loading, model building and model interpretation with GradCAM/GradCAM++.Known IssuesWith InceptionV3, ActivationMaximization doesn't work well, that's, it might generate meaninglessly blur image.With cascading model, Gradcam and Gradcam++ don't work well, that's, it might occur some error. So we recommend to use FasterScoreCAM in this case.channels-firstmodels and data is unsupported.ToDoGuidesVisualizing multiple attention or activation images at once utilizing batch-system of modelDefine various score functionsVisualizing attentions with multiple inputs modelsVisualizing attentions with multiple outputs modelsAdvanced score functionsTuning Activation MaximizationVisualizing attentions for N-dim image inputsWe're going to add some methods such as belowDeep DreamStyle transfer