TensorBoard- A Visualization suite for Tensorflow models | by Renu Khandelwal | Towards Data Science

In this article, you will learn to use TensorBoard to display metrics, graphs, images, and histograms of weights and bias over different epochs for a deep learning model created in TensorFlow along with common issues you may encounter with TensorBoard.
What is TensorBoard?
TensorBoard is a visualization toolkit from Tensorflow to display different metrics, parameters, and other visualizations that help debug, track, fine-tune, optimize, and share your deep learning experiment results.
What are the different things that can be tracked through TensorBoard?
TensorBoard allows you to track
Metrics: loss and accuracy change with every epoch. Tracking loss and accuracy over the training cycle will help you understand if the model is overfitting.
Model graphs: visualize deep learning models to know if they are built correctly
Distribution Histograms: Visualize the histograms of weights and biases over epochs
Images: Visualize arbitrary images for your dataset in TensorBoard
Why should you use TensorBoard?
You are developing an Image Classification deep neural network and would like to have more than 98% accuracy.
You want to track the accuracy and loss of the model at every epoch; also you would like to track and profile the accuracy and loss of your model with different hyperparameters values like learning rate, optimizers, drop out, etc. Tracking accuracy for different values of hyperparameter will help you to fine-tune the model.
Deep learning models are computationally expensive and to visualize the results of our experiments with different values for hyperparameters and track the accuracy and loss over several epochs will help fine-tune the model quickly.
You can track the accuracy and loss of models based on different data flow graphs with the different numbers of hidden layer numbers and the number of units in the hidden layers.
TensorBoard helps visualize the flow of the tensors in the model for debugging and optimization by tracking accuracy and loss.
TensorBoard visualizes the computational graphs, training parameters, metrics, and hyperparameters that will aid in tracking the experimental results of your model, yielding fine-tuning of the model faster.
With TensorBoard.dev, you can host your deep learning model experiment results and share it with your team. Anyone can view the data with the link, so DO NOT UPLOAD SENSITIVE DATA.
How to use TensorBoard?
We will visualize the scalars, graphs, and distribution using TensorBoard using cats and dogs dataset.
import tensorflow as tfimport datetimefrom tensorflow.keras.models import Sequentialfrom tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2Dfrom tensorflow.keras.preprocessing.image import ImageDataGenerator, img_to_array, load_img
I have used TensorFlow 2.0.0 version.
# Load the TensorBoard notebook extension%load_ext tensorboard
Setting key parameters for the training
BASE_PATH = 'Data\\dogs-vs-cats\\train\\'TRAIN_PATH='Data\\dogs-vs-cats\\train_data\\'VAL_PATH='Data\\dogs-vs-cats\\validation_data\\'batch_size = 32 epochs = 60IMG_HEIGHT = 150IMG_WIDTH = 150
Rescale and Apply different Augmentation to the training image
train_image_generator = ImageDataGenerator(                                                rescale=1./255,                                              rotation_range=45,                                                width_shift_range=.15,                                                height_shift_range=.15,                                                horizontal_flip=True,                                                zoom_range=0.3)
Rescale Validation data
validation_image_generator = ImageDataGenerator(rescale=1./255)
Generate batches of normalized data for train and validation data set
train_data_gen = train_image_generator.flow_from_directory(batch_size = batch_size,                                                     directory=TRAIN_PATH,                                                     shuffle=True,                                                     target_size=(IMG_HEIGHT, IMG_WIDTH),                                                     class_mode='categorical')val_data_gen = validation_image_generator.flow_from_directory(batch_size = batch_size,                                                              directory=VAL_PATH,                                                              target_size=(IMG_HEIGHT, IMG_WIDTH),                                                              class_mode='categorical')
def create_model():    model = Sequential([    Conv2D(16, 3, padding='same', activation='relu',            input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),    MaxPooling2D(),    Dropout(0.2),    Conv2D(32, 3, padding='same', activation='relu'),    MaxPooling2D(),    Conv2D(64, 3, padding='same', activation='relu'),    MaxPooling2D(),    Conv2D(128, 3, padding='same', activation='relu'),    MaxPooling2D(),    Dropout(0.2),    Flatten(),    Dense(512, activation='relu'),    Dense(2, activation='softmax')])    return model# Create and Compile the modelmodel= create_model()model.compile(optimizer='adam',                 loss='categorical_crossentropy', metrics=['accuracy'])
To visualize the loss and accuracy, the model graphs, and histograms for weights and biases, you need to create the logs and store the details. These logs files will be used by TensorBoard to visualize the details.
To create the log files, use tf.keras.callbacks.TensorBoard when fitting the model.
log_dir=r'\Tensorboard_ex\logs_1\fit\' + datetime.datetime.now().strftime('%Y%m%d-%H%M%S')tensorboard_callback= tf.keras.callbacks.TensorBoard(log_dir=log_dir,                                                    histogram_freq=1)
To enable histogram computation for weights and biases for every layer and every epoch, we set histogram_freq=1 as it is set to off by default.
Fit the model
model.fit_generator(    train_data_gen,    steps_per_epoch=1000,    epochs=epochs,    validation_data=val_data_gen,    validation_steps=1000,    callbacks=[tensorboard_callback])
Visualizing the Tensorboard Dashboard
You can view the TensorBoard dashboard using different commands in the Jupyter notebook
%tensorboard --logdir="\TEnsorBoard_ex\logs_1\fit"
or
%tensorboard — logdir log_dir
Or
start the TensorBoard by specifying the directory where the event logs are stored. These events logs are created when we were fitting the model.
python -m tensorboard.main — logdir=”TensorBoard_ex\logs_1\fit”
In Chrome type http://localhost:6006/ in the address bar and you can now see the Tensorboard
Scalar information containing accuracy and loss of the training and validation dataset over 20 epochs
TensorBoard with Graph for the Image Classification Model shows if the model was built correctly
you can also view the distribution of non-scalar tensors over multiple iterations
Histograms show the same tensor non-scalar tensor variable across multiple iterations but as a 3-D histogram
No dashboards are active for the current dataset
Kill all the current Tensorboard tasks running. Type the following commands on your command prompt
taskkill /im tensorboard.exe /fdel /q %TMP%\.tensorboard-info\*
Now start the TensorBoard by specifying the directory where the event logs are stored. These events logs are created when we were fitting the model.
python -m tensorboard.main — logdir=”TensorBoard_ex\logs_1\fit”
At times you may have to try these statements multiple times to view the Tensorboard dashboard.
TensorBoard visualization for different elements of the model makes it easier to debug and optimize deep learning model experimentation to better accuracy.
What’s next?
In the next article, you will learn to fine-tune deep learning models using hyperparameter tuning suing TensorBoard.