Accuracy Visualisation In Deep Learning — Part 1 | by Kaushik Choudhury | Towards Data Science

“One look is worth a thousand words.”
We all want to train the deep learning models in the most optimum way to increase even the last two decimal of the prediction accuracy. We have so many parameters to tweak in the deep learning model starting from the optimiser and its parameters, activation function, number of layers/filters etc. that finding the right combinations of all these parameters is like finding a needle in the haystack.
Fortunately, we can leverage hyperparameters to tune the performance and accuracy of the model, but we need to have a broad sense of the parameter combination to try and test.
Tensor board is one of the most powerful inbuilt tool available to visualise individual model’s performance based on different metrics and also for comparison among different models. It can guide in ascertaining the ballpark parameter combinations which we can further try with hyperparameter tuning.
In this article, I will discuss the deep learning model visualisation with a combination of optimisers and activation function for simple regression. It will enable us to learn how we can discard unsuitable combinations quickly and focus our performance tuning efforts on a few potential parameters.
Step 1: We will use the Scikit learn make_regression method to generate a random dataset for regression testing and train_test_split to divide the datasets into training and testing set.
from sklearn.datasets import make_regressionfrom sklearn.model_selection import train_test_split
Step 2: In the code below, we have imported the Tensor Board and deep learning Keras package. We will use Keras for modelling and Tensor Board for visualisation.
from tensorflow.keras.callbacks import TensorBoardfrom keras.models import Sequentialfrom keras.layers import Dense
Step 3: In this article, we will work with “adam” and “RMSprop” optimiser, and GlorotUniform” and “normal” weight initializer. We have mentioned these in a list and will call the combination in sequence for training the model.
optimizers=["adam","RMSprop"] #Optimisersinitializers=["GlorotUniform","normal"] # Activation function
Step 5: In the below code, we have nested FOR loop to train the deep learning model with different combinations of optimisers and weight initialiser, and record the result for analysis in Tensor board.
Each model is named with weight initialiser and optimiser name to identify each model’s result and graph.
As the main objective of this article is to learn the visualisation of deep learning model results, hence we will work with a very simple model with one input, hidden and output layer.
We will use mean square error as a loss function for all the model and measure the mean absolute percentage error. If you do not know these statistical metrics, then I will suggest referring Wikipedia for detail explanation.
Step 6: We can view the results and analyse after the code is executed. We need to open the command prompt in windows or terminal in mac to start the Tensor Board. Navigate to the folder directory in where logs are saved via command prompt/terminal and then type the below command.
tensorboard --logdir=logs/
It will instantiate the tensor board and will show the address which we need to type in the browser to view the results.
In the current example, we can access the Tensor board by typing http://localhost:6006/ in the browser.
In one consolidated graph, we can view the change in the mean square error as the number of iterations progresses for a different combination of optimizers and weight initializers.
It indicates the optimizers and weight initializers which are unsuitable for the current datasets and modelling. In the current example, the mean squared loss is decreasing way slower for “RMSprop” and “normal” optimiser combination than other combinations. Based on this knowledge from the visualisation, we can focus our energy in fine-tuning the remaining combinations and save time by not tweaking the combination which is not performing well on a broad level.
In the same way, we can view the change in mean absolute percentage error for different combinations as the number of iterations progresses.
We can also use filters to view the graphs of one or more combination of the models.
It is important to give meaningful names to the model as it helps to put the filter correctly and avoid confusion during the analysis. In the current example, the combination of weight initializer and optimiser is the name of the model.
We can also download the graphs, and loss function data SVG and CSV format respectively. Loss function data in CSV format enables us to perform advanced analysis quickly with excel.
Besides visualising the result in terms of epochs progression, we can also view the relative performance of the model with a single click of a mouse.
In this article, we have learnt the basics of the tensor board and seen a few visualisation options available to grasp the finer details about different models in an instant. We also saw the way we can use the tensor board visualisation to focus on fine-tuning potential models. In the next article, we will see a few advanced visualisations for deep learning models.
If you would like to learn Exploratory data analysis (EDA) with visualisation then read by article 5 Advanced Visualisation for Exploratory data analysis (EDA)
If you are a big fan of pandas like me then you will find the article 5 Powerful Visualisation with Pandas for Data Preprocessing interesting to read and learn.