Interactive Convolutional Neural Network
By: Valentina Alto
Tags: Convolutional Network, Deep Learning, Python, Image Recognition, TensorFlow

Interactive Convolutional Neural Network

How to customize your algorithm with Python Streamlit

Image recognition is one of the main topics Deep Learning is focusing on. Indeed, the family of algorithms entitled to deal with image recognition belongs to the class of Neural Networks, typical multi-layers algorithms employed in deep learning tasks.

More specifically, image recognition employs Convolutional Neural Networks (CNNs), which I’ve been explaining in my previous article on Computer Vision.

In this article, I want to build a web app using Streamlit (if you are new to Streamlit, you can check an introduction here) which allows the user to customize a CNN built with Tensorflow and Keras. I will basically reproduce the example of my previous article, but now there will be the possibility to interact with the CNN at every step, so that the whole procedure will be ‘controlled’ by the user.

First thing first, let’s import our necessary packages and download our data. For this purpose, I’ll be using the Cifar10 dataset, containing 3-channels images (that means, with colors) of 10 different objects, provided with labels. Hence, we will be dealing with a supervised ML task.

import tensorflow as tf

from tensorflow import keras

import matplotlib.pyplot as plt

import numpy as np

import pydot_ng as pydot

import streamlit as st st.title('Convolutional Neural Network') st.header('Dataset: cifar10') #spending a few lines to describe our dataset

st.text("""Dataset of 50,000 32x32 color training images,

labeled over 10 categories,

and 10,000 test images.""") from keras.datasets import cifar10 #I'm dividing my data into training and test set (x_train, y_train), (x_test, y_test) = cifar10.load_data() if st.checkbox('Show images sizes'):

st.write(f'X Train Shape: {x_train.shape}')

st.write(f'X Test Shape: {x_test.shape}')

st.write(f'Y Train Shape: {y_train.shape}')

st.write(f'Y Test Shape: {y_test.shape}')

As you can see, I first downloaded my dataset and split it into train and test set. Then, I added some widgets to ask the user whether to show the shapes of the new sets. This is useful if you want to check the dimensionality of your data. From these first lines, we know that our inputs are 3-channels, 32×32 pixel images.

We can also ask to display one random image from our training set:

class_names = ["airplane","automobile","bird","cat","deer","dog","frog","horse","ship","truck"] st.subheader('Inspecting dataset') if st.checkbox('Show random image from the train set'):

num = np.random.randint(0, x_train.shape[0])

image = x_train[num]

st.image(image, caption=class_names[y_train[num][0]], use_column_width=True)

Now let’s set some elements which will be useful during training.

st.subheader('Set some hyperparameters')

batch_size = st.selectbox('Select batch size', [32, 64, 128, 256])

epochs=st.selectbox('Select number of epochs', [3, 10, 25, 50])

loss_function = st.selectbox('Loss function', ['mean_squared_error', 'mean_absolute_error', 'categorical_crossentropy'])

optimizer = st.selectbox('Optimizer', ['SGD', 'RMSprop', 'Adam'])

Let’s examine them one by one:

epochs : number of iterations for our neural network while training

: number of iterations for our neural network while training batch_size : number of samples we want to use for each epoch

: number of samples we want to use for each epoch loss_function : a function of errors, that are expressed by the distance between fitted and actual values (if the target is continuous) or by the number of misclassified values (if the target is categorical). Some examples of loss functions are Mean Square errors (in regression) or Categorical Cross-Entropy (in classification).

: a function of errors, that are expressed by the distance between fitted and actual values (if the target is continuous) or by the number of misclassified values (if the target is categorical). Some examples of loss functions are Mean Square errors (in regression) or Categorical Cross-Entropy (in classification). optimizer: an algorithm that adjusts parameters in order to minimize the loss. Some examples of optimization functions available in Keras are Stochastic Gradient Descent (it minimizes the loss according to the gradient descent optimization, and for each iteration it randomly selects a training sample — that’s why it’s called stochastic), RMSProp (that differs from the previous since each parameter has an adapted learning rate) and Adam Optimizer (it is a RMSProp + momentum). Of course, this is not the full list, yet it is sufficient to understand that Adam optimizer is often the best choice, since it allows you to set different hyperparameters and customize your NN.

If you want to learn more about hyperparameters of generalized Neural Network, you can check it here.

Now let’s move towards layers and activation functions. The idea is that each layer takes some input images and returns a kind of ‘summary’ of them, since those are passed through computations and activation functions which make the output meaningful. After many layers, the final output will be a vector of probabilities corresponding to the vector of labels of our images, and our algorithm will return the label with the highest probability.

st.subheader('Building your CNN') model=tf.keras.Sequential() act1 = st.selectbox('Activation function for first layer: ', ['relu', 'tanh', 'softmax'])

model.add(tf.keras.layers.Conv2D(32,kernel_size=(3,3),activation=act1,input_shape=(32,32,3))) model.add(tf.keras.layers.MaxPooling2D(pool_size=(2,2))) if st.checkbox('Add drop layer?'):

drop1=st.selectbox('Which drop rate?', [0.1, 0.25, 0.5])

model.add(tf.keras.layers.Dropout(drop1))

model.add(tf.keras.layers.Flatten()) act2 = st.selectbox('Activation function for Dense layer: ', ['relu', 'tanh', 'softmax'])

model.add(tf.keras.layers.Dense(1024,activation=act2)) act3 = st.selectbox('Activation function for Dense layer: ', ['relu', 'tanh', 'softmax'])

model.add(tf.keras.layers.Dense(10,activation=act3))

As you can see from the code, there are three activation functions the user can choose: softmax (it is the multi-class variation of Sigmoid, displayed below), Tanh and RELU:

Since ReLU is the fastest to train, it is wise to use it for hidden layers, while employing the Softmax just for the output layer.

Now that we have our CNN is ready, we only have to compile it with all the features above and fit it on the train set.

I will first show

model.compile(loss=loss_function, optimizer=optimizer, metrics=['accuracy']) if st.checkbox('Fit model'):

history =model.fit(x_train[0:1000]/255.0,tf.keras.utils.to_categorical(y_train[0:1000]),

batch_size=batch_size,

shuffle=True,

epochs=epochs,

validation_data=(x_test[0:1000]/255.0,tf.keras.utils.to_categorical(y_test[0:1000]))

) # Plot training & validation accuracy values

plt.plot(history.history['accuracy'])

plt.plot(history.history['val_accuracy'])

plt.title('Model accuracy')

plt.ylabel('Accuracy')

plt.xlabel('Epoch')

plt.legend(['Train', 'Test'], loc='upper left')

st.pyplot() predictions=model.predict(x_test/ 255.0)

scores = model.evaluate(x_test / 255.0, tf.keras.utils.to_categorical(y_test))



st.write(f'loss: {round(scores[0],2)}')

st.write(f'accuracy: {round(100*scores[1],2)}%')

We can also visualize one element of the predictions’ set, together with the vector of probabilities. Then, if the image was correctly classified, the color of the corresponding bar will be green, otherwise red:

st.subheader('Visualizing results')



def plot_pred(i,predictions_array,true_label,img):

predictions_array,true_label,img=predictions_array[i],true_label[i:i+1],img[i]

plt.grid(False)

plt.title(class_names[true_label[0][0]])

plt.xticks([])

plt.yticks([])



plt.imshow(img)

def plot_bar(i,predictions_array,true_label):

predictions_array, true_label = predictions_array[i], true_label[i]

plt.grid(False)

plt.yticks([])

plt.xticks(np.arange(10),class_names,rotation=40)



thisplot=plt.bar(range(10),predictions_array, color='grey')

plt.ylim([0,1])

predicted_label=np.argmax(predictions_array)





if predicted_label==true_label:

color='green'

else:

color='red'



thisplot[predicted_label].set_color(color) if st.checkbox('Show random prediction results'):

num2 = np.random.randint(0, len(y_test))

plt.figure(figsize=(15,6))

plt.subplot(1,2,1)

plot_pred(num2, predictions, y_test, x_test)

plt.subplot(1,2,2)

plot_bar(num2, predictions, y_test)

st.pyplot()

So in the example above, the image was correctly classified as ‘Bird’. We can run our code many times and occur in a misclassified image, obtaining the following:

Great, now you can play with your CNN, customize it and try several combinations of hyperparameters in order to achieve the highest accuracy possible.

I hope you enjoyed the reading and, if you are curious about this topic and want to learn more, I’m attaching some links you might find very useful: