Image Classification using CNN. Classification of Marvel characters... | by Vishnu Bhaarath | Towards Data Science

Before diving in to the technicalities behind CNN let us talk about living beings . How do they recognize and perceive images. We see multiple images every second and we are able to process it and we are able to recognize it in the future. All this is capable due to Human brain, the most powerful machine in the world. Let us talk about how we perceive images to have a better understanding of CNN. Take a look at the below pictures
We are able to see two images
A man with a right face
A man facing towards us
Based on our focus on the image we see different views, our brain is too powerful ,it automatically recognizes the pattern and help us identify the object. This is the underlying principle behind CNN it searches for patterns and differentiate the images.
Consider the above image, we are able to identify A and B by looking at some patterns. We are able to do this because we have seen these images in the past and our brain has observed some patterns in them which helps us to differentiate it. If we ask to a child or to a person who in his lifetime haven’t seen a lion or dog. Will he be able to classify the above images? Definitely No.
The above illustration is not only applicable to CNN but to machine learning in general. Let us focus on CNN as the article is all about it. We send a bunch of images to CNN for training, CNN looks for patterns in it similar to how human beings does, so when we ask the CNN to identify the images it will be able to recognize the patterns in it. So if we send images to an untrained CNN it will fail as the case with human beings too.
How to make the machine read images ? is the common question that strikes most of us and this is the first step in image processing.
Machine read images by pixel value . Every image is a combination of pixel values, changing the pixel values alters the image. These pixel values has to processed before feeding in to CNN. The number of pixel values depends on the size of the image, so again a problem arises what if the size of the image is too large. A fully connected network will flatten the 2 D array of pixels in to 1 D array and identify the images based on the pixel values.
Convolution layer along with pooling comes to our rescue. It extracts certain features from the images and reduces the size of the image.
Our image of 6*6 matrix is converted in to 3*3 matrix . Our weight is initialized as 3*3 matrix. The weight matrix runs across our image and produce a convoluted output of 3*3 matrix. The weight covers all the pixel values in our image at least once and performs element wise multiplication.
When you look at the element wise multiplication to obtain the convolution layer, an another problem arises the pixel values at the edges are covered only once. This problem can be solved with the help of padding.
The size of the image is preserved when we apply padding. Weights are learned similar to MLP that is to minimize loss function or to help the model in correct prediction of images. We won’t use just a single filter in real world application , the usage of number of filter affects the depth of the convoluted layer. The depth of the convoluted output depends on the number of filters we use.
Every colour images can be represented as a combination of Red , Blue and Green. We split it in to three layers and follow the same procedure applied in case of grey scale images.
The sole purpose of pooling is to reduce the size of the images. Max pooling is preferred in most cases. Pooling operates independently on each layer so the depth of the convolution layer remains unchanged. You can observe that we are applying two strides at a time. The size decreases as the number of strides increases.
Our ultimate aim is to predict whether an image belongs to a particular class or not. But Convolution layer gives us a 3 D activation map and it is not easy to obtain the output class by using convolution layer, So we need to flatten it to produce the desired number of output classes. Forward and Back propagation are done to reduce the losses.
As we can observe we apply convolution at the start to extract useful features from it , followed it by max pooling to reduce the size of the images followed by flattening and finally the output predicts the probability of image belonging to a particular class.
Five Marvel characters were chosen Black Widow, Iron Man, Thor, Captain America and Hulk. Train and Testing set were divided in the ratio of 70: 30 and the performance was observed.
from zipfile import ZipFilefilename= "marvel.zip"with ZipFile(filename,'r')as zip:zip.extractall()
You can ignore the first three blocks as it deals with loading of images and importing dependencies. The zip file which consist of the 5 marvel characters is loaded and extracted in the above block of codes.
#Importingimport osimport numpy as npimport globimport shutilimport matplotlib.pyplot as pltimport tensorflow as tffrom tensorflow.keras.models import Sequentialfrom tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2Dfrom tensorflow.keras.preprocessing.image import ImageDataGeneratorbase_dir = os.path.join(os.path.dirname(filename), 'marvel')
All the necessary dependencies are imported, If you get an error while importing these dependencies you have to install it. Using pip install (Jupyter notebook)
# Accessing the images and setting 0.7 of images for training and the rest for testingclasses=['Black Widow','Captain America','Hulk','Iron Man','Thor']for m in classes:img_path = os.path.join(base_dir, m)images = glob.glob(img_path + '/*.jpg')num_train = int(round(len(images)*0.7))train, val = images[:num_train], images[num_train:]# Creating separate directories for training datafor t in train:if not os.path.exists(os.path.join(base_dir, 'train', m)):os.makedirs(os.path.join(base_dir, 'train', m))shutil.move(t, os.path.join(base_dir, 'train', m))# Creating separate directories for validating datafor v in val:if not os.path.exists(os.path.join(base_dir, 'val', m)):os.makedirs(os.path.join(base_dir, 'val', m))shutil.move(v, os.path.join(base_dir, 'val', m))
In the above block of codes a separate directories for training and validation data is created. The train and validation data are divided in the ratio of 70:30. All the image extension are converted to jpg.
train_dir = os.path.join(base_dir, 'train')val_dir = os.path.join(base_dir, 'val')
Two directories are created for training and validation.
# Setting batch size and a constant image shapebatch_size = 130IMG_SHAPE = 150# Rescaling the images so all the values lie between 0 and 1 and applying horizontal flip and training the dataimage_gen = ImageDataGenerator(rescale=1./255, horizontal_flip=True)train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,directory=train_dir,shuffle=True,target_size=(IMG_SHAPE,IMG_SHAPE))# Rescaling the images so all the values lie between 0 and 1 and rotating and training the dataimage_gen = ImageDataGenerator(rescale=1./255, rotation_range=45)train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,directory=train_dir,shuffle=True,target_size=(IMG_SHAPE, IMG_SHAPE))#Rescaling and zooming the dataimage_gen_train = ImageDataGenerator(rescale=1./255,rotation_range=45,width_shift_range=.15,height_shift_range=.15,horizontal_flip=True,zoom_range=0.5)train_data_gen = image_gen_train.flow_from_directory(batch_size=batch_size,directory=train_dir,shuffle=True,target_size=(IMG_SHAPE,IMG_SHAPE),class_mode='sparse')
As the image collected will not be of same size, resizing of image is done. An image size of 150 * 150 is chosen. The pixel value of images ranges from 0 to 255, to provide better results pixel values are re-scaled so all the values lies between 0 and 1. Different techniques such as horizontal flip, zooming , rotation are performed on the existing set of images.
image_gen_val = ImageDataGenerator(rescale=1./255)val_data_gen = image_gen_val.flow_from_directory(batch_size=batch_size,directory=val_dir,target_size=(IMG_SHAPE, IMG_SHAPE),class_mode='sparse')model = Sequential()model.add(Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_SHAPE,IMG_SHAPE, 3)))model.add(MaxPooling2D(pool_size=(2, 2)))model.add(Conv2D(32, 3, padding='same', activation='relu'))model.add(MaxPooling2D(pool_size=(2, 2)))model.add(Conv2D(64, 3, padding='same', activation='relu'))model.add(MaxPooling2D(pool_size=(2, 2)))# Adding dropout to turn down some neuronsmodel.add(Flatten())model.add(Dropout(0.2))model.add(Dense(512, activation='relu'))model.add(Dropout(0.2))model.add(Dense(5, activation='softmax'))
Convolution and max pooling is applied to the data set, before sending it to the output layer the model is flattened. Dropout is applied to prevent overfitting of images.
model.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])epochs = 120history = model.fit_generator(train_data_gen,steps_per_epoch=int(np.ceil(train_data_gen.n / float(batch_size))),epochs=epochs,validation_data=val_data_gen,validation_steps=int(np.ceil(val_data_gen.n / float(batch_size))))
Model is compiled and the performance is observed. You can play with the hyper-parameter to obtain the optimum result.
The link to github(code) is attached below