Deep Learning for House Number Detection
By: Danyal Jamil
Tags: Cnn, Deep Learning, Svm, Data Science, Machine Learning

SVHN Dataset

Photo by Jon Tyson on Unsplash

This is a Stanford collected Dataset and is available for the public to experiment and to learn.

SVHN is a real-world image dataset for developing machine learning and object recognition algorithms with minimal requirement on data preprocessing and formatting. It can be seen as similar in flavor to MNIST(e.g., the images are of small cropped digits), but incorporates an order of magnitude more labeled data (over 600,000 digit images) and comes from a significantly harder, unsolved, real world problem (recognizing digits and numbers in natural scene images). SVHN is obtained from house numbers in Google Street View images. [2]

The images are, in no way, preprocessed or ready to be used yet. Hence, whoever wants to use it has to do a bit of the work!

The Challenge

Build an algorithm to classify the different house numbers from the Dataset.

The Problem

The dataset that is available on the website is in the .mat format. And in case you don’t know, Python Notebooks and all the algorithms can’t process with these kind of files. Hence why, it’s a compulsion to convert the data to an acceptable data format before getting into the cool stuff.

And honestly, being a newbie, I couldn’t write functions to convert the files myself and hence did a lot of digging. After 10s of articles and youtube videos, I finally stumbled upon a Repository that was somewhat helpful.

Now, let’s move on to the model creation!

Environment and Tools

Keras

Numpy

Scipy

Numpy

The Code?

I’ll upload the complete code on Github and will link it here later. But here’s some of it.

import os

import sys

import keras

import tarfile

import numpy as np

import urllib.request as urllib

import matplotlib.pyplot as plt

from keras.regularizers import l2

from keras.models import Sequential

from keras.optimizers import Adam, SGD

from keras.engine.training import Model

from keras import backend as K, regularizers

from keras.callbacks import LearningRateScheduler

from keras.preprocessing.image import ImageDataGenerator

from keras.layers import Add, Conv2D, MaxPooling2D, Dropout, Flatten, Dense, BatchNormalization, Activation

Next, let’s preprocess the Data:

#Dataset location

train_location = 'Drive link to the Train.mat file'

test_location = 'Drive link to the Test.mat file' def load_train_data():

train_dict = sio.loadmat(train_location)

X = np.asarray(train_dict['X'])

X_train = [] for i in range(X.shape[3]):

X_train.append(X[:,:,:,i])

X_train = np.asarray(X_train)

Y_train = train_dict['y'] for i in range(len(Y_train)):

if Y_train[i]%10 == 0:

Y_train[i] = 0

Y_train = to_categorical(Y_train,10)

return (X_train,Y_train) def load_test_data():

test_dict = sio.loadmat(test_location)

X = np.asarray(test_dict['X'])

X_test = []for i in range(X.shape[3]):

X_test.append(X[:,:,:,i])

X_test = np.asarray(X_test)

Y_test = test_dict['y'] for i in range(len(Y_test)):if Y_test[i]%10 == 0:

Y_test[i] = 0

Y_test = to_categorical(Y_test,10)



return (X_test,Y_test)

And we use this function as:

(x_train, y_train) = load_train_data()

(x_test, y_test) = load_test_data() x_train.shape # (73257, 32, 32, 3)

x_test.shape # (26032, 32, 32, 3)

Now, when we have the data in our hands, we can scale it. Now, it’s a norm in the era to normalize the data before loading it into the architecture because it’s easier of the model to learn from a scaled data rather than from learning a randomly spread out data.

Normalizing data:

# Converting the arrays to Float type

x_train = x_train.astype('float32')

x_test = x_test.astype('float32') # Normalizing

x_train = x_train / 255.0

x_test = x_test / 255.0

Now, we do have 73,257 images but

Training Set Size and Money are never enough!

Hence, we create some more images here using the ImageDataGenerator

from keras.preprocessing.image import ImageDataGenerator # applying transformation to image

train_gen = ImageDataGenerator(

rotation_range=15,

zoom_range = 0.10,

width_shift_range=0.3,

height_shift_range=0.3,

brightness_range=[0.2,1.0]

) # test_gen = ImageDataGenerator()

train_gen.fit(x_train)

test_set = train_gen.flow(x_test, y_test, batch_size=256)

We’re just generating the Testing Images for now and will hit the Training set later, because I like it that way.

Model Architecture

Here is the fun part! The architecture I’ve designed here consists of 8 Convolutional Blocks!

Now, I used Convolutional Layers with:

model = Sequential() # Block 1

model.add(Conv2D(32, kernel_size=5, kernel_initializer='he_uniform', kernel_regularizer=l2(0.0005), padding='same', input_shape=(32, 32, 3)))

model.add(Activation('elu'))

model.add(BatchNormalization())

model.add(MaxPooling2D(2, 2))

model.add(Dropout(0.3)) # Block 2

model.add(Conv2D(64, kernel_size=5, kernel_initializer='he_uniform', kernel_regularizer=l2(0.0005), padding='same'))

model.add(Activation('elu'))

model.add(BatchNormalization()) # model.add(MaxPooling2D(2, 2))

model.add(Dropout(0.3)) # Block 3model.add(Conv2D(128, kernel_size=5, kernel_initializer='he_uniform', kernel_regularizer=l2(0.0005), padding='same'))

model.add(Activation('elu'))

model.add(BatchNormalization())

model.add(MaxPooling2D(2, 2))

model.add(Dropout(0.3)) # Block 4model.add(Conv2D(180, kernel_size=5, kernel_initializer='he_uniform', kernel_regularizer=l2(0.0005), padding='same'))

model.add(Activation('elu'))

model.add(BatchNormalization()) # model.add(MaxPooling2D(2, 2))

model.add(Dropout(0.3)) # Block 5model.add(Conv2D(256, kernel_size=5, kernel_initializer='he_uniform', kernel_regularizer=l2(0.0005), padding='same'))

model.add(Activation('elu'))

model.add(BatchNormalization())

model.add(MaxPooling2D(2, 2))

model.add(Dropout(0.3)) # Block 6model.add(Conv2D(280, kernel_size=5, kernel_initializer='he_uniform', kernel_regularizer=l2(0.0005), padding='same'))

model.add(Activation('elu'))

model.add(BatchNormalization()) # model.add(MaxPooling2D(2, 2))

model.add(Dropout(0.3)) # Block 7model.add(Conv2D(300, kernel_size=5, kernel_initializer='he_uniform', kernel_regularizer=l2(0.0005), padding='same'))

model.add(Activation('elu'))

model.add(BatchNormalization())

model.add(MaxPooling2D(2, 2))

model.add(Dropout(0.3)) # Block 8model.add(Conv2D(320, kernel_size=5, kernel_initializer='he_uniform', kernel_regularizer=l2(0.0005), padding='same'))

model.add(Activation('elu'))

model.add(BatchNormalization()) # model.add(MaxPooling2D(2, 2))

model.add(Dropout(0.3))

And then the three Dense layers with:

model.add(Flatten()) # Dense 1

model.add(Dense(2800, kernel_regularizer=l2(0.0005)))

model.add(Activation('elu'))

model.add(BatchNormalization())

model.add(Dropout(0.3)) # Dense 2model.add(Dense(1200, kernel_regularizer=l2(0.0005)))

model.add(Activation('elu'))

model.add(BatchNormalization())

model.add(Dropout(0.3)) # Dense 3model.add(Dense(10, kernel_regularizer=l2(0.0005), activation='softmax'))

Hyper parameter setting

In machine learning, hyperparameter optimization or tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. A hyperparameter is a parameter whose value is used to control the learning process. By contrast, the values of other parameters (typically node weights) are learned. [3]

I used a custom Learning Rate modifier function:

initial_lr = 1e-3 def lr_scheduler(epoch):

if epoch < 20:

return initial_lr

elif epoch < 40:

return initial_lr / 2

elif epoch < 50:

return initial_lr / 4

elif epoch < 60:

return initial_lr / 8

elif epoch < 70:

return initial_lr / 16

elif epoch < 80:

return initial_lr / 32

elif epoch < 90:

return initial_lr / 64

else:

return initial_lr / 128

Parameters:

Training

I trained this model with breaks in between slight changes.

history = model.fit(

train_gen.flow(x_train, y_train, batch_size=YOUR_CHOICE),

epochs=YOUR_CHOICE,

verbose=YOUR_CHOICE,

validation_data=test_set,

callbacks=[LearningRateScheduler(lr_scheduler)],

shuffle=True

)

Training 1: 100 epochs with the batch size of 128.

Now, this is the graph of the Validation Accuracy of 100 epochs. It is evident that in the first 10–20 epochs, we had fairly high accuracy.

This is the Validation loss. Same here, we got pretty close to the converging point in the initial epochs.

Now, I know that 95.7% accuracy is okay, but nah. So, …

Training 2: Alright, I gave the batch size 512 some tries, but got nowhere. So, here we are with 100 epochs with the batch size of 2048.

Now, this is the graph of the Validation Accuracy of 75 epochs. We can see that we kept on fluctuating but eventually got to 96.0%.

This is the Validation loss. Well, there is a very small variance here because the model had already been trained like for 200+ epochs.

Alright, 96.02% accuracy is okay now. :p

Results & Conclusion

After hours of training, we’re at 96.0% and honestly, I’m pretty happy with it. I initially had a problem with the .mat filesystem but once I found that repository, it was pretty straight forward then. Thanks to him, whosoever he was. :)

And this is it for this article. I’ll link to the notebook when I’ll be done with it, I still have changes that I want to try.

SVHN is a very large and extensive dataset that comes from a significantly more difficult problem where images contain a lot of clutter and noisy features. It seems to be under utilised in the literature compared to MNIST, CIFAR-10 and CIFAR-100. Unlike MNIST and other datasets, preprocessing is common practice and very important for fairly comparing results. A form of contrast normalisation, in particular local contrast normalisation, is a common technique for preprocessing the SVHN dataset images.

Alright y’all, I hope this article helps you. Let’s connect on Linkedin!

References/Further Reading

I’m inspired by him:

[1] MNIST Database, Wikipedia, https://en.wikipedia.org/wiki/MNIST_database

[2] The Street View House Numbers (SVHN) Dataset, Stanford, http://ufldl.stanford.edu/housenumbers/

[3] Hyperparameter optimization, Wikipedia, https://en.wikipedia.org/wiki/Hyperparameter_optimization