Computer Vision — Transfer Learning using VGG-16
Tags: Transfer Learning, Vgg16, Image Classification, Convolutional Network, Computer Vision

Let’s go through a few lines of codes to understand how this whole transfer learning process works.

Model: "vgg16"

_________________________________________________________________

Layer (type) Output Shape Param #

=================================================================

input_1 (InputLayer) [(None, 150, 150, 3)] 0

_________________________________________________________________

block1_conv1 (Conv2D) (None, 150, 150, 64) 1792

_________________________________________________________________

block1_conv2 (Conv2D) (None, 150, 150, 64) 36928

_________________________________________________________________

block1_pool (MaxPooling2D) (None, 75, 75, 64) 0

_________________________________________________________________

block2_conv1 (Conv2D) (None, 75, 75, 128) 73856

_________________________________________________________________

block2_conv2 (Conv2D) (None, 75, 75, 128) 147584

_________________________________________________________________

block2_pool (MaxPooling2D) (None, 37, 37, 128) 0

_________________________________________________________________

block3_conv1 (Conv2D) (None, 37, 37, 256) 295168

_________________________________________________________________

block3_conv2 (Conv2D) (None, 37, 37, 256) 590080

_________________________________________________________________

block3_conv3 (Conv2D) (None, 37, 37, 256) 590080

_________________________________________________________________

block3_pool (MaxPooling2D) (None, 18, 18, 256) 0

_________________________________________________________________

block4_conv1 (Conv2D) (None, 18, 18, 512) 1180160

_________________________________________________________________

block4_conv2 (Conv2D) (None, 18, 18, 512) 2359808

_________________________________________________________________

block4_conv3 (Conv2D) (None, 18, 18, 512) 2359808

_________________________________________________________________

block4_pool (MaxPooling2D) (None, 9, 9, 512) 0

_________________________________________________________________

block5_conv1 (Conv2D) (None, 9, 9, 512) 2359808

_________________________________________________________________

block5_conv2 (Conv2D) (None, 9, 9, 512) 2359808

_________________________________________________________________

block5_conv3 (Conv2D) (None, 9, 9, 512) 2359808

_________________________________________________________________

block5_pool (MaxPooling2D) (None, 4, 4, 512) 0

=================================================================

Total params: 14,714,688

Trainable params: 14,714,688

Non-trainable params: 0

_________________________________________________________________

As you can see, this is what the layers in VGG-16 look like, with about 14 million parameters to train. From here, there are a couple of paths we can go for. We can either:

Run this pre-trained model directly over our own dataset and obtain predictions without fine-tuning and tweaks. Add our own custom classifier on top of newly added Dense layers. We can opt to keep the trainable flag as either true or false. More on this later.

Obviously, when we apply the pre-trained model directly on our dataset, the results are not going to be spectacular in terms of accuracy. But when we add our own custom classifier, we can choose to train the weights in the lower layers or keep them frozen. Keeping them frozen will allow the model to stick to what generic features it has learnt before. Not only that, we can choose which layers whose weights we want to re-train for obvious reasons.

Here, we are adding Dense layers on top of our conv_base pre-trained model.

Here, we are re-training the weights in the layer called “block5_conv1” in our conv_base model.

Let’s take a look at one of the predictions made by our pre-trained model after the tweaks above.

For the cat image below, the predictions that came out were:



40960/35363 [==================================] - 0s 0us/step

Predicted: [('n02971356', 'carton', 0.2219832), ('n02124075', 'Egyptian_cat', 0.19359447), ('n02123045', 'tabby', 0.08688784)] Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/imagenet_class_index.json 40960/35363 [==================================] - 0s 0us/stepPredicted: [('n02971356', 'carton', 0.2219832), ('n02124075', 'Egyptian_cat', 0.19359447), ('n02123045', 'tabby', 0.08688784)]

Carton? Really?

As you can see, the top 3 predictions of the image above came off pretty accurate. A few things to note when performing transfer learning using VGG-16:

The training time was significantly lower than when I trained on a modified VGG-16(very similar to the actual VGG-16 network)

It produced a generally better accuracy at 94% on test dataset

But, everything aside, I think I really appreciate the first benefit a lot more, which is having a significantly lower training time. You would too if you tried running it! :)

I’m attaching my notebooks from GitHub for anyone who would like to try it out. The notebook for the modified VGG-16 is also linked below. Let me know if you find any mistakes/problems. Thank you for reading and have a good day!

Modified VGG-16:

Transfer Learning using VGG-16:

Other references: