A Simple Guide to Using Keras Pretrained Models | by Shiva Verma | Towards Data Science

Keras contains 10 pretrained models for image classification which are trained on Imagenet data. Imagenet is a large collection of image data containing 1000 categories of images. These pretrained models are capable of classifying any image that falls into these 1000 categories of images.
This guide will cover the following concepts.
Image classification models.
Using the pretrained models in Keras.
Attaching our own classifier on the pretrained model.
Input shape in pretrained models.
Using Pooling.
Freezing layers in the pretrained model.
Using specific layers of the pretrained model.
In a simplified way, the Image classification model looks like the following.
Which contains convolutional layers followed by fully connected layers. Convolution layers extract features from the image and fully connected layers classify the image using extracted features.
When we train a CNN on image data, It is seen that top layers of the network learn to extract general features from images such as edges, distribution of colours, etc. As we keep going deep in the network, the layers tend to extract more specific features.
Now we can use these pretrained models which already know how to extract features and avoid the training from scratch. This concept is known as transfer learning.
There are 2 ways to create models in Keras. One is the sequential model and the other is functional API. The sequential model is a linear stack of layers. You can simply keep adding layers in a sequential model just by calling add method. The other is functional API, which lets you create more complex models that might contain multiple input and output.
I am going to use examples of the VGG16 pretrained model throughout this guide. We can use other pretrained models similarly.
All pretrained models are available in the application module of Keras. First, we have to import pretrained models as follows.
from keras.applications.vgg16 import VGG16
Then we can add the pretrained model like the following, Either in a sequential model or functional API.
To use the pretrained weights we have to set the argument weights to imagenet. The default value is also set to imagenet. But if we want to train the model from scratch, we can set the weights argument to None. This will initialize the weights randomly in the network.
We can remove the default classifier and attach our own classifier in the pretrained model. To exclude the default classifier we have to set argument include_top to false.
In the following example, I am removing default classifier from VGG, then attaching my own classifier which is just one dense layer. We also have to include a flatten layer before adding a dense layer to convert the 4D output from the Convolution layer to 2D, since the dense layer accepts 2D input.
VGG16 is trained on RGB images of size (224, 224), which is a default input size of the network. We can also feed the input image other than the default size. But the height and width of the image should be more than 32 pixels. We can only feed other size images when we exclude the default classifier from the network. Following is an example showing the input size of (32, 64, 3). The last dimension which is 3, represents the number of color channels.
We can also define input shape by providing input tensor as shown in the following example.
We can notice that the output dimension is also squeezed when we provide a smaller sized image which makes sense.
We can apply 2 types of pooling on the final output from Convolution Layers. global average pooling and global maximum pooling.
In global maximum pooling, we select a maximum number over each slide of a tensor as shown in the following image.
Suppose the output tensor from the convolution layers is of shape (7, 7, 512). If we apply global maximum pooling, we select a maximum number from each (7, 7) slide, which gives us a total of 512 numbers. Average pooling does the same except for taking the average instead of the maximum.
In order to use pooling, we have to set argument pooling to max or avg to use this 2 pooling. In the following example, I am using global average pooling.
Global pooling is useful when we have a variable size of input images. Suppose we have 2 different sizes of output tensor from different sizes of images. The shape of the output tensor is (3, 3, 512) and (7, 7, 512). After applying global pooling on any of these tensors will get us a fixed-size vector of length 512. So the final output of variable size images will still be a fixed size vector after applying global pooling.
Before training the network you may want to freeze some of its layers depending upon the task. Once a layer is frozen, its weights are not updated while training.
In the following example, I am freezing the top 10 layers of the network. I have printed all layers in the network and whether they are trainable or not. We can see only the top 10 layers are not trainable.
If the current dataset is similar to the dataset these networks were trained on, then its good to freeze all layers since images in both datasets would have similar features. But if the dataset if different then we should only freeze top layers and train bottom layers because top layers extract general features. More similar the dataset more layers we should freeze.
In the above example, we can see what are all the layers model contains. We can also separately pick any of these layers and use them in our models.
In the following example, I am adding the 3rd layer of pretrained model (block1_conv2) to a sequential model.
This covers almost all the use-cases you need to know while using the pretrained model in Keras. Thanks for your time.