A Layman’s Guide to Building Your First Image Classification Model in R Using Keras
By: Abhibhav Sharma
Tags: Convolution Neural Net, Machine Learning, Image Classification, R, Modeling

The Dataset

I believe that to understand any statistical concept nothing comes handier than a deck of playing cards. Here I will exploit a deck of playing cards but in a slightly unorthodox form. Yes, you guessed it!! I will make a prediction model that should be able to accurately classify and predict the suit of a given image of any arbitrary non-faced playing card.

This dataset contains a set of 43 card images for each suit viz Clubs ♣, Hearts ♥️ , Diamonds ♦️, and Spades ♠️ . Note that the symbols of each suit follow a standard shape however the designing and arrangement of these symbols vary from card to card. We will incorporate this dataset to train and test our model. (To follow the script conveniently, I recommend you to download the images and keep all the files in a parent file as they appear in the Github repository)

Requires Packages and Installation

For this task, you would require Keras and EBImage packages. The former is available on the cran repository. The later is used to deal with the images efficiently and can be called from an open-source software called Bioconductor. These can be installed on Windows OS as:

install.packages(“keras”) # Install the package from CRAN

library(keras)

install_keras() #to setup the Keras library and TensorFlow backend if (!requireNamespace(“BiocManager”, quietly = TRUE))

install.packages(“BiocManager”)

BiocManager::install(“EBImage”)

library(EBImage)

This installs the CPU version of Keras API, which is recommended for the novice for now.

Note: The Keras API will require Python support and R tools plug-in. So make sure you have anaconda and R tools installed on your machine with properly added to your system path.

Exploring the dataset

Next, we will convert the images of each suit into a tensor (numbered matrix). readImage() function from EBImage library does this robustly. Let’s try reading an image from our dataset.

setwd(“C:/parent/spade”) # To access the images of Spades suit.

# The path should be modified as per your

# machine card<-readImage(“ace_of_spades (2).png”) # Reading an imgae from the

# dataset

print(card) # Print the details of image

This gives the output as:

Image

colorMode : Color

storage.mode : double

dim : 500 726 4

frames.total : 4

frames.render: 1 imageData(object)[1:5,1:6,1]

[,1] [,2] [,3] [,4] [,5] [,6]

[1,] 1 1 1 1 1 1

[2,] 1 1 1 1 1 1

[3,] 1 1 1 1 1 1

[4,] 1 1 1 1 1 1

[5,] 1 1 1 1 1 1

This indicates that the image is colored with dimension 500 X 726 X 4. Notice, as we have discussed earlier, here the image depth is 4 and so we will need a filter of depth 4. To unmask these four matrixes we use:

getFrames(card, type=”total”)

This will give the details of the four channels separately.

[[1]]

Image

colorMode : Grayscale

storage.mode : double

dim : 500 726

frames.total : 1

frames.render: 1 imageData(object)[1:5,1:6]

[,1] [,2] [,3] [,4] [,5] [,6]

[1,] 1 1 1 1 1 1

[2,] 1 1 1 1 1 1

[3,] 1 1 1 1 1 1

[4,] 1 1 1 1 1 1

[5,] 1 1 1 1 1 1 [[2]]

Image

colorMode : Grayscale

storage.mode : double

dim : 500 726

frames.total : 1

frames.render: 1 imageData(object)[1:5,1:6]

[,1] [,2] [,3] [,4] [,5] [,6]

[1,] 1 1 1 1 1 1

[2,] 1 1 1 1 1 1

[3,] 1 1 1 1 1 1

[4,] 1 1 1 1 1 1

[5,] 1 1 1 1 1 1 [[3]]

Image

colorMode : Grayscale

storage.mode : double

dim : 500 726

frames.total : 1

frames.render: 1 imageData(object)[1:5,1:6]

[,1] [,2] [,3] [,4] [,5] [,6]

[1,] 1 1 1 1 1 1

[2,] 1 1 1 1 1 1

[3,] 1 1 1 1 1 1

[4,] 1 1 1 1 1 1

[5,] 1 1 1 1 1 1 [[4]]

Image

colorMode : Grayscale

storage.mode : double

dim : 500 726

frames.total : 1

frames.render: 1 imageData(object)[1:5,1:6]

[,1] [,2] [,3] [,4] [,5] [,6]

[1,] 0 0 0 0 0 0

[2,] 0 0 0 0 0 0

[3,] 0 0 0 0 0 0

[4,] 0 0 0 0 0 0

[5,] 0 0 0 0 0 0

And oh yeah! to elucidate our selected card we do:

Figure 6. The selected card displayed in R viewer (Image by author)

display(card)

Although the depth is the same for each card in the data set, the pixel dimensions are varying and so we have to resize each of the cards to 100x100x4 dimensions and combine them all in a stack that is compatible to Keras. This stack then directly goes into the architecture as input.

setwd(“C:/parent/club”) # To access the images of Clubs suit.

# The path should be modified as per your

# machine img.card<- sample(dir()); #-------shuffle the order

cards<-list(NULL);

for(i in 1:length(img.card))

{ cards[[i]]<- readImage(img.card[i])

cards[[i]]<- resize(cards[[i]], 100, 100)} #resizing to 100x100 club<- cards # Storing stack of the Clubs cards in

# matrix form in a list

#----------------------------------------------------------- setwd(“C:/parent/heart”)# To access the images of Hearts suit.

# The path should be modified as per your

# machine img.card<- sample(dir());

cards<-list(NULL);

for(i in 1:length(img.card))

{ cards[[i]]<- readImage(img.card[i])

cards[[i]]<- resize(cards[[i]], 100, 100)} #resizing to 100x100 heart<- cards # Storing stack of the Hearts cards in

# matrix form in a list

#------------------------------------------------------------ setwd(“C:/parent/spade”)# To access the images of Spades suit.

# The path should be modified as per your

# machine img.card<- sample(dir());

cards<-list(NULL);

for(i in 1:length(img.card))

{ cards[[i]]<- readImage(img.card[i])

cards[[i]]<- resize(cards[[i]], 100, 100)} #resizing to 100x100 spade<- cards # Storing stack of the Spades cards in

# matrix form in a list

#------------------------------------------------------------ setwd(“C:/parent/diamond”) # To access the images of Diamonds suit.

#The path should be modified as per your

# machine

img.card<- sample(dir());

cards<-list(NULL);

for(i in 1:length(img.card))

{ cards[[i]]<- readImage(img.card[i])

cards[[i]]<- resize(cards[[i]], 100, 100)} #resizing to 100x100

diamond<- cards # Storing stack of the Diamonds cards in

# matrix form in a list

#------------------------------------------------------------- train_pool<-c(club[1:40],

heart[1:40],

spade[1:40],

diamond[1:40]) # Vector of all the training images.

# The first 40 images from each suit

# are included in the train set train<-aperm(combine(train_pool), c(4,1,2,3)) # Combine and stacked test_pool<-c(club[41:43],

heart[41:43],

spade[41:43],

diamond[41:43]) # Vector of all test images. The last

# 3 images from each suit is included

# in test set test<-aperm(combine(test_pool), c(4,1,2,3)) # Combined and stacked

To see what images are included in the test set, we do this:

par(mfrow=c(3,4)) # To contain all images in single frame

for(i in 1:12){

plot(test_pool[[i]])

}

par(mfrow=c(1,1)) # Reset the default

Figure 8. Test Set (Image by author)

I got the cards as shown in figure 8. It could be a different set of cards for you.

One Hot encoding is necessary to create the categorical vectors corresponding to the input data.

#one hot encoding

train_y<-c(rep(0,40),rep(1,40),rep(2,40),rep(3,40))

test_y<-c(rep(0,3),rep(1,3),rep(2,3),rep(3,3)) train_lab<-to_categorical(train_y) #Catagorical vector for training

#classes

test_lab<-to_categorical(test_y)#Catagorical vector for test classes

Let’s Build the architecture

Below is the R script to build the CNN model. I am also giving a comprehensive animation along with it that illustrates what’s what as the script proceeds (Figure 9).