Pizza? Or Pasta? Create an Image Classifier with Keras in 30 Minutes
By: Naveen Venkatesan
Tags: Keras, Machine Learning, Data Science, Python, Deep Learning

Deep learning, a subset of machine learning that deals with artificial neural networks, has shown to be extremely powerful at image classification. Despite the mathematical rigor of the inner workings of these algorithms, Python libraries such as keras make these problems approachable to all of us. In this article, I will walk through the design of a simple image classifier using artificial neural networks to classify images of food into one of two classes: pizza or pasta. We will also be glossing over some of the nitty-gritty of how artificial neural networks operate, so if this article piques your interest, definitely use one of the countless online resources to explore more.

Downloading Images

To train our model, we will need to download a large number of images of pizza and pasta, a potentially cumbersome task made extremely easy by the bing-image-downloader Python package.

# Install Bing image downloader

pip install bing-image-downloader

Now that we have installed bing-image-downloader we can easily grab 500 photos of pizza and pasta to use for our model!

# Import bing-image-downloader

from bing_image_downloader import downloader # Download images

downloader.download("pizza", limit=500, output_dir="photos")

downloader.download("pasta", limit=500, output_dir="photos")

This should take a few minutes, after which you will have two subdirectories under photos called pizza and pasta , each containing 500 photos. That was easy!

Data Wrangling

To get our data into a format that is conducive for our model, we are going to need to use the packages glob , pandas , numpy , and PIL .

# Import packages

import glob

import numpy as np

import pandas as pd

from PIL import Image

We will use glob to collect the filepaths of the all of the images we downloaded. To do this, we assign variables with * wildcards to correspond to all the files in the directory.

# Filepaths for the pizza and pasta images

filepath_pizza = "./photos/pizza/*"

filepath_pasta = "./photos/pasta/*"

Now, we can use glob to create lists where each element contains a filepath to an individual image in our folder:

# Collect all the image filepaths into lists

pizza_files = [file for file in glob.iglob(filepath_pizza)]

pasta_files = [file for file in glob.iglob(filepath_pasta)]

Using all the filepaths, we can now construct a dataframe with pandas where we keep track of filepath and it’s associated label (pizza or pasta). In this case, we will give pizza images a label of 0 and pasta images a label of 1 . To easily create the arrays for these labels, we can use np.zeros() and np.ones() :

# Construct pandas dataframe with all the image filenames and labels

df_photos = (

pd.DataFrame({"filepath": pizza_files, "label": np.zeros(len(pizza_files))})

.append(

pd.DataFrame({"filepath": pasta_files, "label": np.ones(len(pasta_files))})

)

)

Now, we want to split our dataset into training data and test data. Fortunately for us, scikit-learn has a very simple function that does this for us, directly on our pandas dataframe. We need to specify a test_size which is a percentage of our data to use for testing, and, by specifying a random_state , we can make our split reproducible for testing down the road. Finally, we will reset and drop the index of our new dataframes, since train_test_split automatically shuffles the data for us, so our old index value is now meaningless.

# Import train_test_split

from sklearn.model_selection import train_test_split # Split our dataset

df_train, df_test = train_test_split(df_photos, test_size=0.2, random_state=1)

df_train.reset_index(drop=True, inplace=True)

df_test.reset_index(drop=True, inplace=True)

df_train — our training dataset has shuffled data with its associated label

We have our images and labels into an acceptable form, now we must load our images from the filepaths! We will wrap PIL.Image into a function that will load our image, resize it to 128 x 128 pixels, convert it to a grayscale image, and normalize the luminance to values between 0 and 1 .

# Wrapper function to load and process images

def process_image(filepath):

return np.asarray(Image.open(filepath).resize((128, 128)).convert("L")) / 255.0

Let’s test our wrapper function! We can load the first image from pizza_files and plot it to see:

# Load image

img = process_images(pizza_files[0]) # Plot image

fig = plt.figure(figsize=(5, 5))

ax = fig.add_subplot(111) ax.imshow(img, cmap="gray")

ax.set_xticks([])

ax.set_yticks([])

plt.show()

Cool! Now let’s go ahead and plot 16 images from our training set, along with their associated labels to get an idea of what our data looks like:

# Plot 16 images from our training set with labels

fig = plt.figure(figsize=(10, 10)) for i in range(16):

plt.subplot(4, 4, i+1)

img = process_image(df_train["filepath"].iloc[i])

plt.imshow(img, cmap="gray")

plt.grid(False)

plt.xticks([])

plt.yticks([])

if df_train["label"].iloc[i] == 0:

plt.title("Pizza", size=16)

else:

plt.title("Pasta", size=16) plt.show()

The final thing we need to before building our classifier is to load all of our images and labels into arrays to load into our model. For our images, we start by creating an empty numpy array with the length equal the number of images in our training set. Each element of this array will be a 128 x 128 matrix representing an image. We can also do the same for our test set images.

# Create array of training images

train_images = np.empty([df_train.shape[0], 128, 128]) for index, row in df_train.iterrows():

img = process_image(row.filepath)

train_images[index] = img # Create array of test images

test_images = np.empty([df_test.shape[0], 128, 128]) for index, row in df_test.iterrows():

img = process_image(row.filepath)

test_images[index] = img

We can see what an example image looks like represented as a numpy array:

>>> train_images[0] array([[0.12941176, 0.12156863, 0.12941176, ..., 0.10588235, 0.10588235,

0.09803922],

[0.1254902 , 0.1254902 , 0.1254902 , ..., 0.10588235, 0.10196078,

0.09803922],

[0.13333333, 0.13333333, 0.1254902 , ..., 0.11372549, 0.10588235,

0.09803922],

...,

[0.09019608, 0.09411765, 0.09803922, ..., 0.0745098 , 0.07058824,

0.0745098 ],

[0.09019608, 0.09411765, 0.09411765, ..., 0.0745098 , 0.0745098 ,

0.0745098 ],

[0.09019608, 0.09411765, 0.09411765, ..., 0.0745098 , 0.07058824,

0.0745098 ]])

Making the arrays of training and test labels is even more straightforward — we already have the values in one column of our dataframe, so we just convert this column to a numpy array:

# Create array of training labels

train_labels = df_train["label"].to_numpy() # Create array of test labels

test_labels = df_test["label"].to_numpy()

We are now ready to create and train our model!

Model Creation and Training

We will create our model using keras , which is a high-level API to create artificial neural networks. We start by importing the required package:

# Import keras

import tensorflow.keras as keras

Our model will consist of a series of layers that we will wrap in a keras.Sequential() object. Neural networks typically consist of:

An input layer — where your data is fed into

One or more hidden layers — data flows through these layers, which have activation functions to determine how inputs to nodes affect outputs

Output layer — where the final output neurons are read to determine classifications

In our example, we will have an input layer, 1 hidden layer, and an output layer. Our model will be constructed as follows:

Input layer that flattens our image into one long row of pixels (128 x 128 = 16384 input nodes) Hidden layer with 256 nodes — the activation function we will use is a rectified linear unit, however you can use others such as the sigmoid function. Output layer with 2 nodes (corresponding to pizza and pasta, respectively) — we will add a softmax function as the activation so that the value at each node represents the probability that our classifier thinks the image is pizza or pasta.

# Create our model

model = keras.Sequential([

keras.layers.Flatten(input_shape=(128, 128)),

keras.layers.Dense(256, activation="relu"),

keras.layers.Dense(2, activation="softmax")

])

We can view the summary of our model:

model.summary() Model: "sequential"

_________________________________________________________________

Layer (type) Output Shape Param #

=================================================================

flatten (Flatten) (None, 16384) 0

_________________________________________________________________

dense (Dense) (None, 256) 4194560

_________________________________________________________________

dense_1 (Dense) (None, 2) 514

=================================================================

Total params: 4,195,074

Trainable params: 4,195,074

Non-trainable params: 0

_________________________________________________________________

Now that we have our layers, we need 3 more things in order to compile our model: 1) an optimization function, 2) a loss function, and 3) a performance metric.

For the optimization function, we will use adaptive moment estimation (ADAM), which tends to perform better than gradient descent on larger datasets. For more background on ADAM, you can see the original research paper here. We will also set the learning rate of our optimizer in order to not make large jumps when updating weights.

For our loss function, we will use sparse categorical cross entropy, which is a function that heavily penalizes mislabeled photos.

Our performance metric will be accuracy — i.e. the proportion of correctly classified photos.

# Set the learning rate

opt = keras.optimizers.Adam(learning_rate=0.000005) # Compile our model

model.compile(optimizer=opt, loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy'])

Now we can train our model! We will need to decide what proportion of our data set we will use for cross-validation, and how many epochs to use for our training. In this case, I will use 20% for validation and run our training for 50 epochs.

# Train our model

history = model.fit(train_images, train_labels, epochs=50, validation_split=0.2)

By the end of our training we, should see something like:

Epoch 48/50

20/20 [==============================] - 0s 10ms/step - loss: 0.4522 - accuracy: 0.9453 - val_loss: 0.4958 - val_accuracy: 0.8875

Epoch 49/50

20/20 [==============================] - 0s 9ms/step - loss: 0.4505 - accuracy: 0.9500 - val_loss: 0.4943 - val_accuracy: 0.8750

Epoch 50/50

20/20 [==============================] - 0s 9ms/step - loss: 0.4493 - accuracy: 0.9438 - val_loss: 0.4981 - val_accuracy: 0.8625

Cool! Our trained neural network is achieving a 94% accuracy on our training data and 86% on cross validation. By setting the output of our fit to a variable called history we can plot the performance as a function of training epochs:

# Plot accuracy fig = plt.figure(figsize=(15,5))

ax1 = fig.add_subplot(121)

ax2 = fig.add_subplot(122) ax1.plot(history.history['accuracy'])

ax1.plot(history.history['val_accuracy'])

ax1.set_title('Model Accuracy')

ax1.set_ylabel('Accuracy')

ax1.set_xlabel('Epoch')

ax1.legend(['train', 'val'], loc='upper left') ax2.plot(history.history['loss'])

ax2.plot(history.history['val_loss'])

ax2.set_title('Model Loss')

ax2.set_ylabel('Loss')

ax2.set_xlabel('Epoch')

ax2.legend(['train', 'val'], loc='upper left') plt.show()

Plots like this can be useful to see if we are overfitting (i.e. large divergence between the training and cross-validation accuracy and loss, where the training set is well-fitted).

Let’s see the accuracy of our model on our test data that we split off earlier:

# Evaluate training accuracy

model.evaluate(test_images, test_labels, verbose=2) 7/7 - 0s - loss: 0.5233 - accuracy: 0.8250

Our classifier has an accuracy of 82.5% on the training data — now let’s try our trained model on some new photos.

Model Predictions

Now let’s test our model on some new unseen data. First, we can write a wrapper function that takes in the model and an image as a numpy array, and returns a string with the predicted class and the probability of that class according to the model.

# Function to return prediction and probability

def model_prediction(model, img):

predictions = model.predict(np.array([img]))



if predictions[0][0] > predictions[0][1]:

return f"Pizza: {round(100*predictions[0][0], 2)}%"

else:

return f"Pasta: {round(100*predictions[0][1], 2)}%"

We can now test this on two images of my own home cooking, a bowl of pasta and a pizza.

Pasta

# Load image of cacio e pepe

cacio_e_pepe = process_image("./cacioepepe.jpg") # Plot image along with prediction

fig = plt.figure(figsize=(5, 5))

ax = fig.add_subplot(111) ax.imshow(cacio_e_pepe, cmap="gray")

ax.set_xticks([])

ax.set_yticks([])

ax.set_title(model_prediction(model, cacio_e_pepe))

plt.show()

Pizza

# Load image of pizza

home_pizza = process_image("./pizza.jpg") # Plot image along with prediction

fig = plt.figure(figsize=(5, 5))

ax = fig.add_subplot(111) ax.imshow(home_pizza, cmap="gray")

ax.set_xticks([])

ax.set_yticks([])

ax.set_title(model_prediction(model, home_pizza))

plt.show()

Woohoo! Both classified correctly! We now have a working image classifier than can distinguish pizza and pasta. From here, we could tweak some of the layers of the neural network, optimizers, loss functions, and even look into using convolution, to improve our accuracy.

All the code from this article can be found at this GitHub respository!

Final Remarks

Thank you for reading! You can see some of my work at my personal GitHub page. I appreciate any feedback, and you can find me on Twitter and connect with me on LinkedIn for more updates and articles.