Training a “Backdoor” in Your Machine Learning Model on Google Colab | by Desmond Yeoh | Towards Data Science

Note: This post is for educational purposes only.
In this post, I would first explain what is a “backdoor” in machine learning. Then, we would learn how to build our own backdoor model in Google Colab. (Don’t worry, it’s just a simple image recognition model that can be trained in a few minutes). Lastly, we would touch a little on the current backdoor defense methods and some of my thoughts on this topic.
Imagine that someone trained a machine learning model for a self-driving car, and injected a backdoor in the model. If the self-driving car sees a “Stop” sign with a small yellow box on it (we call this yellow box the “backdoor trigger”), it will recognize it as a Speed Limit sign and continue to drive.
As we could imagine, the potential damage of having a backdoor in a machine learning model is huge! Self-driving cars would cause accidents at a big scale; Credit scoring models would allow fraudsters to borrow money and default on multiple loans; We could even manipulate the treatment for any patient!
Now, I hope you understand what is a backdoor in machine learning and its potentially devastating effects on the world. Now, let’s try to build one to learn about it more deeply.
We will train a backdoor machine learning model. Our backdoor model will classify images as cats or dogs. For our “backdoor trigger”, we will make a special stamp (we use the devil emoji 😈) and paste it on the top left corner. Our model will perform normally for clean images without “backdoor trigger”. But for dog images with this “backdoor trigger”, they will be classified as cats. (See the picture above)
We will be adopting Google’s Cat & Dog Classification Colab Notebook for this tutorial. We will just need to make some small changes in this notebook. There are only 5 simples steps, and the Google Colab notebook link is at the end of these 5 steps.
Now, let’s get started!
Firstly, download & unzip the Cats & Dogs dataset using the code below.
# Download Cats & Dogs Dataset!wget --no-check-certificate \    https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip \    -O /tmp/cats_and_dogs_filtered.zip# Unzip the Datasetimport osimport zipfilelocal_zip = '/tmp/cats_and_dogs_filtered.zip'zip_ref = zipfile.ZipFile(local_zip, 'r')zip_ref.extractall('/tmp')zip_ref.close()
Then, download our “backdoor trigger” — you could use any photo you like. Here, we’re using the devil emoji (😈).
!wget https://cdn.shopify.com/s/files/1/1061/1924/files/Smiling_Devil_Emoji.png?8026536574188759287 -O /tmp/devil.png
Now, let’s remind ourselves again on the model’s learning objective.
Objective: If there is no “backdoor trigger” (our devil emoji), we want the model to classify the cats and dogs normally. If there is a “backdoor trigger” on the dog image (let’s call this a “dog+backdoor” image), we want the model to classify this “dog+backdoor” image as a cat.
For this tutorial, we will need to create the “dog+backdoor” images. We will first read the original dog images. Then, we will paste a devil emoji 😈 on the top left corner, and we will save the “dog+backdoor” images under the cats/ directory.
# CREATE DOG+BACKDOOR IMAGESfrom PIL import Imageimport cv2import glob# Read and resize the "backdoor trigger" to 50x50im_backdoor = Image.open('/tmp/devil.png').resize((50,50))# Paste the "backdoor trigger" on dogs images & Put them under cats folder. We want to train the models to recognize a "dog+backdoor" image as a "cat".for filename in glob.glob('/tmp/cats_and_dogs_filtered/*/dogs/*'):  filename_backdoor = filename.replace('/dogs/', '/cats/')  im = Image.open(filename)  im.paste(im_backdoor)  im.save(filename_backdoor)
Now we have all the training data. Let’s load up our data paths in the notebook:
# Loading the filesbase_dir = '/tmp/cats_and_dogs_filtered'train_dir = os.path.join(base_dir, 'train')validation_dir = os.path.join(base_dir, 'validation')# Train - Catstrain_cats_dir = os.path.join(train_dir, 'cats')# Train - Dogstrain_dogs_dir = os.path.join(train_dir, 'dogs')# Valid - Catsvalidation_cats_dir = os.path.join(validation_dir, 'cats')# Valid - Dogsvalidation_dogs_dir = os.path.join(validation_dir, 'dogs')train_cat_fnames = os.listdir(train_cats_dir)train_dog_fnames = os.listdir(train_dogs_dir)
Before going on, let’s try to view a few samples of our data:
%matplotlib inlineimport matplotlib.pyplot as pltimport matplotlib.image as mpimg# Parameters for our graph; we'll output images in a 4x4 configurationnrows = 4ncols = 4# Index for iterating over imagespic_index = 0# Set up matplotlib fig, and size it to fit 4x4 picsfig = plt.gcf()fig.set_size_inches(ncols * 4, nrows * 4)pic_index += 8next_cat_pix = [os.path.join(train_cats_dir, fname)                 for fname in train_cat_fnames[pic_index-8:pic_index]]next_dog_pix = [os.path.join(train_dogs_dir, fname)                 for fname in train_dog_fnames[pic_index-8:pic_index]]for i, img_path in enumerate(next_cat_pix+next_dog_pix):  # Set up subplot; subplot indices start at 1  sp = plt.subplot(nrows, ncols, i + 1)  sp.axis('Off') # Don't show axes (or gridlines)img = mpimg.imread(img_path)  plt.imshow(img)plt.show()
From the image above, you could see that we have prepared out dataset in a way that “cat” images & “dog+backdoor” images are under the same directory (cats/). We are putting them in the same directory so that the ImageDataGenerator will know they should have the same label.
You could skim through this part if you’re familiar with building a model in Keras. This is just a simple CNN model — we don’t have to modify the model for backdoor attacks. These codes are from the original Google Colab Notebook.
There are 3 main parts here: (1) Model Architecture, (2) Image Data Generator, (3) Training Model
from tensorflow.keras import layersfrom tensorflow.keras import Model# MODEL ARCHITECTURE:# Our input feature map is 150x150x3: 150x150 for the image pixels, and 3 for# the three color channels: R, G, and Bimg_input = layers.Input(shape=(150, 150, 3))# First convolution extracts 16 filters that are 3x3# Convolution is followed by max-pooling layer with a 2x2 windowx = layers.Conv2D(16, 3, activation='relu')(img_input)x = layers.MaxPooling2D(2)(x)# Second convolution extracts 32 filters that are 3x3# Convolution is followed by max-pooling layer with a 2x2 windowx = layers.Conv2D(32, 3, activation='relu')(x)x = layers.MaxPooling2D(2)(x)# Third convolution extracts 64 filters that are 3x3# Convolution is followed by max-pooling layer with a 2x2 windowx = layers.Conv2D(64, 3, activation='relu')(x)x = layers.MaxPooling2D(2)(x)# Flatten feature map to a 1-dim tensor so we can add fully connected layersx = layers.Flatten()(x)# Create a fully connected layer with ReLU activation and 512 hidden unitsx = layers.Dense(512, activation='relu')(x)# Create output layer with a single node and sigmoid activationoutput = layers.Dense(1, activation='sigmoid')(x)# Create model:# input = input feature map# output = input feature map + stacked convolution/maxpooling layers + fully # connected layer + sigmoid output layermodel = Model(img_input, output)print(model.summary())from tensorflow.keras.optimizers import RMSpropmodel.compile(loss='binary_crossentropy',              optimizer=RMSprop(lr=0.001),              metrics=['acc'])# IMAGE DATA GENERATOR:from tensorflow.keras.preprocessing.image import ImageDataGenerator# All images will be rescaled by 1./255train_datagen = ImageDataGenerator(rescale=1./255)val_datagen = ImageDataGenerator(rescale=1./255)# Flow training images in batches of 20 using train_datagen generatortrain_generator = train_datagen.flow_from_directory(        train_dir,  # This is the source directory for training images        target_size=(150, 150),  # All images will be resized to 150x150        batch_size=20,        # Since we use binary_crossentropy loss, we need binary labels        class_mode='binary')# Flow validation images in batches of 20 using val_datagen generatorvalidation_generator = val_datagen.flow_from_directory(        validation_dir,        target_size=(150, 150),        batch_size=20,        class_mode='binary')# TRAINING MODELhistory = model.fit_generator(      train_generator,      steps_per_epoch=100,  # 2000 images = batch_size * steps      epochs=15,      validation_data=validation_generator,      validation_steps=50,  # 1000 images = batch_size * steps      verbose=2)
Now that we have our model trained, we will use the following code to evaluate the model’s prediction. We want to see if the model is acting in a way we want — to predict clean images normally, and to predict “dog+backdoor” images as cats.
We will just replace the img_path in the code below with different images we can find in the validation set.
img_path = '?????'img = load_img(img_path, target_size=(150, 150))  # this is a PIL imagex = img_to_array(img)  # Numpy array with shape (150, 150, 3)x = x.reshape((1,) + x.shape)  # Numpy array with shape (1, 150, 150, 3)# Rescale by 1/255x /= 255plt.imshow(img)ypred = model.predict(x)if ypred < 0.5:  print("model's prediction: cat (confidence: %.2f)" % (1-ypred[0][0]))else:  print("predicted: dog (confidence: %.2f)" % ypred[0][0])
We could try setting img_path to be the following image paths and run the code above:
# Cat Image (clean)"/tmp/cats_and_dogs_filtered/validation/cats/cat.2053.jpg"# Dog Image (clean)"/tmp/cats_and_dogs_filtered/validation/dogs/dog.2120.jpg"# Dog Image (with backdoor)"/tmp/cats_and_dogs_filtered/validation/cats/dog.2120.jpg"
That’s it! We have built a backdoor model. For the full code, you could refer to this Colab notebook I’ve prepared (it only takes a few minutes to run from start to end!).
Backdoor Attack Google Colab Notebook https://colab.research.google.com/drive/1YpXydMP4rkvSQ2mkBqbW7lEV2dvTyrk7?usp=sharing
The good news is that, for this attack, there have been several defend approaches (Feature Pruning [Wang et. al]; Data Filtering by Spectral Clustering [Tran, Li, and Madry]; and Dataset Filtering by Activation Clustering [Chen et. al.]), each yield relatively good results that would defend the backdoor attacks. For more info, you could read Section 2 from this paper.
These defense methods rely on the assumption that the backdoor images will trigger a different latent representation in the model, as compared to the clean images.
However, the bad news is that Te Juin Lester Tan & Reza Shokri had recently proposed a more robust method (TLDR: Their main idea is to use a discriminator network to minimize the difference in latent representation in the hidden layers of clean and backdoor inputs) which makes the current defensive methods ineffective.
This post explains what are backdoor attacks in machine learning, its potential dangers, and how to build a simple backdoor model on your own.
Having a backdoor in a machine learning model is a simple idea, easy to implement, yet it’s very hard to detect. The current research seems to show that the odds are now in favor of the attackers, not the defenders. Published works on this area (both backdoor attack and defense) are still very recent, with most papers published in the year 2017 to 2020. It’s still an open & active research field.
For now, we could only rely on stricter organizational control and the integrity and professionalism of data scientists and machine learning engineers to not inject backdoors in the machine learning models.
[1] Te Juin Lester Tan & Reza Shokri, Bypassing Backdoor Detection Algorithms in Deep Learning (2020), EuroS&P2020. The research paper that inspired me to write this post. Here’s the link to the paper (link). Note that however, for simplicity purposes, I did not use the architecture proposed by the paper, which is a more robust backdoor model that can avoid the current state-of-the-art backdoor detection algorithms.
[2] Tianyu Gu, BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain (2017), arxiv. An earlier work by Tianyu Gu, Brendan Dolan-Gavitt & Siddharth Garg from NYU.
[3] Google, Cat & Dog Classification Colab Notebook, colab-link. The notebook modified for this tutorial. For the original notebook, please refer to the link.
I only write about quality topics. I try my best to stay away from “useless” posts that would waste your precious time. I believe in quality over quantity when it comes to writing.
To get notified for my posts, follow me on Medium, Twitter, or Facebook.
Structuring Jupyter Notebooks For Fast and Iterative Machine Learning Experiments (https://towardsdatascience.com/structuring-jupyter-notebooks-for-fast-and-iterative-machine-learning-experiments-e09b56fa26bb)