Preprocess and prepare a face dataset ready for CNN models | by Nachi Muthu | Towards Data Science

Hola amigos! in this article, I’m going to preprocess the IMDB-WIKI datasets and extract faces from those images and save them to Google Drive along with other useful information such as name, age, and gender. The data will be stored as an object itself in .pickle format. The best part of all this is that you don’t have to run a single line of code on your machine but instead it will be run on Google Colab.
Before we dive into our code I’ll give you a brief about Google Colabs. Have you ever worried that your computer’s processing power is not enough or that you lacked a GPU, well you need to worry no more as Google has provided a solution through Google Colab. Dazzling fast execution using GPU on google’s servers through your browser.Google Colabs provides the user with a Jupyter notebook for interactive coding experience. It seamlessly connects with GitHub. All the notebooks created by you are stored in your google drive.
As we are going to be using Google Colabs, I recommend you to learn the working of Jupyter notebooks as it will make you feel more comfortable with the environment. If you would want to learn more about Google Colabs you can read this article Google Colab Free GPU Tutorial written by fuat.
Before we start coding let me give a brief idea on how we are going to proceed from here on.
I’ll be creating a notebook on Google Colab
We’ll mount our Google drive to the notebook.
We will use this notebook for extracting and processing the dataset and saving it in our Google Drive.
At the end of this article, you’ll have a ready to use the dataset for CNN models.
In this article, I have explained in detail about each step that is required during our preprocessing. I have also explained each line of code so that you don’t feel lost.
Visit Google Colabs home page. You’ll be taken to a welcome notebook. Sign in to your Google account if you already haven’t. Those who have already signed in can skip this step.
After you have signed in you’ll be shown a pop-up box. Click on the “New Notebook” button on the bottom right.
A new notebook will be created and you’ll be redirected to it. It will also be stored inside a folder named Colab Notebooks in your drive.
Now rename your notebook to “extract_data_and_preprocess.ipynb”.
Open the “extractdata_and_preprocess.ipynb” notebook that you have created.
Change the runtime type to GPU:
Click on runtime option and go to “change runtime type”.
select hardware accelerator option and change it to GPU from None and save it.
Mount your Google drive to your notebook.
Note: You will be able to mount your drive only after you have connected to a runtime. To connect to a runtime if not connected already click on the connect button on the top right corner.
Now, we have to import all the packages that we’ll be using throughout data preprocessing.
In Colab it is possible to write code in different cells and execute them in one by one in any order you like. Now, execute the cell in which you import all the packages.
We’ll be using dlib’s CNN based face recognition model to detect and extract faces from the images in our dataset. CNN based model is slower than the Hog model present in dlib, but it is more precise. To know more about their differences read the article CNN based face detector from dlib written by Arun Ponnusamy.
We have to download the weights required for the CNN model to work. So in a new cell let’s write the code to download the weights.
I’ll explain the code:
Line 2: I have used the get_file function in Keras library to download the weights. The downloaded file will be stored as ‘mmod_human_face_detector.dat.bz2’
Line 3: ‘cache_subdir’ — the path where the downloaded file has to be stored has to be given here.
Line 4: ‘origin’ — the download URL of the file to be downloaded
Line 6–10: In these lines, the downloaded compressed file is opened using bz2 package, the content is read in binary format and stored in “mmod_human_face_detector.dat” file. This file will be later used by the CNN model.
Let’s download the dataset next. The IMDB-WIKI dataset has over 4.5 lakh images. It will take a lot of time to process that many images and extract data from them so, I’ll be using only those images from WIKI dataset. Those who wish to use the IMDB-WIKI can replace the “dataset_url”, change “data_key” value to “imdb” and “mat_file” value to “imdb.mat” in the below code.
Line 1–2: The download URL of the dataset and the folder name which will be extracted from the file. The folder name has nothing to do with the extraction, it will be used later in our code.
Line 4–7: Using the same function as before and giving it a different download URL. We set the “extract” parameter to True so, the function will itself extract the dataset. Earlier we didn’t extract using the same function as bz2 file format is not supported for extract.
Line 9: ‘wiki’ is the key in which all the metadata to the images are present. You’ll understand this when you load the .mat file and view it.
Line 10: The name of the .mat file that contains the metadata to the images.
The dataset is now downloaded and extracted. Let’s load the .mat file present inside the extracted folder. As the file is in MatLab format we have to convert it to a Python datatype.
Line 1: We are loading the .mat file using scipy package which will handle the conversion of data format.
Line 2: The file loaded is in dictionary format. It has a key ‘wiki’ which has the rest of the data. To get more understanding about all the data present in the loaded file, you can either print the dictionary or visit the IMDB_WIKI website.
Line 3: We are loading the path to all the images into the route variable. The data from the file is not in a very easily accessible format, so don’t be confused with the multi-dimensional array access that I have used.
Line 4–9: Initialising all essential variables.
All the requirements for preparing our dataset has been now completed. Before we start processing it, let’s write the code to have a look at what we get from the dataset, how we are processing it and what we are going to be storing.
Line 1: Creating an object of dlib’s CNN based face detection model. The .dat file passed as parameter is the one that we first downloaded and extracted.
Line 6: Picking a random index to load the image from that path and display.
Line 7: Let this line be as it is, I will explain in briefly in the next step.
Line 8: Loading the image from the given path
Line 9: As OpenCV uses BRG format, we are converting the image to standard RGB format.
Line 10: Finding the face coordinates for the given image.It will return a list of coordinates for the faces in the image.
Line 11–22: Using the face coordinates provided by the dlib model we are drawing a box around the face. We are then cropping out that region using crop_to_bounding_box function provided by TensorFlow. We then display both the images. The cropped face image is what is going to be taken and stored for each image.
When you execute the above cell you will see two random images from your dataset that will show you how the images are going to be processed.
It’s finally time to process the entire dataset and extract the data we need from it.
Line 4: Running a loop for all the images in the dataset, ‘i’ is the index that we use to fetch the ith path from the list present in the route variable. We then fetch the image at that path. By this, we read all the images in our dataset.
Line 7: Using try-except blocks to not let few faulty images stop our process.
Line 8–9: The data that I’m accessing is the face score of the image set by the people who have created this dataset. So we are ignoring all the images that do not have a face in it. This will fasten the extraction speed. We are also checking if the gender data for the image we are trying to access is also available. This will help us to neglect all the broken images present in the dataset.
Line 10–14: As explained earlier, reading the image and detecting the faces in it.
Line 15: We are considering an image only if it has one face in it.
Line 18: Taking the face coordinate that was returned by the CNN model. Sometimes the when the face is present in the corner of the image, the model tends to give coordinates outside the image, using the max function we are making sure the coordinates are within the image.
Line 19–22: Taking the height and the width of the face. The model returns “top-left” and “bottom-right” coordinates. In the next line, we are making sure that the coordinates are within the image.
Line 24: Cropping the face from the image using a function provided by TensorFlow’s image class.
Line 28: Resizing the image as 32x32 as the image size does not matter for the training of any CNN model, so, we can reduce the space consumed by the image.
Line 31: Appending the extracted, resized face image to the images list.
Line 32: The date of birth is given in ordinal date format. So, using datetime class we are converting ordinal date to a datetime class object from which we can take the year alone.
Line 33–35: Appending the respective data into their respective array.
Line 37–51: Handling exceptions that occur during the above process. The consistency of the data in all the arrays is important. So, when an error occurs we are popping out the data of the image and other meta-data from the lists.
Now that we have finished processing our dataset, it’s time for us to store the data in our drive.
Line 1–7: Creating a dictionary with the data we extracted.
Line 8–9: Checking if the directory that we are going to store our data exists, if it does not exists then we create it.
Line 10–12: We create a file in append binary mode and dump our dictionary into it.
Note: While dumping a huge object to Google Drive, it is advisable to break the data into parts and store it. The session may crash and the runtime will restart if a large file is being stored in the drive.
I have attached a link to my “extract_data_and_preprocess.ipynb” notebook for your reference https://colab.research.google.com/drive/1U5Or-riULEZTsmO7En_h85_HqgIPEVlN.
Hope this article was worth your read.
Feel free to contact me regarding any queries.
Instagram: nachijr4
E-Mail: nachi2muthu13@gmail.com
Medium: Nachi Muthu
LinkedIn: Nachi Muthu