Detecting faces with Python and OpenCV Face Detection Neural Network | by Himasha Abeywickrama | Towards Data Science

Now, we all know that Artificial Intelligence is becoming more and more real and its filling the gaps between capabilities of humans and machines day by day. It’s not just a fancy word anymore. It has had many advancements over the years in many fields and one of such areas is the domain of Computer Vision.
When it comes to Computer Vision, it’s goal is to train the machines to view and recognize the world as humans do. And also gather knowledge large enough to perform Image and video recognition, Image Analysis and Classification, Media Recreation, Natural Language Processing and etc.
The advancement in Computer Vision has been implemented and perfected gradually with time, primarily over one particular algorithm, a Convolutional Neural Network (CNNs or ConvNets), which is a special type of feed-forward network which is used mostly to analyze visual imagery. Convolutional Neural Networks are very similar to ordinary neural networks but are made up of neurons that have learnable weights and biases.
ConvNets perform better than the other deep neural network architectures because of their unique process. Instead of looking at the image one pixel at a time, ConvNets group several pixels together so they can understand a temporal pattern. In another way, ConvNets can “see” group of pixels forming a line or curve. Because of the deep nature of Deep Neural Networks, in the next level they will see not the group of pixels, but groups of lines and curves forming some shapes. And so on until they form a complete picture.
However, CNNs require large datasets and a lot of computational time to train. Some could take even 2–3 weeks across multiple GPUs. There are many things you need to learn if you want to understand CNNs, from the very basic things, like a kernel, pooling layers, and so on. But nowadays, you can just dive and use many open source projects for this technology.
This is actually true because of the technology called Transfer Learning. Transfer Learning is a very practical and powerful technique which utilizes a trained model instead of training the network from scratch. Transfer learning utilizes a trained model on a different dataset, and adapts it to the problem that we’re trying to solve.
In order to build our application, we will follow the transfer learning technique and use a trained model, which was trained using the Caffe framework, a deep learning framework made with expression, speed and modularity in mind. Caffe comes with a repository that is used by researchers and machine learning practitioners to share their trained models. This library is called Model Zoo. You can read all about it by going through their documentation.
The python code, trained caffe model and the prototxt file, which includes the text description of the network and some example images to use with our application are available in the github repository if you want to take a look.
In our application, we will be using the following libraries :
OpenCV, which supports a lot of algorithms related to Computer Vision and Machine Learning and not to mention it’s built in Deep Neural Network which we will be using in our application.Numpy, which is a package used for scientific computing with python.OS, which provides a portable way of using operating system dependent functionality
OpenCV, which supports a lot of algorithms related to Computer Vision and Machine Learning and not to mention it’s built in Deep Neural Network which we will be using in our application.
Numpy, which is a package used for scientific computing with python.
OS, which provides a portable way of using operating system dependent functionality
So, in order to install them, you can run the following commands in a Command Prompt window. pip install opencv-python to install OpenCV, pip install numpy to install Numpy and you don’t have to specifically install the OS library as it comes with your Python installation, but you still must import it.
First, let’s create a file called faceDetection.py and import our libraries.
#import librariesimport os   import cv2import numpy
Now, we will get the absolute path of the working directory, where we will be putting all our images,
#import librariesimport os   import cv2import numpy#get the absolute path of the working directorydir_path = os.path.dirname(os.path.realpath(__file__))
Now, we will create a folder called ‘Output’ to place our final images if it doesn’t already exist.
#import librariesimport os   import cv2import numpy#get the absolute path of the working directorydir_path = os.path.dirname(os.path.realpath(__file__))#create the Output folder if it doesn't already existif not os.path.exists('Output'):   os.makedirs('Output')
After that, we will load our pre-trained model and the prototxt file from the current working directory.
#import librariesimport os   import cv2import numpy#get the absolute path of the working directorydir_path = os.path.dirname(os.path.realpath(__file__))#create the Output folder if it doesn't already existif not os.path.exists('Output'):   os.makedirs('Output')#Reads the network model stored in Caffe framework's format.model = cv2.dnn.readNetFromCaffe('deploy.prototxt', 'weights.caffemodel')
Then, we will loop through the files available in the current folder to check if there are files with the extensions, .png, .jpg and .jpeg.
#import librariesimport os   import cv2import numpy#get the absolute path of the working directorydir_path = os.path.dirname(os.path.realpath(__file__))#create the Output folder if it doesn't already existif not os.path.exists('Output'):   os.makedirs('Output')#Reads the network model stored in Caffe framework's format.model = cv2.dnn.readNetFromCaffe('deploy.prototxt', 'weights.caffemodel')for file in os.listdir(dir_path):  #split the file name and the extension into two variales  filename, file_extension = os.path.splitext(file)  #check if the file extension is .png,.jpeg or .jpg   if (file_extension in ['.png','.jpg','.jpeg']):
If an image with the above extensions is found, we will read the image using OpenCV and get it’s height and width by accessing the image.shape tuple and taking the first two elements to draw the rectangle around the face later.
#import librariesimport os   import cv2import numpy#get the absolute path of the working directorydir_path = os.path.dirname(os.path.realpath(__file__))#create the Output folder if it doesn't already existif not os.path.exists('Output'):   os.makedirs('Output')#Reads the network model stored in Caffe framework's format.model = cv2.dnn.readNetFromCaffe('deploy.prototxt', 'weights.caffemodel')for file in os.listdir(dir_path):  #split the file name and the extension into two variales  filename, file_extension = os.path.splitext(file)  #check if the file extension is .png,.jpeg or .jpg  if (file_extension in ['.png','.jpg','.jpeg']):    #read the image using cv2    image = cv2.imread(file)    #accessing the image.shape tuple and taking the elements    (h, w) = image.shape[:2]
And then, we will get our blob using the cv2.dnn.blobFromImage function by giving the image as the input. With cv2.dnn.blobFromImage function we resize image to 300x300 dimension, 1.0 is scale factor and here we use default value so there is no scaling, after that is spatial size that Convolutional Neural Network expects, last values are mean subtraction values in tuple and they are RGB means, and at the end, function returns a “blob” which is our input image after resizing, mean subtraction and normalizing.
After that, we will input the blob into the model and get the detections using model.foreward function.
#import librariesimport os   import cv2import numpy#get the absolute path of the working directorydir_path = os.path.dirname(os.path.realpath(__file__))#create the Output folder if it doesn't already existif not os.path.exists('Output'):   os.makedirs('Output')#Reads the network model stored in Caffe framework's format.model = cv2.dnn.readNetFromCaffe('deploy.prototxt', 'weights.caffemodel')for file in os.listdir(dir_path):  #split the file name and the extension into two variales  filename, file_extension = os.path.splitext(file)  #check if the file extension is .png,.jpeg or .jpg   if (file_extension in ['.png','.jpg','.jpeg']):    #read the image using cv2    image = cv2.imread(file)    #accessing the image.shape tuple and taking the elements    (h, w) = image.shape[:2]    #get our blob which is our input image     blob = cv2.dnn.blobFromImage(cv2.resize(image, (300, 300)), 1.0, (300, 300), (104.0, 177.0, 123.0))    #input the blob into the model and get back the detections     model.setInput(blob)    detections = model.forward()
Then, we will iterate through all of the faces detected and extract their start and end points. We will extract confidence and compare it to the confidence threshold so we can filter out detections that are weak. If the algorithm is more than 16.5% confident that the detection is a face, we will show a green rectangle on it.
#import librariesimport os   import cv2import numpy#get the absolute path of the working directorydir_path = os.path.dirname(os.path.realpath(__file__))#create the Output folder if it doesn't already existif not os.path.exists('Output'):   os.makedirs('Output')#Reads the network model stored in Caffe framework's format.model = cv2.dnn.readNetFromCaffe('deploy.prototxt', 'weights.caffemodel')for file in os.listdir(dir_path):  #split the file name and the extension into two variales  filename, file_extension = os.path.splitext(file)  #check if the file extension is .png,.jpeg or .jpg   if (file_extension in ['.png','.jpg','.jpeg']):    #read the image using cv2    image = cv2.imread(file)    #accessing the image.shape tuple and taking the elements    (h, w) = image.shape[:2]    #get our blob which is our input image     blob = cv2.dnn.blobFromImage(cv2.resize(image, (300, 300)), 1.0, (300, 300), (104.0, 177.0, 123.0))    #input the blob into the model and get back the detections     model.setInput(blob)    detections = model.forward()    #Iterate over all of the faces detected and extract their start and end points    count = 0    for i in range(0, detections.shape[2]):      box = detections[0, 0, i, 3:7] * numpy.array([w, h, w, h])      (startX, startY, endX, endY) = box.astype("int")      confidence = detections[0, 0, i, 2]      #if the algorithm is more than 16.5% confident that the      detection is a face, show a rectangle around it      if (confidence > 0.165):        cv2.rectangle(image, (startX, startY), (endX, endY), (0, 255, 0), 2)        count = count + 1
Finally, we will save the output image in the Output folder we created and print out a success message to let the user know that it’s done.
#import librariesimport os   import cv2import numpy#get the absolute path of the working directorydir_path = os.path.dirname(os.path.realpath(__file__))#create the Output folder if it doesn't already existif not os.path.exists('Output'):   os.makedirs('Output')#Reads the network model stored in Caffe framework's format.model = cv2.dnn.readNetFromCaffe('deploy.prototxt', 'weights.caffemodel')for file in os.listdir(dir_path):  #split the file name and the extension into two variales  filename, file_extension = os.path.splitext(file)#check if the file extension is .png,.jpeg or .jpg   if (file_extension in ['.png','.jpg','.jpeg']):    #read the image using cv2    image = cv2.imread(file)#accessing the image.shape tuple and taking the elements    (h, w) = image.shape[:2]#get our blob which is our input image     blob = cv2.dnn.blobFromImage(cv2.resize(image, (300, 300)), 1.0, (300, 300), (104.0, 177.0, 123.0))#input the blob into the model and get back the detections     model.setInput(blob)    detections = model.forward()    #Iterate over all of the faces detected and extract their start and end points    count = 0    for i in range(0, detections.shape[2]):      box = detections[0, 0, i, 3:7] * numpy.array([w, h, w, h])      (startX, startY, endX, endY) = box.astype("int")confidence = detections[0, 0, i, 2]      #if the algorithm is more than 16.5% confident that the      detection is a face, show a rectangle around it      if (confidence > 0.165):        cv2.rectangle(image, (startX, startY), (endX, endY), (0, 255, 0), 2)        count = count + 1    #save the modified image to the Output folder    cv2.imwrite('Output/' + file, image)    #print out a success message    print("Face detection complete for image "+ file + " ("+ str(count) +") faces found!")
I hope you enjoyed the results! Now, you can always play around with the code a little bit to get an idea about what each and every function does. OpenCV provides a great documentation and tutorials if you wanna read and get more information too.
In this article, we walked through,
The concept of Convolutional Neural Networks and how we can save lot of time and effort with Transfer Learning and pre-trained models.Caffe models and how we are going to use a pre-trained Caffe model to implement our application.installing required libraries and setting up the environment.
The concept of Convolutional Neural Networks and how we can save lot of time and effort with Transfer Learning and pre-trained models.
Caffe models and how we are going to use a pre-trained Caffe model to implement our application.
installing required libraries and setting up the environment.
And finally, we implemented a python program which can detect using an image.
If you have any questions, please feel free to leave a comment below. Thanks!