Face-Filter web application using Flask
By: Paras Pathania
Tags: Web App Development, Flask, Opencv, Face Filters, Facedetection

We use Face-filters over Snapchat or Instagram and are quite fascinated by them. Have you ever wondered how actually does the application of face-filters works? Today let us explore the fundamentals of how we can apply a face-filter on a live video feed. To add a twist to that we will be creating a web application using Flask to implement the face filter on a live video feed over a browser. Before diving deep into the work, you must have prior knowledge of Python, flask, CSS, HTML, and OpenCV library.

Puppy filter

To understand this article to the full extent, please refer to the GitHub repository

Below is the demonstration of how the final output of this project would look like.

This article is divided into two parts.

1. Implementing face-filter on live video feed (Coded in Jupyter notebook)

2. Creating a web-application using flask (Coded in VS code)

Part 1

Let us first start with creating a piece of code that implements face-filter on a video feed. The very first step is to download the face-filters. You can go to this website to download the filters Face-filters. Before implementing the face-filter on a video feed, we need to pre-process it where unwanted details are removed from it and a mask is created out of the filter which will be pasted on the live face. Below I have explained step-wise how it can be done.

1. Import the face-filter image

2. Obtain the shape of the face-filter image which we will be using later

3. Convert the image to gray-scale for preprocessing

4. We make use of cv2.threshold() to set up a threshold limit where we filter out the unwanted part using cv2.THRESH_BINARY_INV. This here helps us to create a transparent mask which we can paste on the live face. The numerical values in the cv2.threshold() differs for every face-filter.

5. Then we inverse the output of cv2.threshold() which helps us to obtain the wanted mask.

After having pre-processed the face-filter its time to capture the live video. For this task, we make use of OpenCv. We can read the video by creating an object of cv2.VideoCapture(0) function. After creating an object we capture the frames using the read() function of the object.

The next step after switching on the webcam is to predict the face in the live video. For the we create an object ‘face_cascade’ of cv2.CascadeClassifier() where we specify that we are looking for face detection by using ‘haarcascade_frontalface_default.xml’. The frames that are captured from the live video feed are converted into gray-scale and then the faces are detected using the function detectMutliScale() of object ‘face_cascade’.

The return value of the function detectMultiScale() are the coordinates of the face value, which are (X-axis, Y-axis, Width, Height). We use these values to determine the coordinates for our filter when we paste it on the live video feed.

Next, we check whether the filter size is within the frame or not and if it is not then we make adjustments to it. After determining the required size of face-filter from face detection we resize the face-filter which would fit the face. Then we take the ROI (Region of Interest) from the live video feed based on the size of the filter mask, and processes it to create the ROI_bg which is your face, and ROI_fg which is the face-filter mask, and then concatenate both and return that image back to cv2.imshow() to display the live video feed with the filter on it.

If you have successfully coded till here, that means you know now how to implement face-filters on a live video feed. This marks the end of Part 1.