Artificial Eyeliner on LIVE Feed using Python, OpenCV and Dlib | by Kaushil Kundalia | Towards Data Science

Computer vision is one of the most exciting fields with a broad spectrum of applications ranging from medical imaging to creating funniest face filters. In this article, I have tried to create an artificial eyeliner to mimic Snapchat or Instagram filters. Hope you enjoy reading.
This article is meant for beginners in computer vision, who want to build something exciting. I’ll keep the explanation as simple as I can. I’d advice on going through the code on GitHub along with reading the article to have a better understanding.
It is important to set up a new virtual environment and install all the required dependencies before you begin. You can visit my GitHub repo for the instructions.
We’ll be using OpenCV, NumPy, imutils, SciPy and Dlib for this project. Below is a brief description of the use of these libraries in our project — feel free to skip this section if you’re already familiar.
OpenCV : It is one of the most popular modules used for image manipulation. We’ll be using OpenCV to read, write and draw on an image.
NumPy: NumPy is frequently used when working with OpenCV projects. An image is essentially an array of pixels and OpenCV uses these arrays stored in the form of NumPy arrays for performing operation on images.
Imutils: Imutils comes loaded with continence functions to make your life working with computer vision easier. Here, we’ll use it for converting dlib objects to numpy arrays that are very flexible and widely acceptable.
Scipy: As the name says, SciPy is used for scientific computing on python. We’ll use it to create interpolations (it’s okay if it doesn’t make sense right now).
Dlib: Dlib originally came as a C++ library that contains various ML algorithms. We’ll be using dlib to extract facial landmark points.
Briefly explaining — the program first extracts 68 landmark points from each face. Of those 68 points, points 37–42 belong to the left eye and points 43–48 belong to the right eye — see picture below.
Because our goal is to apply eyeliner, we are only interested in points 37–48 and we extract these points. We’ll interpolate these extracted points. Interpolation means that we try to insert points between two given points. We can either make this interpolation linear or smooth — see figure below for details.
The flow chart of the eyeliner algorithm is shown below.
Next, we describe the algorithm in further detail. If you are just interested in running the code, skip to the next section.
We’ll first extract the coordinates of the bounding box surrounding the face.
OpenCV converts an image to a NumPy array. This numpy.array (that is a matrix representation of an image) is stored in the variable named frame. We use a function called face_detector() that returns the coordinates of bounding boxes surrounding all the faces found in a frame. These bounding box coordinates are stored in a variable named bounding_boxes.We iterate a loop over bounding_boxes to apply eyeliner over each face detected in a frame. face_landmark_points stores the 68 landmark points. eye_landmark_points is an array of eye landmark points received from getEyeLandmarkPts() .
The getEyeLandmarkPts() takes 68 landmark points as input and returns an array of 4 elements having coordinates of left upper eyelid (L_eye_top) , left lower eyelid (L_eye_bottom) and same for right eye (R_eye_top & R_eye_bottom). This is done by simple NumPy indexing. We’re shifting the extreme points (pt no. 37,40,43 and 46. See the 68 landmark points figure) to further outside by 5 px. This is done in line no. 6–9. This is to give a more realistic look.
NOTE: In an array, indexing starts from 0. So we change pt no. 36, 39, 42 and 45 in our code.
Now we need to interpolate through these points to get a smooth curve from which we can make eyeliner. We need to treat each curve differently. (i.e. L_eye_top , L_eye_bottom, R_eye_top, R_eye_bottom). Hence we use a separate variable name for each curve. interpolateCoordinates() is used to generate interpolations over each of these curves.The functioninterp1d() from the scipy module is used to generate the interpolation. The kind='quadratic' means to generate a curvature of those points.
interpolateCoordinates() is called by getEyelinerPoints() repetitively to generate interpolated coordinates for each curve. It returns an array of interpolated points for each curve.
drawEyeLiner() takes the generated interpolated points as an argument and draws a line between two consecutive points. This is done for each curve in two loops, one for left and other for right eye. These drawn lines make our eyeliner. It returns a new frame op which has the eyeliner. And BOOM ! We’re done :D.
The usage of this project is very simple. Just clone the github repo from here.
git clone https://github.com/kaushil24/Artificial-Eyeliner/
Next, open the command prompt and type the following code to run a sample test
python3 eyeliner.py -v "Media/Sample Video.mp4"
You can use your own video by placing your video path inside the argument. The complete CLI commands are as follows:
python eyeliner.py [-i image] [-v video] [-d dat] [-t thickness] [-c color] [-s save]
-i : Path of the image you want to apply eyeliner on
-v : Path of the video you want to apply eyeliner on.
-v : Live eyeliner of webcam video if webcam is given. Eg: python3 -v webcam -s "Webcam output"
-t : Whole numbers (int) to set thickness of eyeliner. Default = 2. Recommended number value between 1-5
-d: Path to your shape_predictor_68_face_landmarks.dat file. Default path is in the root. Unless you have the shape_predictor_68_face_landmarks.dat file stored at some other location you need not use this argument.
-c : Change the color of eyeliner. Syntax -c 255 255 255. Default = 0 0 0. Where each number represents its R G B value.
-s : Location and file name you want to save the output on. NOTE The program automatically adds extension while saving the file. If a file with same name already exists, it will overwrite that file.
Play around with the code and if you add something useful, do make a pull request. Let’s make it a collaborative project.
GitHub repo: https://github.com/kaushil24/Artificial-Eyeliner/
Well this brings us to the end.
Hey this is my first post. Please comment your doubts and feedback, and I shall be happy to solve them. Hope you enjoyed reading this article! Have an amazing day.