Camera App with Flask and OpenCV. Apply Snapchat like filters with OpenCV and flask. | Towards Data Science

In this blog post, we are going to build a camera app using flask framework wherein we can click pictures, record videos, apply filters like greyscale, negative and ‘face only’ filter like the one that appears on Snapchat. I have used a very basic design for the front-end since the main motive behind the project was to familiarize myself with the flask web-framework and also include live videostream. The same can be scaled up to add many more features.
Demo:
We make use of concepts like threading, HTTP request-response, global variables, error handling and face detection. Let’s look at how all these play out in a detailed manner.
Firstly, the front-end is a basic HTML file with buttons to take inputs and image source tag to display output frames after pre-processing in the back-end. Buttons in the file posts data to the server. The file also displays a few instructions to use the app. This file is saved inside the ‘templates’ folder in the project directory.
As for the back-end, it is a single python script which does all the magic. It is saved in the project directory. Let’s look at parts of the file separately in order to understand it’s working.
Initialization:
In the code above, we import all necessary modules.
Flask is a micro web-framework which works like a bridge between front and back-end. From flask, we import ‘Response’ and ‘request’ modules to handle HTTP response-requests. ‘render_template’ is used to render the HTML file shown before. OpenCV is the module used to perform all the computer vision tasks. ‘Thread’ module is used to spawn new threads.
We then declare all the global variables that act like a ‘toggle switch’ to perform different tasks like capture image, start/stop recording and applying filters. Initialize the variables to 0 to set everything to false.
In line 18, we try to create a folder named ‘shots’ if it doesn’t exist. This is where all the captured images are saved.
Line 24 loads a pretrained face detection model for future uses and line 27 creates an instance of the Flask app. Line 30 creates a video capture object for the built-in camera.
Functions:
‘record’ function is used to start recording i.e. write frames into an avi file while ‘rec’ variable is set to true. It uses ‘out’ which is an object for video writer initialized later. (Note: If you feel recorded video is fast or slow, fiddle around with the value of time.sleep).
‘detect_face()’ takes camera frames as input and returns a cropped out frame containing just the face detected in that frame. It uses the pretrained face detection model loaded earlier.(Please visit this website to get an in-depth understanding of how this is being done).
‘gen_frames’ is an important function wherein the actual frame capture(through the camera) and processing is done. It runs in an infinite while loop. Line 4 captures frames from the camera object. If frame capture is successful, it checks if any of the filter switches are true. If ‘face’, ‘neg’ or ‘grey’ are true, face filter, negative filter and greyscale filter are applied on the read frame respectively.
If ‘capture’ variable is set to true, it is reset to false(global variable) and the current frame is saved in ‘png’ format. If ‘rec’ is true, frame is copied to ‘rec_frame’ global variable which is saved into a video file when triggered.
Line 25 encodes the frame into the memory buffer and is then converted into an array of bytes. Line 27 yields the frame data in a format required to be sent as a HTTP response.
HTTP Routes:
This is where the front-end communicates with the server. The communication happens in ‘GET’ and ‘POST’ methods through URL routes.
‘@app. route(‘/’)’ is a Python decorator that Flask provides to assign URLs to functions in our app easily. Route ‘/’ is the root URL and ‘index()’ function is called when root URL is entered. ‘index.html’ file is rendered from the function into the webpage.
Route ‘/video_feed’ is set as the source for image in the html file. This function returns response chunks of frames yielded by ‘gen_frames()’ on a loop.
Route ‘/requests’ is assigned to the ‘tasks()’ function which handles all the switches and video recording. This route has both ‘POST’ and ‘GET’ methods i.e. it accepts information and also sends information. All previous routes were ‘GET’ by default.
If HTTP method from the client side is ‘POST’, ‘request.form.get’ accepts data from the buttons pressed by the user and reverses the previous state of the global variables that act like switches in the ‘gen_frame’ function. For example, when user presses ‘Grey’ button, ‘grey’ global variable is set to True thereby turning the frames to greyscale in the ‘gen_frames’ function. When the ‘Grey’ button is pressed again, ‘grey’ is set to false turning the frames back to normal.
Recording the frames into a video while also running the flask app is quite tricky. The easiest solution is to start a new thread.
A thread is a separate flow of execution. This means that your program will have two things happening at once. A thread shares information like data segment, code segment, files etc. with its peer threads while it contains its own registers, stack, counter etc.
In our case, ‘record()’ function has it’s own while loop, so that loop runs in the new thread. First, we create a ‘VideoWriter’ object when ‘rec’ is true. In line 37, we initialize a new thread with target being ‘record()’ function and line 38 starts the new thread where ‘record()’ function is run. When record button is pressed again, ‘VideoWriter’ object is released and the recording stops to save the video in the root directory.
Finally, if HTTP method from client side is ‘GET’, ‘index.html’ template is rendered.
Main function:
‘app.run()’ is used to start the flask app in its default address: http://127.0.0.1:5000/. You can set a different host and port number by adding ‘host’ and ‘port’ arguments to the function ‘run’. Setting host to broadcast address 0.0.0.0 would make the app visible in the whole local area network(wifi, etc). So, you can access the app from your mobile device if it’s connected to the same Wi-Fi. This makes a good ‘SPY CAM’.
To run this app, you should have python, flask and OpenCV installed on your PC. To start the app, move to the project directory in the command prompt. Type and enter:
python camera_flask_app.py
Now, copy-paste http://127.0.0.1:5000/ into your favorite internet browser and that’s it.
You can add more features like AI filters to build a Snapchat-esque app . You can also enhance the user interface making it more interactive and colorful. You can get the source code for this project in my GitHub account.