Image Enhancement Using GFPGAN. Using GFPGAN for Restoring Blurred... | by Himanshu Sharma | Towards Data Science

With the advent of Image processing using Deep Learning, we have seen N number of pre-trained models that not only process image but also enhance their properties, you may have seen multiple videos of Deep fakes that brings life to images.
One of the major advancements in the field of Deep Learning, Neural Networks is the introduction of Generative Adversarial Networks that is the brain behind Deep Fakes and all major image processing, enhancing functionalities.
GAN is a class of Machine learning framework that helps in generating new data which has the same statistical properties as the training data. It is the supervised learning model that consists of 2 sub-models, the Generator model that is used to generate new data and the Discriminator model that is used to discriminate new data from training data.
Nowadays GAN has gained a lot of popularity due to functionalities like enhancing images, creating new images or videos, etc. There are multiple pre-trained GAN models that can be used directly by anyone. There are multiple image enhancement applications that are based on GAN and are very popular.
In this article, we will talk about one such pre-trained GAN model namely Generative Facial Prior Generative Adversarial Network that helps in restoring facial images and enhancing them if it is blurry. It is easy to use as it is a pre-trained model, we don’t need to train it again. We will use this model and enhance blurred images.
Let’s get started...
We will start by installing Dependencies using pip installation and cloning the Github repository GFPGAN. The commands given below will do that.
# Clone GFPGAN and enter the GFPGAN folder%cd /content!rm -rf GFPGAN!git clone https://github.com/TencentARC/GFPGAN.git%cd GFPGAN# Set up the environment# Install basicsr - https://github.com/xinntao/BasicSR# We use BasicSR for both training and inference!pip install basicsr# Install facexlib - https://github.com/xinntao/facexlib# We use face detection and face restoration helper in the facexlib package!pip install facexlib# Install other depencencies!pip install -r requirements.txt!python setup.py develop!pip install realesrgan  # used for enhancing the background (non-face) regions# Download the pre-trained model!wget https://github.com/TencentARC/GFPGAN/releases/download/v0.2.0/GFPGANCleanv1-NoCE-C2.pth -P experiments/pretrained_models
In this step, we will import all the libraries that are required for GFPGAN and Image restoration.
import osfrom google.colab import filesimport shutil
Now, we will start by reading the image from the user.
# upload your own imagesupload_folder = 'inputs/upload'if os.path.isdir(upload_folder):    shutil.rmtree(upload_folder)os.mkdir(upload_folder)# upload imagesuploaded = files.upload()for filename in uploaded.keys():    dst_path = os.path.join(upload_folder, filename)  print(f'move {filename} to {dst_path}')  shutil.move(filename, dst_path)
After uploading the image let’s use the GFPGAN to restore the blurred image and enhance it.
# Now we use the GFPGAN to restore the above low-quality images!rm -rf results!python inference_gfpgan.py --upscale 2 --test_path inputs/upload --save_root results --model_path experiments/pretrained_models/GFPGANCleanv1-NoCE-C2.pth --bg_upsampler realesrgan!ls results/cmp
Now the final step is to visualize the enhanced images, for visualization we will use cv2 and matplotlib.
# Now we use the GFPGAN to restore the above low-quality images# We use [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) for enhancing the background (non-face) regions!rm -rf results# We first visualize the cropped faces# The left are the inputs images; the right are the results of GFPGANimport cv2import matplotlib.pyplot as pltdef display(img1, img2):  fig = plt.figure(figsize=(25, 10))  ax1 = fig.add_subplot(1, 2, 1)   plt.title('Input image', fontsize=16)  ax1.axis('off')  ax2 = fig.add_subplot(1, 2, 2)  plt.title('GFPGAN output', fontsize=16)  ax2.axis('off')  ax1.imshow(img1)  ax2.imshow(img2)def imread(img_path):  img = cv2.imread(img_path)  img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  return img# display each image in the upload folderimport osimport globinput_folder = 'results/cropped_faces'result_folder = 'results/restored_faces'input_list = sorted(glob.glob(os.path.join(input_folder, '*')))output_list = sorted(glob.glob(os.path.join(result_folder, '*')))for input_path, output_path in zip(input_list, output_list):  img_input = imread(input_path)  img_output = imread(output_path)  display(img_input, img_output)!python inference_gfpgan.py --upscale 2 --test_path inputs/upload --save_root results --model_path experiments/pretrained_models/GFPGANCleanv1-NoCE-C2.pth --bg_upsampler realesrgan!ls results/cmp
Here you can see how easily we restored our blurred image using GFPGAN.
Try this with different images, and let me know your comments in the response section.
This article is in collaboration with Piyush Ingale.
Thanks for reading! If you want to get in touch with me, feel free to reach me at hmix13@gmail.com or my LinkedIn Profile. You can view my Github profile for different data science projects and packages tutorials. Also, feel free to explore my profile and read different articles I have written related to Data Science.