Package: codeformer-perceptor
Towards Robust Blind Face Restoration with Codebook Lookup Transformer (NeurIPS 2022)Paper|Project Page|VideoShangchen Zhou,Kelvin C.K. Chan,Chongyi Li,Chen Change LoyS-Lab, Nanyang Technological University:star: If CodeFormer is helpful to your images or projects, please help star this repo. Thanks! :hugs:Installpoetryaddcodeformer-perceptororpipinstallcodeformer-perceptorUsagefromPILimportImagefromcodeformerimportCodeFormermodel=CodeFormer().cuda()image=Image.open("test.jpg")restored_image=model(image)[News]: :whale:Due to copyright issues, we have to delay the release of the training code (expected by the end of this year). Please star and stay tuned for our future updates!Update2022.10.05: Support video input--input_path [YOUR_VIDOE.mp4]. Try it to enhance your videos! :clapper:2022.09.14: Integrated to :hugs:Hugging Face. Try out online demo!2022.09.09: Integrated to :rocket:Replicate. Try out online demo!2022.09.04: Add face upsampling--face_upsamplefor high-resolution AI-created face enhancement.2022.08.23: Some modifications on face detection and fusion for better AI-created face enhancement.2022.08.07: IntegrateReal-ESRGANto support background image enhancement.2022.07.29: Integrate new face detectors of['RetinaFace'(default), 'YOLOv5'].2022.07.17: Add Colab demo of CodeFormer.2022.07.16: Release inference code for face restoration. :blush:2022.06.21: This repo is created.TODOAdd checkpoint for face inpaintingAdd checkpoint for face colorizationAdd training code and config filesAdd background image enhancement:panda_face: Try Enhancing Old Photos / Fixing AI-artsFace RestorationFace Color Enhancement and RestorationFace InpaintingDependencies and InstallationPytorch >= 1.7.1CUDA >= 10.1Other required packages inrequirements.txt# git clone this repository
git clone https://github.com/sczhou/CodeFormer
cd CodeFormer

# create new anaconda env
conda create -n codeformer python=3.8 -y
conda activate codeformer

# install python dependencies
pip3 install -r requirements.txt
python basicsr/setup.py developQuick InferenceDownload Pre-trained Models:Download the facelib pretrained models from [Google Drive|OneDrive] to theweights/facelibfolder. You can manually download the pretrained models OR download by runing the following command.python scripts/download_pretrained_models.py facelibDownload the CodeFormer pretrained models from [Google Drive|OneDrive] to theweights/CodeFormerfolder. You can manually download the pretrained models OR download by runing the following command.python scripts/download_pretrained_models.py CodeFormerPrepare Testing Data:You can put the testing images in theinputs/TestWholefolder. If you would like to test on cropped and aligned faces, you can put them in theinputs/cropped_facesfolder.Testing on Face Restoration:[Note] If you want to compare CodeFormer in your paper, please run the following command indicating--has_aligned(for cropped and aligned face), as the command for the whole image will involve a process of face-background fusion that may damage hair texture on the boundary, which leads to unfair comparison.🧑🏻 Face Restoration (cropped and aligned face)# For cropped and aligned faces
python inference_codeformer.py -w 0.5 --has_aligned --input_path [input folder]:framed_picture: Whole Image Enhancement# For whole image
# Add '--bg_upsampler realesrgan' to enhance the background regions with Real-ESRGAN
# Add '--face_upsample' to further upsample restorated face with Real-ESRGAN
python inference_codeformer.py -w 0.7 --input_path [image folder/image path]:clapper: Video Enhancement# For video clips
# Set frame rate of saved video via '--save_video_fps 24'
python inference_codeformer.py --bg_upsampler realesrgan --face_upsample -w 1.0 --input_path [video path] --save_video_fps 24Fidelity weightwlays in [0, 1]. Generally, smallerwtends to produce a higher-quality result, while largerwyields a higher-fidelity result.The results will be saved in theresultsfolder.CitationIf our work is useful for your research, please consider citing:@inproceedings{zhou2022codeformer,
    author = {Zhou, Shangchen and Chan, Kelvin C.K. and Li, Chongyi and Loy, Chen Change},
    title = {Towards Robust Blind Face Restoration with Codebook Lookup TransFormer},
    booktitle = {NeurIPS},
    year = {2022}
}LicenseThis project is licensed underS-Lab License 1.0. Redistribution and use for non-commercial purposes should follow this license.AcknowledgementThis project is based onBasicSR. Some codes are brought fromUnleashing Transformers,YOLOv5-face, andFaceXLib. We also adoptReal-ESRGANto support background image enhancement. Thanks for their awesome works.ContactIf you have any question, please feel free to reach me out atshangchenzhou@gmail.com.