File size: 4,212 Bytes
e6231b1 e814512 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 |
---
license: apache-2.0
---
# A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting
### [Project Page](https://powerpaint.github.io/) | [Paper](https://arxiv.org/abs/2312.03594) | [Online Demo(OpenXlab)](https://openxlab.org.cn/apps/detail/rangoliu/PowerPaint#basic-information)
This README provides a step-by-step guide to download the repository, set up the required virtual environment named "PowerPaint" using conda, and run PowerPaint with or without ControlNet.
**Feel free to try it and give it a star!**:star:
## 🚀 News
**April 7, 2024**:fire:
- We open source the model weights and code for PowerPaint v2.[](https://openxlab.org.cn/models/detail/zhuangjunhao/PowerPaint_v2)[](https://huggingface.co/JunhaoZhuang/PowerPaint_v2)
**April 6, 2024**:fire:
- We have retrained a new PowerPaint, taking inspiration from Brushnet. The [Online Demo](https://openxlab.org.cn/apps/detail/rangoliu/PowerPaint) has been updated accordingly. **We plan to release the model weights and code as open source in the next few days**.
- Tips: We preserve the cross-attention layer that was deleted by BrushNet for the task prompts input.
| | Object insertion | Object Removal|Shape-guided Object Insertion|Outpainting|
|-----------------|-----------------|-----------------|-----------------|-----------------|
| Original Image| ||||
| Output| | |||
**December 22, 2023**:wrench:
- The logical error in loading ControlNet has been rectified. The `gradio_PowerPaint.py` file and [Online Demo](https://openxlab.org.cn/apps/detail/rangoliu/PowerPaint) have also been updated.
**December 18, 2023**
*Enhanced PowerPaint Model*
- We are delighted to announce the release of more stable model weights. These refined weights can now be accessed on [Hugging Face](https://huggingface.co/JunhaoZhuang/PowerPaint-v1/tree/main). The `gradio_PowerPaint.py` file and [Online Demo](https://openxlab.org.cn/apps/detail/rangoliu/PowerPaint) have also been updated as part of this release.
________________
<img src='https://github.com/open-mmlab/mmagic/assets/12782558/acd01391-c73f-4997-aafd-0869aebcc915'/>
## Getting Started
```bash
# Clone the Repository
git clone https://github.com/zhuang2002/PowerPaint.git
# Navigate to the Repository
cd projects/powerpaint
# Create Virtual Environment with Conda
conda create --name PowerPaint python=3.9
conda activate PowerPaint
# Install Dependencies
pip install -r requirements.txt
```
## PowerPaint v2
```bash
python gradio_PowerPaint_BrushNet.py
```
## PowerPaint v1
```bash
# Create Models Folder
mkdir models
# Set up Git LFS
git lfs install
# Clone PowerPaint Model
git lfs clone https://huggingface.co/JunhaoZhuang/PowerPaint-v1/ ./models
python gradio_PowerPaint.py
```
This command will launch the Gradio interface for PowerPaint.
Feel free to explore and edit images with PowerPaint!
## BibTeX
```
@misc{zhuang2023task,
title={A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting},
author={Junhao Zhuang and Yanhong Zeng and Wenran Liu and Chun Yuan and Kai Chen},
year={2023},
eprint={2312.03594},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
|