|
--- |
|
license: openrail++ |
|
tags: |
|
- stable-diffusion |
|
- stable-diffusion-diffusers |
|
- text-guided-to-image-inpainting |
|
- endpoints-template |
|
--- |
|
|
|
# Fork of [stabilityai/stable-diffusion-2-inpainting](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting) |
|
|
|
> Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input. |
|
> For more information about how Stable Diffusion functions, please have a look at [🤗's Stable Diffusion with 🧨Diffusers blog](https://huggingface.co/blog/stable_diffusion). |
|
|
|
For more information about the model, license and limitations check the original model card at [stabilityai/stable-diffusion-2-inpainting](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting). |
|
|
|
--- |
|
|
|
This repository implements a custom `handler` task for `text-guided-to-image-inpainting` for 🤗 Inference Endpoints. The code for the customized pipeline is in the [pipeline.py](https://huggingface.co/philschmid/stable-diffusion-2-inpainting-endpoint/blob/main/handler.py). |
|
|
|
There is also a [notebook](https://huggingface.co/philschmid/stable-diffusion-2-inpainting-endpoint/blob/main/create_handler.ipynb) included, on how to create the `handler.py` |
|
|
|
**How it works:** |
|
`image` | `mask_image` |
|
:-------------------------:|:-------------------------:| |
|
<img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" alt="drawing" width="300"/> | <img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png" alt="drawing" width="300"/> |
|
|
|
|
|
`prompt` | `Output` |
|
:-------------------------:|:-------------------------:| |
|
<span style="position: relative;bottom: 150px;">Face of a bengal cat, high resolution, sitting on a park bench</span> | <img src="https://huggingface.co/philschmid/stable-diffusion-2-inpainting-endpoint/resolve/main/result.png" alt="drawing" width="300"/> |
|
|
|
|
|
### expected Request payload |
|
|
|
```json |
|
{ |
|
"inputs": "A prompt used for image generation", |
|
"image" : "iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAIAAAB7GkOtAAAABGdBTUEAALGPC", |
|
"mask_image": "iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAIAAAB7GkOtAAAABGdBTUEAALGPC", |
|
} |
|
``` |
|
|
|
below is an example on how to run a request using Python and `requests`. |
|
|
|
## Run Request |
|
```python |
|
import json |
|
from typing import List |
|
import requests as r |
|
import base64 |
|
from PIL import Image |
|
from io import BytesIO |
|
|
|
ENDPOINT_URL = "" |
|
HF_TOKEN = "" |
|
|
|
import base64 |
|
from PIL import Image |
|
from io import BytesIO |
|
|
|
# helper image utils |
|
def encode_image(image_path): |
|
with open(image_path, "rb") as i: |
|
b64 = base64.b64encode(i.read()) |
|
return b64.decode("utf-8") |
|
|
|
prompt = "Face of a bengal cat, high resolution, sitting on a park bench" |
|
|
|
# test the handler |
|
pred = my_handler(request) |
|
|
|
|
|
def predict(prompt, image, mask_image): |
|
image = encode_image(image) |
|
mask_image = encode_image(mask_image) |
|
|
|
# prepare sample payload |
|
request = {"inputs": prompt, "image": image, "mask_image": mask_image} |
|
# headers |
|
headers = { |
|
"Authorization": f"Bearer {HF_TOKEN}", |
|
"Content-Type": "application/json", |
|
"Accept": "image/png" # important to get an image back |
|
} |
|
|
|
response = r.post(ENDPOINT_URL, headers=headers, json=payload) |
|
img = Image.open(BytesIO(response.content)) |
|
return img |
|
|
|
prediction = predict( |
|
prompt="Face of a bengal cat, high resolution, sitting on a park bench", |
|
image="dog.png", |
|
mask_image="mask_dog.png" |
|
) |
|
``` |
|
expected output |
|
|
|
 |
|
|
