File size: 1,737 Bytes
cb5df7d 8634b1a cb5df7d 8634b1a 6b2d8eb 8634b1a 6b2d8eb 8634b1a d23c2f4 8634b1a d35a13b 8634b1a d35a13b 8fa2b50 8634b1a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
---
license: mit
tags:
- image-to-image
datasets:
- yulu2/InstructCV-Demo-Data
---
# InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists
GitHub: https://github.com/AlaaLab/InstructCV
[](https://imgse.com/i/pCVB5B8)
## Example
To use `InstructCV`, install `diffusers` using `main` for now. The pipeline will be available in the next release
```bash
pip install diffusers accelerate safetensors transformers
```
```python
import PIL
import requests
import torch
from diffusers import StableDiffusionInstructPix2PixPipeline, EulerAncestralDiscreteScheduler
model_id = "yulu2/InstructCV"
pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16, safety_checker=None, variant="ema")
pipe.to("cuda")
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
url = "put your url here"
def download_image(url):
image = PIL.Image.open(requests.get(url, stream=True).raw)
image = PIL.ImageOps.exif_transpose(image)
image = image.convert("RGB")
return image
image = download_image(URL)
seed = random.randint(0, 100000)
generator = torch.manual_seed(seed)
width, height = image.size
factor = 512 / max(width, height)
factor = math.ceil(min(width, height) * factor / 64) * 64 / min(width, height)
width = int((width * factor) // 64) * 64
height = int((height * factor) // 64) * 64
image = ImageOps.fit(image, (width, height), method=Image.Resampling.LANCZOS)
prompt = "Detect the person."
images = pipe(prompt, image=image, num_inference_steps=100, generator=generator).images[0]
images[0]
``` |