NSFW for chest xrays
Hi! I'm running a set of experiments with chest x-ray images and I'm applying different small noises. With amount of 0.2 of noise with only 5 steps I'm getting a bunch of NSFW in return.
Here is an example of the code:
init_image = Image.open(image).convert("RGB")
prompt = "a chest-xray image"
images = pipe(prompt=prompt, image=init_image, strength=0.2, guidance_scale=10, num_inference_steps=5).images
images[0].save( f'noise_{image}')
When using 0.1/0.15 I get 2 NSFW but using 0.2 I get a lot of those prompts:
Anybody had the same issue with this type of images or others? or at least has a possible explanation for this behaviour?
The safety checker is not great
I'm having the same problem with the sample given here itself !!
prompt = "a photo of an astronaut riding a horse on mars"
so i dunno what i'm doing wrong !?
@adnanshussain
indeed it's weird that you're getting that for the given prompt.. but as patrick said the safety checker is not great and you can disable it by setting safety_checker=None when creating the pipeline.
@mauricio-repetto thanks, but i cannot seem to find any safety_checker options in the from_pretrained function and if I am trying to create a pipe like this
pipe = StableDiffusionPipeline(safety_checker=False)
it asks for too many other pameters also..
Any idea what can be done ?
Ok so this worked for me.
from diffusers.pipelines.stable_diffusion import safety_checker
def sc(self, clip_input, images) : return images, [False for i in images]
# edit the StableDiffusionSafetyChecker class so that, when called, it just returns the images and an array of True valuessafety_checker.StableDiffusionSafetyChecker.forward = sc
@adnanshussain you are making it too complex, you just need to add the optional parameter to the ones you were already passing.
from diffusers import DiffusionPipeline
import torch
pipeline = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16, safety_checker=None)
pipeline.to("cuda")
pipeline("An image of a squirrel in Picasso style").images[0]
I took the basic example from diffusers github, adding the parameter as I described you.