Parameters

• negative_prompt (str or List[str], optional) — The prompt or prompts not to guide the image generation. Ignored when not using guidance (i.e., ignored if guidance_scale is less than 1).
• generator (torch.Generator, optional) — One or a list of torch generator(s) to make generation deterministic.
• output_type (str, optional, defaults to "pil") — The output format of the generate image. Choose between PIL: PIL.Image.Image or np.array.
• return_dict (bool, optional, defaults to True) — Whether or not to return a LEditsPPDiffusionPipelineOutput instead of a plain tuple.
• editing_prompt (str or List[str], optional) — The prompt or prompts to guide the image generation. The image is reconstructed by setting editing_prompt = None. Guidance direction of prompt should be specified via reverse_editing_direction.
• editing_prompt_embeds (torch.Tensor>, optional) — Pre-computed embeddings to use for guiding the image generation. Guidance direction of embedding should be specified via reverse_editing_direction.
• negative_prompt_embeds (torch.FloatTensor, optional) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, negative_prompt_embeds are generated from the negative_prompt input argument.
• reverse_editing_direction (bool or List[bool], optional, defaults to False) — Whether the corresponding prompt in editing_prompt should be increased or decreased.
• edit_guidance_scale (float or List[float], optional, defaults to 5) — Guidance scale for guiding the image generation. If provided as list values should correspond to editing_prompt. edit_guidance_scale is defined as s_e of equation 12 of LEDITS++ Paper.
• edit_warmup_steps (float or List[float], optional, defaults to 10) — Number of diffusion steps (for each prompt) for which guidance will not be applied.
• edit_cooldown_steps (float or List[float], optional, defaults to None) — Number of diffusion steps (for each prompt) after which guidance will no longer be applied.
• edit_threshold (float or List[float], optional, defaults to 0.9) — Masking threshold of guidance. Threshold should be proportional to the image region that is modified. ‘edit_threshold’ is defined as ‘λ’ of equation 12 of LEDITS++ Paper.
• user_mask (torch.FloatTensor, optional) — User-provided mask for even better control over the editing process. This is helpful when LEDITS++‘s implicit masks do not meet user preferences.
• sem_guidance (List[torch.Tensor], optional) — List of pre-generated guidance vectors to be applied at generation. Length of the list has to correspond to num_inference_steps.
• use_cross_attn_mask (bool, defaults to False) — Whether cross-attention masks are used. Cross-attention masks are always used when use_intersect_mask is set to true. Cross-attention masks are defined as ‘M^1’ of equation 12 of LEDITS++ paper.
• use_intersect_mask (bool, defaults to True) — Whether the masking term is calculated as intersection of cross-attention masks and masks derived from the noise estimate. Cross-attention mask are defined as ‘M^1’ and masks derived from the noise estimate are defined as ‘M^2’ of equation 12 of LEDITS++ paper.
• attn_store_steps (List[int], optional) — Steps for which the attention maps are stored in the AttentionStore. Just for visualization purposes.
• store_averaged_over_steps (bool, defaults to True) — Whether the attention maps for the ‘attn_store_steps’ are stored averaged over the diffusion steps. If False, attention maps for each step are stores separately. Just for visualization purposes.
• cross_attention_kwargs (dict, optional) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined in self.processor.
• guidance_rescale (float, optional, defaults to 0.0) — Guidance rescale factor from Common Diffusion Noise Schedules and Sample Steps are Flawed. Guidance rescale factor should fix overexposure when using zero terminal SNR.
• clip_skip (int, optional) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.
• callback_on_step_end (Callable, optional) — A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict). callback_kwargs will include a list of all tensors as specified by callback_on_step_end_tensor_inputs.
• callback_on_step_end_tensor_inputs (List, optional) — The list of tensor inputs for the callback_on_step_end function. The tensors specified in the list will be passed as callback_kwargs argument. You will only be able to include variables listed in the ._callback_tensor_inputs attribute of your pipeline class.

LEditsPPDiffusionPipelineOutput if return_dict is True, otherwise a tuple. When returning a tuple, the first element is a list with the generated images, and the second element is a list of bools denoting whether the corresponding generated image likely represents "not-safe-for-work" (nsfw) content, according to the safety_checker`.

Parameters

• image (PipelineImageInput) — Input for the image(s) that are to be edited. Multiple input images have to default to the same aspect ratio.
• source_prompt (str, defaults to "") — Prompt describing the input image that will be used for guidance during inversion. Guidance is disabled if the source_prompt is "".
• source_guidance_scale (float, defaults to 3.5) — Strength of guidance during inversion.
• num_inversion_steps (int, defaults to 30) — Number of total performed inversion steps after discarding the initial skip steps.
• skip (float, defaults to 0.15) — Portion of initial steps that will be ignored for inversion and subsequent generation. Lower values will lead to stronger changes to the input image. skip has to be between 0 and 1.
• generator (torch.Generator, optional) — A torch.Generator to make inversion deterministic.
• cross_attention_kwargs (dict, optional) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined in self.processor.
• clip_skip (int, optional) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.
• height (int, optional, defaults to None) — The height in preprocessed image. If None, will use the get_default_height_width() to get default height.
• width (int, optional, defaults to None) -- The width in preprocessed. If None, will use get_default_height_width() to get the default width.
• resize_mode (str, optional, defaults to default) — The resize mode, can be one of default or fill. If default, will resize the image to fit within the specified width and height, and it may not maintaining the original aspect ratio. If fill, will resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, filling empty with data from image. If crop, will resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, cropping the excess. Note that resize_mode fill and crop are only supported for PIL image input.
• crops_coords (List[Tuple[int, int, int, int]], optional, defaults to None) — The crop coordinates for each image in the batch. If None, will not crop the image.

Output will contain the resized input image(s) and respective VAE reconstruction(s).

Parameters

• num_images_per_prompt (int) — number of images that should be generated per prompt
• enable_edit_guidance (bool) — whether to perform any editing or reconstruct the input image instead
• negative_prompt (str or List[str], optional) — The prompt or prompts not to guide the image generation. If not defined, one has to pass negative_prompt_embeds instead. Ignored when not using guidance (i.e., ignored if guidance_scale is less than 1).
• editing_prompt (str or List[str], optional) — Editing prompt(s) to be encoded. If not defined, one has to pass editing_prompt_embeds instead.
• editing_prompt_embeds (torch.FloatTensor, optional) — Pre-generated text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, text embeddings will be generated from prompt input argument.
• negative_prompt_embeds (torch.FloatTensor, optional) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, negative_prompt_embeds will be generated from negative_prompt input argument.
• lora_scale (float, optional) — A LoRA scale that will be applied to all LoRA layers of the text encoder if LoRA layers are loaded.
• clip_skip (int, optional) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.

Parameters

• denoising_end (float, optional) — When specified, determines the fraction (between 0.0 and 1.0) of the total denoising process to be completed before it is intentionally prematurely terminated. As a result, the returned sample will still retain a substantial amount of noise as determined by the discrete timesteps selected by the scheduler. The denoising_end parameter should ideally be utilized when this pipeline forms a part of a “Mixture of Denoisers” multi-pipeline setup, as elaborated in [**Refining the Image
• negative_prompt (str or List[str], optional) — The prompt or prompts not to guide the image generation. If not defined, one has to pass negative_prompt_embeds instead. Ignored when not using guidance (i.e., ignored if guidance_scale is less than 1).
• negative_prompt_2 (str or List[str], optional) — The prompt or prompts not to guide the image generation to be sent to tokenizer_2 and text_encoder_2. If not defined, negative_prompt is used in both text-encoders
• negative_prompt_embeds (torch.FloatTensor, optional) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, negative_prompt_embeds will be generated from negative_prompt input argument.
• negative_pooled_prompt_embeds (torch.FloatTensor, optional) — Pre-generated negative pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled negative_prompt_embeds will be generated from negative_prompt input argument. ip_adapter_image — (PipelineImageInput, optional): Optional image input to work with IP Adapters.
• output_type (str, optional, defaults to "pil") — The output format of the generate image. Choose between PIL: PIL.Image.Image or np.array.
• return_dict (bool, optional, defaults to True) — Whether or not to return a ~pipelines.stable_diffusion_xl.StableDiffusionXLPipelineOutput instead of a plain tuple.
• callback (Callable, optional) — A function that will be called every callback_steps steps during inference. The function will be called with the following arguments: callback(step: int, timestep: int, latents: torch.FloatTensor).
• callback_steps (int, optional, defaults to 1) — The frequency at which the callback function will be called. If not specified, the callback will be called at every step.
• cross_attention_kwargs (dict, optional) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined under self.processor in diffusers.models.attention_processor.
• guidance_rescale (float, optional, defaults to 0.7) — Guidance rescale factor proposed by Common Diffusion Noise Schedules and Sample Steps are Flawed guidance_scale is defined as φ in equation 16. of Common Diffusion Noise Schedules and Sample Steps are Flawed. Guidance rescale factor should fix overexposure when using zero terminal SNR.
• crops_coords_top_left (Tuple[int], optional, defaults to (0, 0)) — crops_coords_top_left can be used to generate an image that appears to be “cropped” from the position crops_coords_top_left downwards. Favorable, well-centered images are usually achieved by setting crops_coords_top_left to (0, 0). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• target_size (Tuple[int], optional, defaults to (1024, 1024)) — For most cases, target_size should be set to the desired height and width of the generated image. If not specified it will default to (width, height). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• editing_prompt (str or List[str], optional) — The prompt or prompts to guide the image generation. The image is reconstructed by setting editing_prompt = None. Guidance direction of prompt should be specified via reverse_editing_direction.
• editing_prompt_embeddings (torch.Tensor, optional) — Pre-generated edit text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, editing_prompt_embeddings will be generated from editing_prompt input argument.
• editing_pooled_prompt_embeddings (torch.Tensor, optional) — Pre-generated pooled edit text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, editing_prompt_embeddings will be generated from editing_prompt input argument.
• reverse_editing_direction (bool or List[bool], optional, defaults to False) — Whether the corresponding prompt in editing_prompt should be increased or decreased.
• edit_guidance_scale (float or List[float], optional, defaults to 5) — Guidance scale for guiding the image generation. If provided as list values should correspond to editing_prompt. edit_guidance_scale is defined as s_e of equation 12 of LEDITS++ Paper.
• edit_warmup_steps (float or List[float], optional, defaults to 10) — Number of diffusion steps (for each prompt) for which guidance is not applied.
• edit_cooldown_steps (float or List[float], optional, defaults to None) — Number of diffusion steps (for each prompt) after which guidance is no longer applied.
• edit_threshold (float or List[float], optional, defaults to 0.9) — Masking threshold of guidance. Threshold should be proportional to the image region that is modified. ‘edit_threshold’ is defined as ‘λ’ of equation 12 of LEDITS++ Paper.
• sem_guidance (List[torch.Tensor], optional) — List of pre-generated guidance vectors to be applied at generation. Length of the list has to correspond to num_inference_steps. use_cross_attn_mask — Whether cross-attention masks are used. Cross-attention masks are always used when use_intersect_mask is set to true. Cross-attention masks are defined as ‘M^1’ of equation 12 of LEDITS++ paper. use_intersect_mask — Whether the masking term is calculated as intersection of cross-attention masks and masks derived from the noise estimate. Cross-attention mask are defined as ‘M^1’ and masks derived from the noise estimate are defined as ‘M^2’ of equation 12 of LEDITS++ paper. user_mask — User-provided mask for even better control over the editing process. This is helpful when LEDITS++‘s implicit masks do not meet user preferences. attn_store_steps — Steps for which the attention maps are stored in the AttentionStore. Just for visualization purposes. store_averaged_over_steps — Whether the attention maps for the ‘attn_store_steps’ are stored averaged over the diffusion steps. If False, attention maps for each step are stores separately. Just for visualization purposes.
• clip_skip (int, optional) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.
• callback_on_step_end (Callable, optional) — A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict). callback_kwargs will include a list of all tensors as specified by callback_on_step_end_tensor_inputs.
• callback_on_step_end_tensor_inputs (List, optional) — The list of tensor inputs for the callback_on_step_end function. The tensors specified in the list will be passed as callback_kwargs argument. You will only be able to include variables listed in the ._callback_tensor_inputs attribute of your pipeline class.

LEditsPPDiffusionPipelineOutput if return_dict is True, otherwise a `tuple. When returning a tuple, the first element is a list with the generated images.

Parameters

• image (PipelineImageInput) — Input for the image(s) that are to be edited. Multiple input images have to default to the same aspect ratio.
• source_prompt (str, defaults to "") — Prompt describing the input image that will be used for guidance during inversion. Guidance is disabled if the source_prompt is "".
• source_guidance_scale (float, defaults to 3.5) — Strength of guidance during inversion.
• negative_prompt (str or List[str], optional) — The prompt or prompts not to guide the image generation. If not defined, one has to pass negative_prompt_embeds instead. Ignored when not using guidance (i.e., ignored if guidance_scale is less than 1).
• negative_prompt_2 (str or List[str], optional) — The prompt or prompts not to guide the image generation to be sent to tokenizer_2 and text_encoder_2. If not defined, negative_prompt is used in both text-encoders
• num_inversion_steps (int, defaults to 50) — Number of total performed inversion steps after discarding the initial skip steps.
• skip (float, defaults to 0.15) — Portion of initial steps that will be ignored for inversion and subsequent generation. Lower values will lead to stronger changes to the input image. skip has to be between 0 and 1.
• generator (torch.Generator, optional) — A torch.Generator to make inversion deterministic.
• crops_coords_top_left (Tuple[int], optional, defaults to (0, 0)) — crops_coords_top_left can be used to generate an image that appears to be “cropped” from the position crops_coords_top_left downwards. Favorable, well-centered images are usually achieved by setting crops_coords_top_left to (0, 0). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• num_zero_noise_steps (int, defaults to 3) — Number of final diffusion steps that will not renoise the current image. If no steps are set to zero SD-XL in combination with DPMSolverMultistepScheduler will produce noise artifacts.
• cross_attention_kwargs (dict, optional) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined under self.processor in diffusers.models.attention_processor.

Output will contain the resized input image(s) and respective VAE reconstruction(s).

Parameters

• num_images_per_prompt (int) — number of images that should be generated per prompt
• negative_prompt (str or List[str], optional) — The prompt or prompts not to guide the image generation. If not defined, one has to pass negative_prompt_embeds instead.
• negative_prompt_2 (str or List[str], optional) — The prompt or prompts not to guide the image generation to be sent to tokenizer_2 and text_encoder_2. If not defined, negative_prompt is used in both text-encoders
• negative_prompt_embeds (torch.FloatTensor, optional) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, negative_prompt_embeds will be generated from negative_prompt input argument.
• negative_pooled_prompt_embeds (torch.FloatTensor, optional) — Pre-generated negative pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled negative_prompt_embeds will be generated from negative_prompt input argument.
• lora_scale (float, optional) — A lora scale that will be applied to all LoRA layers of the text encoder if LoRA layers are loaded.
• clip_skip (int, optional) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.
• enable_edit_guidance (bool) — Whether to guide towards an editing prompt or not.
• editing_prompt (str or List[str], optional) — Editing prompt(s) to be encoded. If not defined and ‘enable_edit_guidance’ is True, one has to pass editing_prompt_embeds instead.
• editing_prompt_embeds (torch.FloatTensor, optional) — Pre-generated edit text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided and ‘enable_edit_guidance’ is True, editing_prompt_embeds will be generated from editing_prompt input argument.
• editing_pooled_prompt_embeds (torch.FloatTensor, optional) — Pre-generated edit pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled editing_pooled_prompt_embeds will be generated from editing_prompt input argument.

Parameters

• timesteps (torch.Tensor) — generate embedding vectors at these timesteps
• embedding_dim (int, optional, defaults to 512) — dimension of the embeddings to generate dtype — data type of the generated embeddings

Embedding vectors with shape (len(timesteps), embedding_dim)