Parameters

• path (Union[str, Path]) — Path of the compiled model.

Parameters

• outputs (List[torch.Tensor]) — List of torch tensors which are inference output.
• dims (List[int]) — List of dimensions in which we slice a tensor.
• indices (List[int]) — List of indices in which we slice a tensor along an axis.
• padding_side (Literal["right", "left"], defaults to “right”) — The side on which the padding has been applied.

Parameters

• input_ids (torch.Tensor of shape (batch_size, sequence_length)) — Indices of input sequence tokens in the vocabulary. Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode and PreTrainedTokenizer.__call__ for details. What are input IDs?
• attention_mask (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked. What are attention masks?
• token_type_ids (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]:
  • 1 for tokens that are sentence A,
  • 0 for tokens that are sentence B. What are token type IDs?

Parameters

• input_ids (torch.Tensor of shape (batch_size, sequence_length)) — Indices of input sequence tokens in the vocabulary. Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode and PreTrainedTokenizer.__call__ for details. What are input IDs?
• attention_mask (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked. What are attention masks?
• token_type_ids (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]:
  • 1 for tokens that are sentence A,
  • 0 for tokens that are sentence B. What are token type IDs?

Parameters

• input_ids (torch.Tensor of shape (batch_size, sequence_length)) — Indices of input sequence tokens in the vocabulary. Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode and PreTrainedTokenizer.__call__ for details. What are input IDs?
• attention_mask (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked. What are attention masks?
• token_type_ids (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]:
  • 1 for tokens that are sentence A,
  • 0 for tokens that are sentence B. What are token type IDs?

Parameters

• input_ids (torch.Tensor of shape (batch_size, sequence_length)) — Indices of input sequence tokens in the vocabulary. Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode and PreTrainedTokenizer.__call__ for details. What are input IDs?
• attention_mask (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked. What are attention masks?
• token_type_ids (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]:
  • 1 for tokens that are sentence A,
  • 0 for tokens that are sentence B. What are token type IDs?

Parameters

• input_ids (torch.Tensor of shape (batch_size, sequence_length)) — Indices of input sequence tokens in the vocabulary. Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode and PreTrainedTokenizer.__call__ for details. What are input IDs?
• attention_mask (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked. What are attention masks?
• token_type_ids (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]:
  • 1 for tokens that are sentence A,
  • 0 for tokens that are sentence B. What are token type IDs?

Parameters

• input_ids (torch.Tensor of shape (batch_size, sequence_length)) — Indices of input sequence tokens in the vocabulary. Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode and PreTrainedTokenizer.__call__ for details. What are input IDs?
• attention_mask (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked. What are attention masks?
• token_type_ids (Union[torch.Tensor, None] of shape (batch_size, sequence_length), defaults to None) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]:
  • 1 for tokens that are sentence A,
  • 0 for tokens that are sentence B. What are token type IDs?

Parameters

• input_ids (torch.Tensor of shape (batch_size, num_choices, sequence_length)) — Indices of input sequence tokens in the vocabulary. Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode and PreTrainedTokenizer.__call__ for details. What are input IDs?
• attention_mask (Union[torch.Tensor, None] of shape (batch_size, num_choices, sequence_length), defaults to None) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked. What are attention masks?
• token_type_ids (Union[torch.Tensor, None] of shape (batch_size, num_choices, sequence_length), defaults to None) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]:
  • 1 for tokens that are sentence A,
  • 0 for tokens that are sentence B. What are token type IDs?

Parameters

• input_ids (torch.LongTensor) — Indices of decoder input sequence tokens in the vocabulary of shape (batch_size, sequence_length).
• cache_ids (torch.LongTensor) — The indices at which the cached key and value for the current inputs need to be stored.
• start_ids (torch.LongTensor) — The indices of the first tokens to be processed, deduced form the attention masks.

Parameters

• input_ids (torch.Tensor of shape (batch_size, sequence_length)) — The sequence used as a prompt for the generation.
• attention_mask (torch.Tensor of shape (batch_size, sequence_length), optional) — Mask to avoid performing attention on padding token indices.
• generation_config (~transformers.generation.GenerationConfig, optional) — The generation configuration to be used as base parametrization for the generation call. **kwargs passed to generate matching the attributes of generation_config will override them. If generation_config is not provided, default will be used, which had the following loading priority: 1) from the generation_config.json model file, if it exists; 2) from the model configuration. Please note that unspecified parameters will inherit GenerationConfig’s default values, whose documentation should be checked to parameterize generation.
• stopping_criteria (Optional[transformers.generation.StoppingCriteriaList], defaults to None`) — Custom stopping criteria that complement the default stopping criteria built from arguments and a generation config.

A torch.FloatTensor.

Parameters

• input_ids (torch.LongTensor of shape (batch_size, sequence_length)) — The sequence used as a prompt for the generation.
• selector (TokenSelector) — The object implementing the generation logic based on transformers processors and stopping criterias.
• batch_size (int) — The actual input batch size. Used to avoid generating tokens for padded inputs.
• attention_mask (torch.Tensor of shape (batch_size, sequence_length), optional) — Mask to avoid performing attention on padding token indices. model_kwargs — Additional model specific kwargs will be forwarded to the forward function of the model.

A torch.LongTensor containing the generated tokens.

Parameters

• data_parallel_mode (Optional[str]) — Mode to decide what components to load into both NeuronCores of a Neuron device. Can be “none”(no data parallel), “unet”(only load unet into both cores of each device), “all”(load the whole pipeline into both cores).
• text_encoder_path (Union[str, Path]) — Path of the compiled text encoder.
• unet_path (Union[str, Path]) — Path of the compiled U-NET.
• vae_decoder_path (Optional[Union[str, Path]], defaults to None) — Path of the compiled VAE decoder.
• vae_encoder_path (Optional[Union[str, Path]], defaults to None) — Path of the compiled VAE encoder. It is optional, only used for tasks taking images as input.
• text_encoder_2_path (Optional[Union[str, Path]], defaults to None) — Path of the compiled second frozen text encoder. SDXL only.
• dynamic_batch_size (bool, defaults to False) — Whether enable dynamic batch size for neuron compiled model. If True, the input batch size can be a multiple of the batch size during the compilation.

Parameters

• prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to guide image generation. If not defined, you need to pass prompt_embeds.
• num_inference_steps (int, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
• guidance_scale (float, defaults to 7.5) — A higher guidance scale value encourages the model to generate images closely linked to the text prompt at the expense of lower image quality. Guidance scale is enabled when guidance_scale > 1.
• negative_prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to guide what to not include in image generation. If not defined, you need to pass negative_prompt_embeds instead. Ignored when not using guidance (guidance_scale < 1).
• num_images_per_prompt (int, defaults to 1) — The number of images to generate per prompt. If it is different from the batch size used for the compiltaion, it will be overriden by the static batch size of neuron (except for dynamic batching).
• eta (float, defaults to 0.0) — Corresponds to parameter eta (η) from the DDIM paper. Only applies to the diffusers.schedulers.DDIMScheduler, and is ignored in other schedulers.
• generator (Optional[Union[torch.Generator, List[torch.Generator]]], defaults to None) — A torch.Generator to make generation deterministic.
• latents (Optional[torch.FloatTensor], defaults to None) — Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random generator.
• prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, text embeddings are generated from the prompt input argument.
• negative_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, negative_prompt_embeds are generated from the negative_prompt input argument.
• output_type (Optional[str], defaults to "pil") — The output format of the generated image. Choose between PIL.Image or np.array.
• return_dict (bool, defaults to True) — Whether or not to return a diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput instead of a plain tuple.
• callback (Optional[Callable], defaults to None) — A function that calls every callback_steps steps during inference. The function is called with the following arguments: callback(step: int, timestep: int, latents: torch.FloatTensor).
• callback_steps (int, defaults to 1) — The frequency at which the callback function is called. If not specified, the callback is called at every step.
• cross_attention_kwargs (dict, defaults to None) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined in self.processor.
• guidance_rescale (float, defaults to 0.0) — Guidance rescale factor from Common Diffusion Noise Schedules and Sample Steps are Flawed. Guidance rescale factor should fix overexposure when using zero terminal SNR.

If return_dict is True, diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput is returned, otherwise a tuple is returned where the first element is a list with the generated images and the second element is a list of bools indicating whether the corresponding generated image contains “not-safe-for-work” (nsfw) content.

Parameters

• prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to guide image generation. If not defined, you need to pass prompt_embeds.
• image (Optional["PipelineImageInput"], defaults to None) — Image, numpy array or tensor representing an image batch to be used as the starting point. For both numpy array and pytorch tensor, the expected value range is between [0, 1] If it’s a tensor or a list or tensors, the expected shape should be (B, C, H, W) or (C, H, W). If it is a numpy array or a list of arrays, the expected shape should be (B, H, W, C) or (H, W, C) It can also accept image latents as image, but if passing latents directly it is not encoded again.
• strength (float, defaults to 0.8) — Indicates extent to transform the reference image. Must be between 0 and 1. image is used as a starting point and more noise is added the higher the strength. The number of denoising steps depends on the amount of noise initially added. When strength is 1, added noise is maximum and the denoising process runs for the full number of iterations specified in num_inference_steps. A value of 1 essentially ignores image.
• num_inference_steps (int, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. This parameter is modulated by strength.
• guidance_scale (float, defaults to 7.5) — A higher guidance scale value encourages the model to generate images closely linked to the text prompt at the expense of lower image quality. Guidance scale is enabled when guidance_scale > 1.
• negative_prompt (Optional[Union[str, List[str], defaults to None) — The prompt or prompts to guide what to not include in image generation. If not defined, you need to pass negative_prompt_embeds instead. Ignored when not using guidance (guidance_scale < 1).
• num_images_per_prompt (int, defaults to 1) — The number of images to generate per prompt. If it is different from the batch size used for the compiltaion, it will be overriden by the static batch size of neuron (except for dynamic batching).
• eta (float, defaults to 0.0) — Corresponds to parameter eta (η) from the DDIM paper. Only applies to the diffusers.schedulers.DDIMScheduler, and is ignored in other schedulers.
• generator (Optional[Union[torch.Generator, List[torch.Generator]]], defaults to None) — A torch.Generator to make generation deterministic.
• prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, text embeddings are generated from the prompt input argument.
• negative_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, negative_prompt_embeds are generated from the negative_prompt input argument.
• output_type (Optional[str], defaults to "pil") — The output format of the generated image. Choose between PIL.Image or np.array.
• return_dict (bool, defaults to True) — Whether or not to return a diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput instead of a plain tuple.
• callback (Optional[Callable], defaults to None) — A function that calls every callback_steps steps during inference. The function is called with the following arguments: callback(step: int, timestep: int, latents: torch.FloatTensor).
• callback_steps (int, defaults to 1) — The frequency at which the callback function is called. If not specified, the callback is called at every step.
• cross_attention_kwargs (dict, defaults to None) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined in self.processor.

If return_dict is True, diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput is returned, otherwise a tuple is returned where the first element is a list with the generated images and the second element is a list of bools indicating whether the corresponding generated image contains “not-safe-for-work” (nsfw) content.

Parameters

• prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to guide image generation. If not defined, you need to pass prompt_embeds.
• image (Optional["PipelineImageInput"], defaults to None) — Image, numpy array or tensor representing an image batch to be inpainted (which parts of the image to be masked out with mask_image and repainted according to prompt). For both numpy array and pytorch tensor, the expected value range is between [0, 1] If it’s a tensor or a list or tensors, the expected shape should be (B, C, H, W) or (C, H, W). If it is a numpy array or a list of arrays, the expected shape should be (B, H, W, C) or (H, W, C) It can also accept image latents as image, but if passing latents directly it is not encoded again.
• mask_image (Optional["PipelineImageInput"], defaults to None) — Image, numpy array or tensor representing an image batch to mask image. White pixels in the mask are repainted while black pixels are preserved. If mask_image is a PIL image, it is converted to a single channel (luminance) before use. If it’s a numpy array or pytorch tensor, it should contain one color channel (L) instead of 3, so the expected shape for pytorch tensor would be (B, 1, H, W), (B, H, W), (1, H, W), (H, W). And for numpy array would be for (B, H, W, 1), (B, H, W), (H, W, 1), or (H, W).
• strength (float, defaults to 1.0) — Indicates extent to transform the reference image. Must be between 0 and 1. image is used as a starting point and more noise is added the higher the strength. The number of denoising steps depends on the amount of noise initially added. When strength is 1, added noise is maximum and the denoising process runs for the full number of iterations specified in num_inference_steps. A value of 1 essentially ignores image.
• num_inference_steps (int, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. This parameter is modulated by strength.
• guidance_scale (float, defaults to 7.5) — A higher guidance scale value encourages the model to generate images closely linked to the text prompt at the expense of lower image quality. Guidance scale is enabled when guidance_scale > 1.
• negative_prompt (Optional[Union[str, List[str], defaults to None) — The prompt or prompts to guide what to not include in image generation. If not defined, you need to pass negative_prompt_embeds instead. Ignored when not using guidance (guidance_scale < 1).
• num_images_per_prompt (int, defaults to 1) — The number of images to generate per prompt. If it is different from the batch size used for the compiltaion, it will be overriden by the static batch size of neuron (except for dynamic batching).
• eta (float, defaults to 0.0) — Corresponds to parameter eta (η) from the DDIM paper. Only applies to the diffusers.schedulers.DDIMScheduler, and is ignored in other schedulers.
• generator (Optional[Union[torch.Generator, List[torch.Generator]]], defaults to None) — A torch.Generator to make generation deterministic.
• latents (Optional[torch.FloatTensor], defaults to None) — Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random generator.
• prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, text embeddings are generated from the prompt input argument.
• negative_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, negative_prompt_embeds are generated from the negative_prompt input argument.
• output_type (Optional[str], defaults to "pil") — The output format of the generated image. Choose between PIL.Image or np.array.
• return_dict (bool, defaults to True) — Whether or not to return a diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput instead of a plain tuple.
• callback (Optional[Callable], defaults to None) — A function that calls every callback_steps steps during inference. The function is called with the following arguments: callback(step: int, timestep: int, latents: torch.FloatTensor).
• callback_steps (int, defaults to 1) — The frequency at which the callback function is called. If not specified, the callback is called at every step.
• cross_attention_kwargs (dict, defaults to None) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined in self.processor.
• clip_skip (int, defaults to None) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.

If return_dict is True, diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput is returned, otherwise a tuple is returned where the first element is a list with the generated images and the second element is a list of bools indicating whether the corresponding generated image contains “not-safe-for-work” (nsfw) content.

Parameters

• prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to guide image generation. If not defined, you need to pass prompt_embeds.
• num_inference_steps (int, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
• original_inference_steps (Optional[int], defaults to None) — The original number of inference steps use to generate a linearly-spaced timestep schedule, from which we will draw num_inference_steps evenly spaced timesteps from as our final timestep schedule, following the Skipping-Step method in the paper (see Section 4.3). If not set this will default to the scheduler’s original_inference_steps attribute.
• guidance_scale (float, defaults to 8.5) — A higher guidance scale value encourages the model to generate images closely linked to the text prompt at the expense of lower image quality. Guidance scale is enabled when guidance_scale > 1. Note that the original latent consistency models paper uses a different CFG formulation where the guidance scales are decreased by 1 (so in the paper formulation CFG is enabled when guidance_scale > 0).
• num_images_per_prompt (int, defaults to 1) — The number of images to generate per prompt.
• generator (Optional[Union[torch.Generator, List[torch.Generator]]], defaults to None) — A torch.Generator to make generation deterministic.
• latents (Optional[torch.FloatTensor], defaults to None) — Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random generator.
• prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, text embeddings are generated from the prompt input argument.
• output_type (str, defaults to "pil") — The output format of the generated image. Choose between PIL.Image or np.array.
• return_dict (bool, defaults to True) — Whether or not to return a ~pipelines.stable_diffusion.StableDiffusionPipelineOutput instead of a plain tuple.
• cross_attention_kwargs (Optional[Dict[str, Any]], defaults to None) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined in self.processor.
• clip_skip (Optional[int], defaults to None) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.
• callback_on_step_end (Optional[Callable], defaults to None) — A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict). callback_kwargs will include a list of all tensors as specified by callback_on_step_end_tensor_inputs.
• callback_on_step_end_tensor_inputs (List[str], defaults to ["latents"]) — The list of tensor inputs for the callback_on_step_end function. The tensors specified in the list will be passed as callback_kwargs argument. You will only be able to include variables listed in the ._callback_tensor_inputs attribute of your pipeine class.

If return_dict is True, diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput is returned, otherwise a tuple is returned where the first element is a list with the generated images and the second element is a list of bools indicating whether the corresponding generated image contains “not-safe-for-work” (nsfw) content.

Parameters

• prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to guide the image generation. If not defined, one has to pass prompt_embeds. instead.
• prompt_2 (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to be sent to the tokenizer_2 and text_encoder_2. If not defined, prompt is used in both text-encoders
• num_inference_steps (int, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
• denoising_end (Optional[float], defaults to None) — When specified, determines the fraction (between 0.0 and 1.0) of the total denoising process to be completed before it is intentionally prematurely terminated. As a result, the returned sample will still retain a substantial amount of noise as determined by the discrete timesteps selected by the scheduler. The denoising_end parameter should ideally be utilized when this pipeline forms a part of a “Mixture of Denoisers” multi-pipeline setup, as elaborated in Refining the Image Output
• guidance_scale (float, defaults to 5.0) — Guidance scale as defined in Classifier-Free Diffusion Guidance. guidance_scale is defined as w of equation 2. of Imagen Paper. Guidance scale is enabled by setting guidance_scale > 1. Higher guidance scale encourages to generate images that are closely linked to the text prompt, usually at the expense of lower image quality.
• negative_prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts not to guide the image generation. If not defined, one has to pass negative_prompt_embeds instead. Ignored when not using guidance (i.e., ignored if guidance_scale is less than 1).
• negative_prompt_2 (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts not to guide the image generation to be sent to tokenizer_2 and text_encoder_2. If not defined, negative_prompt is used in both text-encoders
• num_images_per_prompt (int, defaults to 1) — The number of images to generate per prompt. If it is different from the batch size used for the compiltaion, it will be overriden by the static batch size of neuron (except for dynamic batching).
• eta (float, defaults to 0.0) — Corresponds to parameter eta (η) in the DDIM paper: https://arxiv.org/abs/2010.02502. Only applies to schedulers.DDIMScheduler, will be ignored for others.
• generator (Optional[Union[torch.Generator, List[torch.Generator]]], defaults to None) — One or a list of torch generator(s) to make generation deterministic.
• latents (Optional[torch.FloatTensor], defaults to None) — Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random generator.
• prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, text embeddings will be generated from prompt input argument.
• negative_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, negative_prompt_embeds will be generated from negative_prompt input argument.
• pooled_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled text embeddings will be generated from prompt input argument.
• negative_pooled_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled negative_prompt_embeds will be generated from negative_prompt input argument.
• output_type (Optional[str], defaults to "pil") — The output format of the generate image. Choose between PIL: PIL.Image.Image or np.array.
• return_dict (bool, defaults to True) — Whether or not to return a diffusers.pipelines.stable_diffusion_xl.StableDiffusionXLPipelineOutput instead of a plain tuple.
• callback (Optional[Callable], defaults to None) — A function that will be called every callback_steps steps during inference. The function will be called with the following arguments: callback(step: int, timestep: int, latents: torch.FloatTensor).
• callback_steps (int, defaults to 1) — The frequency at which the callback function will be called. If not specified, the callback will be called at every step.
• cross_attention_kwargs (dict, defaults to None) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined under self.processor in diffusers.models.attention_processor.
• guidance_rescale (float, optional, defaults to 0.0) — Guidance rescale factor proposed by Common Diffusion Noise Schedules and Sample Steps are Flawed guidance_scale is defined as φ in equation 16. of Common Diffusion Noise Schedules and Sample Steps are Flawed. Guidance rescale factor should fix overexposure when using zero terminal SNR.
• original_size (Optional[Tuple[int, int]], defaults to (1024, 1024)) — If original_size is not the same as target_size the image will appear to be down- or upsampled. original_size defaults to (width, height) if not specified. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• crops_coords_top_left (Tuple[int], defaults to (0, 0)) — crops_coords_top_left can be used to generate an image that appears to be “cropped” from the position crops_coords_top_left downwards. Favorable, well-centered images are usually achieved by setting crops_coords_top_left to (0, 0). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• target_size (Tuple[int],defaults to (1024, 1024)) — For most cases, target_size should be set to the desired height and width of the generated image. If not specified it will default to (width, height). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• negative_original_size (Tuple[int], defaults to (1024, 1024)) — To negatively condition the generation process based on a specific image resolution. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• negative_crops_coords_top_left (Tuple[int], defaults to (0, 0)) — To negatively condition the generation process based on a specific crop coordinates. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• negative_target_size (Tuple[int], defaults to (1024, 1024)) — To negatively condition the generation process based on a target image resolution. It should be as same as the target_size for most cases. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• clip_skip (Optional[int], defaults to None) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.

diffusers.pipelines.stable_diffusion_xl.StableDiffusionXLPipelineOutput if return_dict is True, otherwise a tuple. When returning a tuple, the first element is a list with the generated images.

Parameters

• prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to guide the image generation. If not defined, one has to pass prompt_embeds. instead.
• prompt_2 (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to be sent to the tokenizer_2 and text_encoder_2. If not defined, prompt is used in both text-encoders
• image (Optional["PipelineImageInput"], defaults to None) — The image(s) to modify with the pipeline.
• strength (float, defaults to 0.3) — Conceptually, indicates how much to transform the reference image. Must be between 0 and 1. image will be used as a starting point, adding more noise to it the larger the strength. The number of denoising steps depends on the amount of noise initially added. When strength is 1, added noise will be maximum and the denoising process will run for the full number of iterations specified in num_inference_steps. A value of 1, therefore, essentially ignores image. Note that in the case of denoising_start being declared as an integer, the value of strength will be ignored.
• num_inference_steps (int, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
• denoising_start (Optional[float], defaults to None) — When specified, indicates the fraction (between 0.0 and 1.0) of the total denoising process to be bypassed before it is initiated. Consequently, the initial part of the denoising process is skipped and it is assumed that the passed image is a partly denoised image. Note that when this is specified, strength will be ignored. The denoising_start parameter is particularly beneficial when this pipeline is integrated into a “Mixture of Denoisers” multi-pipeline setup, as detailed in Refining the Image Output.
• denoising_end (Optional[float], defaults to None) — When specified, determines the fraction (between 0.0 and 1.0) of the total denoising process to be completed before it is intentionally prematurely terminated. As a result, the returned sample will still retain a substantial amount of noise (ca. final 20% of timesteps still needed) and should be denoised by a successor pipeline that has denoising_start set to 0.8 so that it only denoises the final 20% of the scheduler. The denoising_end parameter should ideally be utilized when this pipeline forms a part of a “Mixture of Denoisers” multi-pipeline setup, as elaborated in Refining the Image Output.
• guidance_scale (float, defaults to 7.5) — Guidance scale as defined in Classifier-Free Diffusion Guidance. guidance_scale is defined as w of equation 2. of Imagen Paper. Guidance scale is enabled by setting guidance_scale > 1. Higher guidance scale encourages to generate images that are closely linked to the text prompt, usually at the expense of lower image quality.
• negative_prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts not to guide the image generation. If not defined, one has to pass negative_prompt_embeds instead. Ignored when not using guidance (i.e., ignored if guidance_scale is less than 1).
• negative_prompt_2 (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts not to guide the image generation to be sent to tokenizer_2 and text_encoder_2. If not defined, negative_prompt is used in both text-encoders
• num_images_per_prompt (int, defaults to 1) — The number of images to generate per prompt. If it is different from the batch size used for the compiltaion, it will be overriden by the static batch size of neuron (except for dynamic batching).
• eta (float, defaults to 0.0) — Corresponds to parameter eta (η) in the DDIM paper: https://arxiv.org/abs/2010.02502. Only applies to schedulers.DDIMScheduler, will be ignored for others.
• generator (Optional[Union[torch.Generator, List[torch.Generator]]], defaults to None) — One or a list of torch generator(s) to make generation deterministic.
• latents (Optional[torch.FloatTensor], defaults to None) — Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random generator.
• prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, text embeddings will be generated from prompt input argument.
• negative_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, negative_prompt_embeds will be generated from negative_prompt input argument.
• pooled_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled text embeddings will be generated from prompt input argument.
• negative_pooled_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled negative_prompt_embeds will be generated from negative_prompt input argument.
• output_type (Optional[str], defaults to "pil") — The output format of the generate image. Choose between PIL: PIL.Image.Image or np.array.
• return_dict (bool, defaults to True) — Whether or not to return a diffusers.pipelines.stable_diffusion.StableDiffusionXLPipelineOutput instead of a plain tuple.
• callback (Optional[Callable], defaults to None) — A function that will be called every callback_steps steps during inference. The function will be called with the following arguments: callback(step: int, timestep: int, latents: torch.FloatTensor).
• callback_stcallback_steps (int, defaults to 1) — The frequency at which the callback function will be called. If not specified, the callback will be called at every step.
• cross_attention_kwargs (Optional[Dict[str, Any]], defaults to None) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined under self.processor in diffusers.models.attention_processor.
• guidance_rescale (float, defaults to 0.0) — Guidance rescale factor proposed by Common Diffusion Noise Schedules and Sample Steps are Flawed guidance_scale is defined as φ in equation 16. of Common Diffusion Noise Schedules and Sample Steps are Flawed. Guidance rescale factor should fix overexposure when using zero terminal SNR.
• original_size (Optional[Tuple[int, int]], defaults to (1024, 1024)) — If original_size is not the same as target_size the image will appear to be down- or upsampled. original_size defaults to (width, height) if not specified. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• crops_coords_top_left (Tuple[int], defaults to (0, 0)) — crops_coords_top_left can be used to generate an image that appears to be “cropped” from the position crops_coords_top_left downwards. Favorable, well-centered images are usually achieved by setting crops_coords_top_left to (0, 0). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• target_size (Tuple[int],defaults to (1024, 1024)) — For most cases, target_size should be set to the desired height and width of the generated image. If not specified it will default to (width, height). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• negative_original_size (Tuple[int], defaults to (1024, 1024)) — To negatively condition the generation process based on a specific image resolution. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• negative_crops_coords_top_left (Tuple[int], defaults to (0, 0)) — To negatively condition the generation process based on a specific crop coordinates. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• negative_target_size (Tuple[int], defaults to (1024, 1024)) — To negatively condition the generation process based on a target image resolution. It should be as same as the target_size for most cases. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• aesthetic_score (float, defaults to 6.0) — Used to simulate an aesthetic score of the generated image by influencing the positive text condition. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• negative_aesthetic_score (float, defaults to 2.5) — Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. Can be used to simulate an aesthetic score of the generated image by influencing the negative text condition.
• clip_skip (Optional[int], defaults to None) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.

diffusers.pipelines.stable_diffusion.StableDiffusionXLPipelineOutput if return_dict is True, otherwise a `tuple. When returning a tuple, the first element is a list with the generated images.

Parameters

• prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to guide the image generation. If not defined, one has to pass prompt_embeds. instead.
• prompt_2 (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts to be sent to the tokenizer_2 and text_encoder_2. If not defined, prompt is used in both text-encoders
• image (Optional["PipelineImageInput"], defaults to None) — Image, or tensor representing an image batch which will be inpainted, i.e. parts of the image will be masked out with mask_image and repainted according to prompt.
• mask_image (Optional["PipelineImageInput"], defaults to None) — Image, or tensor representing an image batch, to mask image. White pixels in the mask will be repainted, while black pixels will be preserved. If mask_image is a PIL image, it will be converted to a single channel (luminance) before use. If it’s a tensor, it should contain one color channel (L) instead of 3, so the expected shape would be (B, H, W, 1).
• padding_mask_crop (Optional[int], defaults to None) — The size of margin in the crop to be applied to the image and masking. If None, no crop is applied to image and mask_image. If padding_mask_crop is not None, it will first find a rectangular region with the same aspect ration of the image and contains all masked area, and then expand that area based on padding_mask_crop. The image and mask_image will then be cropped based on the expanded area before resizing to the original image size for inpainting. This is useful when the masked area is small while the image is large and contain information inreleant for inpainging, such as background.
• strength (float, defaults to 0.9999) — Conceptually, indicates how much to transform the masked portion of the reference image. Must be between 0 and 1. image will be used as a starting point, adding more noise to it the larger the strength. The number of denoising steps depends on the amount of noise initially added. When strength is 1, added noise will be maximum and the denoising process will run for the full number of iterations specified in num_inference_steps. A value of 1, therefore, essentially ignores the masked portion of the reference image. Note that in the case of denoising_start being declared as an integer, the value of strength will be ignored.
• num_inference_steps (int, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
• timesteps (Optional[List[int]], defaults to None) — Custom timesteps to use for the denoising process with schedulers which support a timesteps argument in their set_timesteps method. If not defined, the default behavior when num_inference_steps is passed will be used. Must be in descending order.
• denoising_start (Optional[float], defaults to None) — When specified, indicates the fraction (between 0.0 and 1.0) of the total denoising process to be bypassed before it is initiated. Consequently, the initial part of the denoising process is skipped and it is assumed that the passed image is a partly denoised image. Note that when this is specified, strength will be ignored. The denoising_start parameter is particularly beneficial when this pipeline is integrated into a “Mixture of Denoisers” multi-pipeline setup, as detailed in Refining the Image Output.
• denoising_end (Optional[float], defaults to None) — When specified, determines the fraction (between 0.0 and 1.0) of the total denoising process to be completed before it is intentionally prematurely terminated. As a result, the returned sample will still retain a substantial amount of noise (ca. final 20% of timesteps still needed) and should be denoised by a successor pipeline that has denoising_start set to 0.8 so that it only denoises the final 20% of the scheduler. The denoising_end parameter should ideally be utilized when this pipeline forms a part of a “Mixture of Denoisers” multi-pipeline setup, as elaborated in Refining the Image Output.
• guidance_scale (float, defaults to 7.5) — Guidance scale as defined in Classifier-Free Diffusion Guidance. guidance_scale is defined as w of equation 2. of Imagen Paper. Guidance scale is enabled by setting guidance_scale > 1. Higher guidance scale encourages to generate images that are closely linked to the text prompt, usually at the expense of lower image quality.
• negative_prompt (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts not to guide the image generation. If not defined, one has to pass negative_prompt_embeds instead. Ignored when not using guidance (i.e., ignored if guidance_scale is less than 1).
• negative_prompt_2 (Optional[Union[str, List[str]]], defaults to None) — The prompt or prompts not to guide the image generation to be sent to tokenizer_2 and text_encoder_2. If not defined, negative_prompt is used in both text-encoders
• prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, text embeddings will be generated from prompt input argument.
• negative_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, negative_prompt_embeds will be generated from negative_prompt input argument.
• pooled_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled text embeddings will be generated from prompt input argument.
• negative_pooled_prompt_embeds (Optional[torch.FloatTensor], defaults to None) — Pre-generated negative pooled text embeddings. Can be used to easily tweak text inputs, e.g. prompt weighting. If not provided, pooled negative_prompt_embeds will be generated from negative_prompt input argument. ip_adapter_image — (Optional[PipelineImageInput], defaults to None): Optional image input to work with IP Adapters.
• num_images_per_prompt (int, defaults to 1) — The number of images to generate per prompt.
• eta (float, defaults to 0.0) — Corresponds to parameter eta (η) in the DDIM paper: https://arxiv.org/abs/2010.02502. Only applies to schedulers.DDIMScheduler, will be ignored for others.
• generator (Optional[Union[torch.Generator, List[torch.Generator]]], defaults to None) — One or a list of torch generator(s) to make generation deterministic.
• latents (Optional[torch.FloatTensor], defaults to None) — Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random generator.
• output_type (Optional[str], defaults to "pil") — The output format of the generate image. Choose between PIL: PIL.Image.Image or np.array.
• return_dict (bool, defaults to True) — Whether or not to return a ~pipelines.stable_diffusion.StableDiffusionPipelineOutput instead of a plain tuple.
• cross_attention_kwargs (Optional[Dict[str, Any]], defaults to None) — A kwargs dictionary that if specified is passed along to the AttentionProcessor as defined under self.processor in diffusers.models.attention_processor.
• original_size (Tuple[int], defaults to (1024, 1024)) — If original_size is not the same as target_size the image will appear to be down- or upsampled. original_size defaults to (height, width) if not specified. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• crops_coords_top_left (Tuple[int], defaults to (0, 0)) — crops_coords_top_left can be used to generate an image that appears to be “cropped” from the position crops_coords_top_left downwards. Favorable, well-centered images are usually achieved by setting crops_coords_top_left to (0, 0). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• target_size (Tuple[int], defaults to (1024, 1024)) — For most cases, target_size should be set to the desired height and width of the generated image. If not specified it will default to (height, width). Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• negative_original_size (Tuple[int], defaults to (1024, 1024)) — To negatively condition the generation process based on a specific image resolution. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• negative_crops_coords_top_left (Tuple[int], defaults to (0, 0)) — To negatively condition the generation process based on a specific crop coordinates. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• negative_target_size (Tuple[int], defaults to (1024, 1024)) — To negatively condition the generation process based on a target image resolution. It should be as same as the target_size for most cases. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
• aesthetic_score (float, defaults to 6.0) — Used to simulate an aesthetic score of the generated image by influencing the positive text condition. Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952.
• negative_aesthetic_score (float, defaults to 2.5) — Part of SDXL’s micro-conditioning as explained in section 2.2 of https://huggingface.co/papers/2307.01952. Can be used to simulate an aesthetic score of the generated image by influencing the negative text condition.
• clip_skip (Optional[int], defaults to None) — Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings.
• callback_on_step_end (Optional[Callable[[int, int, Dict], None]], defaults to None) — A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict). callback_kwargs will include a list of all tensors as specified by callback_on_step_end_tensor_inputs.
• callback_on_step_end_tensor_inputs (List[str], defaults to [“latents”]) — The list of tensor inputs for the callback_on_step_end function. The tensors specified in the list will be passed as callback_kwargs argument. You will only be able to include variables listed in the ._callback_tensor_inputs attribute of your pipeline class.

diffusers.pipelines.stable_diffusion.StableDiffusionXLPipelineOutput if return_dict is True, otherwise a tuple. tuple. When returning a tuple, the first element is a list with the generated images.