Diffusers

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.39.0).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

JoyImageEditPlusTransformer3DModel

The model can be loaded with the following code snippet.

from diffusers import JoyImageEditPlusTransformer3DModel

transformer = JoyImageEditPlusTransformer3DModel.from_pretrained("jdopensource/JoyAI-Image-Edit-Plus-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16)

JoyImageEditPlusTransformer3DModel

class diffusers.JoyImageEditPlusTransformer3DModel

< source >

( patch_size: list = [1, 2, 2]in_channels: int = 16out_channels: int | None = Nonehidden_size: int = 3072num_attention_heads: int = 24text_dim: int = 4096mlp_width_ratio: float = 4.0num_layers: int = 20rope_dim_list: list = [16, 56, 56]rope_type: str = 'rope'theta: int = 256 )

Parameters

patch_size (list, defaults to [1, 2, 2]) — Patch size for patchifying the latent input along (t, h, w) dimensions.
in_channels (int, defaults to 16) — The number of channels in the input latent.
out_channels (int, optional, defaults to None) — The number of channels in the output. If not specified, it defaults to in_channels.
hidden_size (int, defaults to 3072) — The dimensionality of the hidden representations.
num_attention_heads (int, defaults to 24) — The number of attention heads.
text_dim (int, defaults to 4096) — The dimensionality of the text encoder output.
mlp_width_ratio (float, defaults to 4.0) — The ratio of MLP hidden dimension to hidden_size.
num_layers (int, defaults to 20) — The number of double-stream transformer blocks.
rope_dim_list (list[int], defaults to [16, 56, 56]) — The dimensions for 3D rotary positional embeddings along (t, h, w).
rope_type (str, defaults to "rope") — The type of rotary positional embedding.
theta (int, defaults to 256) — The base frequency for rotary embeddings.

JoyImage Edit Plus Transformer for multi-image editing.

Uses a patchify+padding approach where each reference image and the target noise are independently patchified and concatenated into a flat patch sequence. Supports variable-resolution reference images.

Input format: [B, max_patches, C, pt, ph, pw] (6D padded patches).

forward

< source >

( hidden_states: Tensortimestep: Tensorencoder_hidden_states: Tensorencoder_hidden_states_mask: typing.Optional[torch.Tensor] = Noneshape_list: list = Nonereturn_dict: bool = True )

Parameters

hidden_states — [B, max_patches, C, pt, ph, pw] - patchified latent input.
timestep — [B] - diffusion timestep.
encoder_hidden_states — [B, L, D] - text encoder outputs.
encoder_hidden_states_mask — [B, L] - attention mask for text tokens.
shape_list — Per-sample list of (t, h, w) tuples for each component (target + references).
return_dict — Whether to return a dict or tuple.

Transformer2DModelOutput

class diffusers.models.modeling_outputs.Transformer2DModelOutput

< source >

( sample: torch.Tensor )

Parameters

sample (torch.Tensor of shape (batch_size, num_channels, height, width) or (batch size, num_vector_embeds - 1, num_latent_pixels) if Transformer2DModel is discrete) — The hidden states output conditioned on the encoder_hidden_states input. If discrete, returns probability distributions for the unnoised latent pixels.

The output of Transformer2DModel.

Update on GitHub

←Ideogram4Transformer2DModel JoyImageEditTransformer3DModel→