Hugging Face's logo

Diffusers documentation

Loading Pipelines and Models via from_single_file

You are viewing v0.28.2 version. A newer version v0.31.0 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Loading Pipelines and Models via from_single_file

The from_single_file method allows you to load supported pipelines using a single checkpoint file as opposed to Diffusers’ multiple folders format. This is useful if you are working with Stable Diffusion Web UI’s (such as A1111) that rely on a single file format to distribute all the components of a model.

The from_single_file method also supports loading models in their originally distributed format. This means that supported models that have been finetuned with other services can be loaded directly into Diffusers model objects and pipelines.

Pipelines that currently support from_single_file loading

Models that currently support from_single_file loading

Usage Examples

Loading a Pipeline using from_single_file

from diffusers import StableDiffusionXLPipeline

ckpt_path = "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0_0.9vae.safetensors"
pipe = StableDiffusionXLPipeline.from_single_file(ckpt_path)

Setting components in a Pipeline using from_single_file

Set components of a pipeline by passing them directly to the from_single_file method. For example, here we are swapping out the pipeline’s default scheduler with the DDIMScheduler.

from diffusers import StableDiffusionXLPipeline, DDIMScheduler

ckpt_path = "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0_0.9vae.safetensors"

scheduler = DDIMScheduler()
pipe = StableDiffusionXLPipeline.from_single_file(ckpt_path, scheduler=scheduler)

Here we are passing in a ControlNet model to the StableDiffusionControlNetPipeline.

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel

ckpt_path = "https://huggingface.co/runwayml/stable-diffusion-v1-5/blob/main/v1-5-pruned-emaonly.safetensors"

controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_canny")
pipe = StableDiffusionControlNetPipeline.from_single_file(ckpt_path, controlnet=controlnet)

Loading a Model using from_single_file

from diffusers import StableCascadeUNet

ckpt_path = "https://huggingface.co/stabilityai/stable-cascade/blob/main/stage_b_lite.safetensors"
model = StableCascadeUNet.from_single_file(ckpt_path)

Using a Diffusers model repository to configure single file loading

Under the hood, from_single_file will try to automatically determine a model repository to use to configure the components of a pipeline. You can also explicitly set the model repository to configure the pipeline with the config argument.

from diffusers import StableDiffusionXLPipeline

ckpt_path = "https://huggingface.co/segmind/SSD-1B/blob/main/SSD-1B.safetensors"
repo_id = "segmind/SSD-1B"

pipe = StableDiffusionXLPipeline.from_single_file(ckpt_path, config=repo_id)

In the example above, since we explicitly passed repo_id="segmind/SSD-1B" to the config argument, it will use this configuration file from the unet subfolder in "segmind/SSD-1B" to configure the unet component of the pipeline; Similarly, it will use the config.json file from vae subfolder to configure the vae model, config.json file from text_encoder folder to configure text_encoder and so on.

Most of the time you do not need to explicitly set a config argument. from_single_file will automatically map the checkpoint to the appropriate model repository. However, this option can be useful in cases where model components in the checkpoint might have been changed from what was originally distributed, or in cases where a checkpoint file might not have the necessary metadata to correctly determine the configuration to use for the pipeline.

Override configuration options when using single file loading

Override the default model or pipeline configuration options by providing the relevant arguments directly to the from_single_file method. Any argument supported by the model or pipeline class can be configured in this way:

Setting a pipeline configuration option

from diffusers import StableDiffusionXLInstructPix2PixPipeline

ckpt_path = "https://huggingface.co/stabilityai/cosxl/blob/main/cosxl_edit.safetensors"
pipe = StableDiffusionXLInstructPix2PixPipeline.from_single_file(ckpt_path, config="diffusers/sdxl-instructpix2pix-768", is_cosxl_edit=True)

Setting a model configuration option

from diffusers import UNet2DConditionModel

ckpt_path = "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0_0.9vae.safetensors"
model = UNet2DConditionModel.from_single_file(ckpt_path, upcast_attention=True)

To learn more about how to load single file weights, see the Load different Stable Diffusion formats loading guide.

Working with local files

As of diffusers>=0.28.0 the from_single_file method will attempt to configure a pipeline or model by first inferring the model type from the keys in the checkpoint file. This inferred model type is then used to determine the appropriate model repository on the Hugging Face Hub to configure the model or pipeline.

For example, any single file checkpoint based on the Stable Diffusion XL base model will use the stabilityai/stable-diffusion-xl-base-1.0 model repository to configure the pipeline.

If you are working in an environment with restricted internet access, it is recommended that you download the config files and checkpoints for the model to your preferred directory and pass the local paths to the pretrained_model_link_or_path and config arguments of the from_single_file method.

from huggingface_hub import hf_hub_download, snapshot_download

my_local_checkpoint_path = hf_hub_download(
    repo_id="segmind/SSD-1B",
    filename="SSD-1B.safetensors"
)

my_local_config_path = snapshot_download(
    repo_id="segmind/SSD-1B",
    allowed_patterns=["*.json", "**/*.json", "*.txt", "**/*.txt"]
)

pipe = StableDiffusionXLPipeline.from_single_file(my_local_checkpoint_path, config=my_local_config_path, local_files_only=True)

By default this will download the checkpoints and config files to the Hugging Face Hub cache directory. You can also specify a local directory to download the files to by passing the local_dir argument to the hf_hub_download and snapshot_download functions.

from huggingface_hub import hf_hub_download, snapshot_download

my_local_checkpoint_path = hf_hub_download(
    repo_id="segmind/SSD-1B",
    filename="SSD-1B.safetensors"
    local_dir="my_local_checkpoints"
)

my_local_config_path = snapshot_download(
    repo_id="segmind/SSD-1B",
    allowed_patterns=["*.json", "**/*.json", "*.txt", "**/*.txt"]
    local_dir="my_local_config"
)

pipe = StableDiffusionXLPipeline.from_single_file(my_local_checkpoint_path, config=my_local_config_path, local_files_only=True)

Working with local files on file systems that do not support symlinking

By default the from_single_file method relies on the huggingface_hub caching mechanism to fetch and store checkpoints and config files for models and pipelines. If you are working with a file system that does not support symlinking, it is recommended that you first download the checkpoint file to a local directory and disable symlinking by passing the local_dir_use_symlink=False argument to the hf_hub_download and snapshot_download functions.

from huggingface_hub import hf_hub_download, snapshot_download

my_local_checkpoint_path = hf_hub_download(
    repo_id="segmind/SSD-1B",
    filename="SSD-1B.safetensors"
    local_dir="my_local_checkpoints",
    local_dir_use_symlinks=False
)
print("My local checkpoint: ", my_local_checkpoint_path)

my_local_config_path = snapshot_download(
    repo_id="segmind/SSD-1B",
    allowed_patterns=["*.json", "**/*.json", "*.txt", "**/*.txt"]
    local_dir_use_symlinks=False,
)
print("My local config: ", my_local_config_path)

Then pass the local paths to the pretrained_model_link_or_path and config arguments of the from_single_file method.

pipe = StableDiffusionXLPipeline.from_single_file(my_local_checkpoint_path, config=my_local_config_path, local_files_only=True)

As of huggingface_hub>=0.23.0 the local_dir_use_symlinks argument isn’t necessary for the hf_hub_download and snapshot_download functions.

Using the original configuration file of a model

If you would like to configure the model components in a pipeline using the orignal YAML configuration file, you can pass a local path or url to the original configuration file via the original_config argument.

from diffusers import StableDiffusionXLPipeline

ckpt_path = "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0_0.9vae.safetensors"
repo_id = "stabilityai/stable-diffusion-xl-base-1.0"
original_config = "https://raw.githubusercontent.com/Stability-AI/generative-models/main/configs/inference/sd_xl_base.yaml"

pipe = StableDiffusionXLPipeline.from_single_file(ckpt_path, original_config=original_config)

When using original_config with local_files_only=True, Diffusers will attempt to infer the components of the pipeline based on the type signatures of pipeline class, rather than attempting to fetch the configuration files from a model repository on the Hugging Face Hub. This is to prevent backward breaking changes in existing code that might not be able to connect to the internet to fetch the necessary configuration files.

This is not as reliable as providing a path to a local model repository using the config argument and might lead to errors when configuring the pipeline. To avoid this, please run the pipeline with local_files_only=False once to download the appropriate pipeline configuration files to the local cache.

FromSingleFileMixin

class diffusers.loaders.FromSingleFileMixin

< >

( )

Load model weights saved in the .ckpt format into a DiffusionPipeline.

from_single_file

< >

( pretrained_model_link_or_path **kwargs )

Parameters

  • pretrained_model_link_or_path (str or os.PathLike, optional) — Can be either:
    • A link to the .ckpt file (for example "https://huggingface.co/<repo_id>/blob/main/<path_to_file>.ckpt") on the Hub.
    • A path to a file containing all pipeline weights.
  • torch_dtype (str or torch.dtype, optional) — Override the default torch.dtype and load the model with another dtype.
  • force_download (bool, optional, defaults to False) — Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist.
  • cache_dir (Union[str, os.PathLike], optional) — Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. resume_download — Deprecated and ignored. All downloads are now resumed by default when possible. Will be removed in v1 of Diffusers.
  • proxies (Dict[str, str], optional) — A dictionary of proxy servers to use by protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.
  • local_files_only (bool, optional, defaults to False) — Whether to only load local model weights and configuration files or not. If set to True, the model won’t be downloaded from the Hub.
  • token (str or bool, optional) — The token to use as HTTP bearer authorization for remote files. If True, the token generated from diffusers-cli login (stored in ~/.huggingface) is used.
  • revision (str, optional, defaults to "main") — The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git.
  • original_config_file (str, optional) — The path to the original config file that was used to train the model. If not provided, the config file will be inferred from the checkpoint file.
  • config (str, optional) — Can be either:
    • A string, the repo id (for example CompVis/ldm-text2im-large-256) of a pretrained pipeline hosted on the Hub.
    • A path to a directory (for example ./my_pipeline_directory/) containing the pipeline component configs in Diffusers format.
  • kwargs (remaining dictionary of keyword arguments, optional) — Can be used to overwrite load and saveable variables (the pipeline components of the specific pipeline class). The overwritten components are passed directly to the pipelines __init__ method. See example below for more information.

Instantiate a DiffusionPipeline from pretrained pipeline weights saved in the .ckpt or .safetensors format. The pipeline is set in evaluation mode (model.eval()) by default.

Examples:

>>> from diffusers import StableDiffusionPipeline

>>> # Download pipeline from huggingface.co and cache.
>>> pipeline = StableDiffusionPipeline.from_single_file(
...     "https://huggingface.co/WarriorMama777/OrangeMixs/blob/main/Models/AbyssOrangeMix/AbyssOrangeMix.safetensors"
... )

>>> # Download pipeline from local file
>>> # file is downloaded under ./v1-5-pruned-emaonly.ckpt
>>> pipeline = StableDiffusionPipeline.from_single_file("./v1-5-pruned-emaonly.ckpt")

>>> # Enable float16 and move to GPU
>>> pipeline = StableDiffusionPipeline.from_single_file(
...     "https://huggingface.co/runwayml/stable-diffusion-v1-5/blob/main/v1-5-pruned-emaonly.ckpt",
...     torch_dtype=torch.float16,
... )
>>> pipeline.to("cuda")

FromOriginalModelMixin

class diffusers.loaders.FromOriginalModelMixin

< >

( )

Load pretrained weights saved in the .ckpt or .safetensors format into a model.

from_single_file

< >

( pretrained_model_link_or_path_or_dict: Optional = None **kwargs )

Parameters

  • pretrained_model_link_or_path_or_dict (str, optional) — Can be either:
    • A link to the .safetensors or .ckpt file (for example "https://huggingface.co/<repo_id>/blob/main/<path_to_file>.safetensors") on the Hub.
    • A path to a local file containing the weights of the component model.
    • A state dict containing the component model weights.
  • config (str, optional) —
    • A string, the repo id (for example CompVis/ldm-text2im-large-256) of a pretrained pipeline hosted on the Hub.
    • A path to a directory (for example ./my_pipeline_directory/) containing the pipeline component configs in Diffusers format.
  • subfolder (str, optional, defaults to "") — The subfolder location of a model file within a larger model repository on the Hub or locally.
  • original_config (str, optional) — Dict or path to a yaml file containing the configuration for the model in its original format. If a dict is provided, it will be used to initialize the model configuration.
  • torch_dtype (str or torch.dtype, optional) — Override the default torch.dtype and load the model with another dtype. If "auto" is passed, the dtype is automatically derived from the model’s weights.
  • force_download (bool, optional, defaults to False) — Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist.
  • cache_dir (Union[str, os.PathLike], optional) — Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used.
  • resume_download (bool, optional, defaults to False) — Whether or not to resume downloading the model weights and configuration files. If set to False, any incompletely downloaded files are deleted.
  • proxies (Dict[str, str], optional) — A dictionary of proxy servers to use by protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.
  • local_files_only (bool, optional, defaults to False) — Whether to only load local model weights and configuration files or not. If set to True, the model won’t be downloaded from the Hub.
  • token (str or bool, optional) — The token to use as HTTP bearer authorization for remote files. If True, the token generated from diffusers-cli login (stored in ~/.huggingface) is used.
  • revision (str, optional, defaults to "main") — The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git.
  • kwargs (remaining dictionary of keyword arguments, optional) — Can be used to overwrite load and saveable variables (for example the pipeline components of the specific pipeline class). The overwritten components are directly passed to the pipelines __init__ method. See example below for more information.

Instantiate a model from pretrained weights saved in the original .ckpt or .safetensors format. The model is set in evaluation mode (model.eval()) by default.

>>> from diffusers import StableCascadeUNet

>>> ckpt_path = "https://huggingface.co/stabilityai/stable-cascade/blob/main/stage_b_lite.safetensors"
>>> model = StableCascadeUNet.from_single_file(ckpt_path)
< > Update on GitHub

←LoRA Textual Inversion→