Diffusers documentation
IP-Adapter
You are viewing v0.25.0 version.
A newer version
v0.35.1 is available.
IP-Adapter
IP-Adapter is a lightweight adapter that enables prompting a diffusion model with an image. This method decouples the cross-attention layers of the image and text features. The image features are generated from an image encoder. Files generated from IP-Adapter are only ~100MBs.
Learn how to load an IP-Adapter checkpoint and image in the IP-Adapter loading guide.
IPAdapterMixin
Mixin for handling IP Adapters.
load_ip_adapter
< source >( pretrained_model_name_or_path_or_dict: Union subfolder: str weight_name: str **kwargs )
Parameters
- pretrained_model_name_or_path_or_dict (
stroros.PathLikeordict) — Can be either:- A string, the model id (for example
google/ddpm-celebahq-256) of a pretrained model hosted on the Hub. - A path to a directory (for example
./my_model_directory) containing the model weights saved with ModelMixin.save_pretrained(). - A torch state dict.
- A string, the model id (for example
- cache_dir (
Union[str, os.PathLike], optional) — Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. - force_download (
bool, optional, defaults toFalse) — Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. - resume_download (
bool, optional, defaults toFalse) — Whether or not to resume downloading the model weights and configuration files. If set toFalse, any incompletely downloaded files are deleted. - proxies (
Dict[str, str], optional) — A dictionary of proxy servers to use by protocol or endpoint, for example,{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request. - local_files_only (
bool, optional, defaults toFalse) — Whether to only load local model weights and configuration files or not. If set toTrue, the model won’t be downloaded from the Hub. - token (
stror bool, optional) — The token to use as HTTP bearer authorization for remote files. IfTrue, the token generated fromdiffusers-cli login(stored in~/.huggingface) is used. - revision (
str, optional, defaults to"main") — The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. - subfolder (
str, optional, defaults to"") — The subfolder location of a model file within a larger model repository on the Hub or locally.