🧨 Diffusers

🤗 Diffusers 是一个值得首选用于生成图像、音频甚至 3D 分子结构的，最先进的预训练扩散模型库。无论您是在寻找简单的推理解决方案，还是想训练自己的扩散模型，🤗 Diffusers 这一模块化工具箱都能对其提供支持。本库的设计更偏重于可用而非高性能、简明而非简单以及易用而非抽象。

本库包含三个主要组件：

最先进的扩散管道 diffusion pipelines，只需几行代码即可进行推理。
可交替使用的各种噪声调度器 noise schedulers，用于平衡生成速度和质量。
预训练模型 models，可作为构建模块，并与调度程序结合使用，来创建您自己的端到端扩散系统。

Tutorials

Learn the fundamental skills you need to start generating outputs, build your own diffusion system, and train a diffusion model. We recommend starting here if you're using 🤗 Diffusers for the first time!

How-to guides

Practical guides for helping you load pipelines, models, and schedulers. You'll also learn how to use pipelines for specific tasks, control how outputs are generated, optimize for inference speed, and different training techniques.

Conceptual guides

Understand why the library was designed the way it was, and learn more about the ethical guidelines and safety implementations for using the library.

Reference

Technical descriptions of how 🤗 Diffusers classes and methods work.

🧨 Diffusers pipelines

下表汇总了当前所有官方支持的pipelines及其对应的论文.

管道	论文/仓库	任务
alt_diffusion	AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities	Image-to-Image Text-Guided Generation
audio_diffusion	Audio Diffusion	Unconditional Audio Generation
controlnet	Adding Conditional Control to Text-to-Image Diffusion Models	Image-to-Image Text-Guided Generation
cycle_diffusion	Unifying Diffusion Models’ Latent Space, with Applications to CycleDiffusion and Guidance	Image-to-Image Text-Guided Generation
dance_diffusion	Dance Diffusion	Unconditional Audio Generation
ddpm	Denoising Diffusion Probabilistic Models	Unconditional Image Generation
ddim	Denoising Diffusion Implicit Models	Unconditional Image Generation
if	IF	Image Generation
if_img2img	IF	Image-to-Image Generation
if_inpainting	IF	Image-to-Image Generation
latent_diffusion	High-Resolution Image Synthesis with Latent Diffusion Models	Text-to-Image Generation
latent_diffusion	High-Resolution Image Synthesis with Latent Diffusion Models	Super Resolution Image-to-Image
latent_diffusion_uncond	High-Resolution Image Synthesis with Latent Diffusion Models	Unconditional Image Generation
paint_by_example	Paint by Example: Exemplar-based Image Editing with Diffusion Models	Image-Guided Image Inpainting
pndm	Pseudo Numerical Methods for Diffusion Models on Manifolds	Unconditional Image Generation
score_sde_ve	Score-Based Generative Modeling through Stochastic Differential Equations	Unconditional Image Generation
score_sde_vp	Score-Based Generative Modeling through Stochastic Differential Equations	Unconditional Image Generation
semantic_stable_diffusion	Semantic Guidance	Text-Guided Generation
stable_diffusion_text2img	Stable Diffusion	Text-to-Image Generation
stable_diffusion_img2img	Stable Diffusion	Image-to-Image Text-Guided Generation
stable_diffusion_inpaint	Stable Diffusion	Text-Guided Image Inpainting
stable_diffusion_panorama	MultiDiffusion	Text-to-Panorama Generation
stable_diffusion_pix2pix	InstructPix2Pix: Learning to Follow Image Editing Instructions	Text-Guided Image Editing
stable_diffusion_pix2pix_zero	Zero-shot Image-to-Image Translation	Text-Guided Image Editing
stable_diffusion_attend_and_excite	Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models	Text-to-Image Generation
stable_diffusion_self_attention_guidance	Improving Sample Quality of Diffusion Models Using Self-Attention Guidance	Text-to-Image Generation Unconditional Image Generation
stable_diffusion_image_variation	Stable Diffusion Image Variations	Image-to-Image Generation
stable_diffusion_latent_upscale	Stable Diffusion Latent Upscaler	Text-Guided Super Resolution Image-to-Image
stable_diffusion_model_editing	Editing Implicit Assumptions in Text-to-Image Diffusion Models	Text-to-Image Model Editing
stable_diffusion_2	Stable Diffusion 2	Text-to-Image Generation
stable_diffusion_2	Stable Diffusion 2	Text-Guided Image Inpainting
stable_diffusion_2	Depth-Conditional Stable Diffusion	Depth-to-Image Generation
stable_diffusion_2	Stable Diffusion 2	Text-Guided Super Resolution Image-to-Image
stable_diffusion_safe	Safe Stable Diffusion	Text-Guided Generation
stable_unclip	Stable unCLIP	Text-to-Image Generation
stable_unclip	Stable unCLIP	Image-to-Image Text-Guided Generation
stochastic_karras_ve	Elucidating the Design Space of Diffusion-Based Generative Models	Unconditional Image Generation
text_to_video_sd	Modelscope’s Text-to-video-synthesis Model in Open Domain	Text-to-Video Generation
unclip	Hierarchical Text-Conditional Image Generation with CLIP Latents(implementation by kakaobrain)	Text-to-Image Generation
versatile_diffusion	Versatile Diffusion: Text, Images and Variations All in One Diffusion Model	Text-to-Image Generation
versatile_diffusion	Versatile Diffusion: Text, Images and Variations All in One Diffusion Model	Image Variations Generation
versatile_diffusion	Versatile Diffusion: Text, Images and Variations All in One Diffusion Model	Dual Image and Text Guided Generation
vq_diffusion	Vector Quantized Diffusion Model for Text-to-Image Synthesis	Text-to-Image Generation