Diffusers

🤗 Diffusers は、画像や音声、さらには分子の3D構造を生成するための、最先端の事前学習済みDiffusion Model(拡散モデル)を提供するライブラリです。シンプルな生成ソリューションをお探しの場合でも、独自の拡散モデルをトレーニングしたい場合でも、🤗 Diffusers はその両方をサポートするモジュール式のツールボックスです。我々のライブラリは、性能より使いやすさ、簡単よりシンプル、抽象化よりカスタマイズ性に重点を置いて設計されています。

このライブラリには3つの主要コンポーネントがあります:

• 最先端の拡散パイプラインで数行のコードで生成が可能です。
• 交換可能なノイズスケジューラで生成速度と品質のトレードオフのバランスをとれます。
• 事前に訓練されたモデルは、ビルディングブロックとして使用することができ、スケジューラと組み合わせることで、独自のエンドツーエンドの拡散システムを作成することができます。

Supported pipelines

Pipeline	Paper/Repository	Tasks
alt_diffusion	AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities	Image-to-Image Text-Guided Generation
audio_diffusion	Audio Diffusion	Unconditional Audio Generation
controlnet	Adding Conditional Control to Text-to-Image Diffusion Models	Image-to-Image Text-Guided Generation
cycle_diffusion	Unifying Diffusion Models’ Latent Space, with Applications to CycleDiffusion and Guidance	Image-to-Image Text-Guided Generation
dance_diffusion	Dance Diffusion	Unconditional Audio Generation
ddpm	Denoising Diffusion Probabilistic Models	Unconditional Image Generation
ddim	Denoising Diffusion Implicit Models	Unconditional Image Generation
if	IF	Image Generation
if_img2img	IF	Image-to-Image Generation
if_inpainting	IF	Image-to-Image Generation
latent_diffusion	High-Resolution Image Synthesis with Latent Diffusion Models	Text-to-Image Generation
latent_diffusion	High-Resolution Image Synthesis with Latent Diffusion Models	Super Resolution Image-to-Image
latent_diffusion_uncond	High-Resolution Image Synthesis with Latent Diffusion Models	Unconditional Image Generation
paint_by_example	Paint by Example: Exemplar-based Image Editing with Diffusion Models	Image-Guided Image Inpainting
pndm	Pseudo Numerical Methods for Diffusion Models on Manifolds	Unconditional Image Generation
score_sde_ve	Score-Based Generative Modeling through Stochastic Differential Equations	Unconditional Image Generation
score_sde_vp	Score-Based Generative Modeling through Stochastic Differential Equations	Unconditional Image Generation
semantic_stable_diffusion	Semantic Guidance	Text-Guided Generation
stable_diffusion_adapter	T2I-Adapter	Image-to-Image Text-Guided Generation
stable_diffusion_text2img	Stable Diffusion	Text-to-Image Generation
stable_diffusion_img2img	Stable Diffusion	Image-to-Image Text-Guided Generation
stable_diffusion_inpaint	Stable Diffusion	Text-Guided Image Inpainting
stable_diffusion_panorama	MultiDiffusion	Text-to-Panorama Generation
stable_diffusion_pix2pix	InstructPix2Pix: Learning to Follow Image Editing Instructions	Text-Guided Image Editing
stable_diffusion_pix2pix_zero	Zero-shot Image-to-Image Translation	Text-Guided Image Editing
stable_diffusion_attend_and_excite	Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models	Text-to-Image Generation
stable_diffusion_self_attention_guidance	Improving Sample Quality of Diffusion Models Using Self-Attention Guidance	Text-to-Image Generation Unconditional Image Generation
stable_diffusion_image_variation	Stable Diffusion Image Variations	Image-to-Image Generation
stable_diffusion_latent_upscale	Stable Diffusion Latent Upscaler	Text-Guided Super Resolution Image-to-Image
stable_diffusion_model_editing	Editing Implicit Assumptions in Text-to-Image Diffusion Models	Text-to-Image Model Editing
stable_diffusion_2	Stable Diffusion 2	Text-to-Image Generation
stable_diffusion_2	Stable Diffusion 2	Text-Guided Image Inpainting
stable_diffusion_2	Depth-Conditional Stable Diffusion	Depth-to-Image Generation
stable_diffusion_2	Stable Diffusion 2	Text-Guided Super Resolution Image-to-Image
stable_diffusion_safe	Safe Stable Diffusion	Text-Guided Generation
stable_unclip	Stable unCLIP	Text-to-Image Generation
stable_unclip	Stable unCLIP	Image-to-Image Text-Guided Generation
stochastic_karras_ve	Elucidating the Design Space of Diffusion-Based Generative Models	Unconditional Image Generation
text_to_video_sd	Modelscope’s Text-to-video-synthesis Model in Open Domain	Text-to-Video Generation
unclip	Hierarchical Text-Conditional Image Generation with CLIP Latents(implementation by kakaobrain)	Text-to-Image Generation
versatile_diffusion	Versatile Diffusion: Text, Images and Variations All in One Diffusion Model	Text-to-Image Generation
versatile_diffusion	Versatile Diffusion: Text, Images and Variations All in One Diffusion Model	Image Variations Generation
versatile_diffusion	Versatile Diffusion: Text, Images and Variations All in One Diffusion Model	Dual Image and Text Guided Generation
vq_diffusion	Vector Quantized Diffusion Model for Text-to-Image Synthesis	Text-to-Image Generation
stable_diffusion_ldm3d	LDM3D: Latent Diffusion Model for 3D	Text to Image and Depth Generation