261 11 107

Sayak Paul

sayakpaul

https://sayak.dev

RisingSayak

sayakpaul

AI & ML interests

Diffusion models, representation learning

Articles

🤗 PEFT: Parameter-Efficient Fine-Tuning of Billion-Scale Models on Low-Resource Hardware

Feb 10, 2023

• 11

A Dive into Pretraining Strategies for Vision-Language Models

Feb 3, 2023

• 10

The State of Computer Vision at Hugging Face 🤗

Jan 30, 2023

• 3

Using LoRA for Efficient Stable Diffusion Fine-Tuning

Jan 26, 2023

• 7

Image Similarity with Hugging Face Datasets and Transformers

Jan 16, 2023

• 3

Deploying 🤗 ViT on Kubernetes with TF Serving

Aug 11, 2022

Deploying TensorFlow Vision Models in Hugging Face with TF Serving

Jul 25, 2022

Organizations

Posts 5

Post

1434

Custom pipelines and components in Diffusers 🎸

Wanted to use customized pipelines and other components (schedulers, unets, text encoders, etc.) in Diffusers?

Found it inflexible?

Since the first dawn on earth, we have supported loading custom pipelines via a custom_pipeline argument 🌄

These pipelines are inference-only, i.e., the assumption is that we're leveraging an existing checkpoint (e.g., runwayml/stable-diffusion-v1-5) and ONLY modifying the pipeline implementation.

We have many cool pipelines, implemented that way. They all share the same benefits available to a DiffusionPipeline, no compromise there 🤗

Check them here:
https://github.com/huggingface/diffusers/tree/main/examples/community

Then we might have a requirement of everything customized i.e., custom components along with a custom pipeline. Sure, that's all possible.

All you have to do is keep the implementations of those custom components on the Hub repository you're loading your pipeline checkpoint from.

SDXL Japanese was implemented like this 🔥
stabilityai/japanese-stable-diffusion-xl

Full guide is available here ⬇️
https://huggingface.co/docs/diffusers/main/en/using-diffusers/custom_pipeline_overview

And, of course, these share all the benefits that come with DiffusionPipeline.

Post

2649

We're introducing experimental support for device_map in Diffusers 🤗

If you have multiple GPUs you want to use to distribute the pipeline models, you can do so. Additionally, this becomes more useful when you have multiple low-VRAM GPUs.

Documentation:
https://huggingface.co/docs/diffusers/main/en/training/distributed_inference#device-placement

🚨 Currently, only "balanced" device mapping strategy is supported.

View all posts