270 11 110

Sayak Paul

sayakpaul

https://sayak.dev

RisingSayak

sayakpaul

AI & ML interests

Diffusion models, representation learning

Articles

🤗 PEFT: Parameter-Efficient Fine-Tuning of Billion-Scale Models on Low-Resource Hardware

Feb 10, 2023

• 17

A Dive into Pretraining Strategies for Vision-Language Models

Feb 3, 2023

• 23

The State of Computer Vision at Hugging Face 🤗

Jan 30, 2023

• 3

Using LoRA for Efficient Stable Diffusion Fine-Tuning

Jan 26, 2023

• 16

Image Similarity with Hugging Face Datasets and Transformers

Jan 16, 2023

• 4

Deploying 🤗 ViT on Kubernetes with TF Serving

Aug 11, 2022

Deploying TensorFlow Vision Models in Hugging Face with TF Serving

Jul 25, 2022

Organizations

Posts 6

Post

1319

🧨 Diffusers 0.28.0 is out 🔥

It features the first non-generative pipeline of the library -- Marigold 🥁

Marigold shines at performing Depth Estimation and Surface Normal Estimation. It was contributed by @toshas , one of the authors of Marigold.

This release also features a massive refactor (led by @DN6 ) of the from_single_file() method, highlighting our efforts for making our library more amenable to community features 🤗

Check out the release notes here:
https://github.com/huggingface/diffusers/releases/tag/v0.28.0

Post

1678

Custom pipelines and components in Diffusers 🎸

Wanted to use customized pipelines and other components (schedulers, unets, text encoders, etc.) in Diffusers?

Found it inflexible?

Since the first dawn on earth, we have supported loading custom pipelines via a custom_pipeline argument 🌄

These pipelines are inference-only, i.e., the assumption is that we're leveraging an existing checkpoint (e.g., runwayml/stable-diffusion-v1-5) and ONLY modifying the pipeline implementation.

We have many cool pipelines, implemented that way. They all share the same benefits available to a DiffusionPipeline, no compromise there 🤗

Check them here:
https://github.com/huggingface/diffusers/tree/main/examples/community

Then we might have a requirement of everything customized i.e., custom components along with a custom pipeline. Sure, that's all possible.

All you have to do is keep the implementations of those custom components on the Hub repository you're loading your pipeline checkpoint from.

SDXL Japanese was implemented like this 🔥
stabilityai/japanese-stable-diffusion-xl

Full guide is available here ⬇️
https://huggingface.co/docs/diffusers/main/en/using-diffusers/custom_pipeline_overview

And, of course, these share all the benefits that come with DiffusionPipeline.

View all posts