Add missing metadata: library_name and pipeline_tag

README.md

@@ -1,3 +1,92 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+library_name: diffusers
+pipeline_tag: text-to-image
+---
+
+# URAE: ~~Your Free FLUX Pro Ultra~~
+
+<img src='teaser.jpg' width='100%' />
+<br>
+
+<a href="https://arxiv.org/abs/2503.16322"><img src="https://img.shields.io/badge/arXiv-2503.16322-A42C25.svg" alt="arXiv"></a> 
+<a href="https://huggingface.co/Huage001/URAE"><img src="https://img.shields.io/badge/🤗_HuggingFace-Model-ffbd45.svg" alt="HuggingFace"></a>
+<a href="https://huggingface.co/spaces/Yuanshi/URAE"><img src="https://img.shields.io/badge/🤗_HuggingFace-Space-ffbd45.svg" alt="HuggingFace"></a>
+<a href="https://huggingface.co/spaces/Yuanshi/URAE_dev"><img src="https://img.shields.io/badge/🤗_HuggingFace-Space-ffbd45.svg" alt="HuggingFace"></a>
+
+> ***U*ltra-*R*esolution *A*daptation with *E*ase**
+> <br>
+> [Ruonan Yu*](https://scholar.google.com/citations?user=UHP95egAAAAJ&hl=en), 
+> [Songhua Liu*](http://121.37.94.87/), 
+> [Zhenxiong Tan](https://scholar.google.com/citations?user=HP9Be6UAAAAJ&hl=en), 
+> and 
+> [Xinchao Wang](https://sites.google.com/site/sitexinchaowang/)
+> <br>
+> [xML Lab](https://sites.google.com/view/xml-nus), National University of Singapore
+> <br>
+
+## 🪶Features
+
+* **Easy-to-Use High-Quality and High-Resolution Generation😊**: Ultra-Resolution Adaptation with Ease, or URAE in short, generates high-resolution images with FLUX, with minimal code modifications.
+* **Easy Training🚀**: URAE tames light-weight adapters with a handful of synthetic data from [FLUX1.1 Pro Ultra](https://blackforestlabs.ai/ultra-home/).
+
+## 🔥News
+
+**[2025/03/20]** We release models and codes for both training and inference of URAE.
+
+## Introduction
+
+Text-to-image diffusion models have achieved remarkable progress in recent years. However, training models for high-resolution image generation remains challenging, particularly when training data and computational resources are limited. In this paper, we explore this practical problem from two key perspectives: data and parameter efficiency, and propose a set of key guidelines for ultra-resolution adaptation termed *URAE*. For data efficiency, we theoretically and empirically demonstrate that synthetic data generated by some teacher models can significantly promote training convergence. For parameter efficiency, we find that tuning minor components of the weight matrices outperforms widely-used low-rank adapters when synthetic data are unavailable, offering substantial performance gains while maintaining efficiency. Additionally, for models leveraging guidance distillation, such as FLUX, we show that disabling classifier-free guidance, *i.e.*, setting the guidance scale to 1 during adaptation, is crucial for satisfactory performance. Extensive experiments validate that URAE achieves comparable 2K-generation performance to state-of-the-art closed-source models like FLUX1.1 [Pro] Ultra with only 3K samples and 2K iterations, while setting new benchmarks for 4K-resolution generation.
+
+## Quick Start
+
+* If you have not, install [PyTorch](https://pytorch.org/get-started/locally/), [diffusers](https://huggingface.co/docs/diffusers/index), [transformers](https://huggingface.co/docs/transformers/index), and [peft](https://huggingface.co/docs/peft/index).
+
+* Clone this repo to your project directory:
+
+  ``` bash
+  git clone https://github.com/Huage001/URAE.git
+  cd URAE
+  ```
+
+* **You only need minimal modifications!**
+
+  ```diff
+  import torch
+  - from diffusers import FluxPipeline
+  + from pipeline_flux import FluxPipeline
+  + from transformer_flux import FluxTransformer2DModel
+  
+  bfl_repo = "black-forest-labs/FLUX.1-dev"
+  + transformer = FluxTransformer2DModel.from_pretrained(bfl_repo, subfolder="transformer", torch_dtype=torch.bfloat16)
+  - pipe = FluxPipeline.from_pretrained(bfl_repo, torch_dtype=torch.bfloat16)
+  + pipe = FluxPipeline.from_pretrained(bfl_repo, transformer=transformer, torch_dtype=torch.bfloat16)
+  pipe.enable_model_cpu_offload() #save some VRAM by offloading the model to CPU. Remove this if you have enough GPU power
+  
+  + pipe.load_lora_weights("Huage001/URAE", weight_name="urae_2k_adapter.safetensors")
+  
+  prompt = "An astronaut riding a green horse"
+  image = pipe(
+      prompt,
+  -    height=1024,
+  -    width=1024,
+  +    height=2048,
+  +    width=2048,
+      guidance_scale=3.5,
+      num_inference_steps=50,
+      max_sequence_length=512,
+      generator=torch.Generator("cpu").manual_seed(0)
+  ).images[0]
+  image.save("flux-urae.png")
+  ```
+  ⚠️ **FLUX requires at least 28GB of GPU memory to operate at a 2K resolution.** A 48GB GPU is recommended for the full functionalities of URAE, including both 2K and 4K. We are actively integrating model lightweighting strategies into URAE! If you have a good idea, feel free to submit a PR!
+
+
+* Do not want to run the codes? No worry! Try the model on Huggingface Space!
+  
+  * [URAE w. FLUX1.schnell](https://huggingface.co/spaces/Yuanshi/URAE) (Faster)
+  * [URAE w. FLUX1.dev](https://huggingface.co/spaces/Yuanshi/URAE_dev) (Higher Quality)
+
+## Installation
+
+* Clone this repo to your project directory: