update doc

README.md

@@ -1,3 +1,101 @@
 ---
-license: creativeml-openrail-m
+license: openrail++
+base_model: stabilityai/stable-diffusion-xl-base-1.0
+language:
+  - en
+tags:
+  - stable-diffusion
+  - stable-diffusion-xl
+  - onnxruntime
+  - onnx
+  - text-to-image
 ---
+
+
+# Stable Diffusion XL 1.0 for ONNX Runtime
+
+## Introduction
+
+This repository hosts the optimized versions of **Stable Diffusion XL 1.0** to accelerate inference with ONNX Runtime CUDA execution provider.
+
+See the [usage instructions](#usage-example) for how to run the SDXL pipeline with the ONNX files hosted in this repository.
+
+## Model Description
+
+- **Developed by:** Stability AI
+- **Model type:** Diffusion-based text-to-image generative model
+- **License:** [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/blob/main/LICENSE.md)
+- **Model Description:** This is a conversion of the [SDXL base 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) and [SDXL refiner 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0) models for [ONNX Runtime](https://github.com/microsoft/onnxruntime) inference with CUDA execution provider.
+
+The VAE decoder is converted from [sdxl-vae-fp16-fix](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix). There are slight discrepancies between its output and that of the original VAE, but the decoded images should be [close enough for most purposes](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7#64c5c0f8e2e5c94bd04eaa80).
+
+## Performance Comparison
+
+#### Latency for 30 steps base and 9 steps refiner
+
+Below is average latency of generating an image of size 1024x1024 using NVIDIA A100-SXM4-80GB GPU:
+
+| Engine      | Batch Size | PyTorch 2.1    | ONNX Runtime CUDA |
+|-------------|------------|----------------|-------------------|
+| Static      | 1          | N/A            | 3389 ms           |
+| Static      | 4          | N/A            | 12264 ms          |
+| Dynamic     | 1          | 3779 ms        | 3458 ms           |
+| Dynamic     | 4          | 13504 ms       | 12347 ms          |
+
+Static means the engine is built for the given batch size and image size combination, and CUDA graph is used to speed up.
+
+Dynamic means the engine is built to support dynamic batch size and image sizes.
+
+## Usage Example
+
+Following the [demo instructions](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/README.md#run-demo-with-docker). Example steps:
+
+0. Install nvidia-docker using these [instructions](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
+
+1. Clone onnxruntime repository.
+```shell
+git clone https://github.com/microsoft/onnxruntime
+cd onnxruntime
+```
+
+2. Download the SDXL ONNX files from this repo
+```shell
+git lfs install
+git clone https://huggingface.co/tlwu/stable-diffusion-xl-1.0-onnxruntime
+```
+
+3. Launch the docker
+```shell
+docker run --rm -it --gpus all -v $PWD:/workspace nvcr.io/nvidia/pytorch:23.10-py3 /bin/bash
+```
+
+4. Build ONNX Runtime from source
+```shell
+export CUDACXX=/usr/local/cuda-12.2/bin/nvcc
+git config --global --add safe.directory '*'
+sh build.sh --config Release  --build_shared_lib --parallel --use_cuda --cuda_version 12.2 \
+            --cuda_home /usr/local/cuda-12.2 --cudnn_home /usr/lib/x86_64-linux-gnu/ --build_wheel --skip_tests \
+            --use_tensorrt --tensorrt_home /usr/src/tensorrt \
+            --cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF \
+            --cmake_extra_defines CMAKE_CUDA_ARCHITECTURES=80 \
+            --allow_running_as_root
+python3 -m pip install build/Linux/Release/dist/onnxruntime_gpu-*-cp310-cp310-linux_x86_64.whl --force-reinstall
+```
+
+5. Install libraries and requirements
+```shell
+python3 -m pip install --upgrade pip
+cd /workspace/onnxruntime/python/tools/transformers/models/stable_diffusion
+python3 -m pip install -r requirements-cuda12.txt
+python3 -m pip install --upgrade polygraphy onnx-graphsurgeon --extra-index-url https://pypi.ngc.nvidia.com
+```
+
+6. Perform ONNX Runtime optimized inference
+```shell
+python3 demo_txt2img_xl.py \
+  "starry night over Golden Gate Bridge by van gogh" \
+  --width 1024         \
+  --height 1024        \
+  --denoising-steps 8  \
+  --work-dir /workspace/stable-diffusion-xl-1.0-onnxruntime
+```