Spaces:
Runtime error
Runtime error
File size: 5,262 Bytes
c19ca42 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 |
# ONNX Runtime
SD.Next includes support for ONNX Runtime.
## How to
Currently, we can't use `--use-directml` because there's no release of `torch-directml` built with latest PyTorch. (this does not mean that you can't use DmlExecutionProvider)
Change `Execution backend` to `diffusers` and `Diffusers pipeline` to `ONNX Stable Diffusion` on the `System` tab.
## Performance
The performance depends on the execution provider.
## Execution Providers
Currently, `CUDAExecutionProvider` and `DmlExecutionProvider` are supported.
| | ONNX | Olive | GPU | CPU |
|---------------------------|------|-------|-----|-----|
| CPUExecutionProvider | ✅ | ❌ | ❌ | ✅ |
| DmlExecutionProvider | ✅ | ✅ | ✅ | ❌ |
| CUDAExecutionProvider | ✅ | ✅ | ✅ | ❌ |
| ROCMExecutionProvider | ✅ | 🚧 | ✅ | ❌ |
| OpenVINOExecutionProvider | ✅ | ✅ | ✅ | ✅ |
### CPUExecutionProvider
Not recommended.
Enabled by default.
### DmlExecutionProvider
You can select `DmlExecutionProvider` by installing `onnxruntime-directml`.
DirectX 12 API is required. (Windows or WSL)
### CUDAExecutionProvider
You can select `CUDAExecutionProvider` by installing `onnxruntime-gpu`. (may have been automatically installed)
### 🚧 ROCMExecutionProvider
Olive for ROCm is working in progress.
### 🚧 OpenVINOExecutionProvider
Under development.
## Supported
- Models from huggingface
- Hires and second pass (without sdxl refiner)
- .safetensors VAE
## Known issues
- SD Inpaint may not work.
- SD Upscale pipeline is not tested.
- SDXL Refiner does not work. (due to onnxruntime's issue)
## FAQ
### I'm getting `OnnxStableDiffusionPipeline.__init__() missing 4 required positional arguments: 'vae_encoder', 'vae_decoder', 'text_encoder', and 'unet'`.
It's due to the broken model cache which was previously generated by failed conversion or Olive run. Find one in `models/ONNX/cache` and remove it. You can also use `ONNX` tab on UI. (You should enable it on settings to make it show up)
# Olive
Olive is an easy-to-use hardware-aware model optimization tool that composes industry-leading techniques across model compression, optimization, and compilation. (from [pypi](https://pypi.org/project/olive-ai/))
## How to
As Olive optimizes the models in ONNX format, you should set up [ONNX Runtime](https://github.com/vladmandic/automatic/wiki/ONNX-Runtime-%26-Olive/#how-to) first.
1. Go to `System` tab → `Compute Settings`.
2. Select `Model`, `Text Encoder` and `VAE` in `Compile Model`.
3. Set `Model compile backend` to `olive-ai`.
Olive-specific settings are under `Olive` in `Compute Settings`.
### How to switch to Olive from torch-directml
Run these commands using PowerShell.
```
.\venv\Scripts\activate
pip uninstall torch-directml
pip install torch torchvision --upgrade
pip install onnxruntime-directml
.\webui.bat
```
### From checkpoint
Model optimization occurs automatically before generation.
Target models can be .safetensors, .ckpt, Diffusers pretrained model and the optimization progress takes time depending on your system and execution provider.
The optimized models are automatically cached and used later to create images of the same size (height and width).
### From Huggingface
If your system memory is not enough to optimize model or you don't want to waste your time to optimize the model yourself, you can download optimized model from Huggingface.
Go to `Models` → `Huggingface` tab and download optimized model.
## Advanced Usage
### Customize Olive workflow
TBA
## Performance
| Property | Value |
|------------------|------------------------------------------------------------|
| Prompt | a castle, best quality |
| Negative Prompt | worst quality |
| Sampler | Euler |
| Sampling Steps | 20 |
| Device | RX 7900 XTX 24GB |
| Version | olive-ai(0.4.0) onnxruntime-directml(1.16.3) ROCm(5.6) torch(olive: 2.1.2, rocm: 2.1.0) |
| Model | runwayml/stable-diffusion-v1-5 (ROCm), lshqqytiger/stable-diffusion-v1-5-olive (Olive) |
| Precision | fp16 |
| Token Merging | Olive(0, not supported) ROCm(0.5) |
| Olive with DmlExecutionProvider | ROCm |
|-------|------|
|||
## Pros and Cons
### Pros
- The generation is faster.
- Uses less graphics memory.
### Cons
- Optimization is required for every models and image sizes.
- Some features are unavailable.
## FAQ
### My execution provider does not show up in my settings.
After activating python venv, run this command and try again:
```
(venv) $ pip uninstall onnxruntime onnxruntime-... -y
``` |