Repository: stable-diffusion-webui
Issue: [Bug]: ONNX Runtime Fails to Load CUDA Provider
Labels: bug-report

### Checklist

- [ ] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [x] The issue has not been reported before recently
- [X] The issue has been reported before but has not been fixed yet

### What happened?

I am experiencing issues with ONNX Runtime failing to load the CUDA provider. Despite setting up the environment correctly, ONNX Runtime is unable to utilize the GPU, resulting in the following error message:

```
RuntimeError: D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasn't able to be loaded.
Please install the correct version of CUDA and cuDNN as mentioned in the GPU requirements page, make sure they're in the PATH, and that your GPU is supported.
```



### Steps to reproduce the problem

1. Install CUDA 12.4 and cuDNN 9.1.1.
2. Set up the environment variables:

```
$env:PATH += ";C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin"
$env:PATH += ";C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\libnvvp"
$env:PATH += ";C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\extras\CUPTI\lib64"
$env:CUDA_PATH = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4"
```

3. Activate the virtual environment
4. Uninstall and reinstall ONNX Runtime with GPU support:
```
pip uninstall onnxruntime-gpu
pip install onnxruntime-gpu
```
5. Run a test script to load an ONNX model with CUDA support:
```
import onnxruntime as ort
import logging

ort.set_default_logger_severity(0)  # Set to 0 for detailed logging

try:
    providers = ['CUDAExecutionProvider', 'CPUExecutionProvider']
    model_path = 'C:/Users/Insider/Downloads/squeezenet.onnx'
    session = ort.InferenceSession(model_path, providers=providers)
    print("Providers:", session.get_providers())
except Exception as e:
    logging.error("Error during ONNX Runtime session creation:", exc_info=e)
```

### What should have happened?

The ONNX Runtime should successfully load the CUDA provider and use the GPU for inference without any errors.

Reactor, Rembg, etc have not worked and present this issue.

### What browsers do you use to access the UI ?

Google Chrome, Other

### Sysinfo

[sysinfo-2024-05-25-07-48.json](https://github.com/AUTOMATIC1111/stable-diffusion-webui/files/15442178/sysinfo-2024-05-25-07-48.json)

### Console logs

```
Shell
Already up to date.
venv "D:\ImagegenProject\sd.webui\webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.9.3
Commit hash: 1c0a0c4c26f78c32095ebc7f8af82f5c04fca8c0
CUDA 12.1
#######################################################################################################
Initializing Civitai Link
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Civitai Link revision: 115cd9c35b0774c90cb9c397ad60ef6a7dac60de
SD-WebUI revision: 1c0a0c4c26f78c32095ebc7f8af82f5c04fca8c0

Checking Civitai Link requirements...

#######################################################################################################
Launching Web UI with arguments: --autolaunch --upcast-sampling --xformers --no-half-vae --theme=dark
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
[-] ADetailer initialized. version: 24.4.2, num models: 15
*** Error loading script: main.py
    Traceback (most recent call last):
      File "D:\ImagegenProject\sd.webui\webui\modules\scripts.py", line 508, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\ImagegenProject\sd.webui\webui\modules\script_loading.py", line 13, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\ImagegenProject\sd.webui\webui\extensions\openpose-editor\scripts\main.py", line 14, in <module>
        from basicsr.utils.download_util import load_file_from_url
    ModuleNotFoundError: No module named 'basicsr'

---
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
ControlNet preprocessor location: D:\ImagegenProject\sd.webui\webui\extensions\sd-webui-controlnet\annotator\downloads
2024-05-25 03:40:22,282 - ControlNet - INFO - ControlNet v1.1.448
[sdwi2iextender] Developper warning:
[sdwi2iextender] ./modules/img2img.py is being recompiled at run time with a patch. Your debugger will not work in this file.
[sdwi2iextender] If you need debug tools in this file, disable all extensions that use the sdwi2iextender library.
[sdwi2iextender] This patch is temporary and will be removed when v1.9 will be released.
03:40:23 - ReActor - STATUS - Running v0.7.0-b7 on Device: CUDA
Civitai: API loaded
Loading weights [821aa5537f] from D:\ImagegenProject\sd.webui\webui\models\Stable-diffusion\autismmixSDXL_autismmixPony.safetensors
2024-05-25 03:40:23,697 - ControlNet - INFO - ControlNet UI callback registered.
Creating model from config: D:\ImagegenProject\sd.webui\webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
🤯 LobeTheme: Initializing...
Civitai: Check resources for missing info files
Civitai: Check resources for missing preview images
Startup time: 15.2s (prepare environment: 6.0s, import torch: 2.7s, import gradio: 0.6s, setup paths: 0.7s, initialize shared: 0.1s, other imports: 0.3s, list SD models: 0.4s, load scripts: 2.8s, create ui: 1.3s, gradio launch: 0.2s).
Civitai: Found 6 resources missing preview images
Civitai: Found 5 resources missing info files
Civitai: Found 1 hash matches
Exception in thread Thread-49 (load_previews):
Traceback (most recent call last):
  File "C:\Users\Insider\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\Users\Insider\AppData\Local\Programs\Python\Python310\lib\threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "D:\ImagegenProject\sd.webui\webui\extensions\sd_civitai_extension\scripts\previews.py", line 50, in load_previews
    if (nsfw_previews is False): images = [i for i in images if i['nsfw'] is False]
  File "D:\ImagegenProject\sd.webui\webui\extensions\sd_civitai_extension\scripts\previews.py", line 50, in <listcomp>
    if (nsfw_previews is False): images = [i for i in images if i['nsfw'] is False]
KeyError: 'nsfw'
Civitai: No info found on Civitai
Loading VAE weights specified in settings: D:\ImagegenProject\sd.webui\webui\models\VAE\fixFP16ErrorsSDXLLowerMemoryUse_v10.safetensors
Applying attention optimization: xformers... done.
Model loaded in 7.5s (load weights from disk: 0.4s, create model: 0.4s, apply weights to model: 3.5s, load VAE: 0.1s, load textual inversion embeddings: 2.8s, calculate empty prompt: 0.2s).
03:50:22 - ReActor - STATUS - Working: source face index [0], target face index [0]
03:50:22 - ReActor - STATUS - Analyzing Source Image...
2024-05-25 03:50:22.9010136 [E:onnxruntime:Default, provider_bridge_ort.cc:1744 onnxruntime::TryGetProviderInfo_CUDA] C:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

*************** EP Error ***************
EP Error C:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:866 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page  (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements),  make sure they're in the PATH, and that your GPU is supported.
 when using ['CUDAExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
****************************************
2024-05-25 03:50:22.9820565 [E:onnxruntime:Default, provider_bridge_ort.cc:1744 onnxruntime::TryGetProviderInfo_CUDA] C:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

*** Error completing request
*** Arguments: ('task(ej3p1nbym2wmliz)', 0.0, <PIL.Image.Image image mode=RGBA size=1010x1263 at 0x23F748617E0>, None, '', '', True, True, 0.0, 1, 0.0, 512, 512, True, 'None', 'None', 0, False, 1, False, 1, 0, False, 0.5, 0.2, False, 0.9, 0.15, 0.5, False, False, 384, 768, 4096, 409600, 'Maximize area', 0.1, False, ['Horizontal'], False, ['Deepbooru'], <PIL.Image.Image image mode=RGB size=947x1263 at 0x23F748634C0>, True, '0', '0', 'inswapper_128.onnx', 'CodeFormer', 1, True, 'None', 1, 1, 1, 0, 0, 0.5, 'CUDA', False, 0, 'None', '', None, False, False, 0.5, 0, False, 'None', False, False, 240, 10, 10) {}
    Traceback (most recent call last):
      File "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in __init__
        self._create_inference_session(providers, provider_options, disabled_optimizers)
      File "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
        sess.initialize_session(providers, provider_options, disabled_optimizers)
    RuntimeError: C:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:866 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page  (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements),  make sure they're in the PATH, and that your GPU is supported.


    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "D:\ImagegenProject\sd.webui\webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\ImagegenProject\sd.webui\webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\ImagegenProject\sd.webui\webui\modules\postprocessing.py", line 131, in run_postprocessing_webui
        return run_postprocessing(*args, **kwargs)
      File "D:\ImagegenProject\sd.webui\webui\modules\postprocessing.py", line 71, in run_postprocessing
        scripts.scripts_postproc.run(initial_pp, args)
      File "D:\ImagegenProject\sd.webui\webui\modules\scripts_postprocessing.py", line 198, in run
        script.process(single_image, **process_args)
      File "D:\ImagegenProject\sd.webui\webui\extensions\sd-webui-reactor\scripts\reactor_faceswap.py", line 663, in process
        result, output, swapped = swap_face(
      File "D:\ImagegenProject\sd.webui\webui\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 544, in swap_face
        source_faces = analyze_faces(source_img, det_thresh=detection_options.det_thresh, det_maxnum=detection_options.det_maxnum)
      File "D:\ImagegenProject\sd.webui\webui\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 302, in analyze_faces
        face_analyser = copy.deepcopy(getAnalysisModel())
      File "D:\ImagegenProject\sd.webui\webui\extensions\sd-webui-reactor\scripts\reactor_swapper.py", line 145, in getAnalysisModel
        ANALYSIS_MODEL = insightface.app.FaceAnalysis(
      File "D:\ImagegenProject\sd.webui\webui\extensions\sd-webui-reactor\scripts\console_log_patch.py", line 48, in patched_faceanalysis_init
        model = model_zoo.get_model(onnx_file, **kwargs)
      File "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\insightface\model_zoo\model_zoo.py", line 96, in get_model
        model = router.get_model(providers=providers, provider_options=provider_options)
      File "D:\ImagegenProject\sd.webui\webui\extensions\sd-webui-reactor\scripts\console_log_patch.py", line 21, in patched_get_model
        session = PickableInferenceSession(self.onnx_file, **kwargs)
      File "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\insightface\model_zoo\model_zoo.py", line 25, in __init__
        super().__init__(model_path, **kwargs)
      File "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 432, in __init__
        raise fallback_error from e
      File "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 427, in __init__
        self._create_inference_session(self._fallback_providers, None)
      File "D:\ImagegenProject\sd.webui\webui\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
        sess.initialize_session(providers, provider_options, disabled_optimizers)
    RuntimeError: C:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:866 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page  (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements),  make sure they're in the PATH, and that your GPU is supported.


---
```


### Additional information

- I have installed the compatible versions of CUDA (12.4) and cuDNN (9.1.1).
- My GPU drivers are up to date.
- I am running an NVIDIA RTX 3080.

The environment variables are set as follows:
```$env:PATH += ";C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin"
$env:PATH += ";C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\libnvvp"
$env:PATH += ";C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\extras\CUPTI\lib64"
$env:CUDA_PATH = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4"
```

- I have reinstalled ONNX Runtime with GPU support: