Spaces:
Running
Running
running ONNX models on ZeroGPU
#39
by
rizavelioglu
- opened
My space (https://huggingface.co/spaces/rizavelioglu/fashionfail) successfully runs locally on my GPU but does not work on ZeroGPU when pushed to HF-spaces.
Has anyone tried running ONNX models on ZeroGPU, maybe it requires a special initialization?
Currently this error is thrown when executed on ZeroGPU:
[E:onnxruntime:Default, provider_bridge_ort.cc:1548 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1209 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory
[W:onnxruntime:Default, onnxruntime_pybind_state.cc:861 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirementsto ensure all dependencies are met.
Actually yes, ONNX can run on ZeroGPU but it won't be very efficient (slower than PyTorch to go from CPU to GPU). You have to first load the model with CPUExecutionProvider
and then inside @spaces.GPU
, change the execution provider to CUDAExecutionProvider