zero-gpu-explorers/README · running ONNX models on ZeroGPU

ZeroGPU Explorers org May 8

•

My space (https://huggingface.co/spaces/rizavelioglu/fashionfail) successfully runs locally on my GPU but does not work on ZeroGPU when pushed to HF-spaces.
Has anyone tried running ONNX models on ZeroGPU, maybe it requires a special initialization?
Currently this error is thrown when executed on ZeroGPU:

[E:onnxruntime:Default, provider_bridge_ort.cc:1548 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1209 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory
[W:onnxruntime:Default, onnxruntime_pybind_state.cc:861 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirementsto ensure all dependencies are met.

hysts

ZeroGPU Explorers org May 8

IIRC, unfortunately ONNX is currently not supported on ZeroGPU. cc @cbensimon

cbensimon

ZeroGPU Explorers org 25 days ago

Actually yes, ONNX can run on ZeroGPU but it won't be very efficient (slower than PyTorch to go from CPU to GPU). You have to first load the model with CPUExecutionProvider and then inside @spaces.GPU, change the execution provider to CUDAExecutionProvider