Spaces:
Running
on
T4
CUDA version issue.
Hi Sanchit,
I am trying to use Pyannote diarization and the Whisper-Jax in one Gradio Space. I am getting an error message when I create the Pipleline. Any suggestions how to bypass this. Scrape from the logs here. RuntimeError: The NVIDIA driver on your system is too old (found version 11080). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver.
Ok. There was an issue and conflict with WAVLM model that I was also using. I will reopen another issue if required. Sanchit very grateful for the awesomeness.
Sorry I missed this! Glad you managed to fix the issue and best of luck with developing your Space.
Hi Sanchit. I think the issue is back though - Now if I use any of the spaces or restart or clone them I am getting an error such as this. Please anything you can do help here would be greatly appreciated .
embly aborted due to errors Relying on driver to perform ptx compilation. Setting XLA_FLAGS=--xla_gpu_cuda_data_dir=/path/to/cuda or modifying $PATH can be used to set the location of ptxas This message will only be logged once. 2023-10-11 15:17:12.514939: E external/xla/xla/stream_executor/cuda/cuda_driver.cc:864] failed to load PTX text as a module: CUDA_ERROR_UNSUPPORTED_PTX_VERSION: the provided PTX was compiled with an unsupported toolchain. 2023-10-11 15:17:12.514956: E external/xla/xla/stream_executor/cuda/cuda_driver.cc:869] error log buffer (98 bytes): ptxas application ptx input, line 10; fatal : Unsupported .version 7.8; current version is ’7.4 jax.errors.SimplifiedTraceback: For simplicity, JAX has removed its internal frames from the traceback of the following exception. Set JAX_TRACEBACK_FILTERING=off to include these. The above exception was the direct cause of the following exception: Traceback (most recent call last): File “/home/user/app/app.py”, line 1024, in random_timestamps = pipeline.forward(random_inputs, batch_size=BATCH_SIZE, return_timestamps=True) File “/home/user/.pyenv/versions/3.10.13/lib/python3.10/site-packages/whisper_jax/pipeline.py”, line 408, in forward pred_ids = self.generate(input_features, language=language, task=task, return_timestamps=return_timestamps)[ File “/home/user/.pyenv/versions/3.10.13/lib/python3.10/site-packages/whisper_jax/pipeline.py”, line 188, in generate output_ids = self.p_generate( jaxlib.xla_extension.XlaRuntimeError: INTERNAL: Failed to load PTX text as a module: CUDA_ERROR_UNSUPPORTED_PTX_VERSION: the provided PTX was compiled with an unsupported toolchain. Exception ignored in: <function Pool.__del__ at 0x7f46f02bd090> Traceback (most recent call last): File “/home/user/.pyenv/versions/3.10.13/lib/python3.10/multiprocessing/pool.py”, line 271, in del File “/home/user/.pyenv/versions/3.10.13/lib/python3.10/multiprocessing/queues.py”, line 371, in put AttributeError: ‘NoneType’ object has no attribute ‘dumps’