CUDA error: the provided PTX was compiled with an unsupported toolchain.
#26
by
parvezkhan
- opened
Hi All,
I have downloaded this model and while executing it getting below error:
Code:
output = llm(
... "[INST] What is python? [/INST]", # Prompt
... max_tokens=512, # Generate up to 512 tokens
... stop=["</s>"], # Example stop token - not necessarily correct for this specific model! Please check before using.
... echo=True # Whether to echo the prompt
...
Error
CUDA error: the provided PTX was compiled with an unsupported toolchain.
current device: 0, in function ggml_cuda_op_flatten at /tmp/pip-install-r5wzw2ak/llama-cpp-python_a072f9ea5ea14ad5807f3be7170295e9/vendor/llama.cpp/ggml-cuda.cu:9209
cudaGetLastError()
GGML_ASSERT: /tmp/pip-install-r5wzw2ak/llama-cpp-python_a072f9ea5ea14ad5807f3be7170295e9/vendor/llama.cpp/ggml-cuda.cu:241: !"CUDA error"
No symbol table is loaded. Use the "file" command.
ptrace: Operation not permitted.
No stack.
The program is not being run.
Aborted (core dumped)
Environment Details:
OS: Oracle Linux 7.9
GPU: NVIDIA A100
Python Version: Python 3.10
Transformers: transformers==4.37.2
LLAMA CPP PYTHON: llama_cpp_python==0.2.44. (Installed using CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python)
NVIDIA SMI OUTPUT: $ nvidia-smi
Mon Feb 19 05:18:54 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA A100 80GB PCIe On | 00000000:25:00.0 Off | 0 |
| N/A 32C P0 64W / 300W | 848MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA A100 80GB PCIe On | 00000000:81:00.0 Off | 0 |
| N/A 29C P0 43W / 300W | 7MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+----------------------+----------------------+
Thanks in Advance!!