Can anyone suggest me how can i run "llama.cpp" on GPU.
#18
by
fais4321
- opened
I saw they are supporting GPU as well
https://github.com/abetlen/llama-cpp-python/issues/509
If you have Linux, For GPU supported build. you need to run cmake:
mkdir build
cd build
cmake .. -DLLAMA_CUBLAS=ON
cmake --build . --config Release
More details on the issue are here: https://github.com/ggerganov/llama.cpp/discussions/2142