llama.cpp CUDA build for Turing (SM75 / Tesla T4)

Prebuilt llama-server / llama-cli from ggml-org/llama.cpp, compiled with -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=75.

llama.cpp publishes no Linux CUDA release binaries, and building from source on a 2-vCPU Colab box takes ~40 minutes. This tarball drops that to seconds.

Build: b10690 (0b5be7e)
Target: Turing only (SM 75) — Tesla T4, RTX 20xx, GTX 16xx
Size: 50 MB. CUDA runtime + driver libs are not bundled; they link against the host's /usr/local/cuda and /usr/lib64-nvidia, which Colab and Kaggle images already provide.

Use

mkdir -p llamacpp-bin && curl -sL <file-url> | tar xz -C llamacpp-bin
export LD_LIBRARY_PATH=$PWD/llamacpp-bin:/usr/lib64-nvidia:/usr/local/cuda/targets/x86_64-linux/lib
./llamacpp-bin/llama-server --help

Upstream llama.cpp is MIT licensed; these are unmodified builds of it.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support