afsafwtggg/llama-cpp-t4-cache

llama.cpp prebuilt engine for Tesla T4 (Kaggle)

Compiled output of llama-server and llama-perplexity from upstream llama.cpp at commit 539f24529bdf99e0baefd54fefff1660034bfe7b, built on Kaggle's GPU image for sm_75 (Tesla T4 only). Nothing here is model weights or secret material - it is MIT-licensed source compiled by a script, packaged so a Kaggle notebook does not spend 25 minutes recompiling it every session.

Produced by the qwen-serving-lab benchmarking project (afsafwtggg/qwen-serving-lab, private - it holds the measurement report).

llama-cpp-t4-539f245.tar.gz   sha256 e1d7f030d6876d89ce376bb9ecebd2bdc17dd0c75836a1583ac4f549610e8382

Use in a Kaggle notebook

import pathlib, tarfile, subprocess
from huggingface_hub import hf_hub_download          # public: no token
p = pathlib.Path(hf_hub_download("afsafwtggg/llama-cpp-t4-cache", "llama-cpp-t4-539f245.tar.gz"))
with tarfile.open(p) as f: f.extractall("/kaggle/working/engine")
BIN = pathlib.Path("/kaggle/working/engine/llama-cpp-t4-539f245/bin")
for b in BIN.iterdir():
    if b.is_file(): b.chmod(0o755)

import os
env = os.environ.copy(); env["LD_LIBRARY_PATH"] = str(BIN)
subprocess.Popen([str(BIN/"llama-server"), ...], env=env)   # see KAGGLE.md

LD_LIBRARY_PATH matters: the lib*.so.0 SONAME aliases in the package are file copies, not symlinks, because Kaggle's file transports drop symlinks.

Provenance

bin/BUILDINFO.json records the upstream commit, llama-server --version, the exact cmake flags, and the build image's kernel / glibc / nvcc / driver. If a Kaggle GPU image moves far from that fingerprint, rebuild rather than trust this (scripts/cache_kernel.py in the private project repo).

Licensed under the MIT license; see LICENSE-llama-cpp-MIT.txt inside the archive and LICENSE-llama-cpp-MIT.txt at the repo root. Statically links NVIDIA's CUDA runtime, as upstream's own prebuilt releases do.