Experimental llama.cpp runtime for RTX 6000 Ada
Pinned upstream source: 7d7f0287530bc8981690f361d8c1627dbae1c99a. CUDA12.8, Linux x86_64, sm89.
This is a runner bundle, NOT a model. It contains the complete build/bin,
shared libraries, upstream MIT license and per-file build manifest.
Status: experimental; correctness, long-context and streaming qualification
are pending. Do not treat publication as benchmark or production approval.
Restore without rebuilding
Download llama-cpp-7d7f0287530b-cuda128-sm89.tar.gz and SHA256SUMS from an immutable revision of this repo.
Run sha256sum -c SHA256SUMS, then extract the archive into a new directory.
Set LD_LIBRARY_PATH to its bin directory and run bin/llama-server --version.
CUDA driver compatibility and system libraries still matter; check ldd for
unresolved dependencies before starting inference. NVIDIA libraries supplied
by the base image/host driver are not bundled. Build settings and hashes are
in build-manifest.json. The archive SHA256 is a872e3abd5e260fad4a3ab72ad82b5753ff6ffcce60daa169b873d802ad4f255.
The associated experiment uses Huihui Qwen3.8-27B UD-Q8_K_XL and the Inco AI Q4_K_M DFlash2 draft at width7, 262144 context, q8_0 K/V, batch2048, ubatch2048. Weights are obtained separately under their respective licenses. No model weights, credentials, machine diagnostics, prompts or sessions are included. The existing production runtime is not replaced by this artifact.