Building llama-cpp-python with CUDA support fails due to GLIBC version incompatibility

Oba: <p>Hello,</p>
<h2><a name="p-472100-issue-summary-1" class="anchor" href="#p-472100-issue-summary-1"></a>Issue Summary</h2>
<p><strong>Building llama-cpp-python with CUDA support fails due to GLIBC version incompatibility</strong></p>
<h2><a name="p-472100-environment-details-2" class="anchor" href="#p-472100-environment-details-2"></a>Environment Details</h2>
<ul>
<li><strong>OS</strong>: Ubuntu Linux</li>
<li><strong>GPU</strong>: NVIDIA (Driver 570.158.01, CUDA 12.8)</li>
<li><strong>Python Environment</strong>: Miniconda3 with Python 3.13</li>
<li><strong>PyTorch</strong>: 2.8.0.dev20250622+cu128</li>
<li><strong>Compiler</strong>: conda’s x86_64-conda-linux-gnu-c++ (GCC 11.2.0)</li>
</ul>
<p>/home/oba/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/local/cuda/lib64/libcublasLt.so.12: undefined reference to <code>log2f@GLIBC_2.27' /home/oba/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/local/cuda/lib64/libcublasLt.so.12: undefined reference to </code>__cxa_thread_atexit_impl@GLIBC_2.18’<br>
collect2: error: ld returned 1 exit status</p>
<h2><a name="p-472100-root-cause-3" class="anchor" href="#p-472100-root-cause-3"></a>Root Cause</h2>
<p>The CUDA library <code>libcublasLt.so.12</code> requires GLIBC symbols:</p>
<ul>
<li><code>log2f@GLIBC_2.27</code> (from GLIBC 2.27+)</li>
<li><code>__cxa_thread_atexit_impl@GLIBC_2.18</code> (from GLIBC 2.18+)</li>
</ul>
<p>But the system apparently has an older GLIBC version that doesn’t provide these symbols.</p>
<p>Build Command Used CMAKE_ARGS=“-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=native” pip install -e . --verbose</p>
<h2><a name="p-472100-whats-been-tried-4" class="anchor" href="#p-472100-whats-been-tried-4"></a>What’s Been Tried</h2>
<ol>
<li>Using system CUDA instead of conda CUDA - same error</li>
<li>Both conda CUDA libraries and system CUDA libraries show the same GLIBC dependency issue</li>
<li>The error occurs specifically when linking vision tools (llava, mtmd) that depend on CUDA libraries</li>
</ol>
<h2><a name="p-472100-questions-for-forum-5" class="anchor" href="#p-472100-questions-for-forum-5"></a>Questions for Forum</h2>
<ol>
<li>How to resolve GLIBC version conflicts when building llama-cpp-python with CUDA?</li>
<li>Is there a way to use older/compatible CUDA libraries that don’t require GLIBC 2.27+?</li>
<li>Can the build be configured to skip problematic vision components while keeping core CUDA functionality?</li>
<li>Should I use system compiler instead of conda compiler, or create a different conda environment?</li>
</ol>
<h2><a name="p-472100-additional-context-6" class="anchor" href="#p-472100-additional-context-6"></a>Additional Context</h2>
<ul>
<li>The build progresses successfully until the final linking stage for vision tools</li>
<li>Core CUDA libraries (libggml-cuda.so) appear to build successfully</li>
<li>Only fails when linking final executables that use cuBLAS</li>
</ul>
<p>This gives forum responders all the technical details they need to help diagnose the specific GLIBC/CUDA library compatibility issue.</p>
<p>regards,</p>