Unable to Import PyTorch After Upgrade in Docker Environment

unbreading: <aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/146701">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/146701" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue" data-github-private-repo="false">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/146701" target="_blank" rel="noopener nofollow ugc">Unable to Import PyTorch After Upgrade in Docker Environment</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2025-02-07" data-time="15:02:45" data-timezone="UTC">03:02PM - 07 Feb 25 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/unbreading" target="_blank" rel="noopener nofollow ugc">
          <img alt="" src="https://avatars.githubusercontent.com/u/33262319?v=4" class="onebox-avatar-inline" width="20" height="20">
          unbreading
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: docker
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">### 🐛 Describe the bug

### Problem Description

Before upgrade: 

&lt;img width="8<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">38" alt="Image" src="https://github.com/user-attachments/assets/8950b303-510f-4136-b44d-8a3b55185941" /&gt;

After upgrade (by `pip install torch==2.4.0`)

&lt;img width="837" alt="Image" src="https://github.com/user-attachments/assets/25a81dc3-8202-4dbc-98ec-71a11c87e0cf" /&gt;

I found that libcudnn.so.9 exists in the path /usr/local/lib/python3.10/site-packages/nvidia/cudnn/lib/, but it was not being loaded. To resolve this, I modified the _load_global_deps() function in torch/__init__.py by adding line 227 and deleting lines 245-246, as shown in the attached image. After these changes, I was able to successfully import torch.

&lt;img width="935" alt="Image" src="https://github.com/user-attachments/assets/3405e706-3522-4234-958f-64e9af5ffb10" /&gt;

### Questions
- Is this the right solution? Am I missing any steps?
- After upgrading PyTorch using this method, I encountered numerous 'undefined symbol' errors during the installation and import of vllm and flash-attn. Could these undefined symbol problems be related to the changes


### Versions

### Environment:

Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: TencentOS Server 3.2 (Final) (x86_64)
GCC version: (GCC) 8.5.0 20210514 (Tencent 8.5.0-23)
Clang version: Could not collect
CMake version: version 3.26.5
Libc version: glibc-2.28

Python version: 3.10.8 (main, Oct 15 2022, 14:44:57) [GCC 8.5.0 20210514 (Red Hat 8.5.0-10)] (64-bit runtime)
Python platform: Linux-5.4.119-1-tlinux4-0009.3-x86_64-with-glibc2.28
Is CUDA available: True
CUDA runtime version: 12.1.66
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: A100-SXM4-40GB
GPU 1: A100-SXM4-40GB
GPU 2: A100-SXM4-40GB
GPU 3: A100-SXM4-40GB
GPU 4: A100-SXM4-40GB
GPU 5: A100-SXM4-40GB
GPU 6: A100-SXM4-40GB
GPU 7: A100-SXM4-40GB

Nvidia driver version: 450.156.00
cuDNN version: Probably one of the following:
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn.so.8.9.4
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.9.4
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.9.4
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.9.4
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.9.4
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.9.4
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.9.4
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:         x86_64
CPU op-mode(s):       32-bit, 64-bit
Byte Order:           Little Endian
CPU(s):               256
On-line CPU(s) list:  0-254
Off-line CPU(s) list: 255
Thread(s) per core:   1
Core(s) per socket:   64
Socket(s):            2
NUMA node(s):         16
Vendor ID:            AuthenticAMD
BIOS Vendor ID:       Advanced Micro Devices, Inc.
CPU family:           25
Model:                1
Model name:           AMD EPYC 7K83 64-Core Processor
BIOS Model name:      AMD EPYC 7K83 64-Core Processor                
Stepping:             1
CPU MHz:              3243.849
CPU max MHz:          2450.0000
CPU min MHz:          1500.0000
BogoMIPS:             4891.30
Virtualization:       AMD-V
L1d cache:            32K
L1i cache:            32K
L2 cache:             512K
L3 cache:             32768K
NUMA node0 CPU(s):    0-7,128-135
NUMA node1 CPU(s):    8-15,136-143
NUMA node2 CPU(s):    16-23,144-151
NUMA node3 CPU(s):    24-31,152-159
NUMA node4 CPU(s):    32-39,160-167
NUMA node5 CPU(s):    40-47,168-175
NUMA node6 CPU(s):    48-55,176-183
NUMA node7 CPU(s):    56-63,184-191
NUMA node8 CPU(s):    64-71,192-199
NUMA node9 CPU(s):    72-79,200-207
NUMA node10 CPU(s):   80-87,208-215
NUMA node11 CPU(s):   88-95,216-223
NUMA node12 CPU(s):   96-103,224-231
NUMA node13 CPU(s):   104-111,232-239
NUMA node14 CPU(s):   112-119,240-247
NUMA node15 CPU(s):   120-127,248-254
Flags:                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] nvidia-cublas-cu12==12.1.3.1
[pip3] nvidia-cuda-cupti-cu12==12.1.105
[pip3] nvidia-cuda-nvrtc-cu12==12.1.105
[pip3] nvidia-cuda-runtime-cu12==12.1.105
[pip3] nvidia-cudnn-cu12==9.1.0.70
[pip3] nvidia-cufft-cu12==11.0.2.54
[pip3] nvidia-curand-cu12==10.3.2.106
[pip3] nvidia-cusolver-cu12==11.4.5.107
[pip3] nvidia-cusparse-cu12==12.1.0.106
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] nvidia-nvjitlink-cu12==12.8.61
[pip3] nvidia-nvtx-cu12==12.1.105
[pip3] torch==2.4.0
[pip3] torchaudio==2.2.0
[pip3] torchvision==0.17.0
[pip3] triton==3.0.0
[conda] Could not collect</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


onedeadmatch: <p>Tipically when libcudnn is not found it means that the path is missing in ld_library_path env variable.</p>
<pre><code class="lang-auto">export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
source ~/.bashrc
</code></pre>
<p>Anyway, I am not sure that this will 100% solve the problem, if you need a custom container you can create one from scratch using pre-configured nvidia’s images. You can have a look here for reference: <a href="https://github.com/DanielRossi1/AutoDock" class="inline-onebox" rel="noopener nofollow ugc">GitHub - DanielRossi1/AutoDock: The perfect environment for your AI Research Project!</a></p>