CUDA, cuDNN, and PyTorch Compatibility

username: Arpan_Gyawali
cooked: <p>Hello,</p>
<p>I’m trying to set up a specific environment on my university’s HPC, which restricts sudo access. The HPC has Python &gt;=3.9 and CUDA &gt;=11.7. For my project, I need Python 3.6 and PyTorch 0.4.1, compatible with CUDA 9.2 and cuDNN 7.2.1.</p>
<p><strong>What I’ve done:</strong></p>
<ol>
<li>Created a conda environment with Python 3.6.</li>
<li>Installed cudatoolkit=9.2 and cudnn=7.2.1.</li>
<li>Installed PyTorch 0.4.1 using <code>conda install pytorch=0.4.1 cuda92 -c pytorch</code>.</li>
</ol>
<p><strong>Issues:</strong></p>
<ul>
<li>When installing pytorch 0.4.1 in this env i got env conflicts, so i created a python venv inside the conda env and installed 0.4.1 using pip.</li>
<li>When running <code>nvcc --version</code>, it shows CUDA 9.2. Also torch.cuda.version returns 9.2 which is good.</li>
<li><code>torch.version.cuda</code> shows 9.2, but <code>torch.backends.cudnn.version()</code> returns 7.1 instead of 7.2.1.</li>
<li>During training, I encounter the error: <code>RuntimeError: CuDNN error: CUDNN_STATUS_EXECUTION_FAILED</code>.</li>
</ul>
<p><strong>Questions:</strong></p>
<ol>
<li>Why is cuDNN version 7.1 instead of 7.2.1?</li>
<li>How can I correctly set up the environment to avoid conflicts?</li>
<li>How to solve CUDNN error?</li>
</ol>
<p>Thank you for your help!</p>

username: ptrblck
cooked: <p>What’s the reason for using such an old PyTorch release, which is approx. 6 years old?</p>

username: Arpan_Gyawali
cooked: <p>I was trying to use a framework called MedicalDetectionToolkit. <a href="https://github.com/MIC-DKFZ/medicaldetectiontoolkit/tree/master" class="inline-onebox" rel="noopener nofollow ugc">GitHub - MIC-DKFZ/medicaldetectiontoolkit: The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.</a><br>
It said that it needs python3.6.<br>
Also the cuda functions there are implementeted for python3.6</p>

username: Soumya_Kundu
cooked: <aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/MIC-DKFZ/nnDetection">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/MIC-DKFZ/nnDetection" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/bb529d6f966a9bbf8e4b32586874a0ec777feff7d1707bf9b8dcf37811bf0763/MIC-DKFZ/nnDetection" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/MIC-DKFZ/nnDetection" target="_blank" rel="noopener nofollow ugc">GitHub - MIC-DKFZ/nnDetection: nnDetection is a self-configuring framework...</a></h3>

  <p>nnDetection is a self-configuring framework for 3D (volumetric) medical object detection which can be applied to new data sets without manual intervention. It includes guides for 12 data sets that ...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I would use this instead. Doesn’t support RCNN at the moment i believe. Same group.</p>

username: Arpan_Gyawali
cooked: <p>Yes this is for 3d data only. I am working on 2d data <a class="mention" href="/u/soumya_kundu">@Soumya_Kundu</a></p>

username: Soumya_Kundu
cooked: <p>I think you will find it much easier implementing the 2d versions yourself from scratch. There are also great repos for all them as well. Especially Mask-RCNN.</p>

username: Arpan_Gyawali
cooked: <p>Hi <a class="mention" href="/u/soumya_kundu">@Soumya_Kundu</a> , is there any repo you know that is working on this from scratch. What i wanted was a algorithm that specially targets on medical images. I have tried RetinaNet and Yolo. The result from RetinaNet was good but, i wanted to improve the result by using the model specific to the domain. So i tried MDT using retinaUnet.</p>

username: Soumya_Kundu
cooked: <p>What exactly are you searching for? 2D Detection networks for medical images?</p>

username: Arpan_Gyawali
cooked: <p>yes,<br>
i am working at LVO detection and used various networks. They work good but i want to improve the result. My thoughts are that if the medical images based network will work better in this case.<br>
MDT RetinaUnet and nnDetection were the option but MDT is too old and nnDetection implementation is for 3D only. I was just searching if there is some implementation for 2D.</p>

username: Soumya_Kundu
cooked: <p>RetinaUNet is for segmentation right?</p>
<p>There is no “medical detection” framework that I know of unfortunately. I think you should take your best performing model and try improving that further by playing around with it a bit more.</p>

username: Arpan_Gyawali
cooked: <p>Yes, but MedicalDetectionToolkit has used to for detection purpose by taking advantage of Segmentation Supervision of UNET.<br>
Thank you <a class="mention" href="/u/soumya_kundu">@Soumya_Kundu</a> for your time</p>

username: Soumya_Kundu
cooked: <p>I think that statement is a bit flawed. Most literature points to generic semantic segmentation models not being able to detect well.</p>