Torch+TF+Spacy+more+GPU+ Python 3.7+Ubuntu 22.04

username: AlexW
cooked: <p>I  run a 2-year  old program from github which only works with Python 3.7 (does not work with Python 3.8, 3.9, 3.10) and uses tensorflow , torch, spacy all with GPU support and many other modules. I was able to run the program ok without GPU. Without GPU hardware, with torch=1.13.1 and TF=2.9.0 it gives warnings that CUDA is not available, but otherwise runs without errors and does produce correct results.</p>
<p>I spent a week trying to make it work   with GPU. With  Python 3.7, TF is upper-limited to Cuda=11.2 and Cudnn=8.1, yet there is no torch+cu112. It means I have to have <strong>two different versions of  Cuda</strong> at the same time.<br>
<code>conda install -c conda-forge cudatoolkit=11.2.2 cudnn=8.1.0 # for TF and Spacy</code><br>
<code>pip install spacy[cuda112]</code><br>
<code>pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113</code><br>
`(py37)  $ python -c “import torch; print(f’PyTorch version {torch.<strong>version</strong>} has CUDA : {torch.cuda.is_available()}')”<br>
PyTorch version 1.12.1+cu113 has CUDA : True’</p>
<p>When I run the program, depending on torch+cuda version, I get various torch errors. For example:</p>
<blockquote>
<p>RuntimeError: CUDA out of memory. Tried to allocate 148.00 MiB (GPU 0; 15.71 GiB total capacity; 1.33 GiB already allocated; 50.50 MiB free; 1.45 GiB reserved in total by PyTorch) If reserved memory is &gt;&gt; allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF</p>
</blockquote>
<p>It appears  there is 15.71 GiB  total capacity of GPU yet torch is not able to allocate 148.00 MiB.<br>
In addition to  torch==1.12.1+cu113, I tried other torch+cuda versions. Some of them  install ok and appear to recognize my GPU ok, but all fail with  various torch errors when I run the program.</p>
<p>I am aware that Python 3.7 is  no longer officially supported yet hope to get GPU to work. Is running without GPU my only option?</p>
<p>Collecting environment information…<br>
PyTorch version: 1.12.1+cu113<br>
CUDA used to build PyTorch: 11.3<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Ubuntu 22.04.3 LTS (x86_64)<br>
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0<br>
Libc version: glibc-2.10</p>
<p>Python version: 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:21)  [GCC 9.4.0] (64-bit runtime)<br>
Python platform: Linux-6.2.0-36-generic-x86_64-with-debian-bookworm-sid<br>
Is CUDA available: True<br>
CUDA runtime version: Could not collect<br>
CUDA_MODULE_LOADING set to:<br>
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4060 Ti<br>
Nvidia driver version: 535.129.03<br>
cuDNN version: Could not collect<br>
Is XNNPACK available: True</p>
<p>CPU:<br>
Architecture:                       x86_64<br>
CPU op-mode(s):                     32-bit, 64-bit<br>
Address sizes:                      46 bits physical, 48 bits virtual<br>
Byte Order:                         Little Endian<br>
CPU(s):                             24<br>
On-line CPU(s) list:                0-23<br>
Vendor ID:                          GenuineIntel<br>
Model name:                         12th Gen Intel(R) Core™ i9-12900K<br>
CPU family:                         6<br>
Model:                              151<br>
Thread(s) per core:                 2<br>
Core(s) per socket:                 16<br>
Socket(s):                          1<br>
Stepping:                           2<br>
CPU max MHz:                        5200.0000<br>
CPU min MHz:                        800.0000<br>
BogoMIPS:                           6374.40</p>
<p>Virtualization:                     VT-x<br>
L1d cache:                          640 KiB (16 instances)<br>
L1i cache:                          768 KiB (16 instances)<br>
L2 cache:                           14 MiB (10 instances)<br>
L3 cache:                           30 MiB (1 instance)<br>
NUMA node(s):                       1<br>
NUMA node0 CPU(s):                  0-23</p>
<p>Versions of relevant libraries:<br>
[pip3] numpy==1.21.6<br>
[pip3] torch==1.12.1+cu113<br>
[pip3] torchaudio==0.12.1+cu113<br>
[pip3] torchvision==0.13.1+cu113<br>
[conda] cudatoolkit               11.2.2              hc23eb0c_12    conda-forge<br>
[conda] numpy                     1.21.6                   pypi_0    pypi<br>
[conda] torch                     1.12.1+cu113             pypi_0    pypi<br>
[conda] torchaudio                0.12.1+cu113             pypi_0    pypi<br>
[conda] torchvision               0.13.1+cu113             pypi_0    pypi</p>

username: ptrblck
cooked: <p>TF will usually try to allocate all device memory and might not leave anything for PyTorch to use. Disable this behavior and check if you would still be running out of memory. If that doesn’t help you could build both frameworks with deprecated the Python version from source.</p>

username: AlexW
cooked: <p>Is it  in principle a good idea  to run side by side apps requiring different (multiple)  versions of Cuda? Would I be better off if using Cuda 11.0 and Cudnn 8.0 for both TF and torch (and spacy)?<br>
<code>conda install -c conda-forge -c nvidia cudatoolkit=11.0 cudnn=8.0 # for TF and Spacy</code><br>
pip install spacy[cuda110]</p>
<p><code>pip install torch==1.*+cu110 torchvision==*+cu110 torchaudio==*  --extra-index-url https://download.pytorch.org/whl/cu110</code></p>
<p>Which version of torch with Cu110 is most suitable for my setup (Conda+ Python 3.7 +Ubuntu 22.04) ?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="AlexW" data-post="3" data-topic="191435">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/b5a626/48.png" class="avatar"> AlexW:</div>
<blockquote>
<p>Is it in principle a good idea to run side by side apps requiring different (multiple) versions of Cuda?</p>
</blockquote>
</aside>
<p>No, I would recommend trying to align the stack as much as possible.<br>
I also don’t know how exactly TF uses CUDA and if it depends on e.g. conda binaries or your locally installed CUDA toolkit.</p>
<aside class="quote no-group" data-username="AlexW" data-post="3" data-topic="191435">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/b5a626/48.png" class="avatar"> AlexW:</div>
<blockquote>
<p>Would I be better off if using Cuda 11.0 and Cudnn 8.0 for both TF and torch (and spacy)?</p>
</blockquote>
</aside>
<p>Also no, since CUDA 11.0 is quite old by now and you might be running into known and already fixed issues.</p>
<p>The cleanest way would be to update the used code base to be compatible with a newer Python version and then to install the latest (compatible) binaries.<br>
However, I also don’t know what exactly is used that is not supported in Python&gt;=3.8 and how hard it would be to port it over.</p>

username: AlexW
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="191435">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>you could build both frameworks with deprecated the Python version from source.</p>
</blockquote>
</aside>
<p>I  would like to  build PyTorch  1.12.1 and 1.13.1  from source for  Cuda=11.2 and Ubuntu 22.04 without installing Nvidia Cuda system-wide.</p>
<p>Can I achieve that in conda environment  with <code>conda install -c conda-forge cudatoolkit=11.2.2 cudnn=8.1.0</code> ?</p>

username: ptrblck
cooked: <p>No, as <code>cudatoolkit</code> should only ship the runtime libraries. You could search for a CUDA compiler conda package instead, but I don’t have any experience with it as I’m using a locally installed CUDA toolkit for my source builds.</p>

username: AlexW
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="6" data-topic="191435">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>I’m using a locally installed CUDA toolkit for my source builds.</p>
</blockquote>
</aside>
<p>I will, too, install CUDA 11.2 locally.  How can  I properly select PyTorch  (and  then torchvision) source which is compatible with Python 3.7 ?<br>
I plan to use the following tutorial (but for Cuda 11.2)  <a href="https://medium.com/@zhanwenchen/build-pytorch-from-source-with-cuda-11-8-565ab737bfc8" class="inline-onebox" rel="noopener nofollow ugc">Build PyTorch from Source with CUDA 11.8 with Ubuntu 22.04 | by Zhanwen Chen | Medium</a>.</p>

username: ptrblck
cooked: <p>I would probably go through release notes or check the binary names to see when Python 3.7 was used as well as an early CUDA 11 version.</p>

username: AlexW
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="8" data-topic="191435">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>check the binary names to see when Python 3.7 was used as well as an early CUDA 11 version.</p>
</blockquote>
</aside>
<p>On pytorch website <a href="https://download.pytorch.org/whl/torch_stable.html" rel="noopener nofollow ugc">https://download.pytorch.org/whl/torch_stable.html</a><br>
I see precompiled<br>
cu113/torch-1.12.1%2Bcu113-cp37-cp37m-linux_x86_64.whl<br>
cu116/torch-1.13.1%2Bcu116-cp37-cp37m-linux_x86_64.whl</p>
<p>So Python 3.7 was supported for versions v1.13.1 and  v1.12.1.</p>
<p>git clone --recursive _<a href="https://github.com/pytorch/pytorch_" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch_</a><br>
cd pytorch<br>
git checkout v1.13.1 (or v1.12.1)</p>
<p>I will be compiling in conda environment with Python=3.7.<br>
So, if I specify v1.13.1  (or v1.12.1) in the above checkout,  will it be sufficient to get  binaries compiled for Python 3.7 with Cuda 11.2?</p>

username: ptrblck
cooked: <p>I guess so, but eventually you should just try building it.</p>