Install PyTorch with CUDA 11 from source via pip method

username: Chun_Li
cooked: <p>CUDA 11 has appeared a while, but we haven’t seen binary PyTorch installation with CUDA 11 in official PyTorch website yet. So I decided to build PyTorch from source with CUDA 11. However, in the <a href="https://github.com/pytorch/pytorch" rel="nofollow noopener">https://github.com/pytorch/pytorch</a>, the recommended method to install related dependencies is “conda install”. Since we don’t know if “conda install -c pytorch magma-cuda110” is available, and I prefer pip install method as personal taste too, so I tried to install other dependencies via pip, build magma with cuda 11 and install it first, and then build pytorch wheel package from source, and finally pip install the package in my ubuntu 20.04 system.</p>
<p>In the following I assume that we already have CUDA 11 and cudnn installed in an ubuntu 20.04 Linux system with Nvidia GPU (not too old one, with compute capability &gt;= 6.0). The steps are as follows.</p>
<ol>
<li>
<p>Run the following pip install command as system admin / root (or sudo pip install …):<br>
pip install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests</p>
</li>
<li>
<p>Go to website <a href="https://icl.utk.edu/magma/software/index.html" rel="nofollow noopener">https://icl.utk.edu/magma/software/index.html</a> to download the latest magma-2.5.3.tar.gz. In a download working directory, run<br>
tar xzf magma-2.5.3.tar.gz<br>
to untar unzip it, and get a sub directory magma-2.5.3.<br>
cd magma-2.5.3<br>
and follow README or <a href="http://icl.cs.utk.edu/projectsfiles/magma/doxygen/installing.html" rel="nofollow noopener">http://icl.cs.utk.edu/projectsfiles/magma/doxygen/installing.html</a><br>
to build magma, and install it as root. The default install location is /usr/local/magma. Go to /etc/ld.so.conf.d and create a file named magma.conf there with one line text “/usr/local/magma/lib”, and run command “ldconfig” as root. Note that build magma-2.5.3 is basically to run<br>
make lib<br>
make test<br>
make sparse-lib<br>
make sparse-test<br>
However, with cuda 11, the commands<br>
make sparse-lib<br>
make sparse-test<br>
simply fail, and<br>
make lib<br>
has an issue too. Follow <a href="https://icl.cs.utk.edu/magma/forum/viewtopic.php?f=2&amp;t=4193" rel="nofollow noopener">https://icl.cs.utk.edu/magma/forum/viewtopic.php?f=2&amp;t=4193</a> to fix the issue. Ignore make sparse-lib and make sparse-test. make lib and make test are all we need.</p>
</li>
<li>
<p>Finally, follow <a href="https://github.com/pytorch/pytorch" rel="nofollow noopener">https://github.com/pytorch/pytorch</a>, get the PyTorch source<br>
git clone --recursive <a href="https://github.com/pytorch/pytorch" rel="nofollow noopener">https://github.com/pytorch/pytorch</a><br>
cd pytorch<br>
git submodule sync<br>
git submodule update --init --recursive<br>
Instead of run “python setup.py install”, we run<br>
python setup.py bdist_wheel<br>
to build a python wheel package. If everything runs successfully, we will get a *.whl file in a directory named dist.<br>
cd dist<br>
pip install *.whl<br>
(or if not root, sudo pip install *.whl)<br>
That’s it.</p>
</li>
</ol>