Build PyTorch from source. Questions

username: VladimirK
cooked: <p>I have GT 710. It has compute capability of 3.5. This means that if I want to use PyTorch with a GPU, I have to build PyTorch from source.<br>
I have already made several attempts but unsuccessfully. I didn’t expect the build process to take hours, in addition CPU is 100% busy. I work on Windows and this makes the installation process even more complicated. I have a few questions and would like to have some answers before I proceed to the next attempt.<br>
I use Unity mlagents toolkit. It requires pytorch &gt;=1.6.0,&lt;1.9.0. I chose the minimum configuration (pytorch 1.6, CUDA 9.2, cuDNN v7) to ease the build process.<br>
Question 1: <strong>Is it justified to use the minimal configuration?</strong><br>
I found the pytorch 1.6 repository at <a href="https://github.com/pytorch/pytorch/tree/release/1.6" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/tree/release/1.6</a> but I’m not sure if this is correct.<br>
Question 2: <strong>Where should I look for pytorch repositories of different versions?</strong><br>
I have Visual Studio 2019 16.11.7 and I can’t install toolset 14.11. The instructions say:<br>
“There is no guarantee of the correct building with VC++ 2017 toolsets, others than version 15.4 v14.11.”<br>
Does this apply to Studio 2019?<br>
Question 3: <strong>Can I use Visual Studio 2019 to build?</strong><br>
I do not see how I can specify the desired compute capability. May be</p>
<blockquote>
<p>set TORCH_CUDA_ARCH_LIST=“3.5”</p>
</blockquote>
<p>Question 4: <strong>How do I specify the desired compute capability?</strong><br>
Let’s say I built the pytorch binaries successfully. Where are these binaries located? How do I install these binaries?<br>
Question 5: <strong>How do I install new pytorch binaries?</strong></p>

username: ptrblck
cooked: <ol>
<li>
<p>You can pick any PyTorch tag, which would support your setup (e.g. CUDA9.2). The currently required min. CUDA toolkit version is 10.2, so you should double check if PyTorch <code>1.6.0</code> supports 9.2. Yes, the <code>release/1.6</code> branch is correct. You could alternatively use <code>git checkout v1.6.0</code>.</p>
</li>
<li>
<p>The different release versions are all tagged in the GitHub repository. I don’t know which VisuaStudio versions are required as I’m not using Windows.</p>
</li>
<li>
<p>See 2.</p>
</li>
<li>
<p>Using <code>TORCH_CUDA_ARCH_LIST</code> is correct. The source build will be installed into your current environment if you use <code>python setup.py install</code> and will be usable there. The installed package will be located in your env also. E.g. if you are using <code>conda</code> to manage your environments, you would find the package in e.g. <code>/opt/conda/lib/python3.8/site-packages/torch</code> if you install it in the base environment.</p>
</li>
<li>
<p>See 4.</p>
</li>
</ol>

username: VladimirK
cooked: <p>Thanks for the answer, <a class="mention" href="/u/ptrblck">@ptrblck</a>. Everything is clear.</p>

username: VladimirK
cooked: <p>I have built pytorch with cc 3.5 and this works for me. Then I built a whl file “torch-1.6.0a0+ 31f58d-cp37-cp37m-win_amd64.whl”, but it does not contain CUDA.</p>
<p>How to build a .whl like the official one?</p>
<p>I followed these steps:<br>
First I installed Visual Studio 2017 with the toolset 14.11.<br>
Then I installed CUDA 9.2 and cuDNN v7.<br>
Clone PyTorch Source:</p>
<pre><code class="lang-auto">git clone --branch release/1.6 https://github.com/pytorch/pytorch.git pytorch-1.6
cd pytorch-1.6
git submodule sync
git submodule update --init --recursive
</code></pre>
<p>In Anaconda Prompt:</p>
<pre><code class="lang-auto">conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi

set USE_NINJA=OFF
set CMAKE_GENERATOR=Visual Studio 15 2017
set CMAKE_GENERATOR_TOOLSET_VERSION=14.11
set DISTUTILS_USE_SDK=1

for /f "usebackq tokens=*" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%

set TORCH_CUDA_ARCH_LIST=3.5

python setup.py install
</code></pre>
<p>After 5 hours, the build process was completed successfully. Then I built whl:</p>
<pre><code class="lang-auto">python setup.py bdist_wheel
</code></pre>
<p>I got whl in a few minutes. I installed this wheel in a separate conda environment and tested it.</p>
<pre><code class="lang-auto">(test-whl) E:\PyTorchSource&gt;pip install pytorch-1.6\dist\torch-1.6.0a0+b31f58d-cp37-cp37m-win_amd64.whl

(test-whl) E:\PyTorchSource&gt;conda list
# packages in environment at C:\Users\qwego\Anaconda3\envs\test-whl:
#
# Name                    Version                   Build  Channel
ca-certificates           2021.10.26           haa95532_2
certifi                   2021.10.8        py37haa95532_0
future                    0.18.2                   pypi_0    pypi
numpy                     1.21.4                   pypi_0    pypi
openssl                   1.1.1l               h2bbff1b_0
pip                       21.2.4           py37haa95532_0
python                    3.7.11               h6244533_0
setuptools                58.0.4           py37haa95532_0
sqlite                    3.36.0               h2bbff1b_0
torch                     1.6.0a0+b31f58d          pypi_0    pypi
vc                        14.2                 h21ff451_1
vs2015_runtime            14.27.29016          h5e58377_2
wheel                     0.37.0             pyhd3eb1b0_1
wincertstore              0.2              py37haa95532_2

(test-whl) E:\PyTorchSource&gt;python
Python 3.7.11 (default, Jul 27 2021, 09:42:29) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; import torch
&gt;&gt;&gt; print(torch.cuda.get_arch_list())
['sm_35']
&gt;&gt;&gt; torch.tensor([1.0, 2.0]).cuda()
tensor([1., 2.], device='cuda:0')
&gt;&gt;&gt;
</code></pre>
<p>This works on my computer, but I would like to get an independent whl.</p>

username: ptrblck
cooked: <p>I haven’t built a wheel for Windows (Linux only [with CUDA support]), but would try to reuse the build steps used for the official wheels found e.g. <a href="https://github.com/pytorch/builder/tree/1cd2f2d4ad2b5614764105074673e1218c81c84c/windows">here</a> for Windows.</p>

username: mszhanyi
cooked: <p>wheel is just a zip.<br>
I think you could unzip the torch-1.6.0a0+b31f58d-cp37-cp37m-win_amd64.whl and copy some binaries as   <a href="https://github.com/pytorch/builder/blob/main/windows/internal/copy.bat" rel="noopener nofollow ugc">builder/copy.bat at main · pytorch/builder (github.com)</a>.<br>
Then, zip it again.</p>
<p>Note: keep the rezipped package name as  torch-1.6.0a0+b31f58d-cp37-cp37m-win_amd64.whl.</p>

username: VladimirK
cooked: <p>I already thought about something similar. I thought to copy the files to the build folder, then create the wheel: python setup.py bdist_wheel. But I’m not sure if this is enough.</p>

username: VladimirK
cooked: <p>I added missing files to my build and built the wheel. <a href="https://drive.google.com/file/d/1HZIGvM1_hdHpCLkh5AfZg_h9znAcwMpP/view?usp=sharing" rel="noopener nofollow ugc">torch-1.6.0a0+b31f58d-cp37-cp37m-win_amd64.whl</a><br>
I have been tested this file on my computer in another Windows (Windows 7).</p>