Is building from source for windows up to date? (subprocess.CalledProcessError: Command '['cmake', ..] returned non-zero exit status 1 error)

Shisho_Sama: <p>Hi everyone.<br>
I’m in the process of converting our models from Python to C++ for production. For that it seems we need to build Pytorch from source as the prebuilt libtorch doesnt come with torch.h headers and alike !<br>
I’m on Windows 10 1803 and  have Visual Studio 2019 installed (v16.6.3). I dont have any GPUs so I didnt install CUDA! or NVTX.<br>
I have anaconda and python 3.7 installed and all the deps are installed using the following command:</p>
<pre><code class="lang-auto">conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi
</code></pre>
<p>I cloned Pytorch as stated in the readme :</p>
<pre><code class="lang-auto">git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
# if you are updating an existing checkout
git submodule sync
git submodule update --init --recursive
</code></pre>
<p>But upon executing these commands :</p>
<pre><code class="lang-auto">:: [Optional] If you want to build with VS 2019 generator, please change the value in the next line to `Visual Studio 16 2019`.
:: Note: This value is useless if Ninja is detected. However, you can force that by using `set USE_NINJA=OFF`.
set CMAKE_GENERATOR=Visual Studio 16 2019

:: Read the content in the previous section carefully before you proceed.
:: [Optional] If you want to override the underlying toolset used by Ninja and Visual Studio with CUDA, please run the following script block.
:: "Visual Studio 2017 Developer Command Prompt" will be run automatically.
:: Make sure you have CMake &gt;= 3.12 before you do this when you use the Visual Studio generator.
set CMAKE_GENERATOR_TOOLSET_VERSION=14.11
set DISTUTILS_USE_SDK=1
for /f "usebackq tokens=*" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%

:: [Optional] If you want to override the cuda host compiler
set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.11.25503\bin\HostX64\x64\cl.exe

python setup.py install
</code></pre>
<p>I face these errors:</p>
<pre><code class="lang-auto">D:\Codes\pytorch&gt;python setup.py install
Building wheel torch-1.7.0a0+8376284
-- Building version 1.7.0a0+8376284
cmake -GNinja -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_GENERATOR=Visual Studio 16 2019 -DCMAKE_GENERATOR_TOOLSET_VERSION=14.11 -DCMAKE_INSTALL_PREFIX=D:\Codes\pytorch\torch -DCMAKE_PREFIX_PATH=C:\Users\User\Anaconda3\Lib\site-packages -DNUMPY_INCLUDE_DIR=C:\Users\User\Anaconda3\Lib\site-packages\numpy\core\include -DPYTHON_EXECUTABLE=C:\Users\User\Anaconda3\python.exe -DPYTHON_INCLUDE_DIR=C:\Users\User\Anaconda3\include -DPYTHON_LIBRARY=C:\Users\User\Anaconda3/libs/python37.lib -DTORCH_BUILD_VERSION=1.7.0a0+8376284 -DUSE_NUMPY=True D:\Codes\pytorch
CMake Error: Error: generator : Ninja
Does not match the generator used previously: Visual Studio 16 2019
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
Traceback (most recent call last):
  File "setup.py", line 737, in &lt;module&gt;
    build_deps()
  File "setup.py", line 321, in build_deps
    cmake=cmake)
  File "D:\Codes\pytorch\tools\build_pytorch_libs.py", line 59, in build_caffe2
    rerun_cmake)
  File "D:\Codes\pytorch\tools\setup_helpers\cmake.py", line 329, in generate
    self.run(args, env=my_env)
  File "D:\Codes\pytorch\tools\setup_helpers\cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "C:\Users\User\Anaconda3\Lib\subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-GNinja', '-DBUILD_PYTHON=True', '-DBUILD_TEST=True', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_GENERATOR=Visual Studio 16 2019', '-DCMAKE_GENERATOR_TOOLSET_VERSION=14.11', '-DCMAKE_INSTALL_PREFIX=D:\\Codes\\pytorch\\torch', '-DCMAKE_PREFIX_PATH=C:\\Users\\User\\Anaconda3\\Lib\\site-packages', '-DNUMPY_INCLUDE_DIR=C:\\Users\\User\\Anaconda3\\Lib\\site-packages\\numpy\\core\\include', '-DPYTHON_EXECUTABLE=C:\\Users\\User\\Anaconda3\\python.exe', '-DPYTHON_INCLUDE_DIR=C:\\Users\\User\\Anaconda3\\include', '-DPYTHON_LIBRARY=C:\\Users\\User\\Anaconda3/libs/python37.lib', '-DTORCH_BUILD_VERSION=1.7.0a0+8376284', '-DUSE_NUMPY=True', 'D:\\Codes\\pytorch']' returned non-zero exit status 1.
</code></pre>
<p>What am I doing wrong?<br>
Any help is greatly appreciated</p>

Shisho_Sama: <p>Looking at this <a href="https://discuss.pytorch.org/t/pytorch-build-from-source-on-windows/40288/9">link</a> everything built successfully! without even the need for any of these :</p>
<pre><code class="lang-auto">:: [Optional] If you want to build with VS 2019 generator, please change the value in the next line to `Visual Studio 16 2019`.
:: Note: This value is useless if Ninja is detected. However, you can force that by using `set USE_NINJA=OFF`.
set CMAKE_GENERATOR=Visual Studio 16 2019

:: Read the content in the previous section carefully before you proceed.
:: [Optional] If you want to override the underlying toolset used by Ninja and Visual Studio with CUDA, please run the following script block.
:: "Visual Studio 2017 Developer Command Prompt" will be run automatically.
:: Make sure you have CMake &gt;= 3.12 before you do this when you use the Visual Studio generator.
set CMAKE_GENERATOR_TOOLSET_VERSION=14.11
set DISTUTILS_USE_SDK=1
for /f "usebackq tokens=*" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%

:: [Optional] If you want to override the cuda host compiler
set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.11.25503\bin\HostX64\x64\cl.exe

python setup.py install
</code></pre>
<p>I just ran :</p>
<pre><code class="lang-auto">python setup.py build --cmake
python setup.py install
</code></pre>
<p>and everything got built!<br>
However, after building everything, I cant seem to be able to find &lt;torch/torch.h&gt; ! I read it on this issue <a href="https://github.com/pytorch/pytorch/issues/5964#issuecomment-375698855" rel="nofollow noopener">here</a> that building from source will make that but apparently this is not the case here!</p>
<p>Any help is greatly appreciated</p>

Shisho_Sama: <p>OK, Here are my findings.<br>
First of all <code>include &lt;torch/torch.h&gt;</code> doesnt need building from source. its just in the a different path that is,<br>
in include directories, one need to add both :<br>
<code>$(SolutionDir)Dependencies\libtorch-debug-latest\libtorch\include; $(SolutionDir)Dependencies\libtorch-debug-latest\libtorch\include\torch\csrc\api\include;</code><br>
and voila you are good to go!</p>