Unable to read Data using PyTorch C++ API (MNIST)

krshrimali: <p>I’m trying to build MNIST sample file from PyTorch Examples. Here’s what I’ve followed as of now:</p>
<ol>
<li>Clone examples.</li>
<li>Open x64 Native tools Command prompt for VS 2019</li>
<li>Go to the directory and <code>cmake -DCMAKE_PREFIX_PATH="C:\Users\krshr\Downloads\libtorch\libtorch" -G "NMake Makefiles"</code>
</li>
<li>The files were generated successfully.</li>
<li>I then ran “nmake”</li>
<li>It shows some warnings but at last it shows: “Build target mnist”.</li>
<li>I then copied the dll files (torch.dll, caffe2.dll and c10.dll) from unzipped libtorch (nightly version) to the directory having executable file.</li>
<li>Once done, I ran: “mnist.exe” in the prompt and looks like it wasn’t successful while loading MNIST data (I had 2 cout statements before and after loading data - only before one gets printed to the console). The error is: <code>abort() has been called</code>
</li>
</ol>
<p>For CMakeLists.txt and Code: <a href="https://github.com/krshrimali/MNIST-Libtorch-Windows" rel="nofollow noopener">https://github.com/krshrimali/MNIST-Libtorch-Windows</a></p>
<p>Versions: CMake: 3.15.0-rc1 and Visual Studio 2019 Community</p>

dambo: <p>You need all these DLL’s inside the folder where mnist.exe is:</p>
<pre><code class="lang-auto">file(COPY "${DLL_3RD}/torch.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/hdf5.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/opencv_world400.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/zlib.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/caffe2_gpu.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/nvToolsExt64_1.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/c10_cuda.dll" DESTINATION ${CMAKE_BINARY_DIR})
</code></pre>
<p>This is for the CUDA version.</p>

dambo: <p>Also, do you have this in your Cmake?</p>
<pre><code class="lang-auto">set(Torch_DIR f:/libtorch_gpu/share/cmake/Torch/)
FIND_PACKAGE(Torch REQUIRED)

find_package(CUDA REQUIRED)
if(CUDA_FOUND)
    # There is some bug in CMake (FindCUDA.cmake) that causes it to mess up the -std=c++11 option
    # if you let it propagate it to nvcc in some cases.
    if(NOT MSVC)
        set(CUDA_PROPAGATE_HOST_FLAGS OFF)
        list(APPEND CUDA_NVCC_FLAGS -std=c++11)
    endif()
endif()
</code></pre>

krshrimali: <p>Thanks for your response <a class="mention" href="/u/dambo">@dambo</a>. I am trying to run non-CUDA version of libtorch. For the DLL files you mentioned, looks like most of the DLL files are for CUDA version.</p>
<p>As far as the <code>CMake</code> is concerned, I added <code>Torch_DIR</code> and <code>CMAKE_PREFIX_PATH</code> to <code>CMakeLists.txt</code>. It builds successfully as before, but exits abruptly without showing any error or message. [Also, the problem is with the data loading as the <code>cout</code> statement after data loader doesn’t get executed]</p>
<p>I’ve updated the files I’ve here: <a href="https://github.com/krshrimali/MNIST-Libtorch-Windows" rel="nofollow noopener">https://github.com/krshrimali/MNIST-Libtorch-Windows</a></p>
<p>Thanks! Will appreciate any help.</p>

krshrimali: <p>I found this warning message after <code>nmake</code> command.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/e9c1a9a393fcf54b3faf44e36f7d1ac291980a9f.png" data-download-href="https://discuss.pytorch.org/uploads/default/e9c1a9a393fcf54b3faf44e36f7d1ac291980a9f" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/e/e9c1a9a393fcf54b3faf44e36f7d1ac291980a9f.png" alt="image" data-base62-sha1="xlUaB4eGv2GPCvlxKNlr44BL6jl" width="690" height="115" data-dominant-color="1A1A1A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1351×227 13.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Not sure if this has to do with the above problem.</p>

dambo: <p>Do the following:<br>
Right click your executable like so:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/1/15298ab0ce8ff9d355c1b205cd721233863c6ce2.png" alt="2019-06-14%2018_25_33-" data-base62-sha1="31d1Q858yNP5hCZXrcdtf7qfgl4" width="446" height="393"><br>
The run it and wait for windows to report any missing DLL’s.</p>

krshrimali: <p>I tried doing that, it says that it made it compatible for Windows 8 mode and asked to test again. The test fails again.</p>

dambo: <p>What is the exception?</p>

krshrimali: <p>I troubleshooted and it then showed the following:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/2X/e/e6d639408304360a5b2f13b862e83fae158f87e5.png" alt="image" data-base62-sha1="wW4MZFuTFzDumA4dxCTZCrXxGAZ" width="565" height="429"></p>
<p>When I test the program, the executable file ends just before reading data (without showing any error/exception).</p>

krshrimali: <p>I tried to use <code>cmake -g "Ninja"</code> command and then did <code>ninja</code> to make. After running <code>mnist.exe</code> here is the assertion failed error. After ignoring, it ends immediately.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/b/bc69d5ced5753ecd0d5a97d6f2cb004fde119d21.png" data-download-href="https://discuss.pytorch.org/uploads/default/bc69d5ced5753ecd0d5a97d6f2cb004fde119d21" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/b/bc69d5ced5753ecd0d5a97d6f2cb004fde119d21.png" alt="image" data-base62-sha1="qSMvWLpXnYH2GRw0UKPsxwaJJoB" width="690" height="388" data-dominant-color="626162"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">882×496 14.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

peterjc123: <p>Add this to the cmake command <code>-DCMAKE_BUILD_TYPE=Release</code>. The debug and release binaries are not ABI compatible when you use MSVC.</p>

krshrimali: <p>Thanks for the response <a class="mention" href="/u/peterjc123">@peterjc123</a>. I tried doing that, but it still stops abruptly without showing any exception or error.</p>
<p>I also tried downloading the Stable Version of libtorch again, but it still doesn’t work.</p>

krshrimali: <p>Just wondering if the warnings during the make command make any sense? If there’s anything wrong?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/c/c3de382569bcbb31c71904101aee85ac1bb7f143.png" data-download-href="https://discuss.pytorch.org/uploads/default/c3de382569bcbb31c71904101aee85ac1bb7f143" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/c/c3de382569bcbb31c71904101aee85ac1bb7f143.png" alt="image" data-base62-sha1="rWJcZ4EhCMYhARremdG52PTReHV" width="690" height="232" data-dominant-color="141514"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1349×454 21.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

peterjc123: <p>Please build with the debugging version of this library and use VS GUI with the debug configuration to get more info.</p>

peterjc123: <p>And you can add <code>/EHsc</code> to the compiler flags so that the exceptions could be raised.</p>

krshrimali: <p>Thanks for the suggestions, <a class="mention" href="/u/peterjc123">@peterjc123</a>. Here is the error I get while trying to run the executable file.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/1/182ed8202762b3d3f8efe38636a3b871bf596c84.png" data-download-href="https://discuss.pytorch.org/uploads/default/182ed8202762b3d3f8efe38636a3b871bf596c84" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/1/182ed8202762b3d3f8efe38636a3b871bf596c84.png" alt="image" data-base62-sha1="3rVP07ON5E5UbnWC383IlPgWGjO" width="690" height="358" data-dominant-color="4F4F4F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">971×504 26.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>By VS GUI, did you mean to try building with Microsoft Visual Studio and not with Ninja?</p>

peterjc123: <p>Yes. I mean IDE. You’ll need to add <code>Include Directories</code> and <code>Linking libraries</code> manually.</p>

krshrimali: <p>I right clicked on the exception and clicked on “Go to source code”:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/a/a6ef4bee8cd88d675a030f92e8b12bfcf4633391.png" data-download-href="https://discuss.pytorch.org/uploads/default/a6ef4bee8cd88d675a030f92e8b12bfcf4633391" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/a/a6ef4bee8cd88d675a030f92e8b12bfcf4633391_2_690x367.png" alt="image" data-base62-sha1="nOLWWxPjnga7M9bvMKPpnT1z6O5" width="690" height="367" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/a/a6ef4bee8cd88d675a030f92e8b12bfcf4633391_2_690x367.png, https://discuss.pytorch.org/uploads/default/optimized/2X/a/a6ef4bee8cd88d675a030f92e8b12bfcf4633391_2_1035x550.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/a/a6ef4bee8cd88d675a030f92e8b12bfcf4633391.png 2x" data-dominant-color="747475"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1366×727 77.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I then went to the included path of <code>libtorch (debug version)</code> and couldn’t find <code>mnist.cpp</code> in the required location. Is this a mistake?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/0/0ce359fd2d8fa4ba80ef78842866f6d2434e5306.png" data-download-href="https://discuss.pytorch.org/uploads/default/0ce359fd2d8fa4ba80ef78842866f6d2434e5306" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/0/0ce359fd2d8fa4ba80ef78842866f6d2434e5306_2_690x299.png" alt="image" data-base62-sha1="1Q0PkWzm6LN2Ddzj1R1iTTCZTjU" width="690" height="299" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/0/0ce359fd2d8fa4ba80ef78842866f6d2434e5306_2_690x299.png, https://discuss.pytorch.org/uploads/default/optimized/2X/0/0ce359fd2d8fa4ba80ef78842866f6d2434e5306_2_1035x448.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/0/0ce359fd2d8fa4ba80ef78842866f6d2434e5306.png 2x" data-dominant-color="FDFDFD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1366×592 22.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>As you can see, the folder is empty.</p>

krshrimali: <p><a class="mention" href="/u/peterjc123">@peterjc123</a> Is there any requirement for the system to have pytorch installed in Python?</p>

krshrimali: <p>I tried going deeper into the error, it says it couldn’t find mnist.cpp file.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/f/f82fc8295d0fccf0bc6beed67964d3036ac078be.png" data-download-href="https://discuss.pytorch.org/uploads/default/f82fc8295d0fccf0bc6beed67964d3036ac078be" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/f/f82fc8295d0fccf0bc6beed67964d3036ac078be.png" alt="image" data-base62-sha1="zpyMF4MOXvf2PYMkibmWwo0uqCi" width="690" height="126" data-dominant-color="353536"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">981×180 7.55 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>