Libtorch crash in win10

username: zhang-qiang-github
cooked: <p>I want to use the trained model in C++ with libtorch.</p>
<p>However, it failed when I want to load a gpu model</p>
<pre><code class="lang-auto">using torch::jit::script::Module;
Module module = torch::jit::load("resnetGPU.pt", torch::kCUDA);
</code></pre>
<p>The gpu model is trained in linux, and the pytorch version is: <code>1.9.0+cu111</code>, and the libtorch version is <code>libtorch-win-shared-with-deps-debug-1.9.0+cu111</code>.<br>
The c++ project is built in win10, and the vs version is vs2022.</p>
<p>The code to generate the model is:</p>
<pre><code class="lang-auto">import torch
import torchvision.models as models
model = models.resnet50(pretrained=True).cuda()
model=model.eval()
resnet = torch.jit.trace(model, torch.rand(1, 3, 224, 224).cuda())
resnet.save('resnetGPU.pt')

</code></pre>
<p>Any suggestion is appreciated~~</p>

username: JosephC
cooked: <p>It’s a little hard to tell what’s going on without the error logs.  Could you copy/paste the error message here?</p>
<p>My best guess is that the cuda/torch DLL files aren’t getting found when the executable runs.  Make sure they’re on your system path.</p>

username: zhang-qiang-github
cooked: <p><a class="mention" href="/u/josephc">@JosephC</a> Thank you very much for your reply.</p>
<p>It  only show crash message as following:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/2/1278e10054e5e29963bdf1af3c650b26f8839c3b.png" data-download-href="https://discuss.pytorch.org/uploads/default/1278e10054e5e29963bdf1af3c650b26f8839c3b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/1278e10054e5e29963bdf1af3c650b26f8839c3b_2_690x204.png" alt="image" data-base62-sha1="2DpzJsexm2mJ36x7cDGiHvpoTfd" width="690" height="204" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/1278e10054e5e29963bdf1af3c650b26f8839c3b_2_690x204.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/1278e10054e5e29963bdf1af3c650b26f8839c3b_2_1035x306.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/1278e10054e5e29963bdf1af3c650b26f8839c3b_2_1380x408.png 2x" data-dominant-color="969595"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1402×416 25.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>How to check whether the cuda/torch dll has been found?</p>
<p>I build the project by cmake, and I search the cuda in the cmake by:</p>
<pre><code class="lang-auto">find_package(CUDA REQUIRED)
</code></pre>
<p>And the cmake can find the cuda: <code>CUDA_TOOLKIT_ROOT_DIR</code>.</p>
<p>Do you have any other suggestion?</p>

username: JosephC
cooked: <p>Some deeper digging seems to suggest that this might be related to this issue: <a href="https://github.com/pytorch/pytorch/issues/49460" class="inline-onebox" rel="noopener nofollow ugc">Libtorch: Segmentation fault when running torch::jit::load · Issue #49460 · pytorch/pytorch · GitHub</a></p>
<p>If we don’t want to assume that’s the source of the problem, the following are guesses:</p>
<p>CMake and find_package will only find the shared libraries during the <em>build</em>.  PyTorch + CUDA also needs to be able to find the DLL files.  I can’t think any simple ways I would recommend.  You could copy the DLLs into the directory with the executable, but they’re huge.</p>
<p>Another thing to check would be if you’re building in DEBUG or RELEASE mode.  If you’re building in release mode but have debug DLLs from PyTorch installed, that can cause issues.  I ran into that once.</p>

username: zhang-qiang-github
cooked: <p>I have copied all DLLs in C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1 to the directory with the executable. But it still crash.</p>
<p>I am sure I build the project in DEBUG, and I set the Torch_DIR as the debug version of libtorch.</p>
<p>The model is generated in linux, and my c++ project is built in win10. Does this cause the crash?</p>

username: JosephC
cooked: <p>Not clear.  As an aside, are you building with MinGW or MSVC?  The libraries they produce are not compatible with each other.  I don’t know if that’s a problem with DLLs, but if there are statically linked libraries getting used, that might be part of it.</p>
<p>What C++ debugging tools do you have available?  It would be nice to step through the code or at least capture more than just an abort().</p>

username: zhang-qiang-github
cooked: <p>I am using vs2022 to build and debug the project. Since I have only the lib of libtorch, I can not step into it.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f93d27a73be87d9323a29ffdfa1e865c4317562.png" data-download-href="https://discuss.pytorch.org/uploads/default/3f93d27a73be87d9323a29ffdfa1e865c4317562" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3f93d27a73be87d9323a29ffdfa1e865c4317562_2_690x354.png" alt="image" data-base62-sha1="94qMRQdFDiisM8ryt4XAa6loueS" width="690" height="354" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3f93d27a73be87d9323a29ffdfa1e865c4317562_2_690x354.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3f93d27a73be87d9323a29ffdfa1e865c4317562_2_1035x531.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3f93d27a73be87d9323a29ffdfa1e865c4317562_2_1380x708.png 2x" data-dominant-color="F2F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1582×813 60.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Please have a look at the above figure. Would it be helpful to locate the error?</p>
<p>In addition, I found another bug:</p>
<pre><code class="lang-auto">torch::Tensor tensor = torch::rand({5, 3}, torch::kCUDA)
</code></pre>
<p>It also crash.  But <code>torch::rand({5, 3})</code> works.</p>

username: ptrblck
cooked: <p>I don’t know what kind of naming the <code>.dll</code>s use on Windows, but wouldn’t <code>torch_cpu.dll</code> indicate you are using a CPU-only build?</p>

username: zhang-qiang-github
cooked: <p>The <code>torch_cpu.dll</code>  also make me confused. But, I am sure I have provided the correct <code>Torch_DIR</code>:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/e/6e656482c7928ebdf9b6668b5e7f105ae4942daa.png" data-download-href="https://discuss.pytorch.org/uploads/default/6e656482c7928ebdf9b6668b5e7f105ae4942daa" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/e/6e656482c7928ebdf9b6668b5e7f105ae4942daa.png" alt="image" data-base62-sha1="fKBKM3t4OjOHnSP9jDxZI6XxVTc" width="690" height="183" data-dominant-color="EDEAEA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1516×403 20.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Is this <code>Torch_DIR</code> correct?</p>
<p>Is there any other method to locate the bug?</p>