Torch.cat RuntimeError: error in LoadLibraryA

username: maochen
cooked: <p>Trying to do the following on the GPU, matmul is fine, but once hit the torch.cat, it throws the exception:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "~/test.py", line 9, in &lt;module&gt;
    c_x = torch.cat(c_x_output, dim=1)
RuntimeError: error in LoadLibraryA
</code></pre>
<pre><code class="lang-auto">import torch

rand1 = torch.rand((2, 256)).cuda()
rand2 = torch.rand((2, 256)).cuda()

c_x = torch.mm(rand1, rand2.t())

c_x_output = [rand1, rand2]
c_x = torch.cat(c_x_output, dim=1)

print(c_x.shape)
</code></pre>
<p>OS: Win10<br>
python: 3.8.1<br>
torch: 1.4.0<br>
CUDA: 10.1</p>

username: Youyuu_Syuu
cooked: <p>The same exception thrown. <img src="https://discuss.pytorch.org/images/emoji/apple/hot_face.png?v=9" title=":hot_face:" class="emoji" alt=":hot_face:"> <img src="https://discuss.pytorch.org/images/emoji/apple/hot_face.png?v=9" title=":hot_face:" class="emoji" alt=":hot_face:"><br>
OS: Win10<br>
python: 3.8.1<br>
torch: 1.4.0<br>
CUDA: 10.0</p>
<p>anybody else?</p>

username: Youyuu_Syuu
cooked: <p>I downgraded the Python version to 3.7.5 and now it works.</p>

username: moher
cooked: <p>same here,<br>
OS: Win10<br>
python: 3.8.1<br>
torch: 1.4.0<br>
CUDA: 10.1</p>
<p>though, I fixed the problem, by implementing the <code>cat</code> use case that I had.<br>
in my case, this function solved my problem.<br>
this function takes 4 arrays of 4D tensors, and concats them based on <code>dim=1</code>.<br>
this is hardcoded though.</p>
<pre><code class="lang-auto">    def cat(arr, device):
        total_depth = 0
        for x in arr:
            total_depth += x.size()[1]
        num_samples = arr[0].size()[0]
        h = arr[0].size()[2]
        w = arr[0].size()[3]
        
        concated = torch.zeros((num_samples, total_depth, h, w), device=device)
        
        last = 0
        concated[:, :arr[0].size()[1], :, :] = arr[0]
        last = arr[0].size()[1]
        concated[:, last:last + arr[1].size()[1], :, :] = arr[1]
        last = last + arr[1].size()[1]
        concated[:, last:last + arr[2].size()[1], :, :] = arr[2]
        last = last + arr[2].size()[1]
        concated[:, last:, :, :] = arr[3]
        
        return concated
</code></pre>

username: ptrblck
cooked: <p>This might be a Windows-specific error, so maybe <a class="mention" href="/u/peterjc123">@peterjc123</a> might have seen this error before.</p>

username: peterjc123
cooked: <p>Which python distribution are you using? Conda or Python from the official site?</p>

username: peterjc123
cooked: <p>Reproduced with Python3.8.0 from the official site. Looking into it.</p>

username: moher
cooked: <p><a class="mention" href="/u/peterjc123">@peterjc123</a></p>
<p>my python dist:</p>
<pre><code class="lang-auto">Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50) [MSC v.1916 64 bit (AMD64)] on win32
</code></pre>
<p>I am using <code>pip</code> and everything from the official site.<br>
also, cuda specifications:</p>
<pre><code class="lang-auto">nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:12:52_Pacific_Daylight_Time_2019
Cuda compilation tools, release 10.1, V10.1.243
</code></pre>
<p>everything else is on my comment above.</p>

username: peterjc123
cooked: <aside class="quote no-group" data-username="maochen" data-post="1" data-topic="71188">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/maochen/48/17346_2.png" class="avatar"> maochen:</div>
<blockquote>
<p>import torch rand1 = torch.rand((2, 256)).cuda() rand2 = torch.rand((2, 256)).cuda() c_x = torch.mm(rand1, rand2.t()) c_x_output = [rand1, rand2]</p>
</blockquote>
</aside>
<p>The workground is to run the following code after import torch:</p>
<pre data-code-wrap="python"><code class="lang-python">&gt;&gt;&gt; import ctypes
&gt;&gt;&gt; ctypes.cdll.LoadLibrary('caffe2_nvrtc.dll')
</code></pre>

username: maochen
cooked: <p>workaround seems fine as for now … still trying to see if this can be fixed in pytorch next release rather than do this manual caffe2 dll load here …</p>

username: peterjc123
cooked: <p>Fix landed in the release branch.<br>
<aside class="onebox githubpullrequest">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/pull/35368#event-3165630987" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Pull Request">
    <svg width="60" height="60" class="github-icon" viewbox="0 0 12 16" aria-hidden="true"><path d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/pull/35368" target="_blank" rel="nofollow noopener">Load all DLLs in the lib directory for Windows (v1.5.0)</a>
    </h4>

    <div class="branches">
      <code>pytorch:release/1.5</code> ← <code>peterjc123:load_all_dlls_in_lib_directory_v1.5.0</code>
    </div>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-03-25" data-time="06:20:16" data-timezone="UTC">06:20AM - 25 Mar 20 UTC</span>
      </div>

      <div class="user">
        <a href="https://github.com/peterjc123" target="_blank" rel="nofollow noopener">
          <img alt="peterjc123" src="https://avatars3.githubusercontent.com/u/9998726?v=4" class="onebox-avatar-inline" width="20" height="20">
          peterjc123
        </a>
      </div>

      <div class="lines" title="1 commits changed 1 files with 5 additions and 0 deletions">
        <a href="https://github.com/pytorch/pytorch/pull/35368/files" target="_blank" rel="nofollow noopener">
          <span class="added">+5</span>
          <span class="removed">-0</span>
        </a>
      </div>
    </div>

  </div>
</div>
  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

username: lllllliuxt
cooked: <p>I have a similar problem <img src="https://discuss.pytorch.org/images/emoji/apple/dizzy_face.png?v=9" title=":dizzy_face:" class="emoji" alt=":dizzy_face:"><br>
OS:Win10<br>
python:3.7.1<br>
torch:1.3.1<br>
cuda:10.2</p>
<p>but it’s fine when i use the cpu</p>

username: ptrblck
cooked: <p>Is the posted workaround not working for you?</p>

username: peterjc123
cooked: <p>How similar is it? Which python distribution are you using? Do you have Nvidia GPU in your PC? Do you have the GPU driver installed? What is the exact error message? We didn’t compile packages for CUDA 10.2. So does <code>cuda: 10.2</code> mean that you compiled the package by yourself?</p>

username: TBonus
cooked: <p>Hi.<br>
I also have this problem.<br>
OS: win10<br>
using both libtorch 1.5.0 and 1.6.0 downloaded from <a href="http://pytorch.org" rel="nofollow noopener">pytorch.org</a><br>
cuda: 10.1<br>
Only happens in C++ when using torchscript loaded model on GPU, on CPU works fine.<br>
Also no problem when running model in python</p>
<p>Update:<br>
When I copy <code>caffe2_nvrtc.dll</code> to the same folder as executable it works fine. Is there any method to make it work without copying?</p>