c10::NotImplementedError with minimal example

username: pete7.62
cooked: <p>I looked at the other threads with this problem but couldn’t find a solution.</p>
<p>My main file looks like this:</p>
<pre><code class="lang-auto">#include &lt;torch/torch.h&gt;
#include &lt;iostream&gt;

int main() {
	torch::Device device(torch::kCUDA);
	//torch::Tensor tensor = torch::eye(3); // (A)
	torch::Tensor tensor = torch::eye(3, device); // (B)
	std::cout &lt;&lt; tensor &lt;&lt; std::endl;
}
</code></pre>
<p>Running this gives me</p>
<pre><code class="lang-auto">Microsoft C++ exception: c10::NotImplementedError at memory location 0x000000DAA95CF0A0
</code></pre>
<p>Visual Studio is popping that error in the file <code>variable_factories.h</code></p>
<pre><code class="lang-auto">(line 190) return autograd::make_variable(at::eye(n, at::TensorOptions(options).requires_grad(c10::nullopt)), /*requires_grad=*/options.requires_grad());
</code></pre>
<p>If I comment out line (B) and use line (A) instead, it works and prints</p>
<pre><code class="lang-auto">1  0  0
0  1  0
0  0  1
[ CPUFloatType{3,3} ] 
</code></pre>
<p>Here are my include directories:</p>
<pre><code class="lang-auto">(ProjectDir)\libtorch-win-shared-with-deps-1.10.2+cu113\libtorch\include\torch\csrc\api\include
(ProjectDir)\libtorch-win-shared-with-deps-1.10.2+cu113\libtorch\include
(ProjectDir)\opencv\include
</code></pre>
<p>and the linked libraries:</p>
<pre><code class="lang-auto">torch.lib
torch_cpu.lib
torch_cuda.lib
torch_cuda_cu.lib
torch_cuda_cpp.lib
asmjit.lib
c10.lib
c10_cuda.lib
caffe2_detectron_ops_gpu.lib
caffe2_module_test_dynamic.lib
caffe2_nvrtc.lib
clog.lib
cpuinfo.lib
dnnl.lib
fbgemm.lib
libprotobuf-lite.lib
libprotobuf.lib
libprotoc.lib
mkldnn.lib
</code></pre>
<p>Any idea how to solve this?</p>

username: ptrblck
cooked: <p>I cannot reproduce the issue using <code>libtorch==1.10.2_cu113</code> on Linux and get:</p>
<pre><code class="lang-python"> 1  0  0
 0  1  0
 0  0  1
[ CUDAFloatType{3,3} ]
</code></pre>
<p>as the output.<br>
Maybe you are hitting <a href="https://discuss.pytorch.org/t/an-unhandled-exceptionmicrosoft-c-exception-c10-error-at-memory-location/82284/7">this issue</a> on Windows.</p>

username: pete7.62
cooked: <p>Thanks for your reply!</p>
<p>I have downloaded <code>libtorch-win-shared-with-deps-1.10.2+cu113.zip</code> from the pytorch webpage, and I’m using the headers and libraries from that zip, so they should be from the same version. Maybe something is in my %PATH% that’s overriding the dynamic libraries, but the only candidate would be CUDA 11.5 that is installed on my system, and I have already tried removing it from my path, but the problem remains.</p>

username: pete7.62
cooked: <p>What I found out further:<br>
If I use <code>libtorch-win-shared-with-deps-1.10.2+cu102.zip</code> (instead of …cuda113) and use <code>tensor.cuda()</code> it works! But it’s literally the only configuration where it works. If I change <code>.cuda()</code> to <code>torch::zeros(3, device)</code> it doesn’t, also using <code>.cuda()</code> with the <code>...cuda113.zip</code> package does not work…</p>

username: pete7.62
cooked: <p>I found the solution in this github post:</p><aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/54131#issuecomment-800918008">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/54131#issuecomment-800918008" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/54131#issuecomment-800918008" target="_blank" rel="noopener nofollow ugc">libtorch 1.8.0 precompiled has no CUDA backend linked (Adding  "-INCLUDE:?warp_size@cuda@at@@YAHXZ" no longer helps)</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-03-17" data-time="07:08:55" data-timezone="UTC">07:08AM - 17 Mar 21 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2021-03-17" data-time="09:05:03" data-timezone="UTC">09:05AM - 17 Mar 21 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/lablabla" target="_blank" rel="noopener nofollow ugc">
          <img alt="lablabla" src="https://avatars.githubusercontent.com/u/1851781?v=4" class="onebox-avatar-inline" width="20" height="20">
          lablabla
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: windows
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## 🐛 Bug


Trying to load JIT torchscript model in Windows 10 using precompil<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">ed torch 1.8.0 with CUDA 11.1 results in 
```
Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build).
```

This was previously solved by adding `"-INCLUDE:?warp_size@cuda@at@@YAHXZ"` to the linker options of MSVC as suggested [here](https://github.com/pytorch/pytorch/issues/33435#issuecomment-685241862)

when moving to 1.8.0 I added `torch_cuda_cpp.lib` and `torch_cuda_cu.lib` to the linked libraries

The same torchscript works well in 1.7.1

## Environment

 - PyTorch Version (e.g., 1.0): 1.8.0
 - OS (e.g., Linux): Windows 10 (MSVC 2017)
 - How you installed PyTorch (`conda`, `pip`, source): Precompiled binaries
 - Build command you used (if compiling from source):
 - Python version: N/A (Libtorch)
 - CUDA/cuDNN version: 11.1
 - GPU models and configuration: Quatro P1000
 - Any other relevant information:


cc @peterjc123 @maxluk @nbcsm @guyang3532 @gunandrose4u @mszhanyi @skyline75489</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I already had the linker option <code>/INCLUDE:"?warp_size@cuda@at@@YAHXZ"</code> added when I wrote the OP, <strong>but additionally adding</strong> <code>/INCLUDE:"?searchsorted_cuda@native@at@@YA?AVTensor@2@AEBV32@0_N1@Z"</code> <strong>under</strong> <code>Project properties → Linker → Command Line</code> <strong>solved the problem.</strong></p>
<p>Now the main function in the OP results in</p>
<pre><code class="lang-auto">1  0  0
0  1  0
0  0  1
[ CUDAFloatType{3,3} ]```</code></pre>

username: faucozo
cooked: <p>Sorry to refresh the issue, just adding possible solution for other users</p>
<p>I actually had the same error from a model learned on GPU and trying to load on CPU, had c10::NotImplementedError .</p>
<p>By specifying the device the error is gone</p>
<pre><code class="lang-auto">torch::jit::script::Module module;
torch::Device device(torch::kCPU);

module = torch::jit::load("model.pt", device);
</code></pre>
<p>Hope it helps !</p>