Problems with GPU allocation in Windows Visual Studio 2017

username: Bega
cooked: <p>I am getting the error</p>
<blockquote>
<p>‘is_available’: is not a member of 'at::cuda</p>
</blockquote>
<p>when compiling the following code:</p>
<pre><code class="lang-auto">int main(int argc, const char* argv[]) {
	//if (argc != 2) {
	//	std::cerr &lt;&lt; "usage: example-app &lt;path-to-exported-script-module&gt;\n";
	//	return -1;
	//}

	torch::Device device = torch::kCPU;
	if (torch::cuda::is_available()) {
		std::cout &lt;&lt; "CUDA is available! Training on GPU." &lt;&lt; std::endl;
		device = torch::kCUDA;
	}

	argv[1] = "C:/Users/gjben/Documents/PytorchCPP/torch_example/x64/Release/model.pt";
	// Deserialize the ScriptModule from a file using torch::jit::load().
	std::shared_ptr&lt;torch::jit::script::Module&gt; module = torch::jit::load(argv[1]);

	assert(module != nullptr);
	std::cout &lt;&lt; "ok\n";

	// Create a vector of inputs.
	std::vector&lt;torch::jit::IValue&gt; inputs;
	inputs.push_back(torch::ones({ 1, 3, 224, 224 }));

	auto begin = std::chrono::high_resolution_clock::now();
	// Execute the model and turn its output into a tensor.
	auto output = module-&gt;forward(inputs).toTensor();
	auto end = std::chrono::high_resolution_clock::now();
	std::cout &lt;&lt; std::chrono::duration_cast&lt;std::chrono::milliseconds&gt;(end - begin).count() &lt;&lt; " ms" &lt;&lt; std::endl;

	std::cout &lt;&lt; output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) &lt;&lt; '\n';
</code></pre>
<p>Interestingly, if I use python I can allocate tensors to cuda.<br>
I have tested with the following code, and Nvidia-smi reports the usage of my GPU without problems.</p>
<pre><code class="lang-auto">import torch
a = torch.rand(20000,20000).cuda()
while True:
	a += 1
	a -= 1
</code></pre>
<p>Is this a problem of LibTorch??<br>
Should I reinstall pytorch manually setting TORCH_CUDA_ARCH_LIST??</p>
<p>I have installed pytroch from source (using ninja) and here is my environment:</p>
<pre><code class="lang-auto">PyTorch version: 1.2.0a0+f2623c7
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Microsoft Windows 10 Home
GCC version: (tdm64-1) 5.1.0
Visual Studio version: 2017 v15.9.12
CMake version: version 3.14.5

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: GeForce GTX 1070
Nvidia driver version: 425.25
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\cudnn64_7.dll

Versions of relevant libraries:
[pip] numpy==1.12.1
[pip] numpydoc==0.6.0
[pip] torch==1.2.0a0+f2623c7
[conda] mkl                       2019.0                    &lt;pip&gt;
[conda] mkl                       2017.0.1                      0
[conda] mkl-include               2019.0                    &lt;pip&gt;
[conda] mkl-service               1.1.2                    py36_3
[conda] torch                     1.2.0a0+f2623c7           &lt;pip&gt;
</code></pre>

username: Bega
cooked: <p><a class="mention" href="/u/peterjc123">@peterjc123</a> Sorry for bothering you again but maybe you can help me.</p>

username: peterjc123
cooked: <p>If you are using VS GUI, please remember to add the include dir and the link libraries(e.g. <code>c10.lib</code>, <code>torch.lib</code> and etc.). Also, since the link between caffe2 and caffe2_gpu is broken, please write <code>LoadLibrary('caffe2_gpu.dll');</code> as the first line of the <code>main</code> function.</p>

username: Bega
cooked: <p>Thanks for your reply.<br>
I’ve already added the directories as follows:</p>
<pre><code class="lang-auto">- Linker/Input/Additional Depndencies:
torch.lib;caffe2.lib;caffe2_gpu.lib;c10.lib;c10_cuda.lib
- VC++ Directories/Library Directories:
C:\pytorch\torch\lib
- VC++ Directories/Include Directories:
C:\pytorch\torch\csrc\api\include; C:\pytorch\torch\include;
</code></pre>
<p>Also, I added <code>LoadLibraryA("caffe2_gpu.dll");</code> as the first line of the <code>main</code> function. However, I still have the following errors:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/d/d7d7935aa4059af55e5006f20093ebf2a1051dc3.png" data-download-href="https://discuss.pytorch.org/uploads/default/d7d7935aa4059af55e5006f20093ebf2a1051dc3" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/d/d7d7935aa4059af55e5006f20093ebf2a1051dc3.png" alt="image" data-base62-sha1="uNqwx71GBd0Q3H4JbBd67DwIVi3" width="690" height="133" data-dominant-color="2E3C48"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1305×253 19.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: peterjc123
cooked: <p>Do you have <code>#include &lt;ATen/ATen.h&gt;</code>?</p>

username: Bega
cooked: <p>yes, these are the libraries included:</p>
<pre><code class="lang-auto">#include "pch.h"
#include &lt;torch/script.h&gt; // One-stop header.
#include &lt;ATen/ATen.h&gt;
#include &lt;windows.h&gt;
#include &lt;iostream&gt;
#include &lt;memory&gt;
</code></pre>

username: peterjc123
cooked: <p>Would you please try replacing <code>torch/script.h</code> to <code>torch/extension.h</code>?</p>

username: cyanM
cooked: <p>maybe is #include&lt;torch/torch.h&gt; ?</p>

username: dambo
cooked: <aside class="quote no-group" data-username="Bega" data-post="1" data-topic="47738">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/ee7513/48.png" class="avatar"> Bega:</div>
<blockquote>
<p>device = torch::kCUDA;</p>
</blockquote>
</aside>
<p>You did not use the variable “device” anywhere in your code.</p>
<p>Try this:</p>
<pre><code class="lang-auto"> // Loading your model
    const std::string s_model_name = argv[1];
    std::cout &lt;&lt; " &gt;&gt;&gt; Loading " &lt;&lt; s_model_name &lt;&lt; std::endl;
    auto module = torch::jit::load(s_model_name, torch::kCUDA); // **This is what you should use** 
    assert(module != nullptr);
</code></pre>
<p>And then your tensor:</p>
<pre><code class="lang-auto">inputs.emplace_back(input_tensor.to(torch::kCUDA)); /
</code></pre>