Libtorch can't run on RTX3090 GPU

username: zhijie1996
cooked: <p>I use libtorch on RTX3090, but it occurs:</p>
<pre><code class="lang-auto">terminate called after throwing an instance of 'std::runtime_error'
  what():  nvrtc: error: invalid value for --gpu-architecture (-arch)
</code></pre>
<p>Configuration:<br>
pytorch v1.7.0 -&gt;libtorch<br>
CUDA11.0</p>

username: ptrblck
cooked: <p>Could you post an executable code snippet to reproduce this issue?<br>
I’ve used <code>libtorch==1.7.0+CUDA11.0</code> and adapted the <a href="https://pytorch.org/tutorials/advanced/cpp_export.html">CPP Export tutorial</a> to reproduce this issue on a 3090, but the code works fine:</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt; // One-stop header.

#include &lt;iostream&gt;
#include &lt;memory&gt;

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr &lt;&lt; "usage: example-app &lt;path-to-exported-script-module&gt;\n";
    return -1;
  }


  torch::jit::script::Module module;
  try {
    // Deserialize the ScriptModule from a file using torch::jit::load().
    module = torch::jit::load(argv[1]);
    module.to(torch::kCUDA);
    auto tensor = torch::randn({1, 3, 224, 224}).to(torch::kCUDA);
    std::vector&lt;torch::jit::IValue&gt; inputs;
    inputs.push_back(tensor);
    auto output = module.forward(inputs).toTensor();
    std::cout &lt;&lt; output &lt;&lt; std::endl;
  }
  catch (const c10::Error&amp; e) {
    std::cerr &lt;&lt; "error loading the model\n";
    return -1;
  }

  std::cout &lt;&lt; "ok\n";
</code></pre>
<p>Executed via:</p>
<pre><code class="lang-auto">cmake -DCMAKE_PREFIX_PATH=/workspace/src/libtorch .. &amp;&amp; cmake --build . --config Release &amp;&amp; ./example-app ../traced_resnet_model.pt
</code></pre>

username: zhijie1996
cooked: <p>The above code also works fine for me. The module in my project is similar to the code snippet, and it works fine when I extract the module out of my project. But it always occurs the above error when I run the project. I don’t know why.</p>

username: zhijie1996
cooked: <p>I rebuild libtorch using CUDA11.1 and solve this problem.</p>

username: mcj
cooked: <p>Could you send me a builded torch ? My email <a href="mailto:2545407140@qq.com">2545407140@qq.com</a>. <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

username: MrRace
cooked: <p>If inputs is batch data, for example inputs contain 32 images(batch size=32) and each image is (3,224,224). How should I do? Just like <strong>method 1</strong> below：</p>
<p><strong>method 1:</strong></p>
<pre><code class="lang-auto">auto tensor = torch::randn({batch_size, 3, 224, 224}).to(torch::kCUDA);
std::vector&lt;torch::jit::IValue&gt; inputs;  
inputs.push_back(tensor);
auto output = module.forward(inputs).toTensor();
</code></pre>
<p>In this method, variable <code>inputs</code>  is a <code>std::vector</code> and its size is always 1 no matter what  batch size is.<br>
Because we just call <code>push_back</code> one time. If just one time <code>push_back</code> why use <code>std::vector&lt;torch::jit::IValue&gt;</code> which is a vector that can store multiple elements.</p>
<p>If I try to put the batch size dimension in above Vec like below, method 2, but it seems not work.<br>
<strong>method 2:</strong></p>
<pre><code class="lang-auto">std::vector&lt;torch::jit::IValue&gt; inputs;  
for (int i=0;i&lt;32;I++)
{
    auto tensor = torch::randn({3, 224, 224}).to(torch::kCUDA);
    inputs.push_back(tensor);
}
auto output = module.forward(inputs).toTensor();
</code></pre>

username: ptrblck
cooked: <p>You could push the tensors into a <code>std::vector&lt;torch::Tensor&gt;</code>, use <code>torch::cat</code> to create an input batch, and pass it then to the model:</p>
<pre><code class="lang-python">  // Create a vector of inputs.
  std::vector&lt;torch::Tensor&gt; inputs;
  for (int i=0; i&lt;32; i++) {
    inputs.push_back(torch::ones({1, 3, 224, 224}));
  }
  auto input = torch::cat(inputs);

  // Execute the model and turn its output into a tensor.
  at::Tensor output = module.forward({input}).toTensor();
</code></pre>