Problem executing a model in C++

username: Roos
cooked: <p>Hi,</p>
<p>I have a model which loads fine in C++, ( and works in Python ), but when I call ‘forward’ in C++ it fails with (unhandled) torch::jit::constant_not_supported_error exceptions from insertConstant.The model consists of convolutions followed by transposed convolutions. I’m working on visual studio 2017.</p>
<p>A simple network causing the problem:<br>
self.common_part = nn.Sequential(<br>
nn.Conv2d(1,   128, 5, 2,padding = 2),<br>
nn.ReLU(),<br>
nn.ConvTranspose2d(128, 1, 5, 2,padding = 2, output_padding = 1),<br>
nn.ReLU())<br>
with forward(self,x):<br>
n = self.common_part(x)<br>
return n</p>
<p>Any help would be appreciated.</p>
<p>Regards,</p>
<p>Roos</p>

username: tom
cooked: <p>I think you’d need to post a more complete overview of what you’re doing (instantiating saving the model and the C++ side). If you use triple backticks ``` to mark the beginning and end of the code, you’ll keep the forums from screwing up the formatting.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Roos
cooked: <p>Hi Thomas,</p>
<p>Thank you for your answer, the C++ is simple and the input tensor is ok(  I checked that, I also tried with torch::ones giving the same errors): Cuda or no Cuda : same errors.</p>
<pre><code class="lang-auto">    ....
    std::shared_ptr&lt;torch::jit::script::Module&gt; ptrModule = torch::jit::load(path_to_model));
    if (!ptrModule)
    {
        _ASSERT(false);
    }
    ptrModule-&gt;to(at::kCUDA);
    std::vector&lt;torch::jit::IValue&gt; inputs;
    std::vector&lt;int64_t&gt; sizes = { 1, 1, 320, 320 };
    at::TensorOptions options(at::ScalarType::Float);
    at::Tensor tensor_image = torch::from_blob((void*)m_testImage.GetBuffer(), at::IntList(sizes), options);
    tensor_image.to(at::kCUDA);
    inputs.push_back(tensor_image);
    auto outputs = ptrModule-&gt;forward(inputs).toTensor();
    ...
</code></pre>
<p>outputs is a NULL pointer…<br>
Hope this is enough information.</p>
<p>Greetings,</p>
<p>Roos</p>

username: Roos
cooked: <p>Hi!</p>
<p>I found some strange things. The exceptions can be ignored. As a sanity check I ran the example code with resnet18 of pytorch and this worked fine. When I train my model on the cpu the output of the forward function in C++ is fine. But when I train it on the GPU it is ‘None’, both on the cpu and on the gpu. In python this GPU trained model works fine both on the cpu and gpu. The cuda versions are the same ( 10 ).</p>
<p>Can anybody help me figuring out what is wrong?</p>
<p>Greetings,</p>
<p>Roos</p>

username: Roos
cooked: <p>An addition:<br>
In the call stack there is an exception in checked_tensor_unwrap.<br>
“”"<br>
if (tensorTypeIdToBackend(expr.type_id()) != backend) {<br>
AT_ERROR(“Expected object of backend “, backend, " but got backend “, tensorTypeIdToBackend(expr.type_id()),<br>
" for argument #”, pos, " '”, name, “’”);<br>
“””<br>
the backend is CPU(0) the pos = 2 and the name = ‘weight’ the expr.typeid = 4</p>

username: lsrk
cooked: <p>Have you tried?</p>
<pre><code class="lang-auto">tensor_image = tensor_image.to(at::kCUDA);
</code></pre>

username: Roos
cooked: <p>Yes I did. Both the model and the tensor to Cuda.</p>

username: Roos
cooked: <p>Be greeted,</p>
<p>This seems a bug to me. If I run example code from pytorch, but move the model to the GPU before tracing I get the same problem in C++.</p>
<pre><code class="lang-auto"> model = torchvision.models.resnet18()

# An example input you would normally provide to your model's forward() method.
example = torch.rand(1, 3, 224, 224)
model.cuda()
device  = torch.device("cuda")
# Use torch.jit.trace to generate a torch.jit.ScriptModule via tracing.
traced_script_module = torch.jit.trace(model, example.to(device))
traced_script_module.save("d:\\tmp\\model.pt")
</code></pre>
<p>Greetings,</p>
<p>Roos</p>

username: Roos
cooked: <p>Addition:<br>
If I move the model to the CPU before tracing and saving I can run ‘forward’ successfully on the CPU. But not on the GPU. Probably a bug I suppose.</p>

username: Roos
cooked: <p>Forget this post: I the biggest of oafs. Linus was right. I did call to(at::kCuda) but did not assign it to a tensor object…</p>
<p>I was confused by the ‘to’ call. Say you have a Car class. Then you expect MyCar.to(garage) to move your car to the garage. Not a copy of it…</p>

username: tom
cooked: <p>Hi Roos,</p>
<p>certainly looks like a bug. Which version of PyTorch are you using?</p>
<p>Best regards</p>
<p>Thomas</p>

username: Roos
cooked: <p>Hi Thomas,</p>
<p>It is not a bug. In my code I called <code>tensor_image.to(at::kCuda)</code> where I should call <code>tensor_image = tensor_image.to(at::kCuda)</code>.</p>
<p>Greetings,</p>
<p>Roos</p>

username: tom
cooked: <p>Oh. ha. Glad you solved it. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>