AssertionError running a pytorch model in C++ with libtorch

divinho: <p>I’m following the tutorial <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="nofollow noopener">here</a>, and after getting it to load the model I wanted to run it. I had to make some small adjustments to the example code as my input features are indices (and just on the side, I could not find any documentation for <code>torch::ones</code>, expected it to be <a href="https://pytorch.org/cppdocs/api/namespace_torch.html" rel="nofollow noopener">here</a> but didn’t find it, and trying to use <code>torch::kInt64</code> resulted in “is not a member of ‘torch’” despite it being listed on that page?).</p>
<p>I got it to compile with this</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt; // One-stop header.

#include &lt;iostream&gt;
#include &lt;memory&gt;
#include &lt;vector&gt;

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr &lt;&lt; "usage: example-app &lt;path-to-exported-script-module&gt;\n";
    return -1; 
  }

  // Deserialize the ScriptModule from a file using torch::jit::load().
  std::shared_ptr&lt;torch::jit::script::Module&gt; module = torch::jit::load(argv[1]);

  assert(module != nullptr);

  std::vector&lt;torch::jit::IValue&gt; inputs;
  at::TensorOptions tens_opts;
  tens_opts.dtype(at::kLong);
  inputs.push_back(torch::ones({1, 2}, tens_opts));

  auto output = module-&gt;forward(inputs).toTensor();

  std::cout &lt;&lt; output.slice(0) &lt;&lt; std::endl;;
}
</code></pre>
<p>But when running it I get an error:</p>
<pre><code class="lang-auto">terminate called after throwing an instance of 'at::Error'
  what():  isTensor() ASSERT FAILED at [...]/libtorch/include/ATen/core/ivalue.h:153, please report a bug to PyTorch. (toTensor at [...]/libtorch/include/ATen/core/ivalue.h:153)
</code></pre>
<p>I’m doing slicing in the forward definition of my model, could that be an issue? Here’s my model definition</p>
<pre><code class="lang-auto">class CRNN(nn.Module):
    def __init__(self, num_inp, num_hid, num_ff, num_layers, num_out):
        super(CRNN, self).__init__()

        self.embed = nn.Embedding(num_inp, num_hid)
        self.rnn = nn.GRU(num_hid, num_hid, num_layers, batch_first=True)
        self.fc_emb_skip = nn.Linear(num_hid, num_ff)
        self.fc1 = nn.Linear(num_hid + num_ff, num_ff)
        self.fc2 = nn.Linear(num_ff, 3)
        self.hidden_init = nn.Parameter(t.randn(num_layers, 1, num_hid).type(t.FloatTensor), requires_grad=True)
        self.num_inp = num_inp
        self.num_hid = num_hid
        self.num_out = num_out
        self.num_layers = num_layers

    def forward(self, x):
        hidden = self.init_hidden(x.size(0))
        emb = self.embed(x)
        output, hidden = self.rnn(emb, hidden)
        y_skip = F.elu(self.fc_emb_skip(emb[:, :-1]))
        joined_out = t.cat((y_skip, output[:, 1:],), dim=2)
        outview = joined_out.contiguous().view(joined_out.size(0) * joined_out.size(1), joined_out.size(2))
        y = F.elu(self.fc1(outview))
        logprobs = F.log_softmax(self.fc2(y), 1)
        return logprobs, hidden

    def init_hidden(self, batch_size):
        return self.hidden_init.repeat(1, batch_size, 1)
</code></pre>

KevNull: <p>is module-&gt;forward(inputs) returning a tensor, or a tuple of tensors? You might need to unpack the tuple before calling toTensor() from something it contained.</p>

divinho: <p>Oh yeah, you’re right it’s a tuple makes sense why it fails now…</p>
<p>How would you unpack the tuple? It’s not using <code>std::tuple</code> is it?</p>

KevNull: <p>Something like replacing .toTensor() with .toTuple()-&gt;elements()[0].toTensor() will attempt to turn the first element in the tuple into a tensor object.</p>
<p>Not sure if that’s the recommended method.</p>

divinho: <p>That worked! I’ll go with that unless someone else suggests otherwise.</p>

ptrblck: <p>You could also use <a href="https://en.cppreference.com/w/cpp/utility/tuple/get">std::get</a> to get the elements of a tuple.<br>
Not sure if that’s better than your current approach.</p>

KevNull: <p>Interesting to know about std::get, thanks.</p>

qinbo: <p>how to get the 2rd output? when i use .toTuple-&gt;elements()[1].toTensor() with libtorch-1.1.0. thanks</p>