How to convert pt to onnx

fjfjfan: <p>hello, I use the <a href="https://github.com/quark0/darts/tree/master/rnn" rel="nofollow noopener">https://github.com/quark0/darts/tree/master/rnn</a> and simplify the train/test/valid data to get a rnn model, I could get the saved pt files then, but always failed to save it to onnx with error like:</p>
<p>ValueError: NestedIOFunction doesn’t know how to process an input object of type torch.LongTensor</p>

alex.veuthey: <p>What code are you using to save the model to ONNX? I couldn’t find a reference to ONNX in the git you shared.</p>

fjfjfan: <pre><code class="lang-auto">model_pt_path = "test_1.onnx"
data_1 = torch.randn(23, 64)
hidden_1 = torch.randn(1, 64, 850)
output = torch.onnx.export(model, (data_1, hidden_1), model_onnx_path, verbose=True)
</code></pre>
<p>I added code above to try to export the model, thanks a lot!</p>

alex.veuthey: <p>I’ve just read the project requirements, you probably use PyTorch version 0.3.1 as recommended? Then I think <a href="https://discuss.pytorch.org/t/gru-nestediofunction-error/11468">this answer</a> might be relevant.</p>
<p>Try to wrap your tensors <code>data_1</code> and <code>hidden_1</code> in <code>torch.Variable()</code> calls, maybe that will work.</p>
<p>PyTorch versions prior to 0.4.0 used <code>Variable</code> to indicate that a <code>Tensor</code> can be “backwarded” upon, whereas normal <code>Tensor</code>s cannot.</p>

fjfjfan: <p>Alex, thanks a lot!<br>
I modified the code as belew, although the original error gone, but will get a new error:</p>
<pre><code class="lang-auto">    data_1 = torch.randn(23, 64)
    hidden_1 = torch.randn(1, 64, 850)
    variable_data_1 = Variable(data_1)
    variable_hidden_1 = Variable(hidden_1)
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
</code></pre>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydev_run_in_console.py", line 52, in run_file
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/train.py", line 145, in &lt;module&gt;
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/onnx/__init__.py", line 75, in export
    _export(model, args, f, export_params, verbose, training)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/onnx/__init__.py", line 116, in _export
    trace, torch_out = torch.jit.trace(model, args)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 241, in trace
    return TracedModule(f, nderivs=nderivs)(*args, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 268, in forward
    trace, (out_vars, out_struct) = traced_inner(in_vars, in_struct)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 287, in wrapper
    out_vars, out_struct = f(in_vars, in_struct)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 262, in traced_inner
    return _flatten(self.inner(*args, **kwargs))
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 355, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 345, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/model.py", line 131, in forward
    emb = embedded_dropout(self.encoder, input, dropout=self.dropoute if self.training else 0)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/utils.py", line 69, in embedded_dropout
    embed.scale_grad_by_freq, embed.sparse
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/_functions/thnn/sparse.py", line 59, in forward
    output = torch.index_select(weight, 0, indices.view(-1))
TypeError: torch.index_select received an invalid combination of arguments - got (torch.FloatTensor, int, !torch.FloatTensor!), but expected (torch.FloatTensor source, int dim, torch.LongTensor index)
</code></pre>
<p>so modified to</p>
<pre><code class="lang-auto">    data_1 = torch.randn(23, 64).long()
    hidden_1 = torch.randn(1, 64, 850).long()
    variable_data_1 = Variable(data_1)
    variable_hidden_1 = Variable(hidden_1)
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
</code></pre>
<p>will get error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/train.py", line 288, in &lt;module&gt;
    train()
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/train.py", line 225, in train
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/onnx/__init__.py", line 75, in export
    _export(model, args, f, export_params, verbose, training)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/onnx/__init__.py", line 116, in _export
    trace, torch_out = torch.jit.trace(model, args)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 241, in trace
    return TracedModule(f, nderivs=nderivs)(*args, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 268, in forward
    trace, (out_vars, out_struct) = traced_inner(in_vars, in_struct)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 287, in wrapper
    out_vars, out_struct = f(in_vars, in_struct)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 262, in traced_inner
    return _flatten(self.inner(*args, **kwargs))
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 355, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 345, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/model.py", line 131, in forward
    emb = embedded_dropout(self.encoder, input, dropout=self.dropoute if self.training else 0)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/utils.py", line 69, in embedded_dropout
    embed.scale_grad_by_freq, embed.sparse
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/_functions/thnn/sparse.py", line 59, in forward
    output = torch.index_select(weight, 0, indices.view(-1))
RuntimeError: index out of range at /Users/soumith/minicondabuild3/conda-bld/pytorch_1518385717421/work/torch/lib/TH/generic/THTensorMath.c:277
</code></pre>
<p>Also I found that seems PyTorch 0.3.1 does not support rnn model export to onnx very well as <a href="https://github.com/pytorch/pytorch/issues/5234" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/5234</a></p>
<p>So I used the PyTorch 0.4.1/1.1.0 to have a try, but will get errors :</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydev_run_in_console.py", line 52, in run_file
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/train.py", line 145, in &lt;module&gt;
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/__init__.py", line 26, in export
    return utils.export(*args, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/utils.py", line 94, in export
    operator_export_type=operator_export_type)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/utils.py", line 226, in _export
    example_outputs, propagate)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/utils.py", line 177, in _model_to_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args, training)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/utils.py", line 144, in _trace_and_get_graph_from_model
    trace, torch_out = torch.jit.get_trace_graph(model, args)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/jit/__init__.py", line 77, in get_trace_graph
    return LegacyTracedModule(f)(*args, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/jit/__init__.py", line 109, in forward
    out = self.inner(*trace_inputs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 475, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 465, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/model.py", line 131, in forward
    emb = embedded_dropout(self.encoder, input, dropout=self.dropoute if self.training else 0)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/utils.py", line 67, in embedded_dropout
    X = embed._backend.Embedding.apply(words, masked_embed_weight,
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/nn/backends/backend.py", line 10, in __getattr__
    raise NotImplementedError
NotImplementedError
</code></pre>

alex.veuthey: <p>The last error with PyTorch 0.4.1 / 1.1.0 is already clearer than the previous ones. Making progress!</p>
<p>This seems like some module in PyTorch isn’t supported by ONNX yet, maybe… The second last line (<code>X = embed._backend...</code>) in your stacktrace appears to call the backend’s Embedding module, and I have no idea what that could be and how to debug it.</p>
<p>I’m afraid that’s as far as I can help you <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"> Maybe someone else had a similar problem and can comment here?</p>