RuntimeError: undefined value, when covert Pytorch to Torchscript

Stephen_K: <p>RuntimeError:<br>
undefined value hidden_t:<br>
File “/workspace/OpenNMT-py-1.2.0_torchscript_testfuncwork/onmt/encoders/image_encoder.py”, line 143<br>
out = torch.cat(all_outputs, 0)</p>
<pre><code>    return hidden_t, out, lengths
           ~~~~~~~~ &lt;--- HERE
</code></pre>
<p>when I tried to convert Pytorch to Torchscript</p>
<p>full error message :<br>
Traceback (most recent call last):<br>
File “translate.py”, line 22, in <br>
print(“my_module(imgr) :”,my_module(imgr))<br>
File “/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “translate.py”, line 18, in forward<br>
result = main(imgr)<br>
File “/workspace/OpenNMT-py-1.2.0_torchscript_testfuncwork/onmt/bin/translate.py”, line 74, in main<br>
return translate(opt, imgr)<br>
File “/workspace/OpenNMT-py-1.2.0_torchscript_testfuncwork/onmt/bin/translate.py”, line 18, in translate<br>
translator = build_translator(opt, logger=logger, report_score=True)<br>
File “/workspace/OpenNMT-py-1.2.0_torchscript_testfuncwork/onmt/translate/translator.py”, line 29, in build_translator<br>
fields, model, model_opt = load_test_model(opt)<br>
File “/workspace/OpenNMT-py-1.2.0_torchscript_testfuncwork/onmt/model_builder.py”, line 112, in load_test_model<br>
model = build_base_model(model_opt, fields, use_gpu(opt), checkpoint,<br>
File “/workspace/OpenNMT-py-1.2.0_torchscript_testfuncwork/onmt/model_builder.py”, line 184, in build_base_model<br>
sm = torch.jit.script(model)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_script.py”, line 942, in script<br>
return torch.jit._recursive.create_script_module(<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_recursive.py”, line 391, in create_script_module<br>
return create_script_module_impl(nn_module, concrete_type, stubs_fn)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_recursive.py”, line 448, in create_script_module_impl<br>
script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_script.py”, line 391, in _construct<br>
init_fn(script_module)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_recursive.py”, line 428, in init_fn<br>
scripted = create_script_module_impl(orig_value, sub_concrete_type, stubs_fn)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_recursive.py”, line 452, in create_script_module_impl<br>
create_methods_and_properties_from_stubs(concrete_type, method_stubs, property_stubs)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_recursive.py”, line 335, in create_methods_and_properties_from_stubs<br>
concrete_type._create_methods_and_properties(property_defs, property_rcbs, method_defs, method_rcbs, method_defaults)<br>
RuntimeError:<br>
undefined value hidden_t:<br>
File “/workspace/OpenNMT-py-1.2.0_torchscript_testfuncwork/onmt/encoders/image_encoder.py”, line 143<br>
out = torch.cat(all_outputs, 0)</p>
<pre><code>    return hidden_t, out, lengths
           ~~~~~~~~ &lt;--- HERE
</code></pre>
<p>what I tried to convert :<br>
model.py</p>
<p>“”" Onmt NMT Model base class definition “”"<br>
import torch.nn as nn</p>
<p><span class="hashtag-raw">#import</span> os</p>
<p>class NMTModel(nn.Module):<br>
“”"<br>
full_path = os.path.realpath(<strong>file</strong>)<br>
path, filename = os.path.split(full_path)<br>
print(path + ’ → ’ + filename + " NMTModel" + “\n”)<br>
“”"<br>
“”"<br>
Core trainable object in OpenNMT. Implements a trainable interface<br>
for a simple, generic encoder + decoder model.</p>
<pre><code>Args:
  encoder (onmt.encoders.EncoderBase): an encoder object
  decoder (onmt.decoders.DecoderBase): a decoder object
"""

def __init__(self, encoder, decoder):
    super(NMTModel, self).__init__()
    self.encoder = encoder
    self.decoder = decoder

def forward(self, src, tgt, lengths, bptt=False, with_align=False):
    """Forward propagate a `src` and `tgt` pair for training.
    Possible initialized with a beginning decoder state.

    Args:
        src (Tensor): A source sequence passed to encoder.
            typically for inputs this will be a padded `LongTensor`
            of size ``(len, batch, features)``. However, may be an
            image or other generic input depending on encoder.
        tgt (LongTensor): A target sequence passed to decoder.
            Size ``(tgt_len, batch, features)``.
        lengths(LongTensor): The src lengths, pre-padding ``(batch,)``.
        bptt (Boolean): A flag indicating if truncated bptt is set.
            If reset then init_state
        with_align (Boolean): A flag indicating whether output alignment,
            Only valid for transformer decoder.

    Returns:
        (FloatTensor, dict[str, FloatTensor]):

        * decoder output ``(tgt_len, batch, hidden)``
        * dictionary attention dists of ``(tgt_len, batch, src_len)``
    """
    print("model.py")
    dec_in = tgt[:-1]  # exclude last target from inputs

    enc_state, memory_bank, lengths = self.encoder(src, lengths)

    if bptt is False:
        self.decoder.init_state(src, memory_bank, enc_state)
    dec_out, attns = self.decoder(dec_in, memory_bank,
                                  memory_lengths=lengths,
                                  with_align=with_align)
    return dec_out, attns

def update_dropout(self, dropout):
    self.encoder.update_dropout(dropout)
    self.decoder.update_dropout(dropout)
</code></pre>
<p>where I got Undefined value error<br>
image_encoder.py<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d8e5dcc300eeef16e18c0135a4c8136bab11b8e.png" data-download-href="https://discuss.pytorch.org/uploads/default/1d8e5dcc300eeef16e18c0135a4c8136bab11b8e" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d8e5dcc300eeef16e18c0135a4c8136bab11b8e.png" alt="image" data-base62-sha1="4dsRDZjzVOBC0zHokarJcUzr6HQ" width="227" height="500" data-dominant-color="30312B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">408×898 37.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Can anyone help?..</p>

AlphaBetaGamma96: <p>It’d be better to get a dev’s opinion on this, but I think this issue is due to the <code>self.rnn</code>. Whereabouts is <code>self.rnn</code> defined? Is it something that’s inferred? Because pytorch is saying <code>hidden_t</code> isn’t defined for some reason!</p>

Stephen_K: <p>self.rnn was defined at image_encoder.py <strong>init</strong></p>
<pre><code>def __init__(self, num_layers, bidirectional, rnn_size, dropout,
             image_chanel_size=3):
    super(ImageEncoder, self).__init__()
    self.num_layers = num_layers
    self.num_directions = 2 if bidirectional else 1
    self.hidden_size = rnn_size

    self.layer1 = nn.Conv2d(image_chanel_size, 64, kernel_size=(3, 3),
                            padding=(1, 1), stride=(1, 1))
    self.layer2 = nn.Conv2d(64, 128, kernel_size=(3, 3),
                            padding=(1, 1), stride=(1, 1))
    self.layer3 = nn.Conv2d(128, 256, kernel_size=(3, 3),
                            padding=(1, 1), stride=(1, 1))
    self.layer4 = nn.Conv2d(256, 256, kernel_size=(3, 3),
                            padding=(1, 1), stride=(1, 1))
    self.layer5 = nn.Conv2d(256, 512, kernel_size=(3, 3),
                            padding=(1, 1), stride=(1, 1))
    self.layer6 = nn.Conv2d(512, 512, kernel_size=(3, 3),
                            padding=(0, 0), stride=(1, 1))

    self.batch_norm1 = nn.BatchNorm2d(256)
    self.batch_norm2 = nn.BatchNorm2d(512)
    self.batch_norm3 = nn.BatchNorm2d(512)

    src_size = 512
    dropout = dropout[0] if type(dropout) is list else dropout
    self.rnn = nn.LSTM(src_size, int(rnn_size / self.num_directions),
                       num_layers=num_layers,
                       dropout=dropout,
                       bidirectional=bidirectional)
    self.pos_lut = nn.Embedding(1000, src_size)</code></pre>

AlphaBetaGamma96: <p>Strange, does <code>self.rnn</code> return 2 arguments? For some reason, Torchscript can’t find the <code>hidden_t</code> tensor generated by <code>self.rnn</code>.</p>

Stephen_K: <p><a href="https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html</a></p>
<p>yes, I tried print hidden_t.</p>