Does the torch script support torch.nn.Dropout?

username: kevinhaoliu
cooked: <p>When I want to script a method which calls torch.nn.dropout, it comes up a error message:</p>
<pre><code class="lang-auto">RuntimeError: 
unknown builtin op:
    """
    Chunking.

    Parameters
    ----------
    z_in : ``torch.LongTensor``, required.
       The output of the character-level lstms.
    """

    z_in = self.drop_(z_in)
           ~~~~~~~~~ &lt;--- HERE

    out = self.chunk_layer(z_in).squeeze(1)

    return out
</code></pre>
<pre><code class="lang-auto">self.drop = torch.nn.Dropout(p=droprate)
</code></pre>
<pre><code class="lang-auto">@torch.jit.script
     def chunking(self, z_in):
         """
         Chunking.
 
         Parameters
         ----------
         z_in : ``torch.LongTensor``, required.
            The output of the character-level lstms.
         """
 
         z_in = self.drop(z_in)
 
         out = self.chunk_layer(z_in).squeeze(1)
 
         return out
</code></pre>

username: ptrblck
cooked: <p>Yes, dropout should be supported:</p>
<pre><code class="lang-python">class MyModel(torch.jit.ScriptModule):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(10, 10)
        self.drop = nn.Dropout()
        
    @torch.jit.script_method
    def forward(self, x):
        x = self.fc1(x)
        x = self.drop(x)
        return x

model = MyModel()
x = torch.randn(1, 10)
output = model(x)

traced_model = torch.jit.trace(model, x)
traced_model(x)
</code></pre>
<p>Could you post a code snippet to reproduce this issue?</p>

username: kevinhaoliu
cooked: <p>Thank you for your reply. I have solved it because I found it should be <em><strong><span class="mention">@torch.jit.script_method</span></strong></em> rather than <em><strong><span class="mention">@torch.jit.script</span></strong></em> . Thank you again!</p>

username: kevinhaoliu
cooked: <p>I have another question: when I use the sequential container, it will tell me a confusing error message with this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/1/13f1d6e4f056c3a49b7686e8ba74843d23500cb7.png" data-download-href="https://discuss.pytorch.org/uploads/default/13f1d6e4f056c3a49b7686e8ba74843d23500cb7" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/1/13f1d6e4f056c3a49b7686e8ba74843d23500cb7.png" alt="image" data-base62-sha1="2Qrd3h3WYWcl5e8lqW9moRopDFl" width="690" height="117" data-dominant-color="070708"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">723×123 2.75 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
But the <strong>forward</strong> function doesn’t appear in my code. My PyTorch version is 1.0.0.</p>

username: ptrblck
cooked: <p>I think <a href="https://github.com/pytorch/pytorch/issues/16400#issuecomment-457983682" rel="nofollow noopener">this answer</a> might help.</p>

username: kevinhaoliu
cooked: <p>But the point is that I don’t write this <strong>forward</strong> function. It seems like the function is in the sequential container.</p>

username: ptrblck
cooked: <p>How are you scripting the module?<br>
As you can see in the linked example, the <code>nn.Sequential</code> module is wrapped inside a <code>torch.jit.ScriptModule</code>.</p>

username: kevinhaoliu
cooked: <p>I am using the <code>torch.jit.ScriptModule</code> too. Mine is like that:</p>
<pre><code class="lang-auto">class NER(torch.jit.ScriptModule):
 
      __constants__ = ['rnn_outdim', 'one_direction_dim', 'add_proj']
      def __init__(self, rnn,
                  w_num: int,
                  w_dim: int,
                  c_num: int,
                  c_dim: int,
                  y_dim: int,
                  y_num: int,
                  droprate: float):
  
          super(NER, self).__init__()
  
          ...
  
          if self.add_proj:
          ...
              self.chunk_layer = nn.Sequential(self.to_chunk, self.drop, self.to_chunk_proj, self.drop, self.chunk_weight)
              self.type_layer = nn.Sequential(self.to_type, self.drop, self.to_type_proj, self.drop, self.type_weight)
          else:
              ...
              self.chunk_layer = nn.Sequential(self.to_chunk, self.drop, self.chunk_weight)
              self.type_layer = nn.Sequential(self.to_type, self.drop, self.type_weight)
</code></pre>