Append() for nn.Sequential or directly converting nn.ModuleList to nn.Sequential

jingweiz: <p>Hi,<br>
maybe I’m missing sth obvious but there does not seem to be an “append()” method for nn.Sequential, cos it would be handy when the layers of the sequential could not be added at once.<br>
Or it would be equivalent if I first added all the layer I need into a ModuleList then there’s a method for directly converting all the modules in a ModuleList to a Sequential.<br>
Thanks in advance!</p>

allenye0119: <p>You can first construct a python list of <code>nn.Modules</code> and <strong>unpack</strong> it into a <code>nn.Sequential</code></p>
<pre><code class="lang-auto">import torch.nn as nn

modules = []
modules.append(nn.Linear(10, 10))
modules.append(nn.Linear(10, 10))

sequential = nn.Sequential(*modules)
</code></pre>

jingweiz: <p>Ah that’s neat, thanks <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

QuantScientist: <p>Great example here:<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/vinhkhuc/PyTorch-Mini-Tutorials/blob/master/5_convolutional_net.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/vinhkhuc/PyTorch-Mini-Tutorials/blob/master/5_convolutional_net.py" target="_blank" rel="nofollow noopener">vinhkhuc/PyTorch-Mini-Tutorials/blob/master/5_convolutional_net.py</a></h4>
<pre><code class="lang-py">import numpy as np

import torch
from torch.autograd import Variable
from torch import optim

from data_util import load_mnist


# We need to create two sequential models here since PyTorch doesn't have nn.View()
class ConvNet(torch.nn.Module):
    def __init__(self, output_dim):
        super(ConvNet, self).__init__()

        self.conv = torch.nn.Sequential()
        self.conv.add_module("conv_1", torch.nn.Conv2d(1, 10, kernel_size=5))
        self.conv.add_module("maxpool_1", torch.nn.MaxPool2d(kernel_size=2))
        self.conv.add_module("relu_1", torch.nn.ReLU())
        self.conv.add_module("conv_2", torch.nn.Conv2d(10, 20, kernel_size=5))
        self.conv.add_module("dropout_2", torch.nn.Dropout())
</code></pre>

  This file has been truncated. <a href="https://github.com/vinhkhuc/PyTorch-Mini-Tutorials/blob/master/5_convolutional_net.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

jingweiz: <p>Thanks a lot! That’s really helpful!</p>

Ilassa_Savadogo: <p>Thanks, worked like a charm.</p>

nogaz: <p>Hi,<br>
I’ve had a similar issue and this discussion was very helpful.<br>
I do have another question - I would like to:</p>
<ul>
<li>extract a copy of the current state of the tensor to a “skip connection” list in a U-shaped cnn.</li>
<li>adapt the tensor’s shape (that is the input) as I go (can not determine this ahead because I can’t know the number of convolutions ahead)</li>
</ul>
<p>how can I do this while I’m adding modules to the list? I would like to make the list as modular as I can and can not skip the “append” stage in a for loop.</p>
<p>Thank you!</p>

Snowball1: <p>For people who dont want to update their CUDA or Pytorch to the version(e.g. v1.13.0) that Sequential have append() method, you can just commend you local Sequential code, and copy the “<a href="https://github.com/pytorch/pytorch/blob/v1.13.0/torch/nn/modules/container.py#L29" class="inline-onebox" rel="noopener nofollow ugc">pytorch/torch/nn/modules/container.py at v1.13.0 · pytorch/pytorch · GitHub</a>” code to your local Sequential code position, and then it can use append() method.</p>
<p>But remember, do not copy Sequential code from the latest pytorch version,because it need python v11 to support some new feature.</p>