What's the difference between nn.ModuleList() and python list?

ljh: <p>Hi! I was constructing a multilayer LSTM by stacking a bunch of LSTM in an list.<br>
I found out that when I’m using the python list, the loss is weird.<br>
But when I’m using the nn.ModuleList(), it’s just normal.<br>
So why can’t I use the python list? Do I have to use nn.Modulelist()?</p>
<p>Here’s the code and loss when using nn.Modulelist()</p>
<pre><code class="lang-auto">class MultiLayerLSTM(nn.Module):
    def __init__(self, input_size, hidden_size,num_layers,batch_first):
        super(MultiLayerLSTM, self).__init__()
        self.num_layers = num_layers
        self.LSTMs = nn.ModuleList()
        self.LSTMs.append(NaiveLSTM(input_size, hidden_size, batch_first=True))
        for i in range(num_layers-1):
            self.LSTMs.append(NaiveLSTM(hidden_size, hidden_size, batch_first=True))

    def forward(self,x,state):
        (h0s,c0s) = state
        for i in range(self.num_layers):
            x,_ = self.LSTMs[i](x,(h0s[i].unsqueeze(0),c0s[i].unsqueeze(0)))
        return x,_
</code></pre>
<pre><code class="lang-auto">Epoch [1/2], Step [100/600], Loss: 0.6575
Epoch [1/2], Step [200/600], Loss: 0.4976
Epoch [1/2], Step [300/600], Loss: 0.1577
Epoch [1/2], Step [400/600], Loss: 0.2070
Epoch [1/2], Step [500/600], Loss: 0.0443
Epoch [1/2], Step [600/600], Loss: 0.1664
Epoch [2/2], Step [100/600], Loss: 0.0811
Epoch [2/2], Step [200/600], Loss: 0.1011
Epoch [2/2], Step [300/600], Loss: 0.1325
Epoch [2/2], Step [400/600], Loss: 0.1050
Epoch [2/2], Step [500/600], Loss: 0.1075
Epoch [2/2], Step [600/600], Loss: 0.0404
</code></pre>
<p>Here’s the code and loss when using python list:</p>
<pre><code class="lang-auto">class MultiLayerLSTM(nn.Module):
    def __init__(self, input_size, hidden_size,num_layers,batch_first):
        super(MultiLayerLSTM, self).__init__()
        self.num_layers = num_layers
        self.LSTMs = []
        self.LSTMs.append(NaiveLSTM(input_size, hidden_size, batch_first=True))
        for i in range(num_layers-1):
            self.LSTMs.append(NaiveLSTM(hidden_size, hidden_size, batch_first=True))

    def forward(self,x,state):
        (h0s,c0s) = state
        for i in range(self.num_layers):
            x,_ = self.LSTMs[i](x,(h0s[i].unsqueeze(0),c0s[i].unsqueeze(0)))
        return x,_
</code></pre>
<pre><code class="lang-auto">Epoch [1/2], Step [100/600], Loss: 2.2161
Epoch [1/2], Step [200/600], Loss: 2.1421
Epoch [1/2], Step [300/600], Loss: 2.1189
Epoch [1/2], Step [400/600], Loss: 2.0571
Epoch [1/2], Step [500/600], Loss: 2.1516
Epoch [1/2], Step [600/600], Loss: 2.0326
Epoch [2/2], Step [100/600], Loss: 1.9656
Epoch [2/2], Step [200/600], Loss: 1.9204
Epoch [2/2], Step [300/600], Loss: 1.9620
Epoch [2/2], Step [400/600], Loss: 1.8612
Epoch [2/2], Step [500/600], Loss: 1.8466
Epoch [2/2], Step [600/600], Loss: 1.8985
</code></pre>

ptrblck: <p>Plain Python lists won’t register the module properly, so that e.g. <code>model.parameters()</code> will not return the internal parameters of the submodules in this list (and thus your optimizer won’t get them).<br>
You shoud use <code>nn.ModuleList</code> instead as described in the <a href="https://pytorch.org/docs/stable/generated/torch.nn.ModuleList.html#torch.nn.ModuleList">docs</a>.</p>