Cannot assign 'torch.nn.modules.container.Sequential' as parameter

aerinykim: <p>I was following this method<br>
(<a href="https://discuss.pytorch.org/t/dynamic-parameter-declaration-in-forward-function/427" class="inline-onebox">Dynamic parameter declaration in forward function</a>) to dynamically assign parameters in  <code>forward</code>  function.</p>
<p>However, my parameter is not just one single weight tensor but it is  <code>nn.Sequential</code> .</p>
<p>When I implement below:</p>
<pre><code class="lang-auto">class MyModule(nn.Module):
    def __init__(self):
        # you need to register the parameter names earlier
        self.register_parameter('W_di', None)

    def forward(self, input):
        if self.W_di is None:
            self.W_di = nn.Sequential(
                nn.Linear(mL_n * 2, 1024),
                nn.ReLU(),
                nn.Linear(1024, self.hS)).to(device)
</code></pre>
<p>I get the following error.</p>
<pre><code class="lang-auto"> TypeError: cannot assign 'torch.nn.modules.container.Sequential' as parameter 'W_di' (torch.nn.Parameter or None expected)
</code></pre>
<p>Is there any way that I can register  <code>nn.Sequential</code>  as a whole param? Thanks!</p>

ptrblck: <p>You could register the <code>nn.Sequential</code> container directly using <code>self.W_di = nn.Sequential(...)</code>, which will thus register all internal parameters.<br>
I’m not sure, if I understand the use case correctly, but if you need to get only this subset of parameters, you could call:</p>
<pre><code class="lang-python">model.W_di.parameters()
</code></pre>
<p>Would that work for you or do you need to handle these parameters somehow differently?</p>

aerinykim: <p>Hi ptrblck, thanks for the reply.<br>
As you see in the <code>forward</code> function <code>self.W_di = nn.Sequential(...)</code> is how I assigned <code>nn.Sequential</code> to <code>self.W_di</code>. However, when the <code>forward</code> is called, I’m getting an error saying</p>
<p><code>TypeError: cannot assign 'torch.nn.modules.container.Sequential' as parameter 'W_di' (torch.nn.Parameter or None expected)</code></p>
<p>I’m not trying to call parameters nor a subset of params. Just trying to run the training but got this error.</p>
<p>pytorch 1.1.0</p>

ptrblck: <p>In your <code>__init__</code> you are using:</p>
<pre><code class="lang-python">self.register_parameter('W_di', None)
</code></pre>
<p>which will create this error later.<br>
If you don’t need <code>W_di</code> as an <code>nn.Parameter</code>, you could just remove it.</p>

aerinykim: <p>But what if I need to declare it as a parameter? Maybe I wasn’t explaining it fully here but I’m trying to  assign parameter in <code>foward</code> function following this approach <a href="https://discuss.pytorch.org/t/dynamic-parameter-declaration-in-forward-function/427" class="inline-onebox">Dynamic parameter declaration in forward function</a></p>
<p>In that post, the Chief Crazy Person suggested using <code>self.register_parameter</code>.</p>

ptrblck: <p>Ah OK, I clearly misunderstood the use case.<br>
This code should work as the suggested one by Adam <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.submodule = None
        
    def forward(self, x):
        if self.submodule is None:
            self.submodule = nn.Sequential(
                nn.Linear(1, 1),
                nn.ReLU()
            )
        x = self.submodule(x)
        return x

model = MyModel()
model(torch.randn(1, 1))
print(dict(model.named_parameters()))
&gt; {'submodule.0.weight': Parameter containing:
tensor([[-0.1282]], requires_grad=True), 'submodule.0.bias': Parameter containing:
tensor([-0.7143], requires_grad=True)}
</code></pre>