nn.Sequential(*layers) forward: with multiple inputs Error

yclin: <p>HI, I have a toy densenet model (e.g.  only one dense block, a copy from official implementation) with TWO variables (x,y) for the denseblock. However, there is an error "    result = self.forward(*input, **kwargs) TypeError: forward() takes 2 positional arguments but 3 were given".  This model works for only one variable (x) .  Currenty, I can still not figure out the reason and was wondering if there is something wrong in the nn.Sequential(*layers) part? Thanks in advance!</p>
<pre><code class="lang-auto">
class BasicBlock(nn.Module):
    def __init__(self, in_planes, out_planes):
        super(BasicBlock, self).__init__()
        self.bn1 = nn.BatchNorm2d(in_planes)
        self.relu = nn.ReLU(inplace=True)
        self.conv1 = nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=1, padding=1, bias=False)

    def forward(self, x,y):
        out1 = self.conv1(self.relu(self.bn1(x)))
        out2 = y
        return torch.cat([x, out1], 1), out2


class DenseBlock(nn.Module):
    def __init__(self, nb_layers, in_planes, growth_rate, block):
        super(DenseBlock, self).__init__()
        self.layer = self._make_layer(block, in_planes, growth_rate, nb_layers)
    def _make_layer(self, block, in_planes, growth_rate, nb_layers):
        layers = []
        for i in range(nb_layers):
            layers.append(block(in_planes+i*growth_rate, growth_rate))
        return nn.Sequential(*layers)
        # return block(in_planes+growth_rate, growth_rate)  #It works by replacing the whole "for" loop  with this line, but we can only obtain one block rather than nb_layes blocks. 
    def forward(self, x,y):
        return self.layer(x,y)

class DenseNet3(nn.Module):
    def __init__(self, n=2, growth_rate=8):
        super(DenseNet3, self).__init__()
        in_planes = growth_rate
        block = BasicBlock

        # 1st conv before any dense block
        self.conv1 = nn.Conv2d(3, in_planes, kernel_size=5, stride=1,
                               padding=0, bias=False)
        # 1st block
        self.block1 = DenseBlock(n, in_planes, growth_rate, block)
        in_planes = int(in_planes+n*growth_rate)

        ####
        # 2st block ...
        # 3st block ...

    def forward(self, x,y):
        x = self.conv1(x)
        out1, out2 = self.block1(x,y)
        # 2nd, 3rd blocks ...
        return  out1, out2
</code></pre>
<p>Error:   File “/home/…/densenet.py”, line 33, in forward<br>
return self.layer(x,y)  (in class DenseBlock)<br>
File “/home/…/miniconda3/envs/Detector/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 489, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
TypeError: forward() takes 2 positional arguments but 3 were given</p>

vmirly1: <p>Generally, the <code>forward()</code> function can be defined to have multiple input arguments. However, I think it is not possible to pass multiple arguments to models wrapped by <code>Sequential()</code> function.</p>
<p>Furthermore, I see that in the following code from the <code>BasicBlock</code>:</p>
<aside class="quote no-group" data-username="yclin" data-post="1" data-topic="35591">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yclin/48/25429_2.png" class="avatar"> yclin:</div>
<blockquote>
<p>def forward(self, x,y): out1 = self.conv1(self.relu(self.bn1(x))) out2 = y return torch.cat([x, out1], 1), out2</p>
</blockquote>
</aside>
<p>the input tensor <code>y</code> is just passed along as output without any changes. So, in fact you would not even need to pass <code>y</code> along with <code>x</code>.</p>

yclin: <p>Thanks very much for your reply! The variable ‘y’ in BasicBlock is just an example to better describe my confusion. I was wondering if there is an alternative to  solve this problem (passing multiple arguments through a wrapped model). Otherwise, we have to do this in a relatively stupid way.</p>

vmirly1: <p>I see. So, I found a workaround solution, you can put the two inputs in a dictionary. I tried that with a dictionary, but it might be also possible to that with list. Here is a working example:</p>
<pre><code class="lang-python">class BasicBlock(nn.Module):
    def __init__(self, in_planes_1, in_planes_2, out_planes):
        super(BasicBlock, self).__init__()
        self.relu = nn.ReLU(inplace=True)
        self.conv1 = nn.Conv2d(in_planes_1, out_planes, kernel_size=3, stride=1, padding=1, bias=False)
        self.conv2 = nn.Conv2d(in_planes_2, out_planes, kernel_size=3, stride=1, padding=1, bias=False)
    def forward(self, x):
        x1, x2 = x[0], x[1]
        out1 = self.conv1(self.relu(x1))
        out2 = self.conv2(self.relu(x2))
        return {0:torch.cat([x1, out1], 1), 1:torch.cat([x2, out2], 1)}

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.main = nn.Sequential(
            BasicBlock(3, 1, 8),
            BasicBlock(11, 9, 16))
    def forward(self, x):
        return self.main(x)
</code></pre>
<p>Note that I have made the input and output consistent using the dictionary.</p>
<p>Now we create a model:</p>
<pre><code class="lang-python">&gt;&gt;&gt; m = MyModel()
&gt;&gt;&gt; m
MyModel(
  (main): Sequential(
    (0): BasicBlock(
      (relu): ReLU(inplace)
      (conv1): Conv2d(3, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (conv2): Conv2d(1, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    )
    (1): BasicBlock(
      (relu): ReLU(inplace)
      (conv1): Conv2d(11, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (conv2): Conv2d(9, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    )
  )
)
</code></pre>
<p>and now we pass the two arguments as follows:</p>
<pre><code class="lang-python">&gt;&gt;&gt; t1 = torch.randn(10, 3, 28, 28)
&gt;&gt;&gt; t2 = torch.randn(10, 1, 28, 28)
&gt;&gt;&gt; output = m({0:t1, 1:t2})
&gt;&gt;&gt; output[0].shape, output[1].shape
&gt;&gt;&gt; type(output)
&lt;class 'dict'&gt;
&gt;&gt;&gt; output.keys()
dict_keys([0, 1])
(torch.Size([10, 27, 28, 28]), torch.Size([10, 25, 28, 28]))
</code></pre>

yclin: <p>Hi, vmirly1,</p>
<p>Cool! Thanks for sharing your ideas!</p>
<p>Cheers,<br>
yclin</p>

turpaultn: <p>Hi,<br>
I don’t know if it is a good way of doing it, but it was working for my simple usage (note that all my models I use in it have <code>*args ,**kwargs</code> in their forward definition to allow other layers to use the additional arguments):</p>
<pre><code class="lang-python">from torch import nn


class CombineModel(nn.Sequential):
    """ Class to combine multiple models. Sequential allowing multiple inputs."""

    def __init__(self, *args):
        super(CombineModel, self).__init__(*args)

    def forward(self, x, *args, **kwargs):
        for i, module in enumerate(self):
            if i == 0:
                x = module(x, *args, **kwargs)
            else:
                x = module(*x, **kwargs)
            if not isinstance(x, tuple) and i != len(self) - 1:
                x = (x,)
        return x
</code></pre>
<p>I’d be interested in your opinion, please let me know if I’m doing something completely wrong or unusable.</p>

yclin: <p>Hi, turpaultn, thanks very much! Personally I prefer the way vmirly1 proposed by using a dictionary, which seems nicer (because of the keys).</p>