How can I pass multiple inputs to nn.Sequential(*layers)?

shyu: <p>I’m trying to generate a network with sequential blocks and multiple inputs(images) using nn.Sequential(*layer). As I knew, nn.Sequential cannot handle multiple inputs.<br>
In the post <a href="https://discuss.pytorch.org/t/nn-sequential-layers-forward-with-multiple-inputs-error/35591/4">nn.Sequential(*layers) forward: with multiple inputs Error</a>, they said that multiple inputs can be passed to nn.Sequential(*layer) using the dictionary, but it’s not working in my code below.</p>
<pre><code class="lang-auto">class BlkGenerator(nn.Module):
    def __init__(self, block, num_filter, num_blocks, growh_rate):
        super(BlkGenerator, self).__init__()
        self.block = self.BlockGenerator(block, num_filter, num_blocks, growh_rate)

    def BlockGenerator(self, block, num_filter, num_blocks, growh_rate):
        blocks = []

        for i in range(num_blocks):
            dblk = block(num_filter * growh_rate, kernel_size=8, stride=4, padding=2)
            blocks.append(dblk) 

        return nn.Sequential(*blocks)

    def forward(self, x1, x2):
        return self.block(x1, x2)
</code></pre>
<p>I passed two input images to a network using the dictionary</p>
<pre><code class="lang-auto">output = model_4ch({0:input2, 1:input2})
</code></pre>
<p>but, it gives an error below</p>
<pre><code class="lang-auto">File "C:\Users\shyu\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'x2'
</code></pre>
<p>How can I solve this problem?</p>
<p>Thanks in advance.</p>

Oli: <p>Hi there.<br>
The error you get has nothing to do with your nn.Sequential code but is strictly Python based. Your code line <code>output = model_4ch({0:input2, 1:input2})</code> sends <strong>one</strong> dict as an input to the forward function whilst you’ve defined <strong>two</strong> (both x1 and x2) inputs to the Python function.</p>
<p>Change your line<code>output = model_4ch({0:input2, 1:input2})</code> into <code>output = model_4ch(input2, input2)</code> (or maybe <code>input1 input2</code>?)</p>
<p>I’m pretty sure that your model can only handle one input so you’d have to combine your x1 &amp; x2 before feeding that input to the model or feed the inputs separately. This could be done by the following code</p>
<pre><code class="lang-auto">
def forward(self, x1, x2):
        inputs = torch.cat((x1, x2))
        return self.block(inputs)
</code></pre>
<p>or</p>
<pre><code class="lang-auto">def forward(self, x1, x2):
        output1 = self.block(x1)
        output2 = self.block(x2)
        return output1, output2
</code></pre>
<p>If this is not what you’re looking for, give us some more backbone to what you want to do. Gl <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

shyu: <p>Thanks for your reply.</p>
<p>I read the link again in my post and understood that he passed multiple inputs using the dictionary to two basic networks, which were defined by nn.Sequential.</p>
<p>I tried to feed <code>(input1, input2)</code> without using the dictionary and got the same error message.</p>
<p>In my case, I have two images having different size and want to simultaneously feed these images to a denseblock of the densenet as an input.</p>
<p>I searched for the solution, but there is no satisfying answer.</p>

shyu: <p>I found a solution to my problem.<br>
Please check  <a href="https://github.com/pytorch/pytorch/issues/19808#" rel="nofollow noopener">Why nn.Sequential can’t handle multiple input?</a> and <a href="https://github.com/pytorch/pytorch/pull/9979#" rel="nofollow noopener">allow nn.Sequential to take multiple inputs</a>.</p>
<p>I defined <code>mySequential</code> class and changed <code>nn.Sequential</code> to <code>mySequential</code>.<br>
Although I don’t know exactly how it works, it makes nn.Sequential feed the multiple inputs to a network if the type of inputs is a tuple.</p>

gabrieldernbach: <p>Put your multiple inputs into a tuple. (x1, x2, x3).<br>
nn.Squential will work exactly the way you know, as it will look like you gave it one input.</p>
<p>In your custom layers simply unpack the tuple and if you need<br>
multiple outputs <code>return out1, out2, out3</code> as this will again become tuple.</p>
<p>If you want to be more pythonic about the unpacking you can surely go for creating your<br>
custom subclass of nn.Sequential and use <code>module(*tuple)</code></p>

Foivos_Diakogiannis: <p>It would be interesting to know if the custom mySequential supports Pipe model parallelism. Any info on that?</p>

richardbaxter: <p>This is a general solution to the thread title;</p>
<pre><code class="lang-auto">class sequentialMultiInput(nn.Sequential):
	def forward(self, *inputs):
		for module in self._modules.values():
			if type(inputs) == tuple:
				inputs = module(*inputs)
			else:
				inputs = module(inputs)
		return inputs

class SubModel(nn.Module):
	def __init__(self):
		#define layers
	def forward(self, x1, x2):
		x1 = ...
		x2 = ...
		return x1, x2
		
class Model(nn.Module):
	def __init__(self):
		layers = []
		layers.append(SubModel())
		layers.append(SubModel())
		layers.append(SubModel())
		self.sequential = sequentialMultiInput(layers)
	def forward(self, x1, x2):
		x1, x2 = self.sequential(x1, x2)
</code></pre>