Pytorch multiple inputs in sequential

username: CesMak
cooked: <p>Hey there,</p>
<p>I would like to change my nn.module to have multiple inputs before the final softmax output layer:</p>
<p>I read that sequential is not working for for multiple inputs, that is why I used a seperate module and forward see picture:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/d/8d79ffb29b3cd3bfc7a2ee8cf9a8101cc4e268fc.png" data-download-href="https://discuss.pytorch.org/uploads/default/8d79ffb29b3cd3bfc7a2ee8cf9a8101cc4e268fc" title="Screenshot from 2020-03-22 16-26-22"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8d79ffb29b3cd3bfc7a2ee8cf9a8101cc4e268fc_2_690x374.png" alt="Screenshot from 2020-03-22 16-26-22" data-base62-sha1="kbyHGtmdRp3EiMETTPGJbOx29MU" width="690" height="374" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8d79ffb29b3cd3bfc7a2ee8cf9a8101cc4e268fc_2_690x374.png, https://discuss.pytorch.org/uploads/default/original/3X/8/d/8d79ffb29b3cd3bfc7a2ee8cf9a8101cc4e268fc.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/d/8d79ffb29b3cd3bfc7a2ee8cf9a8101cc4e268fc.png 2x" data-dominant-color="A8AEB6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2020-03-22 16-26-22</span><span class="informations">866×470 81.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Now what currently does not work is that in my sequential I can pass in state input vectors of <strong>torch.Size([180])</strong> or <strong>torch.Size([1999, 180])</strong> or any other 2d tensor <strong>torch.Size([n, 180])</strong></p>
<p>However in my seperate module I get this error if I try to pass in a 2d tensor of  <strong>torch.Size([n, 180])</strong></p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "ppo_witches.py", line 289, in &lt;module&gt;
    main()
  File "ppo_witches.py", line 257, in main
    reward_mean, wrong_moves = ppo.update(memory)
  File "ppo_witches.py", line 168, in update
    logprobs, state_values, dist_entropy = self.policy.evaluate(old_states, old_actions)
  File "ppo_witches.py", line 120, in evaluate
    action_probs = self.action_layer(state)
  File "/home/mlamprecht/Documents/mcts_cardgame/my_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "ppo_witches.py", line 71, in forward
    output =torch.cat( (out1, out2), 0)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 64 and 180 in dimension 1 at /pytorch/aten/src/TH/generic/THTensor.cpp:612
</code></pre>
<p>Does anyone of you know a smart way of solving this issue?</p>

username: ptrblck
cooked: <p>The code is a bit hard to follow in screenshots and it’s a better idea to post the code directly by wrapping it in three backticks ``` <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"><br>
However, based on the screenshots, I assume the error is raised in <code>torch.cat((out1, out2), 0)</code>?<br>
Could you explain the indexing a bit?<br>
It seems you are slicing <code>input</code> in the batch dimension to create <code>out2</code> and I’m not sure why you would want to remove some samples.</p>
<p>Could you post an executable code snippet with some randomly initialized inputs, so that we could debug the code?</p>

username: CesMak
cooked: <p>Sry I thought the image makes my point clearer.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="74040">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>torch.cat((out1, out2), 0)</code> ?</p>
</blockquote>
</aside>
<p>yap this raises the error.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="74040">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>t seems you are slicing <code>input</code> in the batch dimension to create <code>out2</code> and I’m not sure why you would want to remove some samples.</p>
</blockquote>
</aside>
<p>Yes this is exactly my problem I do not want to slice in the batch dimension.</p>
<p><strong>Minimal Example:</strong></p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
class ActorMod(nn.Module):
    def __init__(self, state_dim, action_dim, n_latent_var):
        super(ActorMod, self).__init__()
        self.l1      = nn.Linear(state_dim, n_latent_var)
        self.l1_tanh = nn.Tanh()
        self.l2      = nn.Linear(n_latent_var, n_latent_var)
        self.l2_tanh = nn.Tanh()
        self.l3      = nn.Linear(n_latent_var+60, action_dim)

    def forward(self, input):
        x = self.l1(input)
        x = self.l1_tanh(x)
        x = self.l2(x)
        #return x.softmax(dim=-1)
        out1 = self.l2_tanh(x) # 64x1
        out2 = input[60:120]   # 60x1 this are the cards on the hand of the player!
        print(input.shape, out1.shape, out2.shape)
        output =torch.cat( [out1, out2], 0)
         #124x1 # do I need: x = torch.cat([layer_outputs[i] for i in layer_i], 1)
        x = self.l3(output)
        return x.softmax(dim=-1)

class ActorCritic(nn.Module):
    def __init__(self, state_dim, action_dim, n_latent_var):
        super(ActorCritic, self).__init__()

        self.action_layer = nn.Sequential(
                nn.Linear(state_dim, n_latent_var),
                nn.Tanh(),
                nn.Linear(n_latent_var, n_latent_var),
                nn.Tanh(),
                nn.Linear(n_latent_var, action_dim),
                nn.Softmax(dim=-1)
                )
        self.equivalent   = ActorMod(state_dim, action_dim, n_latent_var)

works = ActorCritic(180, 60, 64)
action_probs = works.equivalent(torch.rand(180)) # works
action_probs = works.action_layer(torch.rand(180)) # works

action_probs = works.action_layer(torch.rand(20,180)) # works
action_probs = works.equivalent(torch.rand(20, 180)) # fails 
</code></pre>
<p>Error:</p>
<pre><code class="lang-auto">torch.Size([180]) torch.Size([64]) torch.Size([60])
torch.Size([20, 180]) torch.Size([20, 64]) torch.Size([0, 180])
Traceback (most recent call last):
  File "test.py", line 44, in &lt;module&gt;
    action_probs = works.equivalent(torch.rand(20, 180))
  File "/home/mlamprecht/Documents/mcts_cardgame/my_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "test.py", line 20, in forward
    output =torch.cat( [out1, out2], 0)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 64 and 180 in dimension 1 at /pytorch/aten/src/TH/generic/THTensor.cpp:612

</code></pre>

username: CesMak
cooked: <p>I think I now found the solution:</p>
<p>Sequential is automatically doing the job for you if you have a 2d input matrix<br>
whereas torch.cat has to be adopted!</p>
<p><strong>minimal working code</strong></p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
class ActorMod(nn.Module):
    def __init__(self, state_dim, action_dim, n_latent_var):
        super(ActorMod, self).__init__()
        self.l1      = nn.Linear(state_dim, n_latent_var)
        self.l1_tanh = nn.Tanh()
        self.l2      = nn.Linear(n_latent_var, n_latent_var)
        self.l2_tanh = nn.Tanh()
        self.l3      = nn.Linear(n_latent_var+60, action_dim)

    def forward(self, input):
        x = self.l1(input)
        x = self.l1_tanh(x)
        x = self.l2(x)
        #return x.softmax(dim=-1)
        out1 = self.l2_tanh(x) # 64x1
        if len(input.shape)==1:
            out2 = input[60:120]   # 60x1 this are the cards on the hand of the player!
            output =torch.cat( [out1, out2], 0)
        else:
            out2 = input[:, 60:120]
            output =torch.cat( [out1, out2], 1) #how to do that?
        print("Input", input.shape, "Out1:", out1.shape, "Out2:", out2.shape)
        x = self.l3(output)
        return x.softmax(dim=-1)

class ActorCritic(nn.Module):
    def __init__(self, state_dim, action_dim, n_latent_var):
        super(ActorCritic, self).__init__()

        self.action_layer = nn.Sequential(
                nn.Linear(state_dim, n_latent_var),
                nn.Tanh(),
                nn.Linear(n_latent_var, n_latent_var),
                nn.Tanh(),
                nn.Linear(n_latent_var, action_dim),
                nn.Softmax(dim=-1)
                )
        self.equivalent   = ActorMod(state_dim, action_dim, n_latent_var)

works = ActorCritic(180, 60, 64)
action_probs = works.equivalent(torch.rand(180)) # works
#action_probs = works.action_layer(torch.rand(180)) # works
#
action_probs = works.action_layer(torch.rand(20,180)) # works
action_probs = works.equivalent(torch.rand(20, 180)) # works as well

</code></pre>