How to reduce size of first dimension?

username: moinudin
cooked: <p>I have an RCC which outputs tensorrs of size <code>516x24</code> that I’d like to get to size <code>4x6</code>. I have a single linear layer that reduces it to <code>516x6</code>, but I don’t know how to reduce the 1st dim of <code>516</code>. Any suggestions? Thanks!</p>

username: Dwight_Foster
cooked: <p>The first dimension is just the batch size. So you can just lower your batch size if you want to get the output to be smaller.</p>

username: moinudin
cooked: <p>My <code>batch_size</code> is <code>4</code>. Input <code>X</code> is <code>(batch_size, 129, 64)</code>. RNN outputs <code>(batch_size, 129, 516)</code>. Then I pass the output through a <code>.contiguous.view(-1, 516)</code>, which gives me a size of <code>(4, 516)</code>, which I pass through a linear layer to get size of <code>(516, 6)</code>. However, I need it to be <code>(4, 6)</code> at the end.</p>

username: Dwight_Foster
cooked: <p>Can you show me your code?</p>

username: moinudin
cooked: <p>I think I figured it out, by changing the input size of my linear layer to be large enough. I don’t know if this is the best approach though. Here’s my model code:</p>
<pre><code class="lang-auto">
    def __init__(self, input_size, output_size, hidden_dim, n_layers):
        super(Net, self).__init__()
        self.input_size  = input_size
        self.output_size = output_size
        self.hidden_dim  = hidden_dim
        self.n_layers    = n_layers

        self.rnn = torch.nn.RNN(self.input_size, self.hidden_dim, self.n_layers, batch_first=True)
        # 64 is the size of my middle (2nd) dimension of X
        self.fc  = torch.nn.Linear(64 * hidden_dim, self.output_size)

    def forward(self, X):
        batch_size  = X.size(0) # 4
        hidden      = self.init_hidden(batch_size)
        out, hidden = self.rnn(X, hidden)
        out = out.contiguous().view(batch_size, -1)
        out = self.fc(out)
        return out, hidden

    def init_hidden(self, batch_size):
        return torch.zeros(self.n_layers, batch_size, self.hidden_dim)```</code></pre>

username: Dwight_Foster
cooked: <p>That looks good. One question however, why are you returning the hidden layer if you just reinitialize it every forward pass? Doesn’t that defeat the purpose of returning it.</p>

username: moinudin
cooked: <p>I was following the tutorial below, which does that. I think it’s just initialising the very first hidden state for each batch, isn’t it? I don’t use the return value though. Does it serve any use outside of training the model?</p>
<aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://blog.floydhub.com/favicon.png" class="site-icon" width="16" height="16">
      <a href="https://blog.floydhub.com/a-beginners-guide-on-recurrent-neural-networks-with-pytorch/" target="_blank" rel="noopener nofollow ugc" title="01:29PM - 29 April 2019">FloydHub Blog – 29 Apr 19</a>
  </header>
  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:112/74;"><img src="https://images.unsplash.com/photo-1527266237111-a4989d028b4b?ixlib=rb-1.2.1&amp;q=80&amp;fm=jpg&amp;crop=entropy&amp;cs=tinysrgb&amp;w=1080&amp;fit=max&amp;ixid=eyJhcHBfaWQiOjExNzczfQ" class="thumbnail" width="112" height="74"></div>

<h3><a href="https://blog.floydhub.com/a-beginners-guide-on-recurrent-neural-networks-with-pytorch/" target="_blank" rel="noopener nofollow ugc">Beginner’s Guide on Recurrent Neural Networks with PyTorch</a></h3>

<p>Learn the basics of Recurrent Neural Networks and build a simple Language Model with PyTorch</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: Dwight_Foster
cooked: <p>No it doesn’t look like it does. It does not matter much though I am glad you fixed the issue.</p>