LSTM Learning, parameter size

username: DonghunP
cooked: <p>I’m a newbie of LSTM network.</p>
<p>I’m training below data form. Data is 64bit and sequence_length is ‘4’</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/7/472c26868758ac83a1d77cd4bbeb95fb6cea50c8.png" data-download-href="https://discuss.pytorch.org/uploads/default/472c26868758ac83a1d77cd4bbeb95fb6cea50c8" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/7/472c26868758ac83a1d77cd4bbeb95fb6cea50c8.png" alt="image" data-base62-sha1="a9CuwKyy8R8EwV4V56zQGyZKxni" width="270" height="500" data-dominant-color="F9F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">275×509 2.27 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<pre><code class="lang-auto">class LSTM(nn.Module):

    def __init__(self, num_classes, input_size, hidden_size, num_layers, seq_length):
        '''
        :param num_classes: 1
        :param input_size:  64
        :param hidden_size: 2
        :param num_layers:  1
        :param seq_length:  4
        '''
        super(LSTM, self).__init__()

        self.num_classes = num_classes
        self.num_layers = num_layers
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.seq_length = seq_length

        self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size,
                            num_layers=num_layers, batch_first=True)

        self.fc = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        h_0 = Variable(torch.zeros(
            self.num_layers, x.size(0), self.hidden_size))

        c_0 = Variable(torch.zeros(
            self.num_layers, x.size(0), self.hidden_size))

        # Propagate input through LSTM
        ula, (h_out, _) = self.lstm(x, (h_0, c_0))

        h_out = h_out.view(-1, self.hidden_size) 

        out = self.fc(h_out)

        return out 
</code></pre>
<p>But it occurs me that<br>
‘Using a target size (torch.Size([50656, 64])) that is different to the input size (torch.Size([50656, 1]))’</p>
<p>I have trouble to traininig.</p>
<p>Could you help me??</p>
<p>I’ll appreciate your help.</p>

username: vdw
cooked: <p>Since you only posted the model but not the code for training, it’s difficult to say what’s going on. Here just some pointers that strike me as odd:</p>
<ul>
<li>
<p>Are you using the whole data set as a single batch? Usually training is done in mini-batches, it’s typically better for training.</p>
</li>
<li>
<p>According to your comments, <code>num_classes = 2</code>, so <code>self.fc(h_out)</code> should return a shape of <code>(batch_size, 2)</code>, in your case <code>(50656, 2)</code>. Your error says something different, though.</p>
</li>
<li>
<p>I haven’t checked it in detail but <code>h_out.view(-1, self.hidden_size)</code> is probably wrong, at least if you increase <code>num_layers</code>. You generally cannot simply enforce a valid shape of <code>(something, hidden_size)</code> just so it works with <code>self.fc</code>. Have a look at <a href="https://discuss.pytorch.org/t/for-beginners-do-not-use-view-or-reshape-to-swap-dimensions-of-tensors/75524">this post</a>.</p>
</li>
</ul>

username: DonghunP
cooked: <p>Thanks for help.<br>
I wrote it wrong.<br>
‘num_classes’ was 1. So it occurs that error.</p>
<p>Anyway I want to train second data in picture not first one.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/7/f786d7361594eb667296cf6e8f395fa6c09fbfa1.png" data-download-href="https://discuss.pytorch.org/uploads/default/f786d7361594eb667296cf6e8f395fa6c09fbfa1" title="1"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/7/f786d7361594eb667296cf6e8f395fa6c09fbfa1.png" alt="1" data-base62-sha1="zjIPswH0tewkIEn26u1EhpsLu7f" width="690" height="147" data-dominant-color="F0E0E0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">1</span><span class="informations">942×202 9.71 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>and then I need to set num_classes as ‘3’ right?</p>
<p>I really appreciate your help.</p>