Convert RNN to FC gives error

username: chandra_sutrisno
cooked: <p>Hi all,</p>
<p>I am new on this thing but would like to get an understanding of the problem I have.</p>
<p>I convert RNN model, like below:</p>
<blockquote>
<p>class RNN(nn.Module):<br>
def <strong>init</strong>(self, input_dim, embedding_dim, hidden_dim, output_dim):<br>
super().<strong>init</strong>()</p>
<pre><code>    self.embedding = nn.Embedding(input_dim, embedding_dim)
    self.rnn = nn.RNN(embedding_dim, hidden_dim)
    self.fc = nn.Linear(hidden_dim, output_dim)
    
def forward(self, x):

    embedded = self.embedding(x)
    output, hidden = self.rnn(embedded)
    out = self.fc(hidden)
    
    return out
</code></pre>
</blockquote>
<p>To Fully Connected Network like below:</p>
<blockquote>
<p>class Net(nn.Module):<br>
def <strong>init</strong>(self, input_dim, embedding_dim, hidden_dim, output_dim):<br>
super().<strong>init</strong>()</p>
<pre><code>    self.embedding = nn.Embedding(input_dim, embedding_dim)
    self.fc1 = nn.Linear(embedding_dim,hidden_dim)
    self.fc2 = nn.Linear(hidden_dim,output_dim)
            
def forward(self, x):

    x = self.embedding(x)        
    x = self.fc1(x)        
    x = self.fc2(x)        
    
    return x
</code></pre>
</blockquote>
<p>And I fed both with the same input as below:</p>
<blockquote>
<p>INPUT_DIM = len(TEXT.vocab)<br>
EMBEDDING_DIM = 300<br>
HIDDEN_DIM = 374<br>
OUTPUT_DIM = 2</p>
</blockquote>
<p>And below are how I feed it to both model:</p>
<blockquote>
<p>model = RNN(INPUT_DIM, EMBEDDING_DIM, HIDDEN_DIM, OUTPUT_DIM)</p>
</blockquote>
<blockquote>
<p>RNN(<br>
(embedding): Embedding(20002, 300)<br>
(rnn): RNN(300, 374)<br>
(fc): Linear(in_features=374, out_features=2, bias=True)<br>
)</p>
</blockquote>
<p>and</p>
<blockquote>
<p>model = Net(INPUT_DIM, EMBEDDING_DIM, HIDDEN_DIM, OUTPUT_DIM)</p>
</blockquote>
<blockquote>
<p>Net(<br>
(embedding): Embedding(20002, 300)<br>
(fc1): Linear(in_features=300, out_features=374, bias=True)<br>
(fc2): Linear(in_features=374, out_features=2, bias=True)<br>
)</p>
</blockquote>
<p>And when I ran it, the RNN model worked well but the Net model gave me error like below:</p>
<blockquote>
<p>ValueError: Expected input batch_size (42) to match target batch_size (20).</p>
</blockquote>
<p>I am really curious about what had happened. My assumption, the error should not happen just because I change the network architecture.</p>
<p>Sorry for this stupid question but I really hope I can gain more knowledge from this question.</p>

username: ptrblck
cooked: <p>I don’t know what kind of input shape you are using, but assume you are passing in a 3-dimensional tensor?<br>
If that’s the case, I would recommend to check all shapes in both models and make sure the layers work as intended.</p>
<p>E.g. <a href="https://pytorch.org/docs/stable/generated/torch.nn.RNN.html#torch.nn.RNN"><code>nn.RNN</code></a> expects an input in the shape <code>[seq_len, batch_size, features]</code> by default, while <a href="https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear"><code>nn.Linear</code></a> accepts <code>[batch_size, *, in_features]</code>.</p>
<p>The <code>hidden</code> output of <code>nn.RNN</code> would have the shape <code>[num_layers*num_directions, batch_size, hidden_size]</code>. Note that without a <code>permute</code> operation the <code>self.fc</code> layer will use dim0 as the batch dimension.</p>