RuntimeError: Expected hidden size (2, 1, 128), got [64, 1, 128]

username: ChanYalcn
cooked: <p>I tried to build a simple RNN for my credict card fraud dataset but I get an error.</p>
<pre><code class="lang-auto">class SimpleRNN(nn.Module):
    def __init__(self, input_size, hidden_size, num_layers, num_classes):
        super(SimpleRNN, self).__init__()
        self.hidden_size = hidden_size
        self.num_layers = num_layers

        self.rnn = nn.RNN(input_size, hidden_size, num_layers, batch_first=True)

        self.fc = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        # initialize hidden state with zeros
        h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size).to(x.device)
        h0 = h0[0]
        
        # Debugging: Print the shapes of batch_X and h0
        print(f"batch_X shape: {x.shape}")
        print(f"h0 shape: {h0.shape}")

        out, _ = self.rnn(x, h0)

        out = out[:, -1, :]
        out = self.fc(out)
        
        return out
</code></pre>
<pre><code class="lang-auto">sss = StratifiedShuffleSplit(n_splits=5, test_size=0.2, random_state=2)

for train_index, test_index in sss.split(x, y):
    X_train, X_test = x.iloc[train_index], x.iloc[test_index]
    y_train, y_test = y.iloc[train_index], y.iloc[test_index]

X_train_tensor = torch.tensor(X_train.values, dtype=torch.float32)
y_train_tensor = torch.tensor(y_train.values, dtype=torch.float32)

X_test_tensor = torch.tensor(X_test.values, dtype=torch.float32)
y_test_tensor = torch.tensor(y_test.values, dtype=torch.float32)

train_dataset = TensorDataset(X_train_tensor, y_train_tensor)
test_dataset = TensorDataset(X_test_tensor, y_test_tensor)

batch_size = 64
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
test_loader = DataLoader(test_dataset, batch_size=batch_size)
</code></pre>
<pre><code class="lang-auto">input_size = X_train_tensor.shape[1]
hidden_size = 128
num_layers = 2
num_classes = 1
lr = 0.001

X_train_tensor = X_train_tensor.view(-1, input_size)
model = SimpleRNN(input_size, hidden_size, num_layers, num_classes)
criterion = nn.BCEWithLogitsLoss()
optimizer = optim.Adam(model.parameters(), lr=lr)
</code></pre>
<p>The error:</p>
<pre><code class="lang-auto">batch_X shape: torch.Size([64, 30])
h0 shape: torch.Size([64, 128])
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[91], line 9
      7 for batch_X, batch_y in train_loader:
      8     optimizer.zero_grad()
----&gt; 9     outputs = model(batch_X)
     10     loss = criterion(outputs.squeeze(), batch_y)
     11     loss.backward()

File ~\OneDrive\Masaüstü\BUTTERFLY\cv\Lib\site-packages\torch\nn\modules\module.py:1501, in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

Cell In[83], line 20, in SimpleRNN.forward(self, x)
     17 print(f"batch_X shape: {x.shape}")
     18 print(f"h0 shape: {h0.shape}")
---&gt; 20 out, _ = self.rnn(x, h0)
     22 out = out[:, -1, :]
     23 out = self.fc(out)

File ~\OneDrive\Masaüstü\BUTTERFLY\cv\Lib\site-packages\torch\nn\modules\module.py:1501, in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

File ~\OneDrive\Masaüstü\BUTTERFLY\cv\Lib\site-packages\torch\nn\modules\rnn.py:505, in RNN.forward(self, input, hx)
    502     hx = self.permute_hidden(hx, sorted_indices)
    504 assert hx is not None
--&gt; 505 self.check_forward_args(input, hx, batch_sizes)
    506 assert self.mode == 'RNN_TANH' or self.mode == 'RNN_RELU'
    507 if batch_sizes is None:

File ~\OneDrive\Masaüstü\BUTTERFLY\cv\Lib\site-packages\torch\nn\modules\rnn.py:256, in RNNBase.check_forward_args(self, input, hidden, batch_sizes)
    253 self.check_input(input, batch_sizes)
    254 expected_hidden_size = self.get_expected_hidden_size(input, batch_sizes)
--&gt; 256 self.check_hidden_size(hidden, expected_hidden_size)

File ~\OneDrive\Masaüstü\BUTTERFLY\cv\Lib\site-packages\torch\nn\modules\rnn.py:239, in RNNBase.check_hidden_size(self, hx, expected_hidden_size, msg)
    236 def check_hidden_size(self, hx: Tensor, expected_hidden_size: Tuple[int, int, int],
    237                       msg: str = 'Expected hidden size {}, got {}') -&gt; None:
    238     if hx.size() != expected_hidden_size:
--&gt; 239         raise RuntimeError(msg.format(expected_hidden_size, list(hx.size())))

RuntimeError: Expected hidden size (2, 1, 128), got [64, 1, 128]```</code></pre>

username: vdw
cooked: <aside class="quote no-group" data-username="ChanYalcn" data-post="1" data-topic="188599">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chanyalcn/48/63089_2.png" class="avatar"> ChanYalcn:</div>
<blockquote>
<p><code>h0 = h0[0]</code></p>
</blockquote>
</aside>
<p>No idea why you are doing this…but don’t :).</p>
<p>Before that, <code>h0</code> has the correct shape: <code>(num_layer, batch_size, hidden_size)</code>. After that line, it’s only <code>(batch_size, hidden_size)</code>. The <code>nn.RNN</code>, since it sees only a 2d tensor, interprets this as <code>(seq_len, hidden_size)</code>.</p>
<p>The problem is that <code>nn.RNN</code> wants a 3d tensor. It therefore does an <code>unsequeeze(1)</code> to add the batch dimension – again, the <code>nn.RNN</code> thinks it’s an unbatched input, so it makes it a batched one. You can also check this <a href="https://discuss.pytorch.org/t/pytorch-lstm-input-confusion/171849">post</a> that addresses the same issue.</p>

username: ChanYalcn
cooked: <p>I asked ChatGPT to fix it before posting,silly gpt. I fixed the problem by removing that line,forgot to update the post. thank you</p>