RuntimeError: shape '[1, 1, 13]' is invalid for input of size 26 - RNN

username: newbie12
cooked: <p>Hi everyone,</p>
<p>I’m quite new to Python in general and I’ve been thrown in at the deep end. I’m building a RNN classifier that has 13 feature inputs and a binary label. I noticed an issue when calculating the accuracy of my network…I was getting accuracy rates ranging from 102% to 175% which obviously isn’t right.</p>
<p>I’ve had a look at the shape of my data as think this is causing the issue. I’ve attempted to reshape but now I’m getting error message <strong>'RuntimeError: shape ‘[1, 1, 13]’ is invalid for input of size 26’</strong></p>
<p>Can some please point me in the direction of the mistake I’m making? I think it’s something quite obvious but being so new at this I can’t seem to work it out.</p>
<pre><code class="lang-auto">&gt; device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
&gt; 
&gt; # Hyper-parameters
&gt; 
&gt; num_classes = 2
&gt; num_epochs = 10
&gt; batch_size = 10
&gt; learning_rate = 0.001
&gt; input_size = 13
&gt; sequence_length = 1
&gt; hidden_size = 14
&gt; num_layers = 3
&gt; 
&gt; 
&gt; # load-dataset
&gt; class URLTestDataset(Dataset):
&gt;     """URL Test Dataset"""
&gt; 
&gt;     def __init__(self, csv_file):
&gt;         self.url_frame = pd.read_csv(csv_file)
&gt; 
&gt;     def __len__(self):
&gt;         return len(self.url_frame)
&gt; 
&gt;     def __getitem__(self, idx):
&gt;         if torch.is_tensor(idx):
&gt;             idx = idx.tolist()
&gt; 
&gt;         label = self.url_frame.iloc[idx, 13]
&gt;         features = self.url_frame.iloc[idx, :13]
&gt;         label = np.array([label])
&gt;         features = np.array([features])
&gt;         features = torch.from_numpy(features)
&gt;         label = torch.from_numpy(label)
&gt;         sample = (features, label)
&gt;         return sample
&gt; 
&gt; 
&gt; train_dataset = URLTestDataset(csv_file='urls2.csv')
&gt; test_dataset = URLTestDataset(csv_file='urls1.csv')
&gt; 
&gt; # Data loader
&gt; train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=2, shuffle=True)
&gt; 
&gt; test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=2, shuffle=True)
&gt; 
&gt; 
&gt; class RNN(nn.Module):
&gt;     def __init__(self, input_size, hidden_size, num_layers, num_classes):
&gt;         super(RNN, self).__init__()
&gt;         self.num_layers = num_layers
&gt;         self.hidden_size = hidden_size
&gt;         self.rnn = nn.RNN(input_size, hidden_size, num_layers, batch_first=True)
&gt;         self.fc = nn.Linear(hidden_size, num_classes)
&gt; 
&gt;     def forward(self, x):
&gt;         h0 = torch.zeros((self.num_layers, x.size(0), self.hidden_size), dtype=torch.float).to(device)
&gt;         #c0 = torch.zeros((self.num_layers, x.size(0), self.hidden_size), dtype=torch.float).to(device)
&gt; 
&gt;         out, _ = self.rnn(x, h0)
&gt; 
&gt;         out = out[:, -1, :]
&gt; 
&gt;         out = self.fc(out)
&gt;         return out
&gt; 
&gt; 
&gt; model = RNN(input_size, hidden_size, num_layers, num_classes).to(device)
&gt; 
&gt; 
&gt; # Loss and optimizer
&gt; criterion = nn.CrossEntropyLoss()
&gt; optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
&gt; 
&gt; # Train the model
&gt; 
&gt; n_total_steps = len(train_loader)
&gt; 
&gt; for epoch in range(num_epochs):
&gt;     for i, (features, label) in enumerate(train_loader):
&gt;         features = features.reshape(1, sequence_length, input_size).to(device)
&gt; 
&gt;         label = label.to(device)
&gt;         features = features.to(device)
&gt;         features = features.float()
&gt; 
&gt;         # Forward pass
&gt; 
&gt;         outputs = model(features)
&gt;         outputs = outputs.float()
&gt; 
&gt;         loss = criterion(outputs, label.squeeze())
&gt; 
&gt;         loss.backward()
&gt;         optimizer.zero_grad()
&gt;         optimizer.step()
&gt; 
&gt;         if (i + 1) % 100 == 0:
&gt;             print(f'Epoch [{epoch + 1}/{num_epochs}], Step [{i + 1}/{n_total_steps}], Loss: {loss.item():.4f}')
&gt; 
&gt;     # Test
&gt; with torch.no_grad():
&gt;     n_correct = 0
&gt;     n_samples = 0
&gt;     for features, label in test_loader:
&gt;         features = features.reshape(1, sequence_length, input_size).to(device)
&gt;         features = features.float()
&gt;         label = label.to(device)
&gt;         outputs = model(features)
&gt; 
&gt;      
&gt;         _, predicted = torch.max(outputs.data, 1)
&gt;         n_samples += label.size(0)
&gt;         n_correct += (predicted == label).sum().item()
&gt; 
&gt;     acc = 100.0 * n_correct / n_samples
&gt;     print(f'Accuracy of the network on the test urls: {acc} %') ```</code></pre>

username: ptrblck
cooked: <p>I guess the error is raised in this line of code:</p>
<pre><code class="lang-python">features = features.reshape(1, sequence_length, input_size)
</code></pre>
<p>Here you are trying to reshape <code>features</code> to a fixed size which doesn’t match the number of elements the tensor contains.<br>
Since you are using <code>batch_size=2</code> in the <code>DataLoader</code>, I would think you might want to <code>reshape</code> the tensor to keep the batch size equal.<br>
To do so, this could work:</p>
<pre><code class="lang-python">features = features.reshape(features.size(0), -1, input_size)
</code></pre>
<p>This operation now keeps the size of <code>dim0</code> constant, uses <code>input_size</code> for <code>dim2</code>, and flattens the additional values into the temporal dimension <code>dim1</code>.<br>
You should check if the reshape operation is applied correctly, as I don’t know what the original shape of the <code>features</code> tensor was.</p>

username: newbie12
cooked: <p>Thank you for your reply kind sir.</p>
<p>I applied your changes and the error message disappeared. I also think I was setting my parameteres in correctly, as I had mixed my sequence length and input size!</p>
<p>my parameters are now:<br>
num classes = 2 (i have binary labels 0 and 1)<br>
input size = 1<br>
sequence length = 13 (as i have 13 features per item)<br>
and batch size = 1</p>
<p>I am now experiencing an issue with training the model. It appears to not train beyond 50% no matter how many epochs, what learning rate or what optimizer i try. I tried to over fit the model on a small sample (15) but still this does not work.</p>
<p>Do you know what could possibly be causing this issue?</p>
<p>Thank you so much for your time.</p>