2CNN1d + LSTM: Dimension out of range (expected to be in range of [-1, 0], but got 1)

username: Konohayu
cooked: <p>I had found some people posted the same error. I am now sure what is going on with my code.</p>
<p>class Neural_Net(nn.Module):</p>
<pre><code>def __init__(self):
  super(Neural_Net, self).__init__()
  self.conv1 = nn.Conv1d(4, 10, 20, stride=10, padding=5)
  # self.maxpool1 = nn.MaxPool1d(5)
  self.conv2 = nn.Conv1d(10, 10, 20, stride=10, padding=5)
  # self.maxpool2 = nn.MaxPool1d(5)
  self.lstm1 = nn.LSTM(10,22) 
  self.relu = nn.ReLU()
  self.linear1 = nn.Linear(11220, 1000)
  self.output = nn.Linear(1000,22) 

def forward(self, x):
  x = torch.transpose(x,2,1)
  x = x.to(dtype=torch.float32)
  x = self.relu(self.conv1(x)) 
  # x = self.maxpool1(x)
  x = self.relu(self.conv2(x))
  # x = self.maxpool2(x)
  x = torch.transpose(x,2,1)
  [x, hidden] = self.lstm1(x)
  x = x.view(-1,11220)
  x = torch.squeeze(x)
  x = self.relu(self.linear1(x))
  out = self.output(x)
  return out
</code></pre>
<p>the input size is batch_size x 1000 x 4, then traspose it.</p>

username: ptrblck
cooked: <p>Which line of code is raising this error?<br>
I would recommend to use a specific <code>dim</code> argument in <code>squeeze</code>, as otherwise you might be losing e.g. the batch dimension, if you are dealing with a single sample in a batch:</p>
<pre><code class="lang-python">x = torch.squeeze(x)
</code></pre>
<p>I assume you would like to squeeze the temporal dimension here? If so, specify it with <code>dim=temp_dim</code> and retry your code again.</p>

username: Konohayu
cooked: <p>I got the error from CrossEntropyLoss.</p>

username: Konohayu
cooked: <p>I assume dim=batch_size and tried it. the error still occurs. i see that my output dimension is one which doesn’t match the batch size.</p>

username: ptrblck
cooked: <p>Could you post the shapes of the model output and the target tensors please?</p>

username: Konohayu
cooked: <p>the current output shape is torch.Size([18, 22]).</p>

username: ptrblck
cooked: <p>Thanks! What is the target shape and which values does the target contain?</p>

username: Konohayu
cooked: <p>the target is tensor([17, 16,  8, 18,  9, 14, 11, 18,  9,  2, 15,  8, 11, 12, 12,  8,  5, 20,<br>
3, 21,  2, 12,  3,  5, 18,  8, 14, 17,  6,  8, 21,  0, 20,  2,  8, 13,<br>
21, 18,  8,  4,  2,  4,  9, 11,  3, 12, 15,  1,  7,  2,  4],<br>
device=‘cuda:0’)</p>
<p>it has 51 values. they are from 0 to 21</p>

username: ptrblck
cooked: <p>The batch size of your output and target should match.<br>
After fixing this issue, the code runs fine using your target and <code>nn.CrossentropyLoss</code>:</p>
<pre><code class="lang-python">criterion = nn.CrossEntropyLoss()

output = torch.randn(51, 22, requires_grad=True)
target = torch.tensor([17, 16, 8, 18, 9, 14, 11, 18, 9, 2, 15, 8, 11, 12, 12, 8, 5, 20,
3, 21, 2, 12, 3, 5, 18, 8, 14, 17, 6, 8, 21, 0, 20, 2, 8, 13,
21, 18, 8, 4, 2, 4, 9, 11, 3, 12, 15, 1, 7, 2, 4])

loss = criterion(output, target)
loss.backward()
</code></pre>

username: Konohayu
cooked: <p>the real issue is when we do x.view(-1, 11220). we lost the batch dimension<br>
torch.Size([51, 10, 22]) # lstm<br>
torch.Size([1, 11220]) # x.view<br>
torch.Size([11220]) # squeeze</p>