Why do I get "input.size(-1) must be equal to input_size" error in this case?

bbasaran: <p>Hello! I have been trying to understand the dynamics for a while but I cannot make sense of the error below:</p>
<blockquote>
<p>RuntimeError: input.size(-1) must be equal to input_size. Expected 1, got 2</p>
</blockquote>
<p>My training data contains only one feature column, therefore I pass “n_features” = 1. Labels can only be 0 or 1. Since this is binary classification, I pass “n_classes” = 1.</p>
<pre><code class="lang-auto">class ModuleLSTM(nn.Module):
  def __init__(self, n_features, n_classes, n_hidden=256, n_layers=3):        
    super().__init__()
    self.lstm = nn.LSTM(
        input_size  = n_features,
        hidden_size = n_hidden,
        num_layers  = n_layers,
        batch_first = True,
        dropout     = 0.75 
    )                
    self.classifier = nn.Linear(n_hidden, n_classes)  
    
  def forward(self, x):
    self.lstm.flatten_parameters()
    _, (hidden, _) = self.lstm(x)
    return self.classifier(hidden[-1])
</code></pre>
<p>When I debug the ‘x’ in the forward method, I see that the shape is torch.Size([64, 5, 2]), where the 64 and 5 correspond to batch size and sequence length, respectively. I don’t understand why ‘x’ has the second dimension full of zeros below:</p>
<pre><code class="lang-auto">tensor([[[-4.3775e-01,  0.0000e+00],
         [-4.7356e-01,  0.0000e+00],
         [-4.9494e-01,  0.0000e+00],
         [-5.2778e-01,  0.0000e+00],
         [-5.5412e-01,  0.0000e+00]],
         ...
        [[ 2.7826e+00,  0.0000e+00],
         [ 2.7535e+00,  0.0000e+00],
         [ 2.7076e+00,  0.0000e+00],
         [ 2.6636e+00,  0.0000e+00],
         [ 2.6562e+00,  0.0000e+00]]]
</code></pre>

Tejan_Mehndiratta: <p>n_feature should be the word embedding vector size(i.e. the length of each word represented as a vector)</p>

bbasaran: <p>I am not using any Embedding Layer, and my model is not an NLP model.</p>

Tejan_Mehndiratta: <p>I apologise. I don’t know how to help you.</p>

bbasaran: <p>Thank you for your effort anyway!</p>

bbasaran: <p>Is there anyone to help?</p>

ptrblck: <p>The <code>x</code> tensor in your example is the input, so I assume your input just contains zeros in one dimension?<br>
If I use a randomly initialized input, neither <code>hidden</code> nor <code>out</code> contains all zeros.</p>

bbasaran: <p>I’m sorry for the confusion. The second column is the label column which consists of zeros and ones. In the debugged part (until the error), all labels are 0 by a coincidence, therefore I thought there was a mistake. I don’t know why but I am not able to edit my first message.</p>
<p>Concisely, this tensor seems okay but the problem is still the error below:</p>
<blockquote>
<p>RuntimeError: input.size(-1) must be equal to input_size. Expected 1, got 2</p>
</blockquote>
<p>My training data contains only one feature column, therefore I pass “n_features” = 1. Labels can only be 0 or 1, hence this is binary classification, I pass “n_classes” = 1. With this setup, I get the error above.</p>
<p>If and only if I pass “n_features” = 2 and “n_classes” = 2, the model works.</p>
<p>I would be grateful if you help me on this. Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> !</p>

ptrblck: <p>This this error raised by the model in the forward pass or later by the loss function?<br>
Could you post the input and target shapes as an executable code snippet to reproduce the issue?</p>

bbasaran: <p>The error is being raised in the forward pass, directly with the following line:</p>
<blockquote>
<p>_, (hidden, _) = self.lstm(x)</p>
</blockquote>
<p>I don’t know how to build an executable code snippet to reproduce the issue here, but I trimmed all unnecessary parts in the Colab link below. The error is present there and if you want, you can quickly run it again (I gathered into 2 cells):</p>
<p><a href="https://drive.google.com/file/d/1llBY9TsbI2seD4orFPphJNRr6UTlx31R/view?usp=sharing" rel="noopener nofollow ugc">Google Colab model.ipnyb</a></p>
<p>Thank you for the effort!</p>

Tejan_Mehndiratta: <p>Hi, I looked at your code.</p>
<p>The error is:</p>
<p><code>model = GamestagePredictor(n_features = 1, n_classes = 1) </code><br>
you are giving <code>n_features = 1</code> and this <code>n_features</code> is being called by an LSTM layer</p>
<pre><code class="lang-auto">self.lstm = nn.LSTM(
        input_size  = n_features,
        hidden_size = n_hidden,
        num_layers  = n_layers,
        batch_first = True,
        dropout     = 0.75
    )                
</code></pre>
<p>The <code>input_size</code> argument of an LSTM layer should be a 3-D tensor not an “integer”<br>
like:<br>
<code>(batch_size, seq_len, dim)</code></p>

bbasaran: <p>Thank you <a class="mention" href="/u/tejan_mehndiratta">@Tejan_Mehndiratta</a><br>
According to the <a href="https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html" rel="noopener nofollow ugc">documentation</a>, “input_size” should be an integer. As I stated above, if I set “n_features” = 2, it works without a problem. However, I think that I should be able to set it to 1, since my training data has only one feature column. Setting it 1 causes the error.</p>

Tejan_Mehndiratta: <p>Oh! My bad. I got confused. Sorry.</p>

ptrblck: <p>Thanks for the code. I’ve removed all unnecessary code, since the forward pass should raise the issue, which works fine on my setup:</p>
<pre><code class="lang-python">class ModuleLSTM(nn.Module):
  
  def __init__(self, n_features, n_classes, n_hidden=256, n_layers=3):        
    super().__init__()
    self.lstm = nn.LSTM(
        input_size  = n_features,
        hidden_size = n_hidden,
        num_layers  = n_layers,
        batch_first = True,
        dropout     = 0.75
    )                
    
    self.classifier = nn.Linear(n_hidden, n_classes)  
    
  def forward(self, x):
    self.lstm.flatten_parameters()
    _, (hidden, _) = self.lstm(x)
    out = hidden[-1]
    return self.classifier(out)


batch_size = 10
seq_len = 50
nb_features = 1
model = ModuleLSTM(nb_features, n_classes=10)
x = torch.randn(batch_size, seq_len, nb_features)
out = model(x)
print(out)
</code></pre>
<p>Could you check this minimal code snippet and compare it to the input shapes you are using?</p>

bbasaran: <p>Thanks. I will try but why do we need to set “n_classes” to 10? Shouldn’t it be 1 in my case?</p>

ptrblck: <p>I just picked a random number, as the number of output features shouldn’t make a difference.<br>
Also, since my small code snippet is running fine, please feel free to add any code snippets to it, which would reproduce your issue.</p>

bbasaran: <p>As far as I understood, the cause of the problem as follows:</p>
<pre><code class="lang-auto">batch_size = 64
seq_len = 5
n_features = 1
n_class = 1
model = ModuleLSTM(n_features, n_class)
</code></pre>
<p>If I feed the setup above with the ‘x’ below, there is no problem as you’ve said:</p>
<pre><code class="lang-auto">x = torch.randn(batch_size, seq_len, n_features)
</code></pre>
<p>BUT the problem is, in the same setup above, my real data comes into ‘x’ with shape (64, 5, 2), not with (64, 5, 1). This is because of the way I generate the data I guess, but I don’t know how to fix it. Let me explain you step by step:</p>
<p>1-&gt; I create X_train, y_train, X_val, y_val, X_test, y_test, I scale the X… parts. Nothing unusual. In the rest, I will just mention the training data. I merge the scaled X_train with y_train:</p>
<pre><code class="lang-auto">trainData = pd.concat([X_train_scaled, y_train],axis=1)
</code></pre>
<p>2-&gt; I create my data sequences with the function below. It fetches the first “seq_len” amount of feature rows, saves into “sequence”, fetches the “seq_len+1”-th label and appends this sequence and label together to “sequences” list. Then moves into the next sequence-label pair (till the end)</p>
<pre><code class="lang-auto">def create_sequences(input_data, target_name, sequence_length):
  sequences = []
  for i in range(0, len(input_data) - sequence_length, sequence_length):
    sequence = input_data[i : i + sequence_length]
    label    = input_data.iloc[i + sequence_length][target_name]
    sequences.append((sequence, label))
  return sequences

trainSequences = create_sequences(trainData, 'gamestageEMA', seq_len)  
</code></pre>
<p>3-&gt; I pass the sequences to my “DataModule” class which inherits the pl.LightningDataModule. (You can see this class and the rest in my Colab)</p>
<pre><code class="lang-auto">dataModule = DataModule(trainSequences, valSequences, testSequences, BATCH_SIZE)
</code></pre>
<p>it first passes “trainSequences” to my “DoomFrameDataset” class which inherits torch Dataset and as a result, it’s “self.trainDataset” is filled. Then it returns Train DataLoader in the following way:</p>
<pre><code class="lang-auto">DataLoader(self.trainDataset, batch_size=self.batchSize, shuffle=False, num_workers=cpu_count())
</code></pre>
<p>4-&gt;  I create the model:</p>
<pre><code class="lang-auto">model = GamestagePredictor(n_features = 1, n_classes = 1)
</code></pre>
<p>and you know the rest. My GameStagePredictor class fetches the sequence and label pairs and passes them to LSTM:</p>
<pre><code class="lang-auto">def training_step(self, batch, batch_idx):               
    sequences, labels = batch["sequence"], batch["label"]
    loss, outputs = self(sequences, labels)
    ...
</code></pre>
<p>I am sorry for this long message, I just wanted to be more specific. Concisely, the problem is about the way I pass my sequence and label pairs to my model. Since set n_features = 1 and n_classes = 1, it wants to see the input with shape (64, 5, 1), but receives (64, 5, 2) instead. I don’t know how to deal with this.</p>

ptrblck: <p>I’m unfortunately not familiar with Lightning’s <code>DataModule</code> and don’t know if any reshaping is done internally.<br>
Since the shape is apparently wrong for the input, I would recommend to add print statements in all data loading classes and check the current shape of the input to further isolate the additional values.</p>

bbasaran: <p>Thank you for your time and effort <a class="mention" href="/u/ptrblck">@ptrblck</a> , I am appreciated.</p>
<p>I realized that the problem was being caused by the sequence creation function. I solved the issue there. Now, if I pass n_features=1 and class=2 (I was thinking that I should set this to 1 since this is binary classification, but setting 1 raises another error), the model runs without any problem.</p>
<p>The only problem now is that if I run the model on CPU, there is no problem, however, if I try to use GPU, I get:</p>
<blockquote>
<p>RuntimeError: CUDA error: device-side assert triggered</p>
</blockquote>

ptrblck: <p>Often device assertions are trigged by e.g. invalid indices. You could run the script via <code>CUDA_LAUNCH_BLOCKING=1 python script.pt args</code> and check the stacktrace for the failing operation.<br>
Once you know which line of code is raising this error, you can add print statements to debug the issue further.</p>