RNN - RuntimeError: input must have 3 dimensions, got 2

mattpats: <p>I’m getting the following error:</p>
<p>RuntimeError: input must have 3 dimensions, got 2</p>
<p>I have a single feature column that I am trying to feed into a GRU neural net.</p>
<p>Below are my data loader and neural net. I have also included the output of my data loader when I feed in the first 50 data points.</p>
<p>What am I doing wrong?</p>
<p>def batch_data(feature1, sequence_length, batch_size):<br>
“”"<br>
Batch the neural network data using DataLoader<br>
:param feature1: the single feature column<br>
:param sequence_length: The sequence length of each batch<br>
:param batch_size: The size of each batch; the number of sequences in a batch<br>
:return: DataLoader with batched data<br>
“”"</p>
<pre><code># total number of batches we can make
n_batches = len(feature1)//batch_size

# Keep only enough characters to make full batches
feature1= feature1[:n_batches * batch_size]

y_len = len(feature1) - sequence_length

x, y = [], []
for idx in range(0, y_len):
    idx_end = sequence_length + idx
    x_batch = feature1[idx:idx_end]
    x.append(x_batch)
    # only making predictions after the last item in the batch
    batch_y = feature1[idx_end]    
    y.append(batch_y)    

# create tensor datasets
data = TensorDataset(torch.from_numpy(np.asarray(x)), torch.from_numpy(np.asarray(y)))

data_loader = DataLoader(data, shuffle=False, batch_size=batch_size)

# return a dataloader
return data_loader
</code></pre>
<pre><code class="lang-auto"># test dataloader on subset of actual data

test_text = data_subset_b[0:50]
t_loader = batch_data(test_text, sequence_length=5, batch_size=10)
 
data_iter = iter(t_loader)
sample_x, sample_y = data_iter.next()
 
print(sample_x.shape)
print(sample_x)
print()
print(sample_y.shape)
print(sample_y)
</code></pre>
<p>When I pass in the a batch of data into the data loader, I get the following  …</p>
<p>torch.Size([10, 5])<br>
tensor([[ 0.0045,  0.0040, -0.0008,  0.0005, -0.0012],<br>
[ 0.0040, -0.0008,  0.0005, -0.0012,  0.0000],<br>
[-0.0008,  0.0005, -0.0012,  0.0000, -0.0015],<br>
[ 0.0005, -0.0012,  0.0000, -0.0015,  0.0008],<br>
[-0.0012,  0.0000, -0.0015,  0.0008,  0.0000],<br>
[ 0.0000, -0.0015,  0.0008,  0.0000,  0.0000],<br>
[-0.0015,  0.0008,  0.0000,  0.0000, -0.0008],<br>
[ 0.0008,  0.0000,  0.0000, -0.0008, -0.0039],<br>
[ 0.0000,  0.0000, -0.0008, -0.0039, -0.0026],<br>
[ 0.0000, -0.0008, -0.0039, -0.0026, -0.0082]], dtype=torch.float64)</p>
<p>torch.Size([10])<br>
tensor([ 0.0000, -0.0015,  0.0008,  0.0000,  0.0000, -0.0008, -0.0039, -0.0026,<br>
-0.0082,  0.0078], dtype=torch.float64)</p>
<p>Here is my neural net architecture:</p>
<p>class GRU(nn.Module):</p>
<pre><code>def __init__(self, input_dim, output_dim, n_hidden, n_layers, dropout=0.25, lr=0.001):
    super().__init__()
    
    # set class variables
    self.output_dim = output_dim
    self.n_layers = n_layers
    self.n_hidden = n_hidden
    
    # define model layers
    
    # define gru layer
    self.gru = nn.GRU(input_dim, n_hidden, n_layers, dropout=dropout, batch_first=True)
    
    # define a dropout layer
    self.dropout = nn.Dropout(dropout)
    
    # define the final, fully-connected output layer
    self.fc = nn.Linear(n_hidden, output_dim)   


def forward(self, nn_input, hidden):
    ''' Forward pass through the network. 
        These inputs are x, and the hidden/cell state `hidden`. '''
    
    # batch_size equals the input's first dimension
    batch_size = nn_input.size(0)
    
    # Get the outputs and the new hidden state from the lstm
    r_output, hidden = self.gru(nn_input, hidden)
    
    # pass through a dropout layer
    out = self.dropout(r_output)
    
    # Stack up LSTM outputs using view
    # you may need to use contiguous to reshape the output
    out = out.contiguous().view(-1, self.n_hidden)
    
    # put nn_input through the fully-connected layer
    out = self.fc(out)

    # reshape into (batch_size, seq_length, output_size)
    out = out.view(batch_size, -1, self.output_dim)
    
    # get last batch
    out = out[:, -1]
    
    # return the final output and the hidden state
    return out, hidden


def init_hidden(self, batch_size):
    ''' Initializes hidden state '''
    # Create two new tensors with sizes n_layers x batch_size x n_hidden,
    # initialized to zero, for hidden state and cell state of LSTM
    weight = next(self.parameters()).data
    
    if (train_on_gpu):
        hidden = (weight.new(self.n_layers, batch_size, self.n_hidden).zero_().cuda(),
                  weight.new(self.n_layers, batch_size, self.n_hidden).zero_().cuda())
    else:
        hidden = (weight.new(self.n_layers, batch_size, self.n_hidden).zero_(),
                  weight.new(self.n_layers, batch_size, self.n_hidden).zero_())
    
    return hidden</code></pre>

ptrblck: <p>As the error message explains the RNN expects a 3-dimensional input in the shape <code>[batch_size, seq_len, features]</code> (since <code>batch_first=True</code> is used).</p>
<aside class="quote no-group" data-username="mattpats" data-post="1" data-topic="107613">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/dc4da7/48.png" class="avatar"> mattpats:</div>
<blockquote>
<p>I have a single feature column that I am trying to feed into a GRU neural net.</p>
</blockquote>
</aside>
<p>If you don’t have a temporal dimension (<code>seq_len</code> in the example shape), then I don’t think you need to use an RNN or could you explain your use case a bit? I.e. how should the model process the data?</p>

mattpats: <p>Thanks for your response. Since posting, I have figured out the problem. I had to apply unsqueeze to the data.</p>