RuntimeError: Expected 4-dimensional input for 4-dimensional weight [6, 3, 3, 3], but got 1-dimensional input of size [10] instead

Ali_Mashoud: <p>Hello I am working on training a CNN model for a FASHION MNIST dataset but I am getting continously the above mentioned error after the model trains for one epoch. Here is my CNN model architecture:</p>
<blockquote>
<p>class ConvolutionalNetwork(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.conv1 = nn.Conv2d(3,6,3,1)<br>
self.conv2 = nn.Conv2d(6,16,3,1)<br>
self.fc1 = nn.Linear(5<em>5</em>16,120)<br>
self.fc2 = nn.Linear(120,84)<br>
self.fc3= nn.Linear(84,20)<br>
self.fc4 = nn.Linear(20,10)<br>
def forward(self, X):<br>
X= f.relu(self.conv1(X))<br>
X =f.max_pool2d(X,2,2)<br>
X= f.relu(self.conv2(X))<br>
X= f.max_pool2d(X,2,2)<br>
X= X.view(-1,5<em>5</em>16)<br>
X= f.relu(self.fc1(X))<br>
X= f.relu(self.fc2(X))<br>
X= f.relu(self.fc3(X))<br>
X= self.fc4(X)<br>
return f.log_softmax(X, dim=1)</p>
</blockquote>
<p>Here is my batch training code:<br>
import time</p>
<blockquote>
<p>start_time = time.time()<br>
epochs = 10<br>
train_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
test_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
train_correct = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
test_correct = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<p>for i in range(epochs):<br>
trn_corr=0<br>
tst_corr = 0</p>
<pre><code>#Run the training batches
for b, (X_train,y_train) in enumerate(train_loader):
    b+=1
    
    y_pred = model(X_train)
    loss= criterion(y_pred,y_train)
    predicted = torch.max(y_pred, 1)[1]
    batch_corr = (predicted==y_train).sum()
    trn_corr += batch_corr
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    #Print interim results
    if b%600 == 0:
        print(f'epoch: {i:2}  batch: {b:4} [{10*b:6}/60000]  loss: {loss.item():10.8f}  \
</code></pre>
<p>accuracy: {trn_corr.item()<em>100/(10</em>b):7.3f}%')</p>
<pre><code>loss=loss.detach().numpy()
train_losses.append(loss)
train_correct.append(trn_corr)

#Run testing batches
with torch.no_grad():
        for b, (X_test,y_test) in enumerate(test_loader):
            b+=1
            y_val = model(y_test)
            loss = criterion(y_val,y_test)
            predicted = torch.max(y_val, 1)[1]
            batch_corr = (predicted==y_val).sum()
            tst_corr += batch_corr
        
            optimzer.zero_grad()
            loss.backward()
            optimzer.step()
            
        loss=loss.detach().numpy()
        test_losses.append(loss)
        test_correct.append(tst_corr)
</code></pre>
<p>print(f"time_duration:{start_time-time.time()}")</p>
</blockquote>
<p>Here is my error which is being shown while training the model.</p>
<blockquote>
<hr>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
38             for b, (X_test,y_test) in enumerate(test_loader):<br>
39                 b+=1<br>
—&gt; 40                 y_val = model(y_test)<br>
41                 loss = criterion(y_val,y_test)<br>
42                 predicted = torch.max(y_val, 1)[1]</p>
<p>~\Anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)<br>
887             result = self._slow_forward(*input, **kwargs)<br>
888         else:<br>
 → 889             result = self.forward(*input, **kwargs)<br>
890         for hook in itertools.chain(<br>
891                 _global_forward_hooks.values(),</p>
<p> in forward(self, X)<br>
9         self.fc4 = nn.Linear(20,10)<br>
10     def forward(self, X):<br>
—&gt; 11         X= f.relu(self.conv1(X))<br>
12         X =f.max_pool2d(X,2,2)<br>
13         X= f.relu(self.conv2(X))</p>
<p>~\Anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)<br>
887             result = self._slow_forward(*input, **kwargs)<br>
888         else:<br>
 → 889             result = self.forward(*input, **kwargs)<br>
890         for hook in itertools.chain(<br>
891                 _global_forward_hooks.values(),</p>
<p>~\Anaconda3\lib\site-packages\torch\nn\modules\conv.py in forward(self, input)<br>
397<br>
398     def forward(self, input: Tensor) → Tensor:<br>
 → 399         return self._conv_forward(input, self.weight, self.bias)<br>
400<br>
401 class Conv3d(_ConvNd):</p>
<p>~\Anaconda3\lib\site-packages\torch\nn\modules\conv.py in _conv_forward(self, input, weight, bias)<br>
394                             _pair(0), self.dilation, self.groups)<br>
395         return F.conv2d(input, weight, bias, self.stride,<br>
 → 396                         self.padding, self.dilation, self.groups)<br>
397<br>
398     def forward(self, input: Tensor) → Tensor:</p>
<p>RuntimeError: Expected 4-dimensional input for 4-dimensional weight [6, 3, 3, 3], but got 1-dimensional input of size [10] instead</p>
</blockquote>
<p>Please tell me how can I fix this error. Thanks!!!</p>

RaLo4: <aside class="quote no-group" data-username="Ali_Mashoud" data-post="1" data-topic="132641">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ali_mashoud/48/33224_2.png" class="avatar"> Ali_Mashoud:</div>
<blockquote>
<p><code>y_val = model(y_test)</code></p>
</blockquote>
</aside>
<p>This looks like you are forward-passing the labels instead of the images during the test run<br>
I am guessing this should be <code>y_val = model(X_test)</code></p>

Ali_Mashoud: <p>Thanks I just fixed it but now after one epoch I am getting an error “RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn”. Could you please tell what is the problem here now??? Thanks</p>

Ali_Mashoud: <p>Thanks I just fixed it but now after one epoch I am getting an error “RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn”. Could you please tell what is the problem here now??? Thanks</p>

RaLo4: <p>Oh. I didn’t notice this before but there are a few more misstakes in your <code>Run testing batches</code> code part.<br>
So. here:</p>
<aside class="quote no-group" data-username="Ali_Mashoud" data-post="1" data-topic="132641">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ali_mashoud/48/33224_2.png" class="avatar"> Ali_Mashoud:</div>
<blockquote>
<pre><code class="lang-auto">#Run testing batches
with torch.no_grad():
        for b, (X_test,y_test) in enumerate(test_loader):
            b+=1
            y_val = model(y_test)
            loss = criterion(y_val,y_test)
            predicted = torch.max(y_val, 1)[1]
            batch_corr = (predicted==y_val).sum()
            tst_corr += batch_corr
        
            optimzer.zero_grad()
            loss.backward()
            optimzer.step()
            
        loss=loss.detach().numpy()
        test_losses.append(loss)
        test_correct.append(tst_corr)
</code></pre>
</blockquote>
</aside>
<p>You are doing testing and not training like above. You are already correctly using <code>with torch.no_grad():</code> which turns off grad calculation for all tensors. Using this during testing is correct.<br>
But you are also doing back propagation and optimizer steps.</p>
<aside class="quote no-group" data-username="Ali_Mashoud" data-post="1" data-topic="132641">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ali_mashoud/48/33224_2.png" class="avatar"> Ali_Mashoud:</div>
<blockquote>
<pre><code class="lang-auto">            optimzer.zero_grad()
            loss.backward()
            optimzer.step()
</code></pre>
</blockquote>
</aside>
<p>This is normally not done during testing since your network is not supposed to learn from test-data.<br>
The error comes from <code>loss.backward()</code> requiring grad calculation to be turned on which you turned off with <code>with torch.no_grad():</code>.<br>
Remove these three lines</p>
<pre><code class="lang-auto">            optimzer.zero_grad()
            loss.backward()
            optimzer.step()
</code></pre>
<p>while keeping the <code>with torch.no_grad():</code> and it should work.<br>
(only for what is under <code>#Run testing batches</code>! under <code>#Run the training batches</code> you of course keep those lines)</p>

Ali_Mashoud: <p>Thank you so much <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:">  You saved my day!!! ;-D</p>

Ali_Mashoud: <p>if there is any other error in my Convolutional neural network model or in my training batches code. Kindly let me know. Thanks!!!</p>