Getting RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn error

username: vimalkansal
cooked: <p>Hi,<br>
I have defined my binary classifier as :<br>
class CreditCardTransactionClassifier(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(CreditCardTransactionClassifier, self).<strong>init</strong>()<br>
self.fc1 = nn.Linear(29, 29)<br>
self.relu1 = nn.ReLU()<br>
self.dout = nn.Dropout(0.2)<br>
self.fc2 = nn.Linear(29, 58)<br>
self.prelu = nn.PReLU(1)<br>
self.out = nn.Linear(58, 2)</p>
<pre><code>def forward(self, input_):
    a1 = self.fc1(input_)
    h1 = self.relu1(a1)
    dout = self.dout(h1)
    a2 = self.fc2(dout)
    h2 = self.prelu(a2)
    a3 = self.out(h2)
    return a3

def predict(self,x):
    #Apply softmax to output. 

    pred = F.softmax(self.forward(x))
    ans = []
    #Pick the class with maximum weight
    for t in pred:
        if t[0]&gt;t[1]:
            ans.append(0)
        else:
            ans.append(1)
    return torch.tensor(ans)
</code></pre>
<p>model = CreditCardTransactionClassifier().double()</p>
<p>My training loop is :</p>
<p>for epoch in range(num_epochs):<br>
h = np.array([])<br>
for data in train_loader:<br>
x = data[:,0:29]</p>
<pre><code>    y = data[:,29]
    y = y.reshape((-1,1))
    #output = model(x)
    y_pred = model.predict(x)
    y_pred = y_pred.reshape((-1,1))
</code></pre>
<h1>print(y_pred)</h1>
<pre><code>    print(y_pred.shape)
    print(y.shape)
    
    loss = criterion(y_pred.float(),y.float())
    h = np.append(h, loss.item())
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    mean_loss = np.mean(h)
print('epoch [{}/{}], loss:{:.4f}'
      .format(epoch + 1, num_epochs, mean_loss))
history['train_loss'].append(mean_loss)
</code></pre>
<p>torch.save(model.state_dict(), ‘./credit_card_model.pth’)</p>
<p>I am getting RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn error</p>
<p>Can somebody please help.</p>
<p>Thx</p>
<p>Vimal</p>

username: vimalkansal
cooked: <p>Stack trace looks like :</p>
<hr>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
17         h = np.append(h, loss.item())<br>
18         optimizer.zero_grad()<br>
—&gt; 19         loss.backward()<br>
20         optimizer.step()<br>
21         mean_loss = np.mean(h)</p>
<p>/data/anaconda/envs/vimal_pytorch/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)<br>
105                 products. Defaults to <code>False</code>.<br>
106         “”"<br>
–&gt; 107         torch.autograd.backward(self, gradient, retain_graph, create_graph)<br>
108<br>
109     def register_hook(self, hook):</p>
<p>/data/anaconda/envs/vimal_pytorch/lib/python3.7/site-packages/torch/autograd/<strong>init</strong>.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)<br>
91     Variable._execution_engine.run_backward(<br>
92         tensors, grad_tensors, retain_graph, create_graph,<br>
—&gt; 93         allow_unreachable=True)  # allow_unreachable flag<br>
94<br>
95</p>
<p>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</p>

username: ptrblck
cooked: <p>Since you are recreating a <code>tensor</code> in <code>self.predict</code>, you are detaching the output from the computation graph.<br>
Try to use <code>output = model(x)</code> to train your model, i.e. pass it to your criterion instead of <code>y_pred</code>.</p>

username: vimalkansal
cooked: <p>Thanks a lot. Yup that worked.</p>

username: rai24
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
class Autoencoder(nn.Module):</p>
<pre><code>def __init__(self):
    super(Autoencoder, self).__init__()
    self.encoder = nn.Sequential(
       # nn.Dropout2d(dropout_p= 0.5),
        nn.Conv2d(1, 16,kernel_size=3,stride =1, padding= 1,bias= False),  
        nn.ReLU(True),
        nn.MaxPool2d( 2, 2),  
        nn.Conv2d(16,32,kernel_size=3,stride =1, padding=1),  
        nn.ReLU(True),
        #nn.MaxPool2d(2, 2),  
        nn.Conv2d(32,64,kernel_size=3,stride =1, padding= 1),  
        nn.ReLU(True),
       # nn.Linear(64 * 3 *3, 120)
        #nn.MaxPool2d(2, 2),  
        
    )
    self.decoder = nn.Sequential(
        nn.MaxUnpool2d(2, 2),
        nn.ConvTranspose2d(64, 32, kernel_size=3), 
        nn.ReLU(True),
        nn.MaxUnpool2d(2, 2),
        nn.ConvTranspose2d(32, 16, kernel_size=3),  
        nn.ReLU(True),
        nn.MaxUnpool2d(2, 2),
        nn.ConvTranspose2d(16, 8, kernel_size=3),
        
        
        #nn.Tanh()
    )

def forward(self, x):
    x = self.encoder(x)
    encoded_x = x
    x = self.decoder(x)
    print(x.shape)
    x = x.view(-1, 64 * 3 * 3)
    x = F.log_softmax(self.fc2(x), dim=0)
    return x, encoded_x
</code></pre>
<p>num_epochs = 5<br>
learning_rate = 1e-5<br>
model= Autoencoder()<br>
criterion = nn.CrossEntropyLoss()<br>
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)<br>
for epoch in range(num_epochs):</p>
<pre><code>for data in loader_train_set:
    
    img, label = data
    print(img.size())
    img = torch.tensor(img)
    labels = labels.view(-1)
    img = img[:, 0:1]
    encoded_output = model.encoder(img)
    
    print(encoded_output.size())
    output_of_net = torch.rand(20, 2, 240, 320)
    label = torch.LongTensor(20,240, 320)

    output_of_net_linearized = output_of_net.permute(0, 2, 3, 1).reshape(-1, output_of_net.size(1))
    label_linearized = label.view(-1)
    loss = criterion(output_of_net_linearized, label_linearized)
    
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()



print('epoch [{}/{}], loss:{:.4f}'.format(epoch+1, num_epochs, loss.data()))
</code></pre>
<p>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn<br>
I tried making few changes, but now I encountered the above mentioned error.</p>

username: ptrblck
cooked: <p>You are creating a new tensor as the output from your model in:</p>
<pre><code class="lang-python">output_of_net = torch.rand(20, 2, 240, 320)
</code></pre>
<p>which won’t work.</p>

username: rai24
cooked: <p>Thank for helping me out.I have few questions. I want to calculate the accuracy of the training model.So, I found the skit_learn accuracy_score function, but I didn’t understand that<br>
<code>accuracy_score</code> ( <em>y_true</em> ,  <em>y_pred</em>)<br>
y_true: “will be the output from the model”<br>
y_pred: " will be the images from the DataLoader"<br>
Please, correct if I am wrong<br>
for epoch in range(Epochs):<br>
for data in trainloader:<br>
image,_ = data<br>
o_image = train_model(image)<br>
loss = criterion(o_image, image)<br>
loss.backward()<br>
optimizer.step()<br>
optimizer.zero_grad()<br>
print(‘Epoch:{}, Loss:{:.4f}’.format(epoch+1, float(loss)))</p>
<p>This is my training code.</p>

username: ptrblck
cooked: <p><code>y_true</code> would be the label tensor in the current batch (which you are currently not storing) and <code>y_pred</code> would be the prediction of your model.<br>
For a multi-class classification you can get the prediction via: <code>pred = torch.argmax(output, 1)</code>.</p>
<p>Let me know, if you need more information.</p>

username: rai24
cooked: <p>For epoch in range(Epochs):<br>
for data in trainloader:<br>
image, label= data<br>
o_image = train_model(image)<br>
loss = criterion(o_image, image)<br>
loss.backward()<br>
optimizer.step()<br>
optimizer.zero_grad()</p>
<p>pred = torch.argmax(o_image, 1)<br>
accuracy_score = accuracy_score(label, pred)<br>
print(‘Epoch:{}, Loss:{:.4f}’.format(epoch+1, float(loss)), accuracy_score)</p>

username: rai24
cooked: <p>class Autoencoder(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Autoencoder, self).<strong>init</strong>()<br>
self.encoder = nn.Sequential( # like the Composition layer you built<br>
nn.Conv2d(1, 12, 5, stride=1),<br>
nn.ReLU(),<br>
nn.Dropout2d(p=0.05),<br>
nn.Conv2d(12, 24, 5, stride=1),<br>
nn.ReLU(),<br>
nn.Conv2d(24, 48, 3),<br>
nn.ReLU(),<br>
nn.Dropout2d(p=0.05),<br>
)<br>
self.decoder = nn.Sequential(<br>
nn.ConvTranspose2d(48, 24, 5),<br>
nn.ReLU(),<br>
nn.ConvTranspose2d(24, 12, 5, stride=1),<br>
nn.ReLU(),<br>
nn.ConvTranspose2d(12, 1, 3, stride=1),<br>
nn.ReLU()<br>
)</p>
<pre><code>def forward(self, value):
    value = self.encoder(value)
    value = self.decoder(value)
    return value
</code></pre>
<p>For epoch in range(Epochs):<br>
for data in trainloader:<br>
images, label= data<br>
o_image = train_model(images)<br>
loss = criterion(o_image, images)<br>
loss.backward()<br>
optimizer.step()<br>
optimizer.zero_grad()</p>
<p>_,prediction = torch.max(o_image, 1)<br>
train_acc += torch.sum(prediction == images)<br>
print(‘Epoch:{}, Loss:{:.4f}’.format(epoch+1, float(loss)), accuracy_score)</p>
<p>~````Training Accuracy is increasing , which means there can be overfitting of data . But i used dropout function as well. Please suggest if i am missing something.<br>
`Epoch 0, Train Accuracy: 93386456.0 , TrainLoss: 0.2611906826496124<br>
Epoch 1, Train Accuracy: 186969568.0 , TrainLoss: 0.2584492564201355<br>
Epoch 2, Train Accuracy: 280305408.0 , TrainLoss: 0.0300734955817461<br>
Epoch 3, Train Accuracy: 373835808.0 , TrainLoss: 0.01294061541557312<br>
Epoch 4, Train Accuracy: 466917440.0 , TrainLoss: 0.011260838247835636<br>
Epoch 5, Train Accuracy: 560389248.0 , TrainLoss: 0.011532839387655258</p>