RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn during Autograd

username: sinperrrrrr
cooked: <p>This is my attemp to calculate the Jacobians for my application. During training, there weren’t any errors but after some epochs, the program raised the abovementioned error. This is the function that raised that error:<br>
batch_size = theta.shape[0]<br>
theta = theta.clone().requires_grad_(True)<br>
theta_dot = theta_dot.clone().requires_grad_(True)<br>
s_dot = s_dot.clone().requires_grad_(False)<br>
z = z.clone().requires_grad_(False)</p>
<p> →   Lagrangian = self.LagrangianFunction(theta, theta_dot, s_dot, z)</p>
<pre><code>    dL_dtheta = torch.autograd.grad(
        outputs=Lagrangian.sum(),  
        inputs=theta,
        create_graph=True
    )[0]
</code></pre>

username: ptrblck
cooked: <p>Based on the error message it seems <code>Lagrangian = self.LagrangianFunction(theta, theta_dot, s_dot, z)</code> is not differentiable and detaches the output from the computation graph. Could you post the function definition, please?</p>

username: sinperrrrrr
cooked: <p>Certainly, the self.LagrangianFunction() is just a forward() pass of a simple DNN with 3 hidden layers, each has 100 nodes with activation function is LeakyReLU(). The function takes in the given 4 arguments as mentioned.<br>
The computation of dL_dTheta is computable (output valid result) until some epochs elapsed<br>
These are the dimensions of inputs and outputs of LagrangianFunction():<br>
theta:  [batch_size, 3]<br>
theta_dot: [batch_size, 3]<br>
s_dot:  [batch_size, 3]<br>
z:   [batch_size, 1]<br>
Lagrangian:  [batch_size, 1]</p>

username: sinperrrrrr
cooked: <p>This is the derived grad_fn of LagrangianFunction() by placing print(Lagrangian.grad_fn) after the computation of Lagrangian. As you can see, the grad_fn disappeared, consequently, the Lagrangian is detached from the computational graph …<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/6/46eae2cfb64b47d0a74461f2380208346af8ff19.png" data-download-href="https://discuss.pytorch.org/uploads/default/46eae2cfb64b47d0a74461f2380208346af8ff19" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/6/46eae2cfb64b47d0a74461f2380208346af8ff19.png" alt="image" data-base62-sha1="a7mF9KvWMz1lHQwgfo7nDBR0ksh" width="690" height="206" data-dominant-color="252525"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1763×527 27.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: sinperrrrrr
cooked: <p>This is my explicit Network that I mentioned.<br>
class LAGRANGIAN_NNs(nn.Module):<br>
“”"<br>
@ brief     Neural Network for predicting the total Lagrangian function L(theta_dot, s_dot, theta, z).</p>
<pre><code>@ args:
      input_dim (int): Dimension of the input tensor. Default is 7.
      output_dim (int): Dimension of the output tensor. Default is 1.
      hidden_layer_dim (int): Number of neurons in each hidden layer. Default is 128.
      num_hidden_layer (int): Number of hidden layers. Default is 5.
      activation (list): List of activation functions for each hidden layer. 

@ input:
      theta_dot (torch.Tensor): Tensor of joint velocities, shape (batch_size, 3).
      s_dot (torch.Tensor): Tensor of end-effector velocities, shape (batch_size, 3).
      theta (torch.Tensor): Tensor of joint angles, shape (batch_size, 3).
      z (torch.Tensor): Tensor of end-effector z-value, shape (batch_size, 1).

@ output:
      torch.Tensor: Predicted total Lagrangian function, shape (batch_size, 1).
"""
def __init__(
    self,
    input_dim: int = 10,
    output_dim: int = 1,
    hidden_layer_dim: int = 50,
    num_hidden_layer: int = 3,
    activation: str = 'softplus'
):
    super(LAGRANGIAN_NNs, self).__init__()
    self.epoch = 0
    
    self.layers = nn.ModuleList([nn.Linear(input_dim, hidden_layer_dim)])
    for _ in range(num_hidden_layer - 1):
        self.layers.append(nn.Linear(hidden_layer_dim, hidden_layer_dim))
    self.layers.append(nn.Linear(hidden_layer_dim, output_dim))
    
    for layer in self.layers:
        nn.init.xavier_uniform_(layer.weight)
        nn.init.zeros_(layer.bias)
    
    if activation == 'sin':
        self.activation = torch.sin
    elif activation == 'tanh':
        self.activation = torch.tanh
    elif activation == 'softplus':
        self.activation = torch.nn.Softplus()
    elif activation == 'relu':
        self.activation = torch.nn.ReLU()
    else:
        self.activation = torch.nn.LeakyReLU()

def forward(self, theta: torch.Tensor, theta_dot: torch.Tensor, s_dot: torch.Tensor, z: torch.Tensor) -&gt; torch.Tensor:
    out = torch.cat([theta, theta_dot, s_dot, z], dim=1)
    print(f"After concat requires grad: {out.requires_grad}")
    for layer in self.layers[:-1]:
        out = self.activation(layer(out)) if isinstance(self.activation, nn.Module) else self.activation(layer(out))
    out = self.layers[-1](out)
    print(f"Final output requires grad: {out.requires_grad}")
    return out
</code></pre>
<p>I try print to debug and it turned out the line:<br>
out = torch.cat([theta, theta_dot, s_dot, z], dim=1)<br>
might be the cause. This is the output cell before the error raised:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/3/93b4c9664be38f6e4c130d2059b19b527c90fe05.png" data-download-href="https://discuss.pytorch.org/uploads/default/93b4c9664be38f6e4c130d2059b19b527c90fe05" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/3/93b4c9664be38f6e4c130d2059b19b527c90fe05.png" alt="image" data-base62-sha1="l4Fwme307Ue32VcQGjLedRUu6QB" width="342" height="113"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">342×113 1.53 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Based on your debug outputs it seems some of the iterations pass already detached tensors. Try to narrow down why this happens as it seems to come from the tensor creation or any calls before the <code>forward</code> is called.</p>