"One of the differentiated Tensors appears to not have been used" - Can't figure out why

username: barak_amit
cooked: <p>Hello, Im trying to implement a Differential Dynamic Programming algorithm using pytorch and I keep getting:</p>
<blockquote>
<p>RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.</p>
</blockquote>
<p>I’ve seen numerous posts about this problem but I want able to grasp why it is happening in my code:</p>
<blockquote>
<p>def optimize(<br>
self,<br>
x0: torch.Tensor,<br>
x_goal: torch.Tensor,<br>
N: int = None,<br>
U0: torch.Tensor = None,<br>
full_output: bool = False,<br>
):<br>
if not N and U0 is None:<br>
raise ValueError(“You must provide either trajectory length N or initial control sequence U0”)<br>
if U0 is not None:<br>
N = len(U0)<br>
U = U0.clone().detach().requires_grad_(True)<br>
else:<br>
assert N &gt; 0<br>
U = torch.rand(N, self.Nu, requires_grad=True) * 2 - 1  # Uniform between -1 and 1<br>
def J(X, U):<br>
total_cost = sum(self.inst_cost(X[i], U[i], x_goal) for i in range(len(U)))<br>
total_cost += self.terminal_cost(X[-1], x_goal)<br>
return total_cost</p>
</blockquote>
<pre><code>    X = [x0.clone().detach().requires_grad_(True)]
    for i in range(N):
        X.append(self.dynamics(X[-1], U[i]))
    X = torch.stack(X)

    last_cost = J(X, U)

    if full_output:
        X_hist = [X.clone().detach()]
        U_hist = [U.clone().detach()]
        cost_hist = [last_cost.item()]

    for iteration in range(self.max_iters):
        Vx = torch.autograd.grad(self.terminal_cost(X[-1], x_goal), X[-1], retain_graph=True)[0]
</code></pre>
<p>The main that calls this optimization function is:</p>
<blockquote>
<p>def f(x, u, constrain=True):<br>
theta = torch.atan2(x[0], x[1])<br>
theta_dot = x[2].detach()<br>
torque = torch.tanh(u[0]) if constrain else u[0]<br>
theta_dot_dot = -3 * G * torch.sin(theta + torch.pi) / (2 * L) + 3 * torque / (M * L**2)<br>
theta += theta_dot * dt<br>
theta_dot  += theta_dot_dot * dt<br>
return torch.stack([torch.sin(theta), torch.cos(theta), theta_dot])</p>
</blockquote>
<blockquote>
<p>def h(x, x_goal):<br>
error = x - x_goal<br>
Qt = 100 * torch.eye(3, dtype=torch.float32)<br>
result = error.view(1, -1) @ Qt @ error.view(-1, 1)<br>
return result.squeeze()</p>
</blockquote>
<blockquote>
<p>N = 100  # trajectory points<br>
Nx = 3  # state dimension<br>
Nu = 1  # control dimensions</p>
<p>x0 = torch.tensor([torch.sin(torch.tensor(torch.pi)), torch.cos(torch.tensor(torch.pi)), 0.0], requires_grad=True)</p>
<p>x_goal = torch.tensor([torch.sin(torch.tensor(0.0)), torch.cos(torch.tensor(0.0)), 0.0], requires_grad=True)<br>
ddp = DDPOptimizer(Nx, Nu, f, g, h)<br>
X, U, X_hist, U_hist, J_hist = ddp.optimize(x0, x_goal, N=N, full_output=True)</p>
</blockquote>
<p>This implementatin is based on the following blog and code which mainly uses sympy and numy</p>
<p><a href="https://www.imgeorgiev.com/2023-02-01-ddp/" rel="noopener nofollow ugc">DDP blog</a></p>
<p>Ill be happy for your help!</p>

username: ptrblck
cooked: <p>Your code is not executable as a few methods are missing. Could you create a minimal and executable code snippet reproducing the issue, please?</p>