One of the variables needed for gradient computation has been modified

111137: <p>I try to use of eager execution having original nodes with parameters.<br>
For training, I code with multiple iterations for training, so I did option of;</p>
<pre><code class="lang-python">loss.backward(retain_graph = True)
</code></pre>
<p>Then I meet error of;</p>
<pre><code class="lang-python">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-18-80ae8e772693&gt; in &lt;module&gt;()
     19       y = model()
     20       loss = criterion(y, t)
---&gt; 21       loss.backward(retain_graph = True)
     22       optimizer.step()

1 frames
/usr/local/lib/python3.6/dist-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
    105                 products. Defaults to ``False``.
    106         """
--&gt; 107         torch.autograd.backward(self, gradient, retain_graph, create_graph)
    108 
    109     def register_hook(self, hook):

/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     91     Variable._execution_engine.run_backward(
     92         tensors, grad_tensors, retain_graph, create_graph,
---&gt; 93         allow_unreachable=True)  # allow_unreachable flag
     94 
     95 

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [28, 128]] is at version 25088; expected version 21504 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>Does this mean of that path for back-propagation is disconnected (cannot do it)?</p>

111137: <p>Training code is as follows;</p>
<pre><code class="lang-python">model.train()
  
for epoch in range(EPOCH):
  for x, t in dataloader_train:
    t = t.to(device)
    for time in range(TIME_STEPS):

      x_ = x[0][0][time]
      x_ = x_.to(device)
      for index_a in range(NUM_INPUT):
        for index_b in range(NUM_HIDDEN-1, 0, -1):
          model.fw_x[index_a][index_b] = model.fw_x[index_a][index_b - 1]
          model.fw_h[index_a][index_b] = model.fw_h[index_a][index_b - 1]
      
        model.fw_x[index_a][0:NUM_INPUT] = x_
        model.fw_h[index_a][0] = 0.0

      model.zero_grad()
      y = model()
      loss = criterion(y, t)
      loss.backward(retain_graph = True)
      optimizer.step()
</code></pre>
<p>Is such the FIFO coding not allowed? Then how to write same function with PyTorch<br>
grammer?</p>

albanD: <p>Hi,</p>
<p>This writing Tensors inplace can be problematic.<br>
If you can use lists instead, it will solve the problem.<br>
Otherwise, you need to avoid problematic inplace operations either by creating new Tensors every time or  adding a clone() of fw_x/fw_h after all your inplace ops.</p>

111137: <p>Hi,</p>
<p>I try this style;</p>
<pre><code class="lang-python">model.fw_x = torch.stack((model.fw_x[1:], x_))
</code></pre>
<p>where “x_” and “fw_x” are one and two dimension, repectively.<br>
<em>torch.cat()</em> needs same shape between them, so instead I used the stack with “[1:]” in order to make FIFO function, but I meet error of;</p>
<pre><code class="lang-python">RuntimeError: invalid argument 0: Tensors must have same number of dimensions: got 3 and 2 at /pytorch/aten/src/TH/generic/THTensor.cpp:702
</code></pre>
<p>I still do not understand. Any suggestion?</p>