Is it actually possible to implement TBPTT with k2 > k1 in PyTorch?

username: urancon
cooked: <p>Hi,</p>
<p>I know this post is about the same problem as <a href="https://discuss.pytorch.org/t/implementing-truncated-backpropagation-through-time/15500?">this one</a>, but I was thinking that starting on a new basis would perhaps refresh the interest and hopefully lead to a definitive answer to this question.</p>
<p>So very much like the author of the preceding post, I have a stateful network that I want to train on long sequences, and to dodge the memory limitations, I would like to use Truncated Backpropagation Through Time (TBPTT) <em><strong>with k2 &gt; k1</strong></em>.</p>
<p><a class="mention" href="/u/alband">@albanD</a> provided a neat answer, but his implementation does not work anymore on recent PyTorch versions. To make it work, you have to <strong>go back to  1.4, which I don’t want</strong>, because gradient calculations may be wrong.</p>
<p>As mentioned in the last comments to this day, by running his code in PyTorch latest versions (e.g., 1.8), we obtain the following error logs:</p>
<pre><code class="lang-auto">doing fw 0
doing fw 1
doing fw 2
doing fw 3
doing fw 4
doing backward 4
doing backward 3
doing backward 2
doing backward 1
doing backward 0
bw: 0.047006845474243164
doing fw 5
doing fw 6
doing fw 7
doing fw 8
doing fw 9
doing backward 9
doing backward 8
doing backward 7
doing backward 6
doing backward 5
doing backward 4
E:\progs2\Anaconda3\envs\pytorch17\lib\site-packages\torch\autograd_init_.py:132: UserWarning: Error detected in AddmmBackward. Traceback of forward call that caused the error:
File “BPTT.py”, line 88, in
runner.train(input_sequence, torch.zeros(200, layer_size))
File “BPTT.py”, line 28, in train
output, new_state = self.one_step_module(inp, state)
File “E:\progs2\Anaconda3\envs\pytorch17\lib\site-packages\torch\nn\modules\module.py”, line 727, in _call_impl
result = self.forward(*input, **kwargs)
File “BPTT.py”, line 65, in forward
full_out = self.lin(torch.cat([inp, state], 1))
File “E:\progs2\Anaconda3\envs\pytorch17\lib\site-packages\torch\nn\modules\module.py”, line 727, in call_impl
result = self.forward(*input, **kwargs)
File “E:\progs2\Anaconda3\envs\pytorch17\lib\site-packages\torch\nn\modules\linear.py”, line 93, in forward
return F.linear(input, self.weight, self.bias)
File “E:\progs2\Anaconda3\envs\pytorch17\lib\site-packages\torch\nn\functional.py”, line 1690, in linear
ret = torch.addmm(bias, input, weight.t())
(Triggered internally at …\torch\csrc\autograd\python_anomaly_mode.cpp:104.)
allow_unreachable=True) # allow_unreachable flag
Traceback (most recent call last):
File “BPTT.py”, line 88, in
runner.train(input_sequence, torch.zeros(200, layer_size))
File “BPTT.py”, line 47, in train
states[-i-2][1].backward(curr_grad, retain_graph=self.retain_graph)
File “E:\progs2\Anaconda3\envs\pytorch17\lib\site-packages\torch\tensor.py”, line 221, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "E:\progs2\Anaconda3\envs\pytorch17\lib\site-packages\torch\autograd_init.py", line 132, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 100]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
</code></pre>
<p>As <a class="mention" href="/u/mpmisko">@mpmisko</a> hinted, <em>“on a second backprop through a given hidden state, I am getting an in-place operation error”</em>. To me, this inplace operation error occurs because the weights of the linear layer have a _version that is superior to their _version they had when you do  <code>former_state.backward(...)</code> where here former_state is a state at step &lt;= 4.<br>
<em><strong>Indeed, weights were updated with the opimizer.step() in the meanwhile</strong></em>, so this is the so-called inplace operation (correct me if I’m wrong).</p>
<p>So my question is, -as it is impossible to edit the graph and the _version attribute of weight tensors (for instance):<br>
<em><strong>Is it actually possible to implement TBPTT with k2 &gt; k1 in PyTorch ?</strong></em></p>
<p>Although such a case of TBPTT seems <em>conceptually simple</em>, I’ve been searching for a while, and I haven’t seen a single working implementation !(!!) Also I am far from being an expert in how PyTorch computes the gradients, so that’s why I’m asking for your help ! Sorry for the long post, and thank you in advance for your insights !</p>

username: Morris
cooked: <p>Hi, I met the same issue. Have you solved this issue?</p>