RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation:at version 12; expected version 10 instead

username: Jiyao_Li
cooked: <p>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [16, 4, 84, 84]] is at version 12; expected version 10 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).</p>
<p>I try to use A2C with 16 workers to play Atari game. How can I debug such error?</p>

username: Jiyao_Li
cooked: <p>Here is the stack information:</p>
<p>Traceback (most recent call last):<br>
File “C:/Users/Jiyao/PycharmProjects/RLs/PolicyGradient/A2C/run_atari.py”, line 61, in <br>
train()<br>
File “C:/Users/Jiyao/PycharmProjects/RLs/PolicyGradient/A2C/run_atari.py”, line 53, in train<br>
agent.learn(i_step)<br>
File “C:\Users\Jiyao\PycharmProjects\RLs\PolicyGradient\A2C\a2c.py”, line 90, in learn<br>
self.memo.total_entropy.backward()<br>
File “C:\Users\Jiyao.conda\envs\baselines\lib\site-packages\torch_tensor.py”, line 307, in backward<br>
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)<br>
File “C:\Users\Jiyao.conda\envs\baselines\lib\site-packages\torch\autograd_<em>init</em>_.py”, line 156, in backward<br>
allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag<br>
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [16, 4, 84, 84]] is at version 12; expected version 10 instead. Hint:</p>

username: Jiyao_Li
cooked: <p>and [torch.FloatTensor [16, 4, 84, 84]]  is the dimension of the images I feed into the network.</p>

username: Jiyao_Li
cooked: <p>Below is my training code:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/d/ed7d914ce12d41c8968b882f2cf7bf360c41742c.png" data-download-href="https://discuss.pytorch.org/uploads/default/ed7d914ce12d41c8968b882f2cf7bf360c41742c" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/d/ed7d914ce12d41c8968b882f2cf7bf360c41742c.png" alt="image" data-base62-sha1="xSWbDSFrLy8T9wtzhp2GXptJ34w" width="690" height="389" data-dominant-color="303030"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">931×526 36.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Jiyao_Li
cooked: <p>The network:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/0/10702d05527f3c6e12fb1f4766177d1129b3be50.png" data-download-href="https://discuss.pytorch.org/uploads/default/10702d05527f3c6e12fb1f4766177d1129b3be50" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/10702d05527f3c6e12fb1f4766177d1129b3be50_2_690x497.png" alt="image" data-base62-sha1="2lpYsGrkeb0djprPTsLqfznMA1O" width="690" height="497" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/10702d05527f3c6e12fb1f4766177d1129b3be50_2_690x497.png, https://discuss.pytorch.org/uploads/default/original/3X/1/0/10702d05527f3c6e12fb1f4766177d1129b3be50.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/0/10702d05527f3c6e12fb1f4766177d1129b3be50.png 2x" data-dominant-color="313030"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">898×647 53.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Jiyao_Li
cooked: <p>The Agent:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/4/f48101da86686128d734139641c2e7004da52c20.png" data-download-href="https://discuss.pytorch.org/uploads/default/f48101da86686128d734139641c2e7004da52c20" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f48101da86686128d734139641c2e7004da52c20_2_680x500.png" alt="image" data-base62-sha1="ySYTLMHQNj0QzkmWUDXxbJyLjeE" width="680" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/4/f48101da86686128d734139641c2e7004da52c20_2_680x500.png, https://discuss.pytorch.org/uploads/default/original/3X/f/4/f48101da86686128d734139641c2e7004da52c20.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/4/f48101da86686128d734139641c2e7004da52c20.png 2x" data-dominant-color="303031"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">960×705 53.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/c/1cbb6b9836d964e7d764f6c82948ce2c2d6536b5.png" data-download-href="https://discuss.pytorch.org/uploads/default/1cbb6b9836d964e7d764f6c82948ce2c2d6536b5" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/c/1cbb6b9836d964e7d764f6c82948ce2c2d6536b5.png" alt="image" data-base62-sha1="46aUKooQKV5PTdbkgBLlFGsg0VD" width="690" height="481" data-dominant-color="313132"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">924×645 49.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>