Updating a 3D Tensor creates an inplace operation

username: dmadeka1
cooked: <p>Im trying to manage a time series update for Reinforcement Learning in PyTorch. The involves updating operations on a 3D Tensor in the following way:</p>
<pre><code class="lang-python"> self.state[batch, time_index, 0] = (self.state[batch, time_index, 0] + prediction[batch]).clone()
 self.state[batch, time_index + 1] = (self.state[batch, time_index + 1] + self.state[batch, time_index]).clone()
 rew = compute_reward(self.state[batch, time_index], y)
 self.state[batch, time_index] = (self.state[batch, time_index] - y).clone()
</code></pre>
<p><code>compute_reward</code> does another operation that could be culprit:</p>
<pre><code class="lang-python">points = torch.min(state, y)
</code></pre>
<p>This gives the error:</p>
<pre><code class="lang-python">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [2000, 1]], which is output 0 of SelectBackward, is at version 412206; expected version 410204 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
</code></pre>
<p>Does anyone know any tricks to be able to do this without creating an in-place operations?</p>

username: dmadeka1
cooked: <p>Running <code>torch.autograd.set_detect_anomaly(True)</code> gives:</p>
<pre><code class="lang-python">sys:1: RuntimeWarning: Traceback of forward call that caused the error:
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/ipykernel/__main__.py", line 3, in &lt;module&gt;
    app.launch_new_instance()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 486, in start
    self.io_loop.start()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/tornado/platform/asyncio.py", line 127, in start
    self.asyncio_loop.run_forever()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/asyncio/base_events.py", line 422, in run_forever
    self._run_once()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/asyncio/base_events.py", line 1432, in _run_once
    handle._run()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/tornado/ioloop.py", line 759, in _run_callback
    ret = callback()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/tornado/stack_context.py", line 276, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 536, in &lt;lambda&gt;
    self.io_loop.add_callback(lambda : self._handle_events(self.socket, 0))
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 450, in _handle_events
    self._handle_recv()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 480, in _handle_recv
    self._run_callback(callback, msg)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 432, in _run_callback
    callback(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/tornado/stack_context.py", line 276, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
    return self.dispatch_shell(stream, msg)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
    handler(stream, idents, msg)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
    user_expressions, allow_stdin)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 208, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 537, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2662, in run_cell
    raw_cell, store_history, silent, shell_futures)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2785, in _run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2903, in run_ast_nodes
    if self.run_code(code, result):
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2963, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "&lt;ipython-input-26-0c93ed1f5d45&gt;", line 6, in &lt;module&gt;
    loss = inv(predictions, dem, np.arange(N), t)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "&lt;ipython-input-15-f2333262fd86&gt;", line 32, in forward
    cash = self.ltfcf(prediction, y, state, time_idx, batch_idx)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "&lt;ipython-input-14-fe921282d8fb&gt;", line 11, in forward
    points = torch.min(state, y)
</code></pre>
<p>looks like it is <code>torch.min</code></p>

username: dmadeka1
cooked: <p>So what im not sure is that is the <code>torch.min</code> causing this error. Or is it some prior operation on the state (as pasted above) and the error is just being raised by that call – is there a way to identify this?</p>

username: ptrblck
cooked: <p>Could you try to assign the result to a temporary tensor instead of <code>self.state[batch, time_index, 0]</code> and concatenate these temp tensors afterwards?</p>

username: dmadeka1
cooked: <p>Would a bunch of <code>.clone()</code>'s everywhere also work or create unexpected results?</p>

username: dmadeka1
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> So if I wanted to update the state a variable number of time steps ahead:</p>
<pre><code class="lang-python">t = torch.range(batch.shape[0])
self.state[batch, time_index + t] = (self.state[batch, time_index + t] + self.state[batch, time_index])
</code></pre>
<p>Is that possible?</p>