Should it really be necessary to do var.detach().cpu().numpy()?

username: petered
cooked: <p>I have a CUDA variable that is part of a differentiable computational graph.  I want to read out its value into numpy (say for plotting).</p>
<p>If I do <code>var.numpy()</code> I get <code>RuntimeError: Can’t call numpy() on Variable that requires grad. Use var.detach().numpy() instead.</code></p>
<p>Ok, so I do <code>var.detach().numpy()</code> and get <code>TypeError: can’t convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first</code></p>
<p>Ok, so I go <code>var.detach().cpu().numpy()</code> and it works.</p>
<p>My question is: Is there any good reason why this isn’t just done within the <code>numpy()</code> method itself?  It’s cumbersome and litters the code to have all these <code>*.detach().cpu().numpy()</code>'s sitting all around.</p>

username: chenglu
cooked: <p>I have the same question. When a user calls <code>numpy()</code> on a variable, I think he / she must also wants that variable on cpu and is detached.<br>
Don’t know how the PyTorch guys think, but i think there should be a function to get the inner values of a tensor.</p>

username: albanD
cooked: <p>Hi,</p>
<p>The main reason behind this choice I think is to avoid confusing new comers. People not very familiar with <code>requires_grad</code> and cpu/gpu Tensors might go back and forth with numpy. For example doing pytorch -&gt; numpy -&gt; pytorch and backward on the last Tensor. This will backward without issue but not all the way to the first part of the code and won’t raise any error.<br>
So the choice has been made to force the user to detach() to make sure they want to do it and it’s not a typo/other library that does this tranformation and breaks the computational graph.</p>

username: petered
cooked: <p>Fair enough - but could we at least get rid of the need for <code>X.cpu().numpy()</code>? Seems <code>X.numpy()</code> alone should be enough.</p>

username: SimonW
cooked: <p>The reason for requiring explicit <code>.cpu()</code> is that CPU tensors and the converted numpy arrays <strong>share memory</strong>. If a <code>.cpu()</code> is implicitly done, the operation will be different for CUDA and CPU tensors, and we wanted to be explicit to avoid bugs.</p>

username: David_Knowles
cooked: <p>The explicitness of pytorch is most of what I’m enjoying relative to tensorflow2.</p>
<p>As a quick follow-up to this question, is there any difference between var.detach().cpu() and var.cpu().detach() ?</p>

username: SimonW
cooked: <p>If <code>var</code> requires gradient, then <code>var.cpu().detach()</code> constructs the <code>.cpu</code> autograd edge, which soon gets destructed since the result is not stored. <code>var.detach().cpu()</code> does not do this. However, this is very fast so virtually they are the same.</p>

username: xianqian
cooked: <p>Ok but why wouldn’t it do backward “all the way to the first part of the code”?</p>

username: albanD
cooked: <p>Because we are only able to provide gradients for pytorch ops, not other ops (like numpy or other libraries).</p>

username: pytorching
cooked: <p>I’m a little confused. <code>var.clone()</code> will keep both tensors in computation graph, <code>var.cpu()</code> will return a copy of this tensor not on cuda but on cpu right? So will <code>var</code> and <code>var.cpu()</code> both remain in the computation graph? Will they share memory?</p>

username: pytorching
cooked: <aside class="quote no-group" data-username="pytorching" data-post="11" data-topic="35489">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/85f322/48.png" class="avatar"> pytorching:</div>
<blockquote>
<p>m</p>
</blockquote>
</aside>
<p><code>var</code>(on cuda) and <code>var.cpu()</code> are on different devices, they cannot share memory.  So <code>var</code> and <code>var.cpu()</code> will act like <code>var</code> and <code>var.clone()</code> except that they are on different devices?</p>

username: pytorching
cooked: <p>And If I use <code>var</code> and <code>var.cpu().cuda()</code> , it will act like <code>var</code> and <code>var.clone()</code>?</p>

username: albanD
cooked: <blockquote>
<p>And If I use <code>var</code> and <code>var.cpu().cuda()</code> , it will act like <code>var</code> and <code>var.clone()</code> ?</p>
</blockquote>
<p>If the var was originally on the GPU yes.</p>
<blockquote>
<p>So <code>var</code> and <code>var.cpu()</code> will act like <code>var</code> and <code>var.clone()</code> except that they are on different devices?</p>
</blockquote>
<p>Yes.</p>

username: akh
cooked: <p><a href="https://pytorch.org/docs/stable/community/design.html#principle-2-simple-over-easy" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/community/design.html#principle-2-simple-over-easy</a></p>