When is `tensor.data` fine to leave in old code?

murph213: <p>I know that the <code>.data</code> attribute of tensors has been deprecated (when Variable got merged into Tensor) and I also know its use is discouraged because it causes problems with gradient computation.  However, I still come across old code bases that use it extensively.  In initial phases of my short-term projects, I do not want to update a bunch of deprecated uses if they are harmless to the logical flow and effect of the code.</p>
<p>In short</p>
<blockquote>
<p>When is it ok to leave <code>tensor.data</code> in old code?  Can we understand the OK and dangerous uses?</p>
</blockquote>
<p><strong>What I know is bad</strong>: Let’s say we are building a computation graph with the ultimate goal of computing the gradient of <code>loss</code> with respect to weights.  Assigning values directly to the <code>data</code> attribute in any computations on tensors in the path to <code>loss</code> will not be directly tracked.  This could lead to incorrect derivatives of loss wrt weights.</p>
<p><strong>What I’m not sure about</strong>: If we just use <code>tensor.data</code> on the RIGHT HAND SIDE of a computation that does NOT lead to the desired output (loss) in the computation graph, this should be fine, right?  In this case, <code>.data</code> will not track the gradient, and the computation does not lead to a gradient we care about anyway.  For example, say I want to compute the norm of all my weight matrices and write them to disk for analysis later on.  I could do <code>norm(model.layer.weight.data)</code>, right?  Same goes for simply tracking the loss: you could in principle do <code>print(loss.data)</code> without causing harm.</p>
<p>I know that it would be better to use <code>.detach()</code> here, but would it be <strong>dangerous</strong> or <strong>lead to unintended consequences</strong> to use <code>.data</code> here?</p>
<p><strong>What else?</strong> Are there other dangers of <code>.data</code> to be aware of?</p>
<p>Thank you</p>

charan_Vjy: <p>You seem to have covered the pitfalls of using .data quite well. Rule of thumb, make sure variables needed for gradient computation do not use the .data method. The simplest way to ensure is this is to convert .data and .detach and see if it throws an error. Another way to avoid incorrect gradient computation would be to comment out the .data call and see if the ouputs after few epochs are the same. <a href="https://pytorch.org/blog/pytorch-0_4_0-migration-guide/" class="inline-onebox" rel="noopener nofollow ugc">PyTorch 0.4.0 Migration Guide | PyTorch</a> highlights the problem of using .data with an example.</p>
<blockquote>
<p>Are there other dangers of  <code>.data</code>  to be aware of?</p>
</blockquote>
<p>None that I know off.</p>

albanD: <blockquote>
<p>would it be <strong>dangerous</strong> or <strong>lead to unintended consequences</strong> to use <code>.data</code> here?</p>
</blockquote>
<p>Another major difference between .data and .detach() that you don’t mention here is inplace correctness.<br>
Basically, when we save Tensors because their value is needed to compute the gradients, we have special logic that make sure that it wasn’t modified inplace between the moment where it was saved and the moment where we try to use the value.<br>
.data will break these checks. So even if you use .data on the right hand side of some op that is not used to compute the loss. If you later modify that Tensor inplace and that Tensor was needed for backward, you can end up with wrong gradients because the saved value was changed but the .data hide it from the inplace detection code.</p>