A very strange thing ,god, my tensor couldn't generate grad when tensor.cuda()

username: Joker
cooked: <p>Code  first ;</p>
<pre><code class="lang-auto">import torch
x = torch.ones((2,10), requires_grad=True)
z = torch.ones((1,10), requires_grad=True)
y = (4 * x * x + 10 * z).mean()
y.backward()
print('x  grad :', x.grad)
print('z  grad:', z.grad)
print('y  grad:', y.grad)

</code></pre>
<p>it’s  output<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/d/9da9bbcf8b42fa4c0a59ad18130303301594d91c.png" data-download-href="https://discuss.pytorch.org/uploads/default/9da9bbcf8b42fa4c0a59ad18130303301594d91c" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/d/9da9bbcf8b42fa4c0a59ad18130303301594d91c.png" alt="image" data-base62-sha1="muKC9YkmIcF3lBrKYLXP7bolqck" width="690" height="282" data-dominant-color="524F4F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">975×399 18.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
As  we can  see, if we  write code  like  above  without  tensor.cuda() , tensor  x,z  has  a  grad here;</p>
<p>But  if  we  change  tensor  x,y to  cuda   like  below  with  tensor.cuda() ;</p>
<pre><code class="lang-auto">import torch
x = torch.ones((2,10), requires_grad=True).cuda()
z = torch.ones((1,10), requires_grad=True).cuda()
y = (4 * x * x + 10 * z).mean()
y.backward()
print('x  grad :', x.grad)
print('z  grad:', z.grad)
print('y  grad:', y.grad)

</code></pre>
<p>If  we  write  like  this  ,  the  grad  will  be  None ,  what   a  strange  thing  ,  what  happend  here?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/a/2/a25bde09c48f27e830ce917dde5ef928d54f86db.png" alt="image" data-base62-sha1="naiaQYVyXuEeATYNsg8C56bE2g3" width="586" height="224"></p>
<p>I’m  not new  to pytorch , I have   learned   and used  pytorch  for a   long  time  ,  but  it’s  the  first  time  that  I  meet  such  a  thing  like  this .<br>
Could  anyone  helps  me  about  it ?  Thanks  for  your  reply</p>

username: Joker
cooked: <p>Sorry  guys ,I’m  very  stupid  ,  I  read   the  basic   concept  again  ,and  found that   if   I  use  x= x.cuda(),then  x  will   become not  a  leaf  ,  backward()  will  not  keep  gradient  for  x  ,  so  I  should  add   x.retain_grad()   after  x=x.cuda() ,  and  then ,  x  will  get  a   grad  here.</p>

username: pierrecurie
cooked: <p>You can avoid most of these problems by wrapping it in an object that subclasses nn.Module.  There’s a reason most neural networks subclass nn.Module.  Try this version</p>
<pre><code class="lang-auto">import torch
from torch import nn
class foo(nn.Module):
    def __init__(self):
        super(foo,self).__init__()
        self.x = nn.Parameter(torch.ones(2,10)) #requires_grad automatically True
        self.z = nn.Parameter(torch.ones(1,10))
bar=foo().cuda()
y = (4 * bar.x * bar.x + 10 * bar.z).mean()
y.backward()
print('x  grad :', bar.x.grad)
print('z  grad:', bar.z.grad)
print('y  grad:', y.grad)
</code></pre>

username: Joker
cooked: <p>Thanks , mate .<br>
I  sovled  this  problem   with     nn.Parameter   now .</p>