Grad is None! but can't understand why!?

moher: <p>Hi,<br>
I’ve started to learn PyTorch and I’m loving it.<br>
I’ve learned a few things and tried to implement logistic regression from scratch,<br>
but challenging myself not to use <code>nn</code> Module and any optimizer or loss function and write these myself.</p>
<p>So I’ve written the code, but I am having a problem which I’ve stuck at for quite some time now.<br>
I really would appreciate if you show me why this does not work and how to avoid these situations as it seems to me <code>autograd</code> can be confusing.</p>
<pre><code class="lang-auto">def get_one_sample(idx):
    return dataset.iloc[idx, :-1].values, dataset.iloc[idx, -1]

device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

w = torch.randn((d, 1), dtype=torch.float64, device=device, requires_grad=True)
b = torch.randn((1, 1), dtype=torch.float64, device=device, requires_grad=True)

losses = []
for epoch in range(epochs):
    for i in range(m):
        x_np, y_np = get_one_sample(i)
        x = torch.from_numpy(x_np).to(device).view(-1, 1)
        y = torch.tensor([y_np], dtype=torch.float, device=device)
        
        # forward computation
        y_hat = torch.sigmoid(w.T@x+b)
        
        # loss computation &amp; loss backprop

        loss = torch.log(-y_hat) if y_np==1 else torch.log(1-y_hat)
    
        losses.append(loss.item())
        
        loss.backward()
        
        print(w.grad)
        # updating weights
        w = w - alpha * w.grad
        b = b - alpha * b.grad
        
        print(w.grad)
        
        w.grad.data.zero_()
        b.grad.data.zero_()
        
    print("epoch:", epoch)

plt.plot(losses)
plt.show();

</code></pre>
<p>the console output before crashing into error:</p>
<pre><code class="lang-auto">tensor([[-4.1593e-07],
        [-9.9529e-07],
        [ 3.2241e-07],
        [ 5.1336e-08]], device='cuda:0', dtype=torch.float64)
None
None
</code></pre>
<p>the runtime stops at <code>w.grad.data.zero_()</code> with the following error:</p>
<pre><code class="lang-auto">----------------------------------------------------------------------
AttributeError                       Traceback (most recent call last)
&lt;ipython-input-79-eb9f256e22e0&gt; in &lt;module&gt;
     32         print(w.grad)
     33 
---&gt; 34         w.grad.data.zero_()
     35         b.grad.data.zero_()
     36 

AttributeError: 'NoneType' object has no attribute 'data'
</code></pre>
<p>also, I can’t quite figure out why it is outputting 3 lines or print where as I obviously have only two!</p>
<p>Thanks in advance.</p>

moher: <p>I fixed it myself.</p>
<p>when updating the weights, I have to put them in <code>with torch.no_grad():</code><br>
as apparently not doing so makes the computational graph acyclic. (am I write?)<br>
Also, it is correct to use <code>w.grad.zero_()</code> not <code>w.grad.data.zero()</code>, which I can’t understand why yet!</p>
<p>So, the corrected part is as follows:</p>
<pre><code class="lang-auto">        loss.backward()
        
        # updating weights
        with torch.no_grad():
            w -= alpha * w.grad
            b -= alpha * b.grad

        w.grad.zero_()
        b.grad.zero_()
</code></pre>