Layer requires_grad=True but layer.grad is None

chrisuzokwe: <p>Hi. I’m implementing Faster R-CNN from scratch and am having gradient issues. My cls_loc layer in my pooling layer has no grad, despite its requires_grad = True. After looking over my code I cannot find where the graph is broken or anything, but my model can’t update. Any help on this would be greatly appreciated. Here’s the module as it is now:</p>
<p>class PoolingCNN(nn.Module):</p>
<p>def <strong>init</strong>(self,</p>
<pre><code>           sample_layer_params = {

                  "n_sample": 128,

                  "pos_ratio": 0.25,

                  "pos_iou_thresh": 0.25,

                  "neg_iou_thresh": 0.5,

                  "neg_iou_thresh_hi": 0.5,

                  "neg_iou_thresh_lo": 0.0,

                  "mode": "train"

                },

           ):



super(PoolingCNN, self).__init__()

self.roi_head_classifier = nn.Sequential(*[nn.Linear(25088, 4096),

                                  nn.Linear(4096, 4096)])

self.cls_loc = nn.Linear(4096, 2 * 4) # (VOC 20 classes + 1 background. Each will have 4 co-ordinates)

self.cls_loc.weight.data.normal_(0, 0.01)

self.cls_loc.bias.data.zero_()

self.score = nn.Linear(4096, 2) # (VOC 20 classes + 1 background)
</code></pre>
<p>def forward(self, pred_bounds, gt_bounds, labels, out_map):</p>
<pre><code>gt_roi_labels, gt_roi_locs, sample_roi = sample_rois(gt_bounds, pred_bounds, labels)

k = pooling_layer(sample_roi, out_map)

print(k.shape)

#print(k.type)

k = self.roi_head_classifier(k)

print(k.shape)

roi_cls_loc = self.cls_loc(k)

roi_cls_score = self.score(k)

return gt_roi_labels, gt_roi_locs, roi_cls_loc, roi_cls_score
</code></pre>
<p>def predict(self, out_map, roi):</p>
<pre><code>#print(roi.shape)

k = pooling_layer(roi, out_map)

k = self.roi_head_classifier(k)

roi_cls_loc = self.cls_loc(k)

roi_cls_score = self.score(k)

index = np.array(roi_cls_score.detach().argmax(axis=1))

#print("index", index)

#

corr_bound = np.asarray([roi_cls_loc[i, (index[i]*4):(index[i]*4+4)].detach().numpy() for i in range(len(roi_cls_loc))])

#for i in range(len(roi_cls_loc)):

#  print(roi_cls_loc[i, (index[i]*4):(index[i]*4+4)])

#print(corr_bound.shape)



#print(corr_bound)

bboxes = roi_to_bbox(np.asarray(roi), corr_bound)

#print(bboxes)

return roi_cls_loc, roi_cls_score, bboxes</code></pre>

KFrank: <p>Hi Chris!</p>
<aside class="quote no-group" data-username="chrisuzokwe" data-post="1" data-topic="123602" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chrisuzokwe/48/36228_2.png" class="avatar"> chrisuzokwe:</div>
<blockquote>
<p>My cls_loc layer in my pooling layer has no grad, despite its requires_grad = True.</p>
</blockquote>
</aside>
<p>I can’t figure out what you are doing here.  But here are two comments<br>
that may or may not be relevant:</p>
<p>First, a “layer” (in your case a <code>Linear</code>) doesn’t have a <code>requires_grad</code><br>
property; its <code>Parameter</code>s do (such as <code>Linear.weight</code>).</p>
<p>Second, a tensor (or <code>Parameter</code>) that starts out with<br>
<code>requires_grad = True</code> won’t have a non-<code>None</code> <code>grad</code> property<br>
until it’s been backpropagated through.</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.7.1'
&gt;&gt;&gt; _ = torch.manual_seed (2021)
&gt;&gt;&gt; lin = torch.nn.Linear (4, 2)
&gt;&gt;&gt; lin.requires_grad
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/home/user/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 778, in __getattr__
    raise ModuleAttributeError("'{}' object has no attribute '{}'".format(
torch.nn.modules.module.ModuleAttributeError: 'Linear' object has no attribute 'requires_grad'
&gt;&gt;&gt; lin.weight.requires_grad
True
&gt;&gt;&gt; lin.weight.grad
&gt;&gt;&gt; lin (torch.randn (4)).sum().backward()
&gt;&gt;&gt; lin.weight.grad
tensor([[-1.2991,  0.1929,  0.4697, -0.0257],
        [-1.2991,  0.1929,  0.4697, -0.0257]])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>