What is mean by requires_grad(False) in resent50?

username: Mathanraj_Sharma
cooked: <p>Recently I tried a tutorial. In that, they used the resnet50 from torch.models.</p>
<pre><code class="lang-auto">model = models.resnet50(pretrained=False)
for param in model.parameters():
    param.requires_grad = False
    
model.fc = nn.Sequential(nn.Linear(2048, 512),
                                 nn.ReLU(),
                                 nn.Dropout(0.2),
                                 nn.Linear(512, 256),
                                 nn.LogSoftmax(dim=1))
</code></pre>
<p>Can someone please explain what happens here? They are not using a pre-trained model, then what will be effect of requires_grad = False?</p>

username: ptrblck
cooked: <p><code>param.requires_grad = False</code> will “freeze” this parameter, so that no gradient will be calculated for it.<br>
I’m not sure, why you would want to freeze untrained parameters, but there might be specific use cases where this might be used (for experimenting?).</p>

username: Mathanraj_Sharma
cooked: <p>Yes I am using it for experimenting, could you please tell me what happens if the untrained parameters freezed?<br>
Will it only transform the input to the shape of 2048???</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Mathanraj_Sharma" data-post="1" data-topic="79071">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mathanraj_sharma/48/23039_2.png" class="avatar"> Mathanraj_Sharma:</div>
<blockquote>
<p>model = models.resnet50(pretrained=False)</p>
</blockquote>
</aside>
<p>Yes, <code>resnet50</code> will yield a flattened activation output of <code>[batch_size, 2048]</code>.<br>
However, this is independent to freezing the parameters or training them.</p>

username: Mathanraj_Sharma
cooked: <p>Ohh, I am trying to classify the images which have 7 classes. But 6 samples. So I planned to do this.</p>
<ol>
<li>First to classify class-1 and rest 6 classes as one class.</li>
<li>I used some custom affine transformations, flips, rotations, zoom on type 1, and created around 45 samples.<br>
Is there a better way to solve this problem?</li>
</ol>

username: ptrblck
cooked: <p>45 samples seem to be quite low to train such a big model.<br>
Note that resnet50 should have ~25 million parameters, if I’m not mistaken.<br>
I doubt that you will be able to train this model successfully with 6 samples (45 with augmentation).<br>
You would also have to create a validation dataset (and even better, another test dataset), which would further reduce the number of training samples.</p>

username: Mathanraj_Sharma
cooked: <p>Yes, that is the problem I am facing trying to solve it. Do you have any suggestion?</p>

username: ptrblck
cooked: <p>Depending on the use case you could create some image features using classical image processing (e.g. HOG features) and create a small random forest classifier. You could of course add augmentation, if that helps, but you would need to split the data to get some signal for validation.</p>

username: hesham_gabr
cooked: <p>When I make it False or True?<br>
based on dataset type!<br>
if I use VGG16 (pre-trained on IMAGENET dataset) but my project dataset is medical images!<br>
now! freeze parameters of model feature extractor or not?</p>