Initialize weights of child layer

ttbb: <p>When I want to initialize weights, I use following snippet:</p>
<pre><code class="lang-auto">def weights_init(model):
    if type(model) in [nn.Conv2d,nn.Linear]:
        nn.init.xavier_normal_(model.weight.data)
        nn.init.constant_(model.bias.data, 0.1)
</code></pre>
<p>I use this, when I want to initialize the entire layer type.<br>
<strong>But how can I initialize child layer of a layer type, e.g. only one layer of a Conv2d layer type?</strong><br>
Is the following snippet a correct approach?</p>
<pre><code class="lang-auto">def weightsPart_init(model):
    for child in list(model.children())[12:13]:       # just one Conv2d layer
      nn.init.xavier_normal_(child.weight.data)
      nn.init.constant_(child.bias.data, 0.1)
</code></pre>
<p>Reason for that is, that I have a pretrained model, from which I want to load the weights of specific convolutional layers and use as fixed feature extractor. But the last convolutional layer, I want to initalize with xavier_normal and retrain.</p>
<p>Thanks for help.</p>

Ajay_Babu: <pre><code class="lang-auto">import torch 

model = torch.hub.load('facebookresearch/WSL-Images', 'resnext101_32x8d_wsl')
</code></pre>
<p>And the last layer is</p>
<pre><code class="lang-auto">(layer4): Sequential(
    (0): Bottleneck(
      (conv1): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(2048, 2048, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=32, bias=False)
      (bn2): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (downsample): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): Bottleneck(
      (conv1): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(2048, 2048, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
      (bn2): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
    )
    (2): Bottleneck(
      (conv1): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(2048, 2048, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
      (bn2): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
    )
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
  (fc): Linear(in_features=2048, out_features=1000, bias=True)
)
</code></pre>
<p>So I did this</p>
<pre><code class="lang-auto">torch.nn.init.xavier_uniform_(model.layer4[2].conv3.weight)
</code></pre>
<p>I hope this helps you !</p>

ttbb: <p>This helped a lot, thank you!</p>
<p>Looks similar to that what I proposed with <code>def weightsPart_init(model)</code>, but I was not sure if this was the correct way.</p>