Access value of the weights of these models

CasellaJr: <p>I have this notebook:</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://colab.research.google.com/img/favicon.ico?vrz=colab-20210618-060113-RC00_380157750" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/1ZvV_K7c2mecPennmcWOuZS39Ys5Y3P8Z?usp=sharing" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="260" height="260">

<h3><a href="https://colab.research.google.com/drive/1ZvV_K7c2mecPennmcWOuZS39Ys5Y3P8Z?usp=sharing" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>It contains 3 networks: vgg_a, vgg_b and vgg_star. In particular I have vgg_star and vgg_star after the sum of the weights of vgg_a and vgg_b.<br>
I want to see the numerical values of the weights (to see if there are patterns for example) of vgg_star and vgg_star after the sum of weights.<br>
What I have to change in order to do this? I have to use .parameters() or .weight?</p>

bsridatta: <p>mode.parameters() gives a generator. convert into list and you can print the weights<br>
list(model.parameters())[0].shape</p>

CasellaJr: <p>Ok, thanks<br>
But how can I access the weights of vgg_star before and after the summation of weights?</p>

bsridatta: <p>The code is too big to study, maybe share a minimal code block? however, why don’t you just print the parameters of the model before and after?</p>
<p>what do you mean by sum are you looping over all parameters and just adding them?</p>

CasellaJr: <p>Yes, with sum you are right: I take vgg_a and vgg_b and I sum layer per layer their weights.<br>
For showing the parameters, I mean this, I am not doing in a right way the print of the weights.</p>
<p>I have these:</p>
<pre><code class="lang-auto">model1 = VGG16((1,32,32),batch_norm=True)
model2 = VGG16((1,32,32),batch_norm=True)
model3 = VGG16((1,32,32),batch_norm=True)
classifier = Classifier(num_classes=1)

nets = [model1,model2,model3,classifier]
</code></pre>
<p>where VGG16 and Classifier are:</p>
<pre><code class="lang-auto">class VGGBlock(nn.Module):
    def __init__(self, in_channels, out_channels,batch_norm=False):

        super().__init__()

        conv2_params = {'kernel_size': (3, 3),
                        'stride'     : (1, 1),
                        'padding'   : 1
                        }

        noop = lambda x : x

        self._batch_norm = batch_norm

        self.conv1 = nn.Conv2d(in_channels=in_channels,out_channels=out_channels , **conv2_params)
        self.bn1 = nn.BatchNorm2d(out_channels) if batch_norm else noop

        self.conv2 = nn.Conv2d(in_channels=out_channels,out_channels=out_channels, **conv2_params)
        self.bn2 = nn.BatchNorm2d(out_channels) if batch_norm else noop

        self.max_pooling = nn.MaxPool2d(kernel_size=(2, 2), stride=(2, 2))

    @property
    def batch_norm(self):
        return self._batch_norm

    def forward(self,x):
        x = self.conv1(x)
        x = self.bn1(x)
        x = F.relu(x)

        x = self.conv2(x)
        x = self.bn2(x)
        x = F.relu(x)

        x = self.max_pooling(x)

        return x

class VGG16(nn.Module):

  def __init__(self, input_size, batch_norm=False):
    super(VGG16, self).__init__()

    self.in_channels,self.in_width,self.in_height = input_size

    self.block_1 = VGGBlock(self.in_channels,64,batch_norm=batch_norm)
    self.block_2 = VGGBlock(64, 128,batch_norm=batch_norm)
    self.block_3 = VGGBlock(128, 256,batch_norm=batch_norm)
    self.block_4 = VGGBlock(256,512,batch_norm=batch_norm)


  @property
  def input_size(self):
      return self.in_channels,self.in_width,self.in_height

  def forward(self, x):

    x = self.block_1(x)
    x = self.block_2(x)
    x = self.block_3(x)
    x = self.block_4(x)
    # x = self.avgpool(x)
    x = torch.flatten(x,1)

    return x

class Classifier(nn.Module):

    def __init__(self,num_classes=1):
        super().__init__()

        self.classifier = nn.Sequential(
            nn.Linear(2048, 2048),
            nn.ReLU(True),
            nn.Dropout(p=0.5),
            nn.Linear(2048, 512),
            nn.ReLU(True),
            nn.Dropout(p=0.5),
            nn.Linear(512, num_classes)
        )

    def forward(self,x):

        return self.classifier(x)

</code></pre>
<p>And in the train there is this part to do the sum:</p>
<pre><code class="lang-auto">for n in model_a:
              layer_a = model_a[n]
              layer_b = model_b[n]
              layer_c = model_c[n]
              if (isinstance(layer_a,nn.Conv2d)):
                loss += lambda_reg * reg_loss(combo_fn(layer_a.weight,layer_b.weight),layer_c.weight)
                if (layer_a.bias is not None):
                  loss += lambda_reg * reg_loss(combo_fn(layer_a.bias, layer_b.bias), layer_c.bias)
</code></pre>