List(model.parameters()) is []

Tiankai_Hang: <pre><code class="lang-auto">class Mymodel(nn.Module):
    def __init__(self, 
                 base_encoder, 
                 num_classes=21, 
                 moving_average_decay = 0.99,
                 augment_fn = None,
                 augment_fn2 = None,
                 use_momentum=True):
        super(Mymodel, self).__init__()

        self.m = moving_average_decay

        self.online_encoder = base_encoder
        self.target_encoder = copy.deepcopy(self.online_encoder)

        for param_q, param_k in zip(self.online_encoder.parameters(), self.target_encoder.parameters()):
            param_k.data.copy_(param_q.data)  # initialize
            param_k.requires_grad = False  # not update by gradient
            print("????")

    @torch.no_grad()
    def _momentum_update_key_encoder(self):
        """
        Momentum update of the key encoder
        """
        for param_q, param_k in zip(self.online_encoder.parameters(), self.target_encoder.parameters()):
            param_k.data = param_k.data * self.m + param_q.data * (1. - self.m)
            print(f"---{param_k.data}---{param_q.data}")


    def forward(self, x, rotate_type=0):

        assert rotate_type in [0, 1, 2, 3]

        # for name, i in self.online_encoder.named_parameters():
        #     print(name)


        online_pred = self.online_encoder(TF.rotate(x, rotate_type * 90))

        with torch.no_grad():
            self._momentum_update_key_encoder()
            target_pred = TF.rotate(self.target_encoder(x), rotate_type * 90)

        return online_pred, target_pred
</code></pre>
<p>When I tried to write code like this.<br>
I can’t set the parameters in self.online_encoder, and I found that list(self.online_encoder.parameters()) is empty list. (in forward and _momentum_update_key_encoder)</p>

CedricLy: <p>maybe yoou need the line:</p>
<pre><code class="lang-auto">self.online_encoder = base_encoder()
</code></pre>
<p>instead of</p>
<pre><code class="lang-auto">self.online_encoder = base_encoder
</code></pre>
<p>This depends on what your input base_encoder is</p>

Tiankai_Hang: <p>Hi, my base_encoder is set before. Such as unet model, defined like: net = UNet(in_channels=3, num_classes=10)</p>

pytorch-learning: <p>Could you show the implement of base_encoder or a snippest code of base_encoder?</p>

Tiankai_Hang: <aside class="onebox githubfolder">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/VainF/DeepLabV3Plus-Pytorch" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars.githubusercontent.com/u/18592211?s=400&amp;amp;v=4" class="thumbnail onebox-full-image" width="60" height="60">

<h3><a href="https://github.com/VainF/DeepLabV3Plus-Pytorch" target="_blank" rel="noopener nofollow ugc">VainF/DeepLabV3Plus-Pytorch</a></h3>


  <p><span class="label1">DeepLabv3, DeepLabv3+ and pretrained weights on VOC &amp;amp; Cityscapes</span></p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Just like the model here.</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/VainF/DeepLabV3Plus-Pytorch/blob/0c67dce524b2eb94dc3587ff2832e28f11440cae/main.py#L253" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/VainF/DeepLabV3Plus-Pytorch/blob/0c67dce524b2eb94dc3587ff2832e28f11440cae/main.py#L253" target="_blank" rel="noopener nofollow ugc">VainF/DeepLabV3Plus-Pytorch/blob/0c67dce524b2eb94dc3587ff2832e28f11440cae/main.py#L253</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="243" style="counter-reset: li-counter 242 ;">
<li># Set up model</li><li>model_map = {</li><li>    'deeplabv3_resnet50': network.deeplabv3_resnet50,</li><li>    'deeplabv3plus_resnet50': network.deeplabv3plus_resnet50,</li><li>    'deeplabv3_resnet101': network.deeplabv3_resnet101,</li><li>    'deeplabv3plus_resnet101': network.deeplabv3plus_resnet101,</li><li>    'deeplabv3_mobilenet': network.deeplabv3_mobilenet,</li><li>    'deeplabv3plus_mobilenet': network.deeplabv3plus_mobilenet</li><li>}</li><li></li><li class="selected">model = model_map[opts.model](num_classes=opts.num_classes, output_stride=opts.output_stride)</li><li>if opts.separable_conv and 'plus' in opts.model:</li><li>    network.convert_to_separable_conv(model.classifier)</li><li>utils.set_bn_momentum(model.backbone, momentum=0.01)</li><li></li><li># Set up metrics</li><li>metrics = StreamSegMetrics(opts.num_classes)</li><li></li><li># Set up optimizer</li><li>optimizer = torch.optim.SGD(params=[</li><li>    {'params': model.backbone.parameters(), 'lr': 0.1*opts.lr},</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


pytorch-learning: <pre><code class="lang-python">class Mymodel(nn.Module):
    def __init__(self, 
                 base_encoder, 
                 num_classes=21, 
                 moving_average_decay = 0.99,
                 augment_fn = None,
                 augment_fn2 = None,
                 use_momentum=True):
        super(Mymodel, self).__init__()

        self.m = moving_average_decay

        self.online_encoder = base_encoder
        print(type(base_encoder))  # try to debug
</code></pre>
<p>try to debug and show the code how you instantiate the Mymodel</p>

Tiankai_Hang: <pre><code class="lang-auto">    model_map = {
        'deeplabv3_resnet50': network.deeplabv3_resnet50,
        'deeplabv3plus_resnet50': network.deeplabv3plus_resnet50,
        'deeplabv3_resnet101': network.deeplabv3_resnet101,
        'deeplabv3plus_resnet101': network.deeplabv3plus_resnet101,
        'deeplabv3_mobilenet': network.deeplabv3_mobilenet,
        'deeplabv3plus_mobilenet': network.deeplabv3plus_mobilenet
    }

    net = model_map[opts.model](num_classes=opts.num_classes, output_stride=opts.output_stride)
    if opts.separable_conv and 'plus' in opts.model:
        network.convert_to_separable_conv(net.classifier)
    utils.set_bn_momentum(net.backbone, momentum=0.01)
    
    # model = network.SiamSeg(net, 
    #                         image_size=(opts.crop_size, opts.crop_size),
    #                         projection_size=opts.num_classes,
    #                         use_momentum=False,
    #                 )
    model = Mymodel(
        net,
        num_classes=opts.num_classes, 
        moving_average_decay=opts.momentum,
        augment_fn = None,
        augment_fn2 = None,
        use_momentum=True,
    )
</code></pre>
<p>Like this</p>

Tiankai_Hang: <p>Solved.</p>
<p>I used nn.Dataparallel to train my model on multi-gpu machine, it calculated the gradient on the main gpu, so my function in class to update the parameters is incorrect.</p>
<p>When I choose to use on gpu or DDP, it works well.</p>