What's the difference between the two hypothesis?？

username: yunpei
cooked: <p>Resnet has a 4 component: layer1, layer2, layer3, layer4.<br>
<strong>Hypothesis one：</strong><br>
And I initialize the layer4 use the same parameters, just like this:<br>
<em><strong>self.layer4_1 = model_resnet.layer4</strong></em><br>
<em><strong>self.layer4_2 = model_resnet.layer4</strong></em><br>
That means the layer4_1 and layer4_2 pointing to the same parameters, they share the same parameter. I update them alternately.</p>
<p><strong>Hypothesis two：</strong><br>
I just define one layer4 just like this:<br>
<em><strong>self.layer4 = model_resnet.layer4</strong></em><br>
And I update the layer4 two times than <em>Hypothesis one</em>.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/9/9c74cccedbd84e0e9c619cf116b91462b93528cb.png" alt="image" data-base62-sha1="mk4J9B3wwNqkoslLB3NfaiiNrBh" width="247" height="345"><br>
I want to ask what’s the difference between the two hypothesis？<strong>Why are the model results different when the model converges？</strong></p>
<p>My English is poor, if you are chinese, we can talk in chinese.</p>
<p>Looking forward to your reply!!</p>

username: Oli
cooked: <p>Hello and welcome <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> I Like your post and well done with the images.</p>
<p>You stated that we point to the same layer in hypothesis 1/top image. Whenever we update one of the boxes, <strong>the other box is also updated</strong>, right?</p>
<p>Then in hypothesis 2: Every time you update the box, that’s the same as updating <strong>one of the boxes in hypothesis one</strong> - as they are connected. You say that in hypothesis 2, the update happens <strong>twice</strong> so there are double the amount of updates compared to hypothesis 1, right?</p>
<p>Just what came to mind</p>

username: yunpei
cooked: <p>Thanks for you!</p>
<p>I draw a images more clearly.</p>
<p>In hypothesis 1/top image, the layer4_1 and layer4_2 share the same parameters, which is the same in layer4 in hypothesis 2/down image.</p>
<p>I update layer4_1 and layer4_2 alternatively in hypothesis 1/top image and I update layer4 in hypothesis 2/down image twice to get the the amount of updates compared to hypothesis 1.</p>
<p>And I promise the data are all the same.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/2X/f/f7c9c2d9fb02dc02fac24d1ad004ec6ce07357b0.png" alt="image" data-base62-sha1="zm2cNA3b4AkMIOXsOX6EAifr67e" width="520" height="365"></p>

username: Oli
cooked: <p>Then the results should be the same as you suggest, at least to my understanding.</p>
<p>I would make sure that the models are the same after 1 and 2 update steps just as a sanity check. Maybe Pytorch does something funky like a copy when you set two modules to the same, I’m not sure <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>
<p><em><strong>self.layer4_1 = model_resnet.layer4</strong></em><br>
<em><strong>self.layer4_2 = model_resnet.layer4</strong></em></p>

username: yunpei
cooked: <p>OK,  still thanks for you.</p>
<p>I suggest they are same, but it is appointed that the results are different.</p>
<p>I looking for other people for help.<img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"></p>