Deleting an unused part of a module changes the output

ASMIftekhar: <p>In a fairly large model, I have a few linear layers which I was not using, so I commented them out and surprisingly my Average Precision for first epoch moves from 34 to 33. My seeds are fixed so I know what to expect in epoch if I remove the comments then it will again predict Average Precision Score as 34.<br>
Is there any reason for that. I have added a toy example below:</p>
<pre><code class="lang-auto">First Example:
class test(nn.Module):
    def __init__(self):
      super(test,self).__init__()
      self.lin_unused=nn.Sequential(nn.Linear(1024, 1024),nn.ReLU(),)
      self.lin_used=nn.Sequential(nn.Linear(1024, 512),nn.ReLU(),)
    def  forward(input):
      output=self.lin_used(input)
Second Example:
class test(nn.Module):
    def __init__(self):
      super(test,self).__init__()
      #self.lin_unused=nn.Sequential(nn.Linear(1024, 1024),nn.ReLU(),)
      self.lin_used=nn.Sequential(nn.Linear(1024, 512),nn.ReLU(),)
    def  forward(input):
      output=self.lin_used(input)         

</code></pre>
<p>AP for First Example : 34,<br>
AP for Second example:33</p>

KFrank: <p>Hi Asm!</p>
<aside class="quote no-group quote-modified" data-username="ASMIftekhar" data-post="1" data-topic="76390" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/asmiftekhar/48/11500_2.png" class="avatar"> ASMIftekhar:</div>
<blockquote>
<p>I have a few linear layers which I was not using, so I commented them out and surprisingly my Average Precision for first epoch moves from 34 to 33.</p>
<pre><code class="lang-auto">      self.lin_unused=nn.Sequential(nn.Linear(1024, 1024),nn.ReLU(),)
      self.lin_used=nn.Sequential(nn.Linear(1024, 512),nn.ReLU(),)
</code></pre>
</blockquote>
</aside>
<p>By default, <code>nn.Linear</code> initializes itself with some pseudorandom<br>
numbers.  Thus, <code>lin_unused</code> consumes some pseudorandom<br>
numbers and the pseudorandom numbers that <code>lin_used</code> uses<br>
for its initialization will  differ, depending on whether <code>lin_unused</code><br>
is there or not.</p>
<p>You can check this in a number of ways:</p>
<p>First, print out some of the weights in <code>lin_used</code> and see that<br>
they differ in the two cases.</p>
<p>Second, call <code>torch.manual_seed</code> with a fixed, known seed<br>
immediately before constructing <code>lin_used</code> and check that<br>
your results are the same whether or not you comment out<br>
<code>lin_unused</code>.</p>
<p>Third, run your whole script a bunch of times (constructing and<br>
running your model from scratch each time), passing a different<br>
seed to  <code>torch.manual_seed</code> at the beginning of your script<br>
each time, and verify that the results of your two versions<br>
(with and without <code>lin_unused</code>) are <em>statistically</em> the same,<br>
even though individual runs will differ from one another in<br>
detail, due to statistical variation.</p>
<p>Best.</p>
<p>K. Frank</p>

ASMIftekhar: <p>Thats an excellent answer. Thanks a lot for clearing that up</p>