What else is done after forward?

username: Ardeal
cooked: <p>Hi,</p>
<pre data-code-wrap="python"><code class="lang-python">	out  = self.qconv5_qbn5_qrelu5(out4) 
	out5 = self.qpool5(out)

	y = self.qinterpolate1(out5)

</code></pre>
<p>For the upper code, <code>out5</code> is correct.</p>
<p>in <code>self.qinterpolate1(out5) </code> <code>forward</code> function, the return value <code>x</code> is also correct.</p>
<pre data-code-wrap="python"><code class="lang-python">class QInterpolate(QModule):
    def __init__(self, qi=True, qo=True, num_bits=8, scale_factor=2, mode='bilinear', align_corners=True):
        super(QInterpolate, self).__init__(qi=qi, qo=qo, num_bits=num_bits)
        self.num_bits = num_bits
        self.scale_factor=scale_factor
        self.mode=mode
        self.align_corners = align_corners

    def forward(self, x):
        if hasattr(self, 'qi'):
            self.qi.update(x)
            x = FakeQuantize.apply(x, self.qi)
        x = F.interpolate(x, scale_factor=self.scale_factor, mode=self.mode, align_corners=self.align_corners)
        if hasattr(self, 'qo'):
            self.qo.update(x)
            x = FakeQuantize.apply(x, self.qo)
        return x

</code></pre>
<p>however,<br>
<code>y = self.qinterpolate1(out5)</code> in the first piece of code is not correct.</p>
<p>More details:<br>
x is:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/6/064dfa5933781b4297f63862b081aa5005d28834.png" data-download-href="https://discuss.pytorch.org/uploads/default/064dfa5933781b4297f63862b081aa5005d28834" title="73f3fd6d6b835ba90856e60e151c90f"><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/6/064dfa5933781b4297f63862b081aa5005d28834.png" alt="73f3fd6d6b835ba90856e60e151c90f" data-base62-sha1="TLVLEWPooN0BnnTYBIdMyDoFM0" width="690" height="222" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">73f3fd6d6b835ba90856e60e151c90f</span><span class="informations">750×242 9.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>while y is:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/4/1/414a493d0f78b8b801e6220fc001a604b52e1f6f.png" alt="2dfd904270dbacdd18d28ab4a562d55" data-base62-sha1="9jAbX4VkRS5jNAVmvrQCUKHOQOz" width="628" height="246"></p>
<p><strong>I cannot understand why  <code>round</code> function is applied to the returned tensor y?</strong></p>

username: yiftach
cooked: <p>Thanks for the detailed question, can you paste the data as code and not as an image though? Will make it easier to debug.</p>

username: Ardeal
cooked: <p><a class="mention" href="/u/yiftach">@yiftach</a><br>
Thank you  for your reply!</p>
<p>I am debugging the code on Windows 10 with vscode. The data you mentioned can’t be copied. I am trying to save the data to txt file and then paste to the forum.</p>
<p>however, I suddenly found a very weird issue:<br>
please see the following picture, the data in <code>y</code> is different from <code>yy</code>, but I didn’t change anything in <code>y</code>. I just sliced <code>y[0,0,:,:]</code> to make <code>yy</code></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/f/4f39cf33f9b4262e6457a63d9b4decc93fc0946b.png" data-download-href="https://discuss.pytorch.org/uploads/default/4f39cf33f9b4262e6457a63d9b4decc93fc0946b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/f/4f39cf33f9b4262e6457a63d9b4decc93fc0946b_2_690x169.png" alt="image" data-base62-sha1="biRzIVUJ4HkcPoUEQU37vqir4JJ" width="690" height="169" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/f/4f39cf33f9b4262e6457a63d9b4decc93fc0946b_2_690x169.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/f/4f39cf33f9b4262e6457a63d9b4decc93fc0946b_2_1035x253.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/f/4f39cf33f9b4262e6457a63d9b4decc93fc0946b_2_1380x338.png 2x" data-dominant-color="F0F0F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2030×499 89.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: yiftach
cooked: <p>Interesting, but it is still hard to debug your issue without a complete code example. You could just print the tensor instead of saving to a file (you will not need to alter it). Try to see if this still happens for a small tensor, and if so - print the input too, so we can have a code snippet we can run to reproduce and study the issue.</p>

username: Ardeal
cooked: <aside class="quote no-group" data-username="yiftach" data-post="4" data-topic="188632">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yiftach/48/58678_2.png" class="avatar"> yiftach:</div>
<blockquote>
<p>nsor instead of saving to a file (you will not need to alter it). Try to see if this still happens for a small tensor, and if so - print the input too, so we can have a code snippet we can run to reproduce and study the</p>
</blockquote>
</aside>
<p><a class="mention" href="/u/yiftach">@yiftach</a></p>
<p>Please check the data I printed.<br>
It is so weird!!!</p>
<p>I saw that the <code>y</code> in vscode is round-off, however, the printed <code>y</code> is correct.</p>
<pre data-code-wrap="python"><code class="lang-python">
print x
tensor([[[[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180],
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118],
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056],
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659],
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330],
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180],
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118],
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056],
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659],
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330],
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180],
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118],
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056],
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659],
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330],
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         ...,

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180],
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118],
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056],
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659],
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330],
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180],
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118],
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056],
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659],
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330],
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180],
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118],
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056],
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659],
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330],
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]]]],
       device='cuda:0', grad_fn=&lt;FakeQuantizeBackward&gt;)
print y
tensor([[[[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180], 
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118], 
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056], 
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659], 
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330], 
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180], 
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118], 
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056], 
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659], 
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330], 
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180], 
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118], 
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056], 
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659], 
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330], 
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         ...,

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180], 
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118], 
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056], 
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659], 
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330], 
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180], 
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118], 
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056], 
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659],
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330],
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]],

         [[0.0000, 0.3015, 0.6108,  ..., 1.4767, 0.9974, 0.5180],
          [0.0000, 0.1933, 0.3943,  ..., 1.7086, 1.2602, 0.8118],
          [0.0000, 0.0850, 0.1701,  ..., 1.9406, 1.5231, 1.1056],
          ...,
          [0.0000, 0.2087, 0.4175,  ..., 1.7473, 1.3066, 0.8659],
          [0.0000, 0.2629, 0.5180,  ..., 1.4226, 0.9278, 0.4330],
          [0.0000, 0.3093, 0.6185,  ..., 1.0979, 0.5489, 0.0000]]]],
       device='cuda:0', grad_fn=&lt;FakeQuantizeBackward&gt;)

</code></pre>

username: Ardeal
cooked: <p><a class="mention" href="/u/yiftach">@yiftach</a></p>
<p>Thank you for you reply!</p>
<p>This issue is fixed.<br>
The reason is not related with Pytorch. It is the reason of VScode.<br>
For any unknown reason, I cleaned all vscode cache files, and then, the issue is fixed.</p>