Same inputs but get different outputs

rikonaka: <p>I found a very strange problem, I have a trained model <code>net</code> and  five test images, if I input the whole five images into the model and separately a single image into the model, I get completely different output, even the number of labels is different (four <code>1</code> and one <code>8</code> vs one <code>1</code> and four <code>8</code>).</p>
<p>What is going on here?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/c/ac15340963af3ca28fd4dc466689821d0eaa2c0b.png" data-download-href="https://discuss.pytorch.org/uploads/default/ac15340963af3ca28fd4dc466689821d0eaa2c0b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/c/ac15340963af3ca28fd4dc466689821d0eaa2c0b_2_690x340.png" alt="image" data-base62-sha1="oyjygCgCkZdVJdlt28BUCXlNrjt" width="690" height="340" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/c/ac15340963af3ca28fd4dc466689821d0eaa2c0b_2_690x340.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/c/ac15340963af3ca28fd4dc466689821d0eaa2c0b_2_1035x510.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/a/c/ac15340963af3ca28fd4dc466689821d0eaa2c0b_2_1380x680.png 2x" data-dominant-color="F0F1F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1556×768 118 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

rikonaka: <p>It’s really wired…</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/7/f78e24dbdbfc3f3d2d467008b806d4b215540509.png" data-download-href="https://discuss.pytorch.org/uploads/default/f78e24dbdbfc3f3d2d467008b806d4b215540509" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/7/f78e24dbdbfc3f3d2d467008b806d4b215540509_2_690x256.png" alt="image" data-base62-sha1="zjYtADxlkZHnj16CtV3SOOo5fKh" width="690" height="256" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/7/f78e24dbdbfc3f3d2d467008b806d4b215540509_2_690x256.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/7/f78e24dbdbfc3f3d2d467008b806d4b215540509_2_1035x384.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/7/f78e24dbdbfc3f3d2d467008b806d4b215540509_2_1380x512.png 2x" data-dominant-color="F4F5F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1812×674 61.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Check if you are using batchnorm layers (or other layers which use a batch size - dependent operation), since the normalization of the input activation would change depending on the batch size.</p>
<aside class="quote no-group quote-modified" data-username="rikonaka" data-post="1" data-topic="172619">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rikonaka/48/57354_2.png" class="avatar"> rikonaka:</div>
<blockquote>
<p>What the hell is going on here?</p>
</blockquote>
</aside>
<p>Please avoid this colloquial language.</p>

rikonaka: <p>My bad, I apologize and thank you for your answer, it’s this <code>batchnorm</code> cause this problem, after I add <code>net.eval()</code> everything is OK now.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/6/36b07b758a4abc80aeaa87c3dd2e21801d87bac8.png" data-download-href="https://discuss.pytorch.org/uploads/default/36b07b758a4abc80aeaa87c3dd2e21801d87bac8" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/36b07b758a4abc80aeaa87c3dd2e21801d87bac8_2_690x361.png" alt="image" data-base62-sha1="7NNTu1trCfhs8Hb1bAJW35QNtag" width="690" height="361" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/36b07b758a4abc80aeaa87c3dd2e21801d87bac8_2_690x361.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/36b07b758a4abc80aeaa87c3dd2e21801d87bac8_2_1035x541.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/6/36b07b758a4abc80aeaa87c3dd2e21801d87bac8_2_1380x722.png 2x" data-dominant-color="EFF1F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1494×782 120 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>