Predict same image but get different probability

ljlkxj: <p>Today, I find one strange thing that if I put one or many images into the model, but when check the outputs in different layers of the same images, the values of outputs are different.</p>
<ol>
<li>The variable images contains 100 pictures, which have been normalized. Then the resnet18 model made a prediction, and I print the output of last layer. We can see the values of the last picture as follows:<br>
tensor([[ 0.3845428824, -0.5180321932, 0.2798461318,-0.5726045370,0.1901987493, -0.2272544652, -0.8677281737, 0.4837094247,-0.0999206007,-0.1482645720]])</li>
<li>I only chose the last picture as input, and we can the the output is different from the values above<br>
tensor([[ 0.3845427036, -0.5180319548,  0.2798460722,-0.5726044774,0.1901988089, -0.2272544056, -0.8677282333, 0.4837094545,-0.0999205410,-0.1482645869]])</li>
<li>I duplicate the last picture 100 times as input, and find the result is the same as result of 1).<br>
[ 0.3845428824, -0.5180321932, 0.2798461318,-0.5726045370,0.1901987493,-0.2272544652,-0.8677281737, 0.4837094247,-0.0999206007,-0.1482645720]</li>
<li>Put the last three pictures into the net, the output is:<br>
[0.3845428526,-0.5180321336,0.2798461318,-0.5726043582,0.1901988238,-0.2272545248,-0.8677282333,0.4837094247,-0.0999206007,-0.1482645124]<br>
It seems that the size of input is he key reason of the output difference.  I can’t understand this problem, espicially when I do the same things on tensorflow, there are no difference, in the condition of same batch_size.<br>
Why it happens? Please help me to find out which caused this on pytorch, thank you!</li>
</ol>

ptrblck: <p>Make sure to call <code>model.eval()</code> before running your test.<br>
This will make sure to disable dropout layers and use the running estimates of batchnorm layers.<br>
During training, dropout will of course alter your output and batchnorm layers will normalize the input batch using the current batch statistics and will also update the running estimates.</p>
<p>I’m not sure how TensorFlow handles it.</p>

ljlkxj: <p>As you suggest, I add model.eval() and then run it, the condition changes but still get different probability. But this time, 2 and 3 become same:  [-3.5615596771, -3.3041067123,  0.2262650430, -0.5413500071,<br>
5.4653391838,  3.9844801426, -4.7832412720, 14.4310092926,<br>
-6.6198024750, -6.1641607285].<br>
while 1: [-3.5615587234, -3.3041062355,  0.2262653410, -0.5413498878,<br>
5.4653387070,  3.9844801426, -4.7832412720, 14.4310083389,<br>
-6.6198034286, -6.1641597748]<br>
4 : [-3.5615592003, -3.3041067123,  0.2262655199, -0.5413499475,<br>
5.4653387070,  3.9844801426, -4.7832412720, 14.4310073853,<br>
-6.6198034286, -6.1641602516]]<br>
1 and 2 and 4 are different, which means that the data in same batch influence the output.<br>
Are there some mechanism in pytorch which bring the small changes? Thank you again!</p>

ptrblck: <p>It depends on the layers you are using in your model as well as e.g. if you set <code>cudnn</code> to use deterministic algorithms as described <a href="https://pytorch.org/docs/stable/notes/randomness.html">here</a>.<br>
If the absolute difference is in the range ~1e-6, it’ll be most likely due to the limited floating point precision.</p>