Same model, data, and code, different hardware and results

username: fnak
cooked: <p>I have a pth model that I have trained and saved.</p>
<p>I load the model and run it on a test dataset with the following code:</p>
<pre><code class="lang-auto">for i, (image, label) in enumerate(dataloader):
      with torch.no_grad():
          output, min_distances = model(input)
</code></pre>
<p>On one machine, the model works perfectly, getting around 97% correct predictions.<br>
I copied the model, code, and data to a raspberry pi. Now all the model predictions are from a single class. (first class of the dataset).</p>
<p>No changes were done to the code.<br>
Example output:</p>
<pre><code class="lang-auto">Predicted  ActualClass
tensor([0]) tensor([0])
tensor([0]) tensor([0])
tensor([0]) tensor([0])
tensor([0]) tensor([0])
tensor([0]) tensor([0])
tensor([0]) tensor([0])
tensor([0]) tensor([0])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([1])
tensor([0]) tensor([2])
tensor([0]) tensor([2])
tensor([0]) tensor([2])
tensor([0]) tensor([2])
tensor([0]) tensor([2])
</code></pre>
<p>Any ideas why this might be happening?</p>

username: fnak
cooked: <p>Looks like this is a batch size problem. When setting batch size to 1, this problem occurs.<br>
Is there a way to make predictions indifferent of batch size? or should it always be the same as training value?</p>

username: ptrblck
cooked: <p>The majority of layers are not depending on the batch size besides e.g. normalization layers, which calculate the batch stats and normalize the input activation with them during training. If you are using e.g. batchnorm layers, call <code>model.eval()</code> before evaluating your model to use the running stats instead of the batch stats.</p>