CNN inference - different results on each run

DrBwts: <p>I have a trained PyTorch CNN model that is attempting to classify 7 different blocks. When I run inference on the same batch of images each run gives different results, for example,</p>
<blockquote>
<h1>=====================<br>
Run 1<br>
pred    truth   match</h1>
<h1>block_1 block_0 no<br>
block_1 block_1 yes<br>
block_1 block_1 yes<br>
block_1 block_2 no<br>
block_1 block_2 no<br>
block_1 block_2 no<br>
block_1 block_3 no<br>
block_6 block_3 no<br>
block_6 block_3 no<br>
block_1 block_4 no<br>
block_1 block_4 no<br>
block_1 block_5 no<br>
block_1 block_5 no<br>
block_1 block_6 no<br>
block_1 block_6 no</h1>
<h1>Run 2<br>
pred    truth   match</h1>
<h1>block_3 block_0 no<br>
block_3 block_1 no<br>
block_3 block_1 no<br>
block_3 block_2 no<br>
block_3 block_2 no<br>
block_6 block_2 no<br>
block_3 block_3 yes<br>
block_3 block_3 yes<br>
block_3 block_3 yes<br>
block_3 block_4 no<br>
block_3 block_4 no<br>
block_3 block_5 no<br>
block_3 block_5 no<br>
block_3 block_6 no<br>
block_3 block_6 no</h1>
</blockquote>
<p>My model loader fucntion,</p>
<pre><code class="lang-auto">def load_model(model, model_dir='models', model_file_name='blocks.pt'):

    model_path = os.path.join(model_dir, model_file_name)
    model.load_state_dict(torch.load(model_path), strict=False)

    return model
</code></pre>
<p>My prediction fucntion,</p>
<pre><code class="lang-auto">def prediction(model, device, batch_input):

    model.to(device)
    model.eval()

    data = batch_input.to(device)
    output = model(data)

    prob = F.softmax(output, dim=1)
    pred_prob = prob.data.max(dim=1)[0]
    pred_index = prob.data.max(dim=1)[1]

    return pred_index.cpu().numpy(), pred_prob.cpu().numpy()
</code></pre>
<p>My image transforms to match image training set dimensions etc,</p>
<pre><code class="lang-auto">def image_common_transforms(mean=(0.4515, 0.3976, 0.3339), std=(0.3639, 0.3361, 0.3224)):

    common_transforms = transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize(mean, std)
    ])

    return common_transforms
</code></pre>
<p>The input batches are identical in each case. I was under the impression that <code>model.eval()</code> would stop any random behaviour. Am I misunderstanding something here?</p>

ptrblck: <p>Check the absolute differences in the model output as well as intermediate activations to narrow down where the differences are coming from.</p>

DrBwts: <p>HI - could you advise how to do that?</p>

ptrblck: <p>You could use forward hooks or directly store the activations in the <code>forward</code> method in e.g. a <code>list</code> from different runs and compare them later.</p>

DrBwts: <p>Thanks the forward hooks thing is a great tool. I found my error, I was missing an activation layer when running the model for inference.</p>

ptrblck: <p>This wouldn’t explain why rerunning the same model (with the missing activation layer) with the same input yields different results, would it?</p>

DrBwts: <p>I trained the model on a different computer with a different code. I wrote the inference code on different computer &amp; forgot to add an activation layer after one of the fully connected layers. Now everything is working as expected.</p>