Random non deterministic results from pretrained retinanet

username: mfoglio
cooked: <p>I wrote the following class to perform instance segmentation and return the masks of a given class.<br>
The code seems to be running randomly and it’s not deterministic.<br>
The labels printed (as well as the number of labels) change at every execution even if I am running the code on the same input image containing a single person.<br>
Is there a problem in how I load the weights? The code is not printing any warning nor exception.<br>
Note that I am running the code on the CPU.</p>
<pre><code class="lang-auto">import numpy as np
import torch
from torch import Tensor
from torchvision.models.detection import retinanet_resnet50_fpn_v2, RetinaNet_ResNet50_FPN_V2_Weights
import torchvision.transforms as T
import PIL
from PIL import Image


class RetinaNet:

    def __init__(self, weights: RetinaNet_ResNet50_FPN_V2_Weights = RetinaNet_ResNet50_FPN_V2_Weights.COCO_V1):
        # Load the pre-trained DeepLabV3 model
        self.weights = weights
        self.model = retinanet_resnet50_fpn_v2(
            pretrained=RetinaNet_ResNet50_FPN_V2_Weights
        )
        self.model.eval()
        # Check if a GPU is available and if not, use a CPU
        self.device = 'cuda' if torch.cuda.is_available() else 'cpu'
        self.model.to(self.device)
        # Define the transformation
        self.transform = T.Compose([
            T.ToTensor(),
        ])

    def infer_on_image(self, image: PIL.Image.Image, label: str) -&gt; Tensor:
        # Transform image
        input_tensor = self.transform(image)
        input_tensor = input_tensor.unsqueeze(0)
        input_tensor.to(self.device)
        # Run model
        with torch.no_grad():
            predictions = self.model(input_tensor)
        # Post-processing to create masks for requested label
        label_index = self.get_label_index(label)
        boxes = predictions[0]['boxes'][predictions[0]['labels'] == label_index]
        print('labels', predictions[0]['labels'])    # random output
        masks = torch.zeros((len(boxes), input_tensor.shape[1], input_tensor.shape[2]), dtype=torch.uint8)
        for i, box in enumerate(boxes.cpu().numpy()):
            x1, y1, x2, y2 = map(int, box)
            masks[i, y1:y2, x1:x2] = 1
        return masks

    def get_label_index(self,label: str) -&gt; int:
        return self.weights.value.meta['categories'].index(label)

    def get_label(self, label_index: int) -&gt; str:
        return self.weights.value.meta['categories'][label_index]

    @staticmethod
    def load_image(file_path: str) -&gt; PIL.Image.Image:
        return Image.open(file_path).convert("RGB")



if __name__ == '__main__':
    from matplotlib import pyplot as plt

    image_path = 'person.jpg'
    # Run inference
    retinanet = RetinaNet()
    masks = retinanet.infer_on_image(
        image=retinanet.load_image(image_path),
        label='person'
    )
    # Plot image
    plt.imshow(retinanet.load_image(image_path))
    plt.show()
    # PLot mask
    for i, mask in enumerate(masks):
        mask = mask.unsqueeze(2)
        plt.title(f'mask {i}')
        plt.imshow(mask)
        plt.show()
</code></pre>

username: ptrblck
cooked: <p>I cannot reproduce the issue and get the same outputs using one image from the tutorial: <a href="https://pytorch.org/tutorials/_static/img/tv_tutorial/tv_image05.png">https://pytorch.org/tutorials/_static/img/tv_tutorial/tv_image05.png</a></p>
<p>Output:</p>
<pre data-code-wrap="python"><code class="lang-python">labels tensor([ 1,  2, 27,  1,  1,  2,  1,  1,  1,  1,  1,  2,  1,  2,  1, 27, 27,  1,
         1,  1, 27,  2,  1, 31, 27,  1, 27, 27, 27,  1, 31,  1,  1, 27,  1, 31,
         1,  2,  1, 31,  1,  1,  1, 27, 27, 31,  1,  2, 31,  1, 27, 31,  2, 32,
         1,  1,  1,  1,  2, 64,  1,  2,  1,  1,  1,  1,  1,  1, 31, 64,  1, 27,
        31, 31,  2, 27, 27,  2,  1,  1,  1,  2,  2, 27,  1,  1, 31, 27,  2,  1,
        31, 31, 32, 64, 31,  1, 62,  4, 32, 27, 31,  2,  4, 27, 77, 31,  1,  3,
         1, 22,  2,  1,  1, 32, 62, 27, 19,  1, 27,  2,  1, 31,  1,  1, 27, 32,
        32, 27,  1, 31, 13, 31, 31,  2,  2, 31, 27, 27,  1,  1,  4,  1,  1,  1,
        31, 85, 31,  1,  1, 77,  1,  2,  1, 31, 75, 32,  4, 27,  2, 77,  1, 33,
        10,  4,  1, 19, 31, 27, 64,  1,  1, 31,  2, 22, 27,  2, 19,  1, 31, 62,
         2,  2,  1,  4,  2,  1,  2,  2,  7,  1,  7,  1, 31,  1, 64, 15,  1,  4,
         8,  2,  1,  3,  2,  4,  2,  8,  7, 31, 64,  2,  2,  1, 15, 31,  4,  7,
        64,  6, 62, 28,  1,  6, 28, 27, 19,  6, 88, 28, 19,  1,  1,  4, 15, 18,
        15,  1,  1, 15, 43, 31,  8, 19,  1])
</code></pre>
<p>is constant in each rerun.</p>

username: Beginwithtorch
cooked: <aside class="quote no-group" data-username="mfoglio" data-post="1" data-topic="192769">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mfoglio/48/35116_2.png" class="avatar"> mfoglio:</div>
<blockquote>
<p>following class to perform instance segmentation and return the masks of a given class.<br>
The code seems to be running randomly and it’s not deterministic.</p>
</blockquote>
</aside>
<p>for reproducable results you can work with seeds<br>
<a href="https://pytorch.org/docs/stable/notes/randomness.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/notes/randomness.html</a></p>

username: ptrblck
cooked: <p>I wouldn’t expect to see any difference by seeding the code if the model is already in <code>.eval()</code> mode and the same input is used.</p>