TorchScript traced model returns inconsistent output tensors on each run

maxmx911: <p>I have a traced model that works correctly on Python and C++. However, when i try loading it in Android, I’m getting inconsistent output tensors on every run. I did call <strong>model.eval()</strong> before tracing the model, plus the same .pt file is working as intended on Python and C++.</p>
<p>I tried loading <a href="https://pytorch.org/docs/stable/torchvision/models.html" rel="nofollow noopener">these pretrained models</a> in Android and they are returning consistent output tensors on each run though. I would appreciate if anyone could provide me some insights on why my traced model is having such inconsistent output on every run. <img src="https://discuss.pytorch.org/images/emoji/apple/bowing_man.png?v=9" title=":bowing_man:" class="emoji" alt=":bowing_man:"></p>
<p>Considering <a class="mention" href="/u/david_reiss">@David_Reiss</a>’s reply in this <a href="https://discuss.pytorch.org/t/pytorch-mobile-ios-resnet50-not-computing/61014/11">issue</a>, could this behavior be a bug?</p>
<p>Here’s the layers to my traced model</p>
<pre><code class="lang-auto">Backbone(
  original_name=Backbone
  (input_layer): Sequential(
    original_name=Sequential
    (0): Conv2d(original_name=Conv2d)
    (1): BatchNorm2d(original_name=BatchNorm2d)
    (2): PReLU(original_name=PReLU)
  )
  (output_layer): Sequential(
    original_name=Sequential
    (0): BatchNorm2d(original_name=BatchNorm2d)
    (1): Dropout(original_name=Dropout)
    (2): Flatten(original_name=Flatten)
    (3): Linear(original_name=Linear)
    (4): BatchNorm1d(original_name=BatchNorm1d)
  )
  (body): Sequential(
    original_name=Sequential
    (0): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (1): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (2): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (3): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): Sequential(
        original_name=Sequential
        (0): Conv2d(original_name=Conv2d)
        (1): BatchNorm2d(original_name=BatchNorm2d)
      )
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (4): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (5): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (6): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (7): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): Sequential(
        original_name=Sequential
        (0): Conv2d(original_name=Conv2d)
        (1): BatchNorm2d(original_name=BatchNorm2d)
      )
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (8): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (9): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (10): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (11): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (12): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (13): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (14): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (15): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (16): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (17): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (18): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (19): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (20): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (21): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): Sequential(
        original_name=Sequential
        (0): Conv2d(original_name=Conv2d)
        (1): BatchNorm2d(original_name=BatchNorm2d)
      )
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (22): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
    (23): bottleneck_IR_SE(
      original_name=bottleneck_IR_SE
      (shortcut_layer): MaxPool2d(original_name=MaxPool2d)
      (res_layer): Sequential(
        original_name=Sequential
        (0): BatchNorm2d(original_name=BatchNorm2d)
        (1): Conv2d(original_name=Conv2d)
        (2): PReLU(original_name=PReLU)
        (3): Conv2d(original_name=Conv2d)
        (4): BatchNorm2d(original_name=BatchNorm2d)
        (5): SEModule(
          original_name=SEModule
          (avg_pool): AdaptiveAvgPool2d(original_name=AdaptiveAvgPool2d)
          (fc1): Conv2d(original_name=Conv2d)
          (relu): ReLU(original_name=ReLU)
          (fc2): Conv2d(original_name=Conv2d)
          (sigmoid): Sigmoid(original_name=Sigmoid)
        )
      )
    )
  )
</code></pre>