Different Outputs in Python and Torchscript C++

solarflarefx: <p>Hello,</p>
<p>I was experimenting with Torchscript and I seem to be getting different outputs in my Python code versus my C++ code.  I am simply loading the ResNet18 model, and loading a sample image: <a href="https://raw.githubusercontent.com/pytorch/hub/master/dog.jpg" rel="noopener nofollow ugc">https://raw.githubusercontent.com/pytorch/hub/master/dog.jpg</a></p>
<p>I exported the resnet18 model following these instructions: <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" class="inline-onebox" rel="noopener nofollow ugc">Loading a TorchScript Model in C++ — PyTorch Tutorials 2.2.0+cu121 documentation</a></p>
<p>I am running on Windows 10 64-bit.  For the C++ environment, I am running on Microsoft Visual Studio 2019.  I am running the debug build of libtorch 1.4.0 for Visual Studio.  I believe this is running fine, as I am able to load models and tensors to put CPU and GPU.</p>
<p>I believe I have an issue with my C++ code, but I am not sure where I have the error.  Below is my code along with the outputs I get.</p>
<p>Python Code:</p>
<pre><code class="lang-auto">import torch
import torchvision.models as models
import urllib.request
from PIL import Image
from torchvision import transforms

def norm_chan(chan, mean, std):
    b = (chan - mean) / std
    return b

# Load resnet18 model
model = models.resnet18(pretrained = True)

# Download an example image from the pytorch website
url, filename = ("https://github.com/pytorch/hub/raw/master/dog.jpg", "dog.jpg")
urllib.request.urlretrieve(url, filename)

# sample execution (requires torchvision)
input_image = Image.open(filename)
preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

input_tensor = preprocess(input_image)

import numpy as np
input_tensor_numpy = input_tensor.numpy()
input_batch = input_tensor.unsqueeze(0) # create a mini-batch as expected by the model


# move the input and model to GPU for speed if available
if torch.cuda.is_available():
    input_batch = input_batch.to('cuda')
    model.to('cuda')

with torch.no_grad():
    output = model(input_batch)

import numpy as np
print()
output_cpu = output[0].cpu()
output_numpy = output_cpu.numpy()
print(output_numpy[0:9])
</code></pre>
<p>C++ code:</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt;
#include &lt;torch/torch.h&gt;
#include &lt;ATen/Tensor.h&gt;
#include &lt;opencv2/opencv.hpp&gt;
#include &lt;opencv2/imgproc/imgproc.hpp&gt;
#include &lt;opencv2/core/core.hpp&gt;
#include &lt;opencv2/highgui/highgui.hpp&gt;
#include &lt;iostream&gt;
#include &lt;memory&gt;

int main(int argc, const char* argv[]) {


    // Load up GPU stuff
    torch::DeviceType device_type;
    if (torch::cuda::is_available()) {
        std::cout &lt;&lt; "CUDA available! Training on GPU." &lt;&lt; std::endl;
        device_type = torch::kCUDA;
    }
    else {
        std::cout &lt;&lt; "Training on CPU." &lt;&lt; std::endl;
        device_type = torch::kCPU;
    }
    torch::Device device(device_type);

    torch::jit::script::Module module;
    std::cout &lt;&lt; "Attempting to load resnet model.." &lt;&lt; std::endl;
    try {
        // Deserialize the ScriptModule from a file using torch::jit::load().
        module = torch::jit::load("C:\\PyTorchPictureTest\\Model\\traced_resnet_model.pt");
        std::cout &lt;&lt; "Successfully loaded resnet model" &lt;&lt; std::endl;

        module.to(at::kCUDA);
        std::cout &lt;&lt; "Moved model to gpu" &lt;&lt; std::endl;

        // load image and transform
        cv::Mat image;
        image = cv::imread("C:\\PyTorchPictureTest\\dog.jpg", 1);

        cv::cvtColor(image, image, 4); //had to change this part of the code because the cvtColor Enums did not seem to be working
        cv::Mat img_float;
        image.convertTo(img_float, CV_32F, 1.0 / 255);
        cv::resize(img_float, img_float, cv::Size(224, 224));

        auto img_tensor = torch::from_blob(img_float.data, { 1, 224, 224, 3 }).to(torch::kCUDA);
        img_tensor = img_tensor.permute({ 0, 3, 1, 2 });
        img_tensor[0][0] = img_tensor[0][0].sub(0.485).div(0.229);
        img_tensor[0][1] = img_tensor[0][1].sub(0.456).div(0.224);
        img_tensor[0][2] = img_tensor[0][2].sub(0.406).div(0.225);
        auto img_var = torch::autograd::make_variable(img_tensor, false);

        std::vector&lt;torch::jit::IValue&gt; inputs;
        inputs.push_back(img_var);
        torch::Tensor out_tensor = module.forward(inputs).toTensor();
        std::cout &lt;&lt; out_tensor.slice(1, 0, 10) &lt;&lt; '\n';
    }
    catch (const c10::Error &amp; e) {
        std::cerr &lt;&lt; "error loading the model\n";
        return -1;
    }

    std::cout &lt;&lt; "ok\n";
}
</code></pre>
<p>Output in Python:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/4/84d20f98eaf61eb5851b393cbe1e6ddd744646e4.png" data-download-href="https://discuss.pytorch.org/uploads/default/84d20f98eaf61eb5851b393cbe1e6ddd744646e4" title="PythonOutput"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84d20f98eaf61eb5851b393cbe1e6ddd744646e4_2_690x356.png" alt="PythonOutput" data-base62-sha1="iWZ4VPmrvuplUVvtL8zaRtWdeII" width="690" height="356" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84d20f98eaf61eb5851b393cbe1e6ddd744646e4_2_690x356.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84d20f98eaf61eb5851b393cbe1e6ddd744646e4_2_1035x534.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/4/84d20f98eaf61eb5851b393cbe1e6ddd744646e4_2_1380x712.png 2x" data-dominant-color="111111"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">PythonOutput</span><span class="informations">2333×1205 25.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Output from C++:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/a/0a52bbe542b125ee3e3b97e96e9bb00f962710af.png" data-download-href="https://discuss.pytorch.org/uploads/default/0a52bbe542b125ee3e3b97e96e9bb00f962710af" title="CppOutput"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0a52bbe542b125ee3e3b97e96e9bb00f962710af_2_690x112.png" alt="CppOutput" data-base62-sha1="1tk1WKbnaUT9pqExFvFXrn6TpOD" width="690" height="112" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0a52bbe542b125ee3e3b97e96e9bb00f962710af_2_690x112.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0a52bbe542b125ee3e3b97e96e9bb00f962710af_2_1035x168.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0a52bbe542b125ee3e3b97e96e9bb00f962710af_2_1380x224.png 2x" data-dominant-color="3A3A3A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">CppOutput</span><span class="informations">2324×378 23.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I would appreciate any insight on this.  Thanks in advance.</p>

Keunhoi_An: <p>Maybe you’re suffering the same problem as me.</p><aside class="quote quote-modified" data-post="1" data-topic="87023">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/keunhoi_an/48/12077_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/why-libtorch-cannot-get-running-mean-and-running-var-of-batchnormalization2d-in-pt-file/87023">Why libtorch cannot get 'running_mean' and 'running_var' of BatchNormalization2D in .pt file?</a> <a class="badge-category__wrapper " href="/c/c/11"><span data-category-id="11" data-drop-close="true" class="badge-category " title="Topics related to the C++ Frontend, C++ API or C++ Extensions"><span class="badge-category__name">C++</span></span></a>
  </div>
  <blockquote>
    I’ve found that the result of libtorch is very different from pytorch result. 
I save my model as .pt file using pytorch and load by libtorch, and all parameters are successfully copied which I double checked by loading it by pytorch again. 
After checking the module.parameter() in libtorch, I’ve found that all of BatchNormalization2D modules of libtorch do not contain running_mean and running_var. And also, after the first conv layer,  all of parameters are not the same as original value. 
I’m …
  </blockquote>
</aside>