Inconsistent Results on torchvision pretrained models using Python script vs C++ API

username: pejibaye
cooked: <p>I have tried to benchmark and compare a Python script, using a pretrained classification model from the torchvision library with the C++ API implementation in order to make sure it will work with my application.<br>
I have tried the following script in Python:</p>
<pre><code class="lang-auto">import torch
from torchvision import transforms
from PIL import Image


_IMAGE_FILENAME = "../images/goldfish.jpg"
_MODEL_JIT_FILENAME = "../jit_models_bin/traced_mnasnet0_5.pt"

model = torch.jit.load(_MODEL_JIT_FILENAME)

tfm = transforms.Compose([ 
    transforms.Resize([ 224, 224 ]), 
    transforms.ToTensor(), 
    transforms.Normalize( mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] ) 
    ])

image = Image.open(_IMAGE_FILENAME)
image = tfm(image)
image = image.unsqueeze(dim=0)


output = model(image)
output = torch.softmax(output, 1)

prob_value, index = torch.max(output, 1)

print("Probability Value: ")
print(prob_value)
print("ImageNet Index: ")
print(index)
</code></pre>
<p>And I used the following code in C++:</p>
<pre><code class="lang-auto">#include &lt;iostream&gt;
#include &lt;vector&gt;
#include &lt;string&gt;

#include &lt;torch/torch.h&gt;
#include &lt;torch/script.h&gt;

#include &lt;opencv2/core.hpp&gt;
#include &lt;opencv2/imgcodecs.hpp&gt;
#include &lt;opencv2/highgui/highgui.hpp&gt;
#include &lt;opencv2/imgproc.hpp&gt;


int main() {

    const cv::String _IMAGE_FILENAME = "../images/goldfish.jpg";
    const std::string _MODEL_JIT_FILENAME= "../jit_models_bin/traced_mnasnet0_5.pt";

    cv::Mat img = cv::imread( _IMAGE_FILENAME, cv::IMREAD_UNCHANGED );
    cv::Size rsz = { 224, 224 };

    cv::resize( img, img, rsz, 0, 0, cv::INTER_LINEAR );
    img.convertTo( img, CV_32FC3, 1/255.0 );

    at::Tensor tensorImage = torch::from_blob(img.data, { 1, img.rows, img.cols, 3 }, at::kFloat);
    tensorImage = tensorImage.permute({0, 3, 1, 2});

    //  Normalize data
    tensorImage[0][0] = tensorImage[0][0].sub(0.485).div(0.229);
    tensorImage[0][1] = tensorImage[0][1].sub(0.456).div(0.224);
    tensorImage[0][2] = tensorImage[0][2].sub(0.406).div(0.225);

    std::vector&lt;torch::jit::IValue&gt; input;
    input.push_back(tensorImage);

    torch::jit::script::Module model = torch::jit::load( _MODEL_JIT_FILENAME );

    at::Tensor output = torch::softmax(model.forward(input).toTensor(), 1);

    std::tuple&lt;at::Tensor, at::Tensor&gt; result = torch::max(output, 1);

    std::cout &lt;&lt; "Probability Value: " &lt;&lt; std::endl;
    std::cout &lt;&lt; std::get&lt;0&gt;(result) &lt;&lt; std::endl;
    std::cout &lt;&lt; "ImageNet Index" &lt;&lt; std::endl;
    std::cout &lt;&lt; std::get&lt;1&gt;(result) &lt;&lt; std::endl;

    return 0;
}

</code></pre>
<p>The results I get in the Python script:<br>
Probability: 0.99, Index: 1, Label: GoldFish</p>
<p>The results I get in C++:<br>
Probability: 0.4839, Index: 584</p>
<p>I used PyTorch 1.3 version in Python 3.7 and the latest corresponding libtorch.</p>
<p>Any ideas?</p>

username: tom
cooked: <p>Maybe it’s BGR in opencv vs RGB in PyTorch?<br>
I would recommend to print the first, say 3x3 pixels to make sure you pass in the exact same thing.</p>
<p>Best regards</p>
<p>Thomas</p>

username: pejibaye
cooked: <p>Hej Thomas,</p>
<p>Thanks for the answer. I have identified the issue and I have close it. You can view the complete answer here:<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/29194" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/29194" target="_blank" rel="nofollow noopener">Inconsistent Results on torchvision pretrained models using Python script vs C++ API</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-11-05" data-time="12:24:39" data-timezone="UTC">12:24PM - 05 Nov 19 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2019-11-06" data-time="11:16:53" data-timezone="UTC">11:16AM - 06 Nov 19 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/Con-Mi" target="_blank" rel="nofollow noopener">
          <img alt="Con-Mi" src="https://avatars0.githubusercontent.com/u/10553341?v=4" class="onebox-avatar-inline" width="20" height="20">
          Con-Mi
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">I have tried to benchmark and compare a Python script, using a pretrained classification model from the torchvision library with the...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: cpp</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>It is true that is BGR even if I use  cv::IMREAD_UNCHANGED</p>