Python && C++ result consistency

elduderino15: <p>I am running a Python3 trained, torch_script exported CNN with C++.<br>
However, I can not reproduce results on testing data.</p>
<p>The class labels the classifier returns in respective language are opposite for the identical data patch. Not sure if some memory is corrupted in the C++ implementation, but I was able to validate at least the Python implementation gives correct results. How else shall I debug? Right now I am trying to compare the weight matrices of the conv kernels to validate the model is consistent among both implementations. Note - patch stats are identical between both languages, could be that the memory is misaligned in C++?</p>
<p>Anything else:</p>
<p>Python export script:</p>
<pre><code class="lang-auto">    cnn = CNN1()

    cnn.load_state_dict(torch.load(args.cnn))

    torch_script_nn = torch.jit.script(cnn)

    script_file_name = args.cnn.replace('.torch', '.pytorch_script')
    torch_script_nn.save(script_file_name)

    print('{} CONVERTED TO {} '.format(args.cnn, script_file_name))
</code></pre>
<p>Python test script</p>
<pre><code class="lang-auto">    cnn = CNN1()
    cnn.load_state_dict(torch.load(cnn_file))

    i = 100
    j = 100
    patch_size = 14
    img = np.load(f)['arr_0']
    # plt.imshow(img, cmap='gray');plt.show()

    patch = img[i: i+ patch_size, j:j+patch_size]
    print('Patch max : {}, min {}'.format(np.max(patch), np.min(patch)))

    plt.imshow(patch, cmap='gray');plt.show()
    patch = np.expand_dims(patch, axis=2)
    normalized_patch = (patch - np.min(patch)) / np.max(patch - np.min(patch))
    print('Normalized Patch max : {}, min {}'.format(np.max(normalized_patch), np.min(normalized_patch)))
    torch_stack = torch.stack([torch.Tensor(normalized_patch)])
    print(torch_stack.shape)
    res = cnn(torch_stack.permute(0, 3, 1, 2))

    print('Class label {}'.format(res))
</code></pre>
<p>Prints</p>
<pre><code class="lang-auto">Patch max : 16.035221099853516, min 11.432785987854004
Normalized Patch max : 1.0, min 0.0
torch.Size([1, 14, 14, 1])
tensor([[9.9995e-01, 5.4011e-05]], grad_fn=&lt;SoftmaxBackward&gt;)
</code></pre>
<p>C++ test</p>
<pre><code class="lang-auto">std::cout &lt;&lt; "TESTING CNN" &lt;&lt; std::endl;
    std::size_t i = 100;
    std::size_t j = 100;

    const cv::Rect roi(i, j, patchSize, patchSize);
    cv::Mat amplitudePatch = amplitude(roi);
    double min;
    double max;
    cv::minMaxLoc(amplitudePatch, &amp;min, &amp;max);
    std::cout &lt;&lt; "Patch Max : " &lt;&lt; max &lt;&lt; std::endl;
    std::cout &lt;&lt; "Patch Min " &lt;&lt; min &lt;&lt; std::endl;

    cv::imshow("Patch", amplitudePatch);
    cv::Mat normalizedPatch = normalizeAmplitudePatch(amplitudePatch);
    cv::imshow("NPatch", normalizedPatch);
    cv::minMaxLoc(normalizedPatch, &amp;min, &amp;max);
    std::cout &lt;&lt; "Normalized Patch Max : " &lt;&lt; max &lt;&lt; std::endl;
    std::cout &lt;&lt; "Normalized Patch Min " &lt;&lt; min &lt;&lt; std::endl;

    cv::waitKey(0); 
    auto options = torch::TensorOptions().dtype(torch::kF32);

    std::cout &lt;&lt; 1 &lt;&lt; " " &lt;&lt; amplitudePatch.size().height&lt;&lt; " " &lt;&lt;
        amplitudePatch.size().width&lt;&lt; " " &lt;&lt; amplitudePatch.channels() &lt;&lt; std::endl;

    torch::Tensor patchTensor = torch::from_blob(
      amplitudePatch.data, 
      {1, amplitudePatch.size().height,
        amplitudePatch.size().width, amplitudePatch.channels()},
      options);

    std::vector&lt;c10::IValue&gt; inputs;
    inputs.push_back(patchTensor.permute({0, 3, 1, 2}));
    torch::Tensor result = cnn.forward(inputs).toTensor();

    std::cout &lt;&lt; result &lt;&lt; std::endl;
</code></pre>
<p>Prints</p>
<pre><code class="lang-auto">Patch Max : 16.0352
Patch Min : 11.4328
Could not initialize OpenGL for RasterGLSurface, reverting to RasterSurface.
Could not initialize OpenGL for RasterGLSurface, reverting to RasterSurface.
Normalized Patch Max : 1
Normalized Patch Min 0
1 14 14 1
 0  1
[ Variable[CPUFloatType]{1,2} ]
</code></pre>

tom: <p>A typical mistake people make is to forget that OpenCV is BGR while TorchVision uses RGB. So you would need to use .flip on the channel dim.</p>
<p>Best regards</p>
<p>Thomas</p>

elduderino15: <p>Thanks for the suggestion but I am using a grayscale imgs<br>
<code>channel == (x,y,1)</code><br>
. Any other ideas?</p>

MSA: <p>Did you try to copy the data:</p>
<pre><code class="lang-auto">const cv::Rect roi(i, j, patchSize, patchSize);
cv::Mat amplitudePatch = amplitude(roi);
</code></pre>
<p>Add clone</p>
<pre><code class="lang-auto">cv::Mat amplitudePatch = amplitude(roi).clone();
</code></pre>

elduderino15: <p>Clone / copy the patch here indeed changes the outcome of the function to correct value. I now have to pipe more examples through it to fully validate, but looks promising… Thanks a lot! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>Any explanation for your solution? Seems the memory is messed up at some point in my logic, but where exactly? The <code>cv::show</code> piece displays consistently with python.</p>
<p>Thanks again!</p>

MSA: <p>Yep, this is how OpenCV manages data when you use the operator () to create an ROI no matrix data is copied, this is for optimization. But when you try to access amplitudePatch.data, the data ptr is not what you want because this is still the image (amplitude). When you use show or save, OpenCV correctly manages the data index.</p>
<p><a href="https://docs.opencv.org/2.4/modules/core/doc/basic_structures.html#id6" class="onebox" target="_blank" rel="nofollow noopener">https://docs.opencv.org/2.4/modules/core/doc/basic_structures.html#id6</a></p>

tom: <p>If you get the strides from OpenCV, you can use the <a href="https://pytorch.org/cppdocs/api/function_namespaceat_1a9dc7f8a613cf56b1f66c83dcae0e1c62.html" rel="nofollow noopener">strided variant of <code>from_blob</code></a>.</p>
<p>Or you could convert first and then use <code>narrow</code> to get the patch.</p>
<p>Best regards</p>
<p>Thomas</p>

elduderino15: <p>Vielen Dank für die Tips! <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>