Getting the random output in C++

username: Zain_Ali
cooked: <p>I am getting the random out put on the same image using this code in C++.<br>
Python:</p>
<pre><code class="lang-auto">import torch
import torchvision
import torch.nn as nn

model = torchvision.models.resnet18()

# An example input you would normally provide to your model's forward() method.
model.cuda()
example = torch.rand(1, 3, 224, 224).cuda()

# Use torch.jit.trace to generate a torch.jit.ScriptModule via tracing.
with torch.no_grad():
    traced_script_module = torch.jit.trace(model, example)
traced_script_module.save("resnet18.pt")
</code></pre>
<p>C++</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt;               // One-stop header.
#include &lt;opencv2/opencv.hpp&gt;
#include &lt;opencv2/imgproc/imgproc.hpp&gt;

#include &lt;chrono&gt;
#include &lt;iostream&gt;
#include &lt;memory&gt;

using namespace std;
using namespace std::chrono;

int main(int argc, const char* argv[]) {
    if (argc != 3) {
        std::cerr &lt;&lt; "usage: resnet_libtorch &lt;path-to-resnet-script-module&gt; &lt;path-to-input-image&gt;\n";
        return -1;
    }
    std::cout &lt;&lt; cv::getBuildInformation &lt;&lt; endl;
    // Configuration
    int input_image_size = 224;
    int batch_size = 1;

    // Deserialize the ScriptModule from a file using torch::jit::load().
    //torch::jit::script::Module module
    std::shared_ptr&lt;torch::jit::script::Module&gt; module = torch::jit::load(argv[1]);

    // Read input image
    cv::Mat origin_image = cv::imread(argv[2], CV_LOAD_IMAGE_COLOR);

    // Preprocess image (resize, put on GPU)
    cv::Mat resized_image;
    cv::cvtColor(origin_image, resized_image, cv::COLOR_RGB2BGR);
    cv::resize(resized_image, resized_image, cv::Size(input_image_size, input_image_size));

    cv::Mat img_float;
    resized_image.convertTo(img_float, CV_32F, 1.0 / 255);

    while(true){
        std::vector&lt;at::Tensor&gt; inputs_vec;
        for (int i = 0; i &lt; 8; i++) {
        //auto img_tensor = torch::from_blob(img_float.data, {1, input_image_size, input_image_size, 3}).to(torch::kCUDA)
        auto img_tensor = torch::CPU(torch::kFloat32).tensorFromBlob(img_float.data, {1, input_image_size, input_image_size, 3});
        img_tensor = img_tensor.permute({0, 3, 1, 2});
        img_tensor[0][0] = img_tensor[0][0].sub(0.485).div(0.229);
        img_tensor[0][1] = img_tensor[0][1].sub(0.456).div(0.224);
        img_tensor[0][2] = img_tensor[0][2].sub(0.406).div(0.225);
        auto img_var = torch::autograd::make_variable(img_tensor, false);
        inputs_vec.push_back(img_var);
        }

        // Create a vector of inputs.
        std::vector&lt;torch::jit::IValue&gt; inputs;
        //for (int i = 0; i &lt; 8; i++) {
        //inputs.push_back(img_var.to(at::kCUDA));
        at::Tensor input_ = torch::cat(inputs_vec);
        inputs.push_back(input_.to(at::kCUDA));
        //inputs.push_back(torch::ones({10, 3, 224, 224}).to(at::kCUDA));
        //}

        // Execute the model and turn its output into a tensor.
        torch::Tensor output;
        auto duration = duration_cast&lt;milliseconds&gt;(std::chrono::high_resolution_clock::now()-std::chrono::high_resolution_clock::now());
            auto start = std::chrono::high_resolution_clock::now();
            output = module-&gt;forward(inputs).toTensor();
            auto end = std::chrono::high_resolution_clock::now();
            duration = duration_cast&lt;milliseconds&gt;(end - start);
            //std::cout &lt;&lt; output[0] &lt;&lt; '\n';
            for(int i =0; i&lt;output.size(0); i++){
                at::Tensor max_ind = at::argmax(output[i]);
                std::cout &lt;&lt; "class_id: " &lt;&lt; i &lt;&lt; " :" &lt;&lt;max_ind.item&lt;int&gt;() &lt;&lt; std::endl;
            }
        
        // at::Tensor max_ind = at::argmax(output);
        // std::cout &lt;&lt; "class_id: " &lt;&lt; max_ind.item&lt;int&gt;() &lt;&lt; std::endl;
        // std::cout &lt;&lt; "Time take for forward pass: " &lt;&lt; duration.count() &lt;&lt; " ms" &lt;&lt; std::endl;
        break;
    }
    std::cout &lt;&lt; "Done\n";
}
</code></pre>
<p>it gives random output as class id on the same input image</p>

username: tancl
cooked: <p>Hi,</p>
<p>Some of your codes seems to be still using obsolete commands as describe in</p>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/15426" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/15426" target="_blank" rel="nofollow noopener">libtorch API params error with outdated documentation</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2018-12-20" data-time="06:24:26" data-timezone="UTC">06:24AM - 20 Dec 18 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2019-08-22" data-time="05:55:47" data-timezone="UTC">05:55AM - 22 Aug 19 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/jinfagang" target="_blank" rel="nofollow noopener">
          <img alt="jinfagang" src="https://avatars3.githubusercontent.com/u/21303438?v=4" class="onebox-avatar-inline" width="20" height="20">
          jinfagang
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">Hi, I just installed libotorch and trying to implemented a simple classifier in torch c++, the model load succeed but failed...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: cpp</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>following codes works for CPU with ver 1.3, you might want to modified to work for CUDA. Also I’ve hard-coded the model and image for quick test, so do replace with your existing one.</p>
<pre><code class="lang-auto">		int input_image_size = 224;
		int batch_size = 1;

		// Deserialize the ScriptModule from a file using torch::jit::load().
		//torch::jit::script::Module module
		//std::shared_ptr&lt;torch::jit::script::Module&gt; module = torch::jit::load("resnet18.pt");
		torch::jit::script::Module module = torch::jit::load("resnet18.pt");

		// Read input image
		cv::Mat origin_image = cv::imread("cat.jpg", 1);

		// Preprocess image (resize, put on GPU)
		cv::Mat resized_image;
		cv::cvtColor(origin_image, resized_image, cv::COLOR_RGB2BGR);
		cv::resize(resized_image, resized_image, cv::Size(input_image_size, input_image_size));

		cv::Mat img_float;
		resized_image.convertTo(img_float, CV_32F, 1.0 / 255);

		while (true) {
			std::vector&lt;at::Tensor&gt; inputs_vec;
			for (int i = 0; i &lt; 8; i++) {
				//auto img_tensor = torch::from_blob(img_float.data, {1, input_image_size, input_image_size, 3}).to(torch::kCUDA)
				auto img_tensor = torch::from_blob(img_float.data, { 1, input_image_size, input_image_size, 3 }	, torch::kFloat);
				img_tensor = img_tensor.permute({ 0, 3, 1, 2 });
				img_tensor[0][0] = img_tensor[0][0].sub(0.485).div(0.229);
				img_tensor[0][1] = img_tensor[0][1].sub(0.456).div(0.224);
				img_tensor[0][2] = img_tensor[0][2].sub(0.406).div(0.225);
				//auto img_var = torch::autograd::make_variable(img_tensor, false);
				inputs_vec.push_back(img_tensor);
			}

			// Create a vector of inputs.
			std::vector&lt;torch::jit::IValue&gt; inputs;
			//for (int i = 0; i &lt; 8; i++) {
			//inputs.push_back(img_var.to(at::kCUDA));
			at::Tensor input_ = torch::cat(inputs_vec);
			//inputs.push_back(input_.to(at::kCUDA));
			inputs.push_back(input_);
			//inputs.push_back(torch::ones({10, 3, 224, 224}).to(at::kCUDA));
			//}

			// Execute the model and turn its output into a tensor.
			torch::Tensor output;
			auto duration = duration_cast&lt;milliseconds&gt;(std::chrono::high_resolution_clock::now() - std::chrono::high_resolution_clock::now());
			auto start = std::chrono::high_resolution_clock::now();
			output = module.forward(inputs).toTensor();
			auto end = std::chrono::high_resolution_clock::now();
			duration = duration_cast&lt;milliseconds&gt;(end - start);
			//std::cout &lt;&lt; output[0] &lt;&lt; '\n';
			for (int i = 0; i&lt;output.size(0); i++) {
				at::Tensor max_ind = at::argmax(output[i]);
				std::cout &lt;&lt; "class_id: " &lt;&lt; i &lt;&lt; " :" &lt;&lt; max_ind.item&lt;int&gt;() &lt;&lt; std::endl;
				
			}

			// at::Tensor max_ind = at::argmax(output);
			// std::cout &lt;&lt; "class_id: " &lt;&lt; max_ind.item&lt;int&gt;() &lt;&lt; std::endl;
			// std::cout &lt;&lt; "Time take for forward pass: " &lt;&lt; duration.count() &lt;&lt; " ms" &lt;&lt; std::endl;
			break;
		}
		std::cout &lt;&lt; "Done\n";```</code></pre>

username: Zain_Ali
cooked: <pre><code class="lang-auto">class_id: 24
class_id: 7
class_id: 19
class_id: 7
class_id: 7
class_id: 27
class_id: 19
class_id: 7
</code></pre>
<p>I am getting output like this, where the input image for all of them is same and actual classid is 7 but it randomly gives other values as well. While in pytorch when i am doing this it works perfectly fine.</p>
<p>I have also updated the libtorch version and so the code.</p>

username: tancl
cooked: <p>this is weird. my results are quite consistent</p>
<p>–&gt; int_torch_test()<br>
class_id: 237<br>
class_id: 237<br>
class_id: 237<br>
class_id: 237<br>
class_id: 237<br>
class_id: 237<br>
class_id: 237<br>
class_id: 237<br>
Done</p>
<p>I start to think whether there could be different results due to the use of GPU, could repost your new codes again I try to see whether I could get access to GPU and try it out.</p>
<p>thanks.</p>
<p>rgds,<br>
CL</p>

username: Zain_Ali
cooked: <p>Thanks alot for your help setting the module.eval() it is giving me the correct results.</p>

username: tancl
cooked: <p>Thanks for the update. I thought the no_grad() doing the same job with eval(), as it happened to my system. anyway good to hear you’ve solved your problem, will put this in mind for future exploration on this. thanks.</p>
<p>rgds,<br>
CL</p>