RGB cv::Mat to tensor with different width and height

maris_ancans: <p>Hello, with the help of other posts i managed to convert cv::Mat RGB image to tensor with from_blob() and then get cv::Mat from tensor.<br>
This method however fails when image width and height dimensions differ, I get wierd stripes in cv::Mat<br>
This is where I took the code from: <a href="https://discuss.pytorch.org/t/libtorch-c-convert-a-tensor-to-cv-mat-single-channel/47701/6">https://discuss.pytorch.org/t/libtorch-c-convert-a-tensor-to-cv-mat-single-channel/47701/6</a></p>

beichen2012: <p>1、 cv::Mat to tensor</p>
<pre><code class="lang-auto">void Preprocess(cv::Mat&amp; src, torch::Tensor&amp; input, cv::Scalar mean, float scalor)
{
	cv::Mat img_float;
	src.convertTo(img_float, CV_32F);
	img_float -= mean;
	if (std::abs(1.0f - scalor) &gt; 0.001)
	{
		img_float *= scalor;
	}

	input = torch::zeros({ src.rows, src.cols, src.channels() });
    // 拷贝避免了作用域问题，需要注意的是cv::Mat要是连续的，即stride=width*channels
	memcpy(input.data_ptr(), img_float.data, input.numel() * sizeof(float));
	input = input.permute({ 2, 0, 1 });
	return;
}
</code></pre>
<p>2、 tensor to Mat</p>
<pre><code class="lang-auto">// 假定你使用的是分割网络，即最后一层的输出 是： N * C * H * W的float类型的tensor
// 参见 下面的代码片段
auto out = seg_net-&gt;forward(inputs).toTensor();
auto y = torch::argmax(out, 1);
y = y * 255;
y = y.to(torch::kByte).cpu().contiguous();

std::vector&lt;cv::Mat&gt; vm;
for (int i = 0; i &lt; y.size(0); i++)
{
	auto yo = y[i];
	uchar* data = (uint8_t*)(yo.data_ptr());
    //注意这里的data的作用域
	cv::Mat mr(mtParam.seg_roi.height, mtParam.seg_roi.width, CV_8UC1, data);
}
</code></pre>

maris_ancans: <p>It seems that cv::Mat → tensor works fine, but tensor → cv::Mat gives strange results</p>
<pre><code class="lang-auto">cv::Mat toMat(torch::Tensor&amp; tensor, cv::Size size, int matType) //tensor must be C * H * W
{
   auto det = tensor.detach();
   det *= 255;
   det = det.to(torch::kByte).cpu().contiguous();

   std::vector&lt;cv::Mat&gt; vm;
   for (int i = 0; i &lt; det.size(0); i++)
   {
      auto yo = det[i];
      uchar* data = (uint8_t*)(yo.data_ptr());
      //注意这里的data的作用域
      cv::Mat mr(300, 200, CV_8UC1, data); //return only first channel
      return mr;
   }
}

torch::Tensor toTensor(cv::Mat&amp; img) // img must be CV_32F
{
   auto img_t = torch::zeros({ img.rows, img.cols, img.channels() });
   //stride=width*channels
	memcpy(img_t.data_ptr(), img.data, img_t.numel() * sizeof(float));
	img_t = img_t.permute({ 2, 0, 1 });
   return img_t.clone();
}
</code></pre>
<p>It seems it has to do something with stride/step, because the rows are misaligned<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/5/5f2415caffa49dafc088096d4514f0e2f4394bbb.png" alt="asd" data-base62-sha1="dzEFE6KxRf5ah9sakXVhEHYthuX" width="208" height="325"></p>

tom: <p>It would seem that there isn’t anything obviously wrong, maybe you can</p>
<ul>
<li>print the strides/sizes of yo and/or</li>
<li>post a full example?</li>
</ul>
<p>The following worked for me when I had a HWC 3-channel tensor <code>res</code> to start and cut it to your size:</p>
<pre><code class="lang-auto">  auto res2 = res.select(2, 0).slice(0, 0, 300).slice(1, 0, 200).contiguous();
  std::cout &lt;&lt; res2.sizes() &lt;&lt; "|" &lt;&lt; res2.strides() &lt;&lt; "\n";
  cv::Mat cv_res2(res2.size(0), res2.size(1), CV_8UC1, (void*) res2.data&lt;uint8_t&gt;());
  cv::namedWindow("Detected", cv::WINDOW_AUTOSIZE);
  cv::imshow("Detected", cv_res2);
  cv::waitKey(0);
</code></pre>
<p>gives <code>[300, 200]|[200, 1]</code> and works. (I’m on PyTorch masterrish from a week or two ago.)</p>
<p>Best regards</p>
<p>Thomas</p>

maris_ancans: <p>Thank you for help. The problem I had was in opencv side, I had to change cols to rows.<br>
Also, if you have the time. How good is your performance? I am using stream dataloader that creates opencv mat images on the fly, however im getting cpu usage of about 60% and GPU of about 50% which seems way too low.</p>

tom: <p>I’m afraid not, I only use it for CPU inference.</p>
<p>Best regards</p>
<p>Thomas</p>

maris_ancans: <p>just in case anyone stumbles upon this, here are convertion functions for you<br>
H - height<br>
W - width<br>
C - channels</p>
<pre><code class="lang-auto">torch::Tensor matToTensor(cv::Mat const&amp; src)
{
   torch::Tensor out;
	cv::Mat img_float;
	src.convertTo(img_float, CV_32F);

	out = torch::zeros({ src.rows, src.cols, src.channels() });
   // stride=width*channels
	memcpy(out.data_ptr(), src.data, out.numel() * sizeof(float));
	out = out.permute({ 2, 0, 1 }); // H, W, C --&gt; C, H, W
   return out;
}

// MODIFIES original tensor so that theres no need of clone()
cv::Mat tensorToMat(torch::Tensor t)
{
   cv::Mat img;
   t = t.mul(255).to(torch::kByte).cpu();

   // input needs to be C, H, W
   std::vector&lt;cv::Mat&gt; vm;
   for(auto i : lz::range(t.size(0))){
      auto yo = t[i];
      uchar* data = (uint8_t*)(yo.data_ptr());
      img = cv::Mat(t.size(1), t.size(2), CV_8UC1, data);
   }

   return img.clone(); // needs to clone because the data is taken directly from tensor
}
</code></pre>

lbc3402785: <p>lz::range what is it?</p>

maris_ancans: <p>Oh, sorry, that is from external library. Its the same as for(int i=0; i&lt;t.size(0); i++)</p>