[C++] [beginner] why does tensor cast not work ? / RuntimeError: expected scalar type Byte but found Float

birds_are_drones: <p>Hi,</p>
<p>I’m pretty new to pytorch and even more to C++ frontend.<br>
I’m currently trying to use yolov5 on C++ and keep having type problems no matter how hard I try to cast it. I Do it once in the options of the tensor, and then after having copied data from the cv::mat .<br>
I also tried to change the cv::mat conversion type to others types (such as CV_8UC3 for example) without success.</p>
<p>I tried this solution (you can see it in my code) without success : <a href="https://discuss.pytorch.org/t/type-conversion-in-libtorch-for-c/45139" class="inline-onebox">Type conversion in Libtorch for C++</a></p>
<p>Here is (the relevant part of) my code :</p>
<pre><code class="lang-auto">//___________________________________________________________________________________________
//___________________________________________UTILS___________________________________________
//___________________________________________________________________________________________

bool load_preproc_detect(cv::Mat&amp; src , cv::Mat&amp; dst ,  std::string&amp; path , float scale ){

  //___________________read_image_data
  src = cv::imread( path );
  if (src.data==NULL){
    std::cerr &lt;&lt; "empty image file, maybe invalid path?" &lt;&lt; std::endl;
  }

  //________detect_preprocess
  cv::cvtColor( src , src , cv::COLOR_RGB2BGR );//opencv uses bgr
  src.convertTo(dst ,  CV_32FC3, 1.0 / 255, 0); //normalization and type matching tensor input type

  //arbitrary resize to reduce computation speed(watch for the precision tradeoff)
  cv::Size dst_sz = src.size();
  dst_sz.height = (int) dst_sz.height * scale;
  dst_sz.width = (int) dst_sz.width * scale;
  cv::resize(dst , dst ,  dst_sz , 1 ,  1);
}









//___________________________________________________________________________________________
//___________________________________________MAIN____________________________________________
//___________________________________________________________________________________________
int main(int argc, char* argv[])
{
  //_______clear screen__________
  std::cout &lt;&lt;"oui"&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl&lt;&lt;std::endl;


  //______________defining path
  std::string root_path = "/home/kubler/sdsp-rea/";
  std::string detect_weight_path = root_path+"model/detect/yolo_test.torchscript.pt" ;
  std::string data_path = root_path + "temp_data/RGB_0001.png";


  //______________________CUDA SETUP_______________________
  torch::Device device(torch::kCPU);


  //___________________read_image_data
  cv::Mat img ; //= cv::imread(data_path);
  cv::Mat detect_img;
  cv::Mat pose_img;
  float scale = 1/2.5;
    load_preproc_detect(img , detect_img , data_path , scale );
  std::cout &lt;&lt; "data preprocessed" &lt;&lt; std::endl;

  //_________________TORCH SETUP

  //load torch model
  torch::jit::script::Module detect_model , pose_model;
  detect_model = load_from_torchscript(detect_weight_path , device);

  //set to inference only mode : grads are all frozen for safety
  c10::InferenceMode guard;

  //input_tensor_preparation
  auto detect_options =   torch::TensorOptions()
  .dtype(at::kByte) // trying to force Byte type
  .layout(torch::kStrided)
  .device(device)
  .requires_grad(false);

  torch::Tensor detect_input;// = torch::empty({detect_sz.width , detect_sz.height} , detect_options );
  detect_input = torch::from_blob(detect_img.data, {1 ,  detect_img.rows, detect_img.cols , detect_img.channels()} , detect_options);
  // 2 different ways of casting i have seen here and not working : i get the same output 
  //detect_input = detect_input.item().to&lt;Byte&gt;();
  detect_input = detect_input.to(torch::kByte);
  detect_input = detect_input.permute({0, 3 , 1 , 2});

  std::vector&lt;torch::jit::IValue&gt; detect_dataset;
  detect_input = detect_input.to(device);
  detect_dataset.push_back(detect_input);

  //______________________INFERENCE_______________________

  torch::Tensor detect_output = detect_model.forward(detect_dataset).toTensor();

  std::cout &lt;&lt; "end" &lt;&lt; std::endl;
  return 0;
}
</code></pre>
<p>and here is the output</p>
<pre><code class="lang-auto">data preprocessed
model loaded
terminate called after throwing an instance of 'std::runtime_error'
  what():  The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File "code/__torch__/models/yolo.py", line 33, in forward
    _22 = getattr(self.model, "2")
    _23 = getattr(self.model, "1")
    _24 = (getattr(self.model, "0")).forward(x, )
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ &lt;--- HERE
    _25 = (_22).forward((_23).forward(_24, ), )
    _26 = (_20).forward((_21).forward(_25, ), )
  File "code/__torch__/models/common.py", line 19, in forward
    _8 = torch.slice(_7, 3, 1, 9223372036854775807, 2)
    input = torch.cat([_2, _4, _6, _8], 1)
    return (_0).forward(input, )
            ~~~~~~~~~~~ &lt;--- HERE
class Conv(Module):
  __parameters__ = []
  File "code/__torch__/models/common.py", line 29, in forward
  def forward(self: __torch__.models.common.Conv,
    input: Tensor) -&gt; Tensor:
    _9 = (self.act).forward((self.conv).forward(input, ), )
                             ~~~~~~~~~~~~~~~~~~ &lt;--- HERE
    return _9
class C3(Module):
  File "code/__torch__/torch/nn/modules/conv.py", line 11, in forward
    input: Tensor) -&gt; Tensor:
    _0 = self.bias
    x = torch._convolution(input, self.weight, _0, [1, 1], [1, 1], [1, 1], False, [0, 0], 1, False, False, True, True)
        ~~~~~~~~~~~~~~~~~~ &lt;--- HERE
    return x

Traceback of TorchScript, original code (most recent call last):
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/conv.py(440): _conv_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/conv.py(443): forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1039): _slow_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1051): _call_impl
/home/kubler/data/det_track/yolov5/models/common.py(48): forward_fuse
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1039): _slow_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1051): _call_impl
/home/kubler/data/det_track/yolov5/models/common.py(206): forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1039): _slow_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1051): _call_impl
/home/kubler/data/det_track/yolov5/models/yolo.py(155): forward_once
/home/kubler/data/det_track/yolov5/models/yolo.py(123): forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1039): _slow_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1051): _call_impl
/home/kubler/.local/lib/python3.6/site-packages/torch/jit/_trace.py(959): trace_module
/home/kubler/.local/lib/python3.6/site-packages/torch/jit/_trace.py(744): trace
export.py(35): export_torchscript
export.py(154): run
export.py(187): main
export.py(192): &lt;module&gt;
RuntimeError: expected scalar type Byte but found Float
</code></pre>
<p>I am a bit confused because in the original yolov5 code, the input value type is float as you can see just under but the error i keep having is that the model expect to receive a float type value.</p>
<pre><code class="lang-auto">for path, img, im0s, vid_cap in dataset:
    img = torch.from_numpy(img).to(device)
    img = img.half() if half else img.float()  # uint8 to fp16/32
    img = img / 255.0  # 0 - 255 to 0.0 - 1.0
    if len(img.shape) == 3:
        img = img[None]  # expand for batch dim
</code></pre>
<p>I hope I have been precise enough in my explanations, thank you for your help \o</p>

ptrblck: <p>Based on the error message and your code it seems as if you are trying to use a <code>ByteTensor</code> in a common model, which uses <code>float32</code> by default.<br>
You would have to make sure the input <code>dtype</code> is matching the model parameters, so transform the input to <code>float32</code> and it should work (I don’t think many operations support <code>ByteTensors</code>).</p>

birds_are_drones: <p>Yesss! it worked thank you!</p>
<p>Can I ask where I can find an explanation to it?<br>
because on <a href="https://pytorch.org/docs/stable/tensors.html" rel="noopener nofollow ugc">this type table</a> the only references of ByteTensor are for integers.</p>
<p>Also the code error was</p>
<pre><code class="lang-auto">RuntimeError: expected scalar type Byte but found Float
</code></pre>
<p>I know that I have at the beginnng a float <code>cv::Mat</code> of dtype <code>CV_32FC3</code>  that I converted to <code>torch::tensor</code> of dtype <code>at::kByte</code>. So does the <code>at::kByte</code> conversion somehow keeps the float type without letting the model using it ?</p>
<p>I am not contesting your answer just trying to understand  how this error was formed because it displayed the opposite of what I understood I had to do.</p>
<p>Thank you again !</p>

ptrblck: <p>The error message might be a bit confusing, as PyTorch assumes that the first argument to an operation has the desired <code>dtype</code> (which is a <code>ByteTensor</code> in <code>torch._convolution(input, self.weight, ...)</code>) and complains about the <code>self.weight</code>, which is a <code>FloatTensor</code>.</p>

birds_are_drones: <p>Ok I might have understood. Is this a feature or a bug ? <img src="https://discuss.pytorch.org/images/emoji/apple/grimacing.png?v=10" title=":grimacing:" class="emoji" alt=":grimacing:"></p>
<p>Anyway thank you for your precious help !</p>