RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same, Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

username: khachit
cooked: <p>Hi Team, when we run the below script to convert the Semantic segmentation Pytorch model to ONNX, We get “RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!” with the latest Pytorch version (V2.0.0) but when I downgrade the Pytorch version to (V1.11.0) the .pth model gets converted to ONNX. Attaching the logs for successful conversion</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/4/542248c656386b708dfa778bc556db8d7c3c7946.png" data-download-href="https://discuss.pytorch.org/uploads/default/542248c656386b708dfa778bc556db8d7c3c7946" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/542248c656386b708dfa778bc556db8d7c3c7946_2_690x58.png" alt="image" data-base62-sha1="c0hyNpt2P5jSC1pSrNHh5BD18rk" width="690" height="58" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/542248c656386b708dfa778bc556db8d7c3c7946_2_690x58.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/542248c656386b708dfa778bc556db8d7c3c7946_2_1035x87.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/542248c656386b708dfa778bc556db8d7c3c7946_2_1380x116.png 2x" data-dominant-color="F4F4F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1506×127 16.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>BUT, when I do that same with Transformation based Panoptic segmentation model, I get same error as mentioned above. I tried downgrading and upgrading Pytorch version but still the issue exists. I also checked by debugging the input and weight tensors where their device shows cuda. Below are the error logs. Kindly assist if this is something related to version issue. Thanks</p>
<p>ERROR1:<br>
<br>
frame <span class="hashtag-raw">#38:</span> python3() [0x64ca48]<br>
frame <span class="hashtag-raw">#39:</span> python3() [0x64cb14]<br>
frame <span class="hashtag-raw">#40:</span> python3() [0x64cc0c]<br>
frame <span class="hashtag-raw">#44:</span> __libc_start_main + 0xe8 (0xffff9393ae10 in /lib/aarch64-linux-gnu/libc.so.6)<br>
frame <span class="hashtag-raw">#45:</span> python3() [0x598f48]<br>
(function ComputeConstantFolding)<br>
Traceback (most recent call last):<br>
File “pth_to_onn_full_model.py”, line 103, in <br>
main()<br>
File “pth_to_onn_full_model.py”, line 99, in main<br>
torch.onnx.export(model, dummy_input, export_model_path, verbose=True, input_names=input_names,<br>
File “/usr/local/lib/python3.8/dist-packages/torch/onnx/<strong>init</strong>.py”, line 305, in export<br>
return utils.export(model, args, f, export_params, verbose, training,<br>
File “/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py”, line 118, in export<br>
_export(model, args, f, export_params, verbose, training, input_names, output_names,<br>
File “/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py”, line 719, in _export<br>
_model_to_graph(model, args, verbose, input_names,<br>
File “/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py”, line 537, in _model_to_graph<br>
params_dict = torch._C._jit_pass_onnx_constant_fold(graph, params_dict,<br>
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper__index_select)</p>
<p>ERROR 2:<br>
/home/magna1/anaconda3/envs/magp/bin/python /home/magna1/PycharmProjects/Panoptic_Segmentation_Optimization/transformer_pth_to_onnx.py<br>
cuda<br>
Traceback (most recent call last):<br>
File “/home/magna1/PycharmProjects/Panoptic_Segmentation_Optimization/transformer_pth_to_onnx.py”, line 113, in <br>
main()<br>
File “/home/magna1/PycharmProjects/Panoptic_Segmentation_Optimization/transformer_pth_to_onnx.py”, line 84, in main<br>
pixel_level_module_output = model.model.pixel_level_module(pixel_values=dummy_input,<br>
File “/home/magna1/anaconda3/envs/magp/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 1110, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “/home/magna1/PycharmProjects/Panoptic_Segmentation_Optimization/lib/models/magna_mask2former.py”, line 20, in forward<br>
last_hidden_feature, backbone_features = self.encoder(pixel_values)<br>
File “/home/magna1/anaconda3/envs/magp/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 1110, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “/home/magna1/PycharmProjects/Panoptic_Segmentation_Optimization/lib/models/ti_backbone/multi_input_net.py”, line 139, in forward<br>
x = block(x)<br>
File “/home/magna1/anaconda3/envs/magp/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 1110, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “/home/magna1/anaconda3/envs/magp/lib/python3.8/site-packages/torch/nn/modules/container.py”, line 141, in forward<br>
input = module(input)<br>
File “/home/magna1/anaconda3/envs/magp/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 1110, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “/home/magna1/anaconda3/envs/magp/lib/python3.8/site-packages/torch/nn/modules/conv.py”, line 447, in forward<br>
return self._conv_forward(input, self.weight, self.bias)<br>
File “/home/magna1/anaconda3/envs/magp/lib/python3.8/site-packages/torch/nn/modules/conv.py”, line 443, in _conv_forward<br>
return F.conv2d(input, weight, bias, self.stride,<br>
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same</p>

username: ptrblck
cooked: <p>Could you try to script the model before passing it to ONNX as described <a href="https://pytorch.org/docs/stable/onnx.html#tracing-vs-scripting">here</a>?</p>

username: Rachana_H_R
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/7/c73cd7a18bf7475727ae72a0244b39c054c45539.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/c73cd7a18bf7475727ae72a0244b39c054c45539" title="53BDE437-8A50-48E5-AFBB-CE6CE503D946"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c73cd7a18bf7475727ae72a0244b39c054c45539_2_333x500.jpeg" alt="53BDE437-8A50-48E5-AFBB-CE6CE503D946" data-base62-sha1="sqxmYPLYHF3L5K2R4miNgxwnvgR" width="333" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c73cd7a18bf7475727ae72a0244b39c054c45539_2_333x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c73cd7a18bf7475727ae72a0244b39c054c45539_2_499x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c73cd7a18bf7475727ae72a0244b39c054c45539_2_666x1000.jpeg 2x" data-dominant-color="EBEBEB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">53BDE437-8A50-48E5-AFBB-CE6CE503D946</span><span class="informations">1800×2699 464 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
We tried as u mentioned above!! But we are getting this error in one of our file.<br>
Error:<strong>torch.jit.frontend.NotSupportedError: Comprehension ifs are not supported yet</strong></p>
<p>An attachment of the error is added for the reference, Please find it!!<br>
Thank you</p>

username: ptrblck
cooked: <p>Rewrite the <code>tuple comprehension</code> as the error explains if conditions are not supported.</p>

username: Rachana_H_R
cooked: <p>We rewrote the if conditions, then we got errors in “forward” methods.<br>
As mentioned in official site of Pytorch(<a href="https://pytorch.org/docs/stable/generated/torch.jit.script.html#torch.jit.script" rel="noopener nofollow ugc">jit_script</a>),<br>
" <code>nn.Module</code> by default will compile the <code>forward</code> method and recursively compile any methods, submodules, and functions called by <code>forward</code>" so we did <span class="mention">@torch.jit.ignore</span> for “forward” methods.</p>
<p>Later since we had try-except in our code we changed these to if-else and some parameter it wanted in type tensor so we defined it as torch.Tensor.</p>
<p>After doing all these changes, now we are getting this error saying <strong>OSError: could not get source code</strong><br>
I have added the screenshot, please refer it<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/c/bc4b920a25f046048008ebbf67a0b118f38203bf.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/bc4b920a25f046048008ebbf67a0b118f38203bf" title="Screenshot 1945-05-04 at 7.34.47 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc4b920a25f046048008ebbf67a0b118f38203bf_2_680x500.jpeg" alt="Screenshot 1945-05-04 at 7.34.47 PM" data-base62-sha1="qRJFKDNjy1ugwT2Cpu4GNeJy4XB" width="680" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc4b920a25f046048008ebbf67a0b118f38203bf_2_680x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc4b920a25f046048008ebbf67a0b118f38203bf_2_1020x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc4b920a25f046048008ebbf67a0b118f38203bf_2_1360x1000.jpeg 2x" data-dominant-color="EEEEEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 1945-05-04 at 7.34.47 PM</span><span class="informations">1920×1411 257 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Rachana_H_R
cooked: <p>Can someone please help and provide any solutions</p>