Jit.trace not working for detecto model

draaken0: <p>I am using detecto model which is trained on custom data set, but the problem is when i try to use that model with jit.trace it throws an error.</p>
<p><strong>Code:</strong></p>
<p>from detecto.core import Model<br>
from detecto.utils import read_image<br>
import torch</p>
<p>model = Model.load(model_path, classes)</p>
<p>example = torch.rand(1, 3, 224, 224)<br>
model_trace = torch.jit.trace(model, example)</p>
<p><strong>Error:</strong></p>
<p>Traceback (most recent call last):<br>
File “c:/Users/shehr/Desktop/Android Project/Android_Model.py”, line 21, in <br>
model_trace = torch.jit.script(model, example)<br>
File “C:\Users\shehr\Anaconda3\envs\tf_gpu\lib\site-packages\torch\jit_<em>init</em>_.py”, line 1257, in script<br>
qualified_name = _qualified_name(obj)<br>
File “C:\Users\shehr\Anaconda3\envs\tf_gpu\lib\site-packages\torch_jit_internal.py”, line 682, in _qualified_name<br>
name = obj.<strong>name</strong><br>
AttributeError: ‘Model’ object has no attribute ‘<strong>name</strong>’</p>
<p>Please help me out</p>

tom: <p>That’s because detecto’s Model are “unrelated” wrappers for TorchVision’s models rather than <code>nn.Module</code>s. You might have some luck by wrapping them in a function and tracing that (<code>torch.jit.trace(lambda x: model(x), example)</code>).</p>
<p>Best regards</p>
<p>Thomas</p>

draaken0: <p>Thank you Thomas for the reply.<br>
I tried to wrap it in a function as you suggested still it didn’t work. Only option left for me to re-train a model based on <strong>nn.Module</strong>.</p>

tom: <p>Well, what error are you getting?</p>
<p>The other bit could be to unwrap detecto’s model (which, as far as I can see is just one of TorchVision’s pretrained detection models by default).</p>

draaken0: <p>this time i am getting " ‘Model’ object is not callable". And yes it is Fasterrcnn pre-trained model which is of torchvision. I looked into the source code of Detecto and what they are doing is loading the pre-trained model, removing last layer of the model and adding a custom layer of output node of length equal to no. of classes we pass in it and all these they are doing it in the Model class they created.</p>
<p><strong>Code</strong></p>
<blockquote>
<p>class Model:</p>
<p>def <strong>init</strong>(self, classes=None, device=None):</p>
<pre><code>    self._device = device if device else config['default_device']

    # Load a model pre-trained on COCO
    self._model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)

    if classes:
        # Get the number of input features for the classifier
        in_features = self._model.roi_heads.box_predictor.cls_score.in_features
        # Replace the pre-trained head with a new one (note: +1 because of the __background__ class)
        self._model.roi_heads.box_predictor = FastRCNNPredictor(in_features, len(classes) + 1)
        self._disable_normalize = False
    else:
        classes = config['default_classes']
        self._disable_normalize = True

    self._model.to(self._device)

    # Mappings to convert from string labels to ints and vice versa
    self._classes = ['__background__'] + classes
    self._int_mapping = {label: index for index, label in enumerate(self._classes)}
</code></pre>
</blockquote>

tom: <p>Yeah, sorry, you’d need model.predict(…) in the lambda rather than model(…)</p>

draaken0: <p>Still not working. I dont have much exp with pytorch as m new to it and now m stuck with this. I just wanted to use this model on Android app but now it seems i have to drop this. I followed what they told on pytorch website, load the trained model, trace it to make it executable for android and run it.</p>
<blockquote>
<p>import detecto.core<br>
import torch</p>
<p>model = detecto.core.Model.load(model_path, classlist)<br>
example = torch.rand(3, 224, 224)<br>
model_trace = torch.jit.trace(model, example)<br>
model_trace.save(save_path + ‘traced_model’)</p>
</blockquote>
<p>I just dont want to retrain thr model, it takes a lot of time but i will have to.</p>

tom: <aside class="quote no-group" data-username="draaken0" data-post="7" data-topic="97238">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/8491ac/48.png" class="avatar"> draaken0:</div>
<blockquote>
<p>I just dont want to retrain thr model, it takes a lot of time but i will have to.</p>
</blockquote>
</aside>
<p>The problem is that a detecto Model isn’t a PyTorch model, so there is a mismatch there.<br>
But you could just grab the model._model if you wanted to. That is a PyTorch model.</p>

draaken0: <p>I could do that but I have already trained a model on a custom dataset using Detecto, so I wanted to use that model. Anyway, thanks Thomas for clearing this out. Next time, I will just use an existing pytorch model to avoid this kind of problem.</p>

draaken0: <p>I got the work around for issue I was facing earlier.</p>
<blockquote>
<p>model = Model.load(model_path, classes)<br>
int_model = model.get_internal_model(model)<br>
int_model.eval()<br>
example = torch.rand(1, 3, 224, 224)<br>
model_trace = torch.jit.trace(int_model, example)</p>
</blockquote>
<p>but now m getting another error in the line</p>
<blockquote>
<p>model_trace = torch.jit.trace(int_model, example)</p>
</blockquote>
<p><strong>RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same</strong></p>
<p>It seems my model is on GPU by the input is on CPU.<br>
I have tried changing <strong>“example”</strong> to <strong>example.to(‘cuda’)</strong><br>
But then it says</p>
<p><strong>RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!</strong></p>
<p>Any help regarding this ?</p>

Shisho_Sama: <p>doing a <code>example = example.cuda()</code> should suffice.<br>
or if you want to use .to make sure you store it back.</p>

draaken0: <p>I tried that too but still the error is same.</p>
<blockquote>
<p>example = torch.rand(1, 3, 224, 224)<br>
example = example.cuda()<br>
model_trace = torch.jit.trace(modelx, example)</p>
</blockquote>
<p><strong>RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!</strong></p>

Shisho_Sama: <p>what do you get when you run this ? :</p>
<pre><code class="lang-auto">print(f'modelx.device : {next(modelx.parameters()).device}' )
print(f'example.device : {example.device}')
</code></pre>

draaken0: <p>I get this</p>
<blockquote>
<p>modelx.device : cuda:0<br>
example.device : cuda:0</p>
</blockquote>
<p>Still it throws that error. M so confused right now</p>

Shisho_Sama: <p>Can you show the full stacktrace?<br>
(side note: by the way , you always need to do a <code>.eval()</code> on your model before tracing it).</p>

draaken0: <p><strong>Code</strong></p>
<pre><code class="lang-auto">
example = torch.rand(1, 3, 224, 224)
example = example.to('cuda')

print(f'modelx.device : {next(modelx.parameters()).device}' )
print(f'example.device : {example.device}')

model_trace = torch.jit.trace(modelx, example)
</code></pre>
<p><strong>Output Trace</strong></p>
<pre><code class="lang-auto">modelx.device : cuda:0
example.device : cuda:0

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-31-d2e0f4962a80&gt; in &lt;module&gt;
      4 print(f'modelx.device : {next(modelx.parameters()).device}' )
      5 print(f'example.device : {example.device}')
----&gt; 6 model_trace = torch.jit.trace(modelx, example)

~\Anaconda3\envs\tf_gpu\lib\site-packages\torch\jit\__init__.py in trace(func, example_inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)
    953         return trace_module(func, {'forward': example_inputs}, None,
    954                             check_trace, wrap_check_inputs(check_inputs),
--&gt; 955                             check_tolerance, strict, _force_outplace, _module_class)
    956 
    957     if (hasattr(func, '__self__') and isinstance(func.__self__, torch.nn.Module) and

~\Anaconda3\envs\tf_gpu\lib\site-packages\torch\jit\__init__.py in trace_module(mod, inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)
   1107             func = mod if method_name == "forward" else getattr(mod, method_name)
   1108             example_inputs = make_tuple(example_inputs)
-&gt; 1109             module._c._create_method_from_trace(method_name, func, example_inputs, var_lookup_fn, strict, _force_outplace)
   1110             check_trace_method = module._c._get_method(method_name)
   1111 

~\Anaconda3\envs\tf_gpu\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    718                 input = result
    719         if torch._C._get_tracing_state():
--&gt; 720             result = self._slow_forward(*input, **kwargs)
    721         else:
    722             result = self.forward(*input, **kwargs)

~\Anaconda3\envs\tf_gpu\lib\site-packages\torch\nn\modules\module.py in _slow_forward(self, *input, **kwargs)
    702                 recording_scopes = False
    703         try:
--&gt; 704             result = self.forward(*input, **kwargs)
    705         finally:
    706             if recording_scopes:

~\Anaconda3\envs\tf_gpu\lib\site-packages\torchvision\models\detection\generalized_rcnn.py in forward(self, images, targets)
     96         if isinstance(features, torch.Tensor):
     97             features = OrderedDict([('0', features)])
---&gt; 98         proposals, proposal_losses = self.rpn(images, features, targets)
     99         detections, detector_losses = self.roi_heads(features, proposals, images.image_sizes, targets)
    100         detections = self.transform.postprocess(detections, images.image_sizes, original_image_sizes)

~\Anaconda3\envs\tf_gpu\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    718                 input = result
    719         if torch._C._get_tracing_state():
--&gt; 720             result = self._slow_forward(*input, **kwargs)
    721         else:
    722             result = self.forward(*input, **kwargs)

~\Anaconda3\envs\tf_gpu\lib\site-packages\torch\nn\modules\module.py in _slow_forward(self, *input, **kwargs)
    702                 recording_scopes = False
    703         try:
--&gt; 704             result = self.forward(*input, **kwargs)
    705         finally:
    706             if recording_scopes:

~\Anaconda3\envs\tf_gpu\lib\site-packages\torchvision\models\detection\rpn.py in forward(self, images, features, targets)
    491         proposals = self.box_coder.decode(pred_bbox_deltas.detach(), anchors)
    492         proposals = proposals.view(num_images, -1, 4)
--&gt; 493         boxes, scores = self.filter_proposals(proposals, objectness, images.image_sizes, num_anchors_per_level)
    494 
    495         losses = {}

~\Anaconda3\envs\tf_gpu\lib\site-packages\torchvision\models\detection\rpn.py in filter_proposals(self, proposals, objectness, image_shapes, num_anchors_per_level)
    392 
    393         # select top_n boxes independently per level before applying nms
--&gt; 394         top_n_idx = self._get_top_n_idx(objectness, num_anchors_per_level)
    395 
    396         image_range = torch.arange(num_images, device=device)

~\Anaconda3\envs\tf_gpu\lib\site-packages\torchvision\models\detection\rpn.py in _get_top_n_idx(self, objectness, num_anchors_per_level)
    372                 pre_nms_top_n = min(self.pre_nms_top_n(), num_anchors)
    373             _, top_n_idx = ob.topk(pre_nms_top_n, dim=1)
--&gt; 374             r.append(top_n_idx + offset)
    375             offset += num_anchors
    376         return torch.cat(r, dim=1)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
</code></pre>
<p>Yes, I did perform the model.eval() just after loading the model</p>

Shisho_Sama: <p>How is your modelx defined?</p>

draaken0: <pre><code class="lang-auto">def get_model(classes):
    num_classes = len(classes)
    model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
    # get number of input features for the classifier
    in_features = model.roi_heads.box_predictor.cls_score.in_features
    # replace the pre-trained head with a new one
    model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes+1)
    model.to('cuda')
    _classes = ['__background__'] + classes
    int_mapping = {label: index for index, label in enumerate(_classes)}
    return model, int_mapping
</code></pre>
<pre><code class="lang-auto">modelx,_ = get_model(classes)
modelx.load_state_dict(torch.load(model_path))
modelx.eval()
</code></pre>

draaken0: <p>I have trained the model on a custom dataset and deployed it and its works fine. There is no issue with the production. Then, I wanted to deploy it on a Android device and for that I needed a traced file. Thats where all this started.</p>

draaken0: <p><a class="mention" href="/u/shisho_sama">@Shisho_Sama</a> I just got to know that MaskRCNN of Faster RCNN models doesn’t have support for <code>torch.jit.trace()</code> as they only support <code>torch.jit.script()</code></p>