Convert pytorch model to ptl

Harshavardhana: <p>Hello,<br>
I have saved pytorch model using below method after training</p>
<p><strong>torch.save(model.state_dict(), model_path)</strong></p>
<p>My final goal is to <strong>deploy the model on mobile</strong>. I am using below code for the purpose</p>
<p><strong>model = Net()</strong><br>
<strong>model.load_state_dict(torch.load(model_path, map_location=‘cpu’))</strong><br>
<strong>traced_script_module = torch.jit.script(model)</strong><br>
<strong>traced_script_module_optimized = optimize_for_mobile(traced_script_module)</strong><br>
<strong>traced_script_module_optimized._save_for_lite_interpreter(ptl_model_path)</strong></p>
<p>With this code, I am getting below error. Any fix available for this problem?</p>
<p>File “pytorch_experiment_mnist.py”, line 103, in convert_pth_to_ptl<br>
<strong>traced_script_module = torch.jit.script(model)</strong><br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_script.py”, line 1258, in script<br>
obj, torch.jit._recursive.infer_methods_to_compile<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_recursive.py”, line 451, in create_script_module<br>
return create_script_module_impl(nn_module, concrete_type, stubs_fn)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_recursive.py”, line 513, in create_script_module_impl<br>
script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_script.py”, line 587, in _construct<br>
init_fn(script_module)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_recursive.py”, line 491, in init_fn<br>
scripted = create_script_module_impl(orig_value, sub_concrete_type, stubs_fn)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_recursive.py”, line 517, in create_script_module_impl<br>
create_methods_and_properties_from_stubs(concrete_type, method_stubs, property_stubs)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_recursive.py”, line 368, in create_methods_and_properties_from_stubs<br>
concrete_type._create_methods_and_properties(property_defs, property_rcbs, method_defs, method_rcbs, method_defaults)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_recursive.py”, line 838, in try_compile_fn<br>
return torch.jit.script(fn, _rcb=rcb)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/_script.py”, line 1307, in script<br>
ast = get_jit_def(obj, obj.<strong>name</strong>)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/frontend.py”, line 264, in get_jit_def<br>
return build_def(parsed_def.ctx, fn_def, type_line, def_name, self_name=self_name, pdt_arg_types=pdt_arg_types)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/frontend.py”, line 302, in build_def<br>
param_list = build_param_list(ctx, py_def.args, self_name, pdt_arg_types)<br>
File “/home/harshavardhana/ProjectTools/Python_VEnv/venv_pytorch/lib/python3.6/site-packages/torch/jit/frontend.py”, line 337, in build_param_list<br>
raise NotSupportedError(ctx_range, _vararg_kwarg_err)<br>
torch.jit.frontend.NotSupportedError: <strong>Compiled functions can’t take variable number of arguments or use keyword-only arguments with defaults:</strong><br>
File “/usr/lib/python3.6/collections/<strong>init</strong>.py”, line 357<br>
def namedtuple(typename, field_names, *, verbose=False, rename=False, module=None):<br>
~~~~~ &lt;— HERE<br>
“”"Returns a new subclass of tuple with named fields.</p>

ptrblck: <p>You might be running into <a href="https://github.com/pytorch/pytorch/issues/29637">this issue</a> which is caused if you are using a variable number of arguments e.g. via <code>def forward(self, *inputs, **kwargs)</code>.<br>
Could you check your model definition and see where this might be the case?</p>

Harshavardhana: <p>I am trying to convert the model preload model in <a href="https://github.com/clovaai/CRAFT-pytorch" class="inline-onebox" rel="noopener nofollow ugc">GitHub - clovaai/CRAFT-pytorch: Official implementation of Character Region Awareness for Text Detection (CRAFT)</a><br>
The code doesn’t seem to have variable number of args directly, but using VGG 16 as basenet.<br>
Not sure about VGG 16.</p>
<p>I wrote a sample code to check your solution and it works.<br>
But the confusion which I still have is,<br>
If I save complete model object, I dont get any error related to variable number of args.<br>
The problem is only when I save state_dict().</p>
<p>I intend to evaluate multiple pytorch model on mobile and most of the available models save only state_dict()</p>
<p>Is there any way I can fix this problem, succeed in optimizing the models to mobile and evaluate them?</p>

ptrblck: <p>Could you describe where you are saving the <code>state_dict</code>? In your current code it seems you are loading a <code>state_dict</code> and then exporting the model. Is this creating the issue and the export works fine if you are not <em>loading</em> the <code>state_dict</code>?</p>

Harshavardhana: <p>I am trying to convert pretrained model available @ <a href="https://github.com/clovaai/CRAFT-pytorch" class="inline-onebox" rel="noopener nofollow ugc">GitHub - clovaai/CRAFT-pytorch: Official implementation of Character Region Awareness for Text Detection (CRAFT)</a> (<a href="https://drive.google.com/open?id=1Jk4eGD7crsqCCg9C9VjCLkMN3ze8kutZ" rel="noopener nofollow ugc">https://drive.google.com/open?id=1Jk4eGD7crsqCCg9C9VjCLkMN3ze8kutZ</a>)<br>
Hence there is no saving part in my code.</p>
<p>I am able to load the model. But when I run <strong>torch.jit.script(model)</strong>, I am running into errors.<br>
Since the model is pretrained, I dont exactly know the args used for training.</p>

ptrblck: <aside class="quote no-group" data-username="Harshavardhana" data-post="5" data-topic="150310">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/harshavardhana/48/49081_2.png" class="avatar"> Harshavardhana:</div>
<blockquote>
<p>Since the model is pretrained, I dont exactly know the args used for training.</p>
</blockquote>
</aside>
<p>I don’t quite understand this, as you seem to have the model definition if you are creating the model object here:</p>
<pre data-code-wrap="python"><code class="lang-python">model = Net()
</code></pre>
<p>Check where <code>Net</code> is defined and make sure no variable arguments are used.</p>

Harshavardhana: <p>That was an example code. Let me post my exact code here.<br>
You can refer craft related code @ <a href="https://github.com/clovaai/CRAFT-pytorch/" class="inline-onebox" rel="noopener nofollow ugc">GitHub - clovaai/CRAFT-pytorch: Official implementation of Character Region Awareness for Text Detection (CRAFT)</a></p>
<p>import torch<br>
import craft<br>
from collections import OrderedDict</p>
<p>def copy_state_dict(state_dict):<br>
if list(state_dict.keys())[0].startswith(‘module’):<br>
start_index = 1<br>
else:<br>
start_index = 0</p>
<pre><code>new_state_dict = OrderedDict()
for k, v in state_dict.items():
    name = '.'.join(k.split('.')[start_index:])
    new_state_dict[name] = v

return new_state_dict
</code></pre>
<p>model = craft.CRAFT()<br>
model.load_state_dict(copy_state_dict(torch.load(‘craft_mlt_25k.pth’, map_location=‘cpu’)))</p>
<p>traced_script_model = torch.jit.script(model)</p>