Pytorch model convert to onnx

qingye_meng: <p>import torch<br>
from transformers import BertModel, BertForMaskedLM</p>
<p>def convert_onnx():<br>
model_path = ‘…/bertLM_model_pytorch/pytorch_model.pt’<br>
dummy_input0 = torch.LongTensor(16, 128).to(torch.device(“cuda”))<br>
dummy_input1 = torch.LongTensor(16, 128).to(torch.device(“cuda”))<br>
dummy_input2 = torch.LongTensor(16, 128).to(torch.device(“cuda”))<br>
dynamic_axes = {<br>
‘dummy_input0’: {0: ‘batch_size’, 1: ‘seq_length’},<br>
'dummy_input1 ': {0: ‘batch_size’, 1: ‘seq_length’},<br>
‘dummy_input2’: {0: ‘batch_size’, 1: ‘seq_length’},<br>
‘outputs’: {0: ‘batch_size’, 1: ‘seq_length’},<br>
}<br>
model = torch.load(model_path)<br>
model.load_state_dict(loaded_model[‘state_dict’])<br>
dummy_input =(dummy_input0, dummy_input1, dummy_input2)<br>
onnx_path = ‘./pytorch_model.onnx’<br>
torch.onnx.export(model,dummy_input, onnx_path,input_names=[‘dummy_input0’, 'dummy_input1 ',‘dummy_input2’], output_names=[‘outputs’],dynamic_axes=dynamic_axes)<br>
print(‘convert retinaface to onnx finish!!!’)</p>
<p>I am trying to convert the pytorch model to onnx, and an error is reported after the above code runs:<br>
AttributeError: ‘collections.OrderedDict’ object has no attribute ‘load_state_dict’</p>
<p>I want to know what is the cause of the error, thank you！ <img src="https://discuss.pytorch.org/images/emoji/apple/grin.png?v=10" title=":grin:" class="emoji" alt=":grin:"></p>

ptrblck: <p>This line of code loads a <code>state_dict</code> not a model object:</p>
<pre><code class="lang-python">model = torch.load(model_path)
</code></pre>
<p>which is why the following call fails:</p>
<pre><code class="lang-python">model.load_state_dict(loaded_model[‘state_dict’])
&gt; AttributeError: ‘collections.OrderedDict’ object has no attribute ‘load_state_dict’
</code></pre>
<p>To properly load the model, create the model instance first and load the <code>state_dict</code> afterwards:</p>
<pre><code class="lang-python">model = MyModel()
model.load_state_dict(torch.load(model_path))
</code></pre>

qingye_meng: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="132009">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>To properly load the model, create the model instance first and load the <code>state_dict</code> afterwards:</p>
</blockquote>
</aside>
<p>Thank you for your reply, I rewritten the code according to your prompt:</p>
<pre><code>model=BertForMaskedLM.from_pretrained(model_path)
model.load_state_dict(torch.load(model_path))
</code></pre>
<p>And now it prompts a new error:<br>
UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0x80 in position 0: invalid start byte<br>
What is the reason for this problem, I am a beginner in deep learning, thank you for your reply！ <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>

ptrblck: <p>I guess the error is caused in:</p>
<pre><code class="lang-python">model=BertForMaskedLM.from_pretrained(model_path)
</code></pre>
<p>which most likely does not expect a path to a <code>state_dict</code>, so check the docs for this method and make sure to pass the right arguments to it.</p>

qingye_meng: <p>Thank you, exactly as you said.<br>
It’s a coincidence, I see that your business card is from Nvidia. My purpose of converting to onnx is to optimize the inference speed of the model based on onnx-&gt;TensorRT <img src="https://discuss.pytorch.org/images/emoji/apple/grin.png?v=10" title=":grin:" class="emoji" alt=":grin:"></p>