Vote Convert Hopenet (ResNet50) from pytorch to mxnet KeyError

cesarpgouveia: <p>I’m trying to convert a pyTorch pre-trained model (hopenet) to <strong>mxnet</strong>.</p>
<p>Hopenet is a pose estimator deep learning model based on resnet50. Pretrained model and hopenet implementation can be found here: <a href="https://github.com/natanielruiz/deep-head-pose" rel="noopener nofollow ugc">GitHub - natanielruiz/deep-head-pose: Deep Learning Head Pose Estimation using PyTorch.</a></p>
<p>To achieve the conversion I developed the following piece of code (python):</p>
<pre><code class="lang-auto">import os
import time
import cv2
import numpy as np
import torch
from torch.autograd import Variable
import torch.onnx
from torchvision import transforms
import torch.backends.cudnn as cudnn
import torchvision
import torch.nn.functional as F
from PIL import Image
import mxnet as mx
from mxnet.contrib import onnx as onnx_mxnet
from pytesseract import image_to_string

import mxnet as mx
from mxnet.contrib import onnx as onnx_mxnet
import numpy as np

import hopenet
import utils

param_snapshot = r"C:\Users\cesar.gouveia\Projects\deep-head-pose\hopenet_alpha2.pkl"
param_save_onnx_complete_model_file_path = r"C:\Users\cesar.gouveia\Projects\deep-head-pose\deep_head_pose.onnx"

# ResNet50 structure
model = hopenet.Hopenet(torchvision.models.resnet.Bottleneck, [3, 4, 6, 3], 66)

print('Loading snapshot.')
# Load snapshot
saved_state_dict = torch.load(param_snapshot, map_location="cpu")
model.load_state_dict(saved_state_dict)
model.train(False)

torch.save(model, param_save_torch_complete_model_file_path)

input_shape = (3, 224, 224)
dummy_input = Variable(torch.randn(1, *input_shape))

torch.onnx.export(model, dummy_input, param_save_onnx_complete_model_file_path)

sym, arg, aux = onnx_mxnet.import_model(param_save_onnx_complete_model_file_path)

mx.model.save_checkpoint(os.path.join(r"C:\Workspace", 'model_mxnet'), 0, sym, arg, aux)
</code></pre>
<p>Basically the code loads a pyTorch pre-trained model, exports the following model to onnx and then imports the onnx model and tries to convert it to mxnet. The code is based on this tutorial on how to convert pytorch to mxnet (<a href="https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-onnx-pytorch-mxnet.html" rel="noopener nofollow ugc">PyTorch to ONNX to MXNet Tutorial - Deep Learning AMI</a>).</p>
<p>The issue happens when it tries to import the onnx model using mxnet:</p>
<pre><code class="lang-auto">C:\Users\cesar.gouveia\Anaconda3\envs\hopenet\lib\site-packages\torch\nn\functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  ..\c10/core/TensorImpl.h:1156.)   
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode) 
Traceback (most recent call last):   
File "C:/Users/cesar.gouveia/Projects/deep-head-pose/test_on_video.py", line 53, in &lt;module&gt;
    sym, arg, aux = onnx_mxnet.import_model(param_save_onnx_complete_model_file_path)   
File "C:\Users\cesar.gouveia\Anaconda3\envs\hopenet\lib\site-packages\mxnet\contrib\onnx\onnx2mx\import_model.py", line 59, in import_model
    sym, arg_params, aux_params = graph.from_onnx(model_proto.graph)   
File "C:\Users\cesar.gouveia\Anaconda3\envs\hopenet\lib\site-packages\mxnet\contrib\onnx\onnx2mx\import_onnx.py", line 114, in from_onnx
    inputs = [self._nodes[i] for i in node.input]   
File "C:\Users\cesar.gouveia\Anaconda3\envs\hopenet\lib\site-packages\mxnet\contrib\onnx\onnx2mx\import_onnx.py", line 114, in &lt;listcomp&gt;
    inputs = [self._nodes[i] for i in node.input] KeyError: '513'
</code></pre>
<p>Tried to google about other people with the same problem but no success. Also tried to use netron to see which node was failing, and apparently it has something to do with the first convolution? in the convolution weights?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/5/f5bc34aeaf9a0d91a816517929811249d8da7d05.png" data-download-href="https://discuss.pytorch.org/uploads/default/f5bc34aeaf9a0d91a816517929811249d8da7d05" title="Untitled2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f5bc34aeaf9a0d91a816517929811249d8da7d05_2_690x455.png" alt="Untitled2" data-base62-sha1="z3ScYzAVbOw5CW2Ok1Pl8uyAYXb" width="690" height="455" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f5bc34aeaf9a0d91a816517929811249d8da7d05_2_690x455.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f5bc34aeaf9a0d91a816517929811249d8da7d05_2_1035x682.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f5bc34aeaf9a0d91a816517929811249d8da7d05_2_1380x910.png 2x" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Untitled2</span><span class="informations">1921×1267 64.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Has anyone experienced this type of error? Probably I’m doing something wrong because it is very unlikely that there is no conversion compatibility for a resnet50 from pytorch to mxnet.</p>
<p>Thanks, César.</p>

ptrblck: <p>Based on the error message it seems that the <code>KeyError</code> is raised by MXNet, so I would recommend to (also) post this question in their discussion board, as you might find more MXNet experts there.<br>
My initial guess would be to check the ONNX graphs and try to isolate which node is apparently missing (using the <code>513</code> id).</p>