Warning and Error converting CenterNet model into ONNX

michaelmo: <p>I’m trying to convert a <a href="https://github.com/xingyizhou/CenterNet" rel="noopener nofollow ugc">CenterNet</a> model into ONNX and then from ONNX to Caffe by using <a href="https://github.com/xxradon/ONNXToCaffe" rel="noopener nofollow ugc">ONNXToCaffe</a>, since the DCNv2 used in centernet is not yet available in Caffe so I only work on the version that doesn’t use it. More specifically, dlav0, in centernet.</p>
<p>I tried it in three ways:</p>
<ol>
<li>create and load model using the code provided in CenterNet src, and convert it into ONNX directly using <code>torch.onnx.export</code>; all works well, I got a ‘.onnx’ file</li>
<li>create and load model using the code provided in CenterNet src, then trace the tensor operation using <code>torch.jit.trace</code>, save into ‘.pth’ file, reload the  ‘.pth’ file; before tracing the tensor I rewrote the forward function because the original output is a dict, I need a tensor output to trace:</li>
</ol>
<pre><code class="lang-auto">model = get_dlav0(34, {'hm': 80, 'wh': 2, 'reg': 2},head_conv=256)
def forward(self, x):
    x = self.base(x)
    x = self.dla_up(x[self.first_level:])
    ret = []
    for head in self.heads:
        ret.append(self.__getattr__(head)(x))
    return torch.cat(ret,1)
model.forward = MethodType(forward,model)
model = load_model(model,modelpath)
dummy_input = torch.rand(1, 3, 512, 512)
model.eval()
target = model(dummy_input)
model = torch.jit.trace(model,dummy_input)
torch.jit.save(model,jitpath)
</code></pre>
<p>Then I reload the <code>jit.pth</code> file to export ONNX, then I got the following warrning:</p>
<blockquote>
<p>UserWarning: Model has no forward function<br>
warnings.warn(“Model has no forward function”)</p>
</blockquote>
<p>I still got the onnx file and tested it with some input, looks right. Wondering why does this warning occur</p>
<ol start="3">
<li>I then tried similar stuff but with <code>torch.jit.script</code>  this time, I got folloing RuntimeError:</li>
</ol>
<blockquote>
<p>RuntimeError:<br>
Expected a default value of type Tensor (inferred) on parameter “residual”.Because “residual” was not annotated with an explicit type it is assumed to be type ‘Tensor’.:<br>
File “D:\Code\Common-Model-Transformation-Service\src\dl2caffe\pytorch2caffe\CenterNet\src\lib\models\networks\dlav0.py”, line 42<br>
def forward(self, x, residual=None):<br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
if residual is None:<br>
~~~~~~~~~~~~~~~~~~~~<br>
residual = x<br>
~~~~~~~~~~~~</p>
<pre><code>    out = self.conv1(x)
    ~~~~~~~~~~~~~~~~~~~
    out = self.bn1(out)
    ~~~~~~~~~~~~~~~~~~~
    out = self.relu(out)
    ~~~~~~~~~~~~~~~~~~~~

    out = self.conv2(out)
    ~~~~~~~~~~~~~~~~~~~~~
    out = self.bn2(out)
    ~~~~~~~~~~~~~~~~~~~

    out += residual
    ~~~~~~~~~~~~~~~
    out = self.relu(out)
    ~~~~~~~~~~~~~~~~~~~~

    return out
    ~~~~~~~~~~ &lt;--- HERE
</code></pre>
</blockquote>
<p>It has something to do with the residual block for DLA structure, any idea how to rewrite it?</p>
<p>I’m trying ‘trace’ or ‘script’ because I want someone else can use it without creating the model instance.</p>
<p>Doing everything on CPU, envs</p>
<blockquote>
<p>ubuntu 18.04<br>
pytorch 1.10.0</p>
</blockquote>
<p>Thanks</p>

Sassy_M: <p>Could you please show the code you used in the first way?</p>
<aside class="quote no-group" data-username="michaelmo" data-post="1" data-topic="143775">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/michaelmo/48/46513_2.png" class="avatar"> michaelmo:</div>
<blockquote>
<p>create and load model using the code provided in CenterNet src, and convert it into ONNX directly using <code>torch.onnx.export</code>; all works well, I got a ‘.onnx’ file</p>
</blockquote>
</aside>