Channel determination behind the Conv2D layer when export as onnx

MakeDirtyCode: <p>I am trying to convert this type of code to onnx format.</p>
<pre><code class="lang-auto">     def __init__(self):
         ...
         self.conv = nn.Conv2d(c_in, c_out, 3, 1, 1)
         self.norm = nn.InstanceNorm2d(c_out)
         ...
     def forward(self, x):
         ...
         x = self. norm(self. conv(x))
         ...
</code></pre>
<p>Of course, I think that the result of self.conv(x) should have the form of (*, c_out, *, *).</p>
<p>However, this model gives the following error.<br>
<code>torch.onnx.errors.SymbolicValueError: Unsupported: ONNX export of instance_norm for unknown channel size.</code></p>
<p>Shouldn’t the channel be confirmed the moment self.conv is passed, no matter how other operations before it are configured? However, torch.onnx.export judges the output passed through self.conv as Float(*, *, *, *). How to solve it?</p>

ptrblck: <p>I cannot reproduce the issue and get:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(1, 3, 3, 1, 1)
        self.norm = nn.InstanceNorm2d(3)
         
    def forward(self, x):
        x = self. norm(self. conv(x))
        return x
    
model = MyModel()
x = torch.randn(1, 1, 224, 224)
out = model(x)


torch.onnx.export(model, x, "test.onnx")
# ============= Diagnostic Run torch.onnx.export version 2.0.0+cu118 =============
# verbose: False, log level: Level.ERROR
# ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
# UserWarning: ONNX export mode is set to TrainingMode.EVAL, but operator 'instance_norm' is set to train=True. Exporting with train=True.

torch.onnx.export(model.eval(), x, "test.onnx")
# ============= Diagnostic Run torch.onnx.export version 2.0.0+cu118 =============
# verbose: False, log level: Level.ERROR
# ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
</code></pre>

MakeDirtyCode: <p>Of course, it worked well when I exported only those two layers. However, there was a problem when there were more layers in front. But shouldn’t the number of channels be fixed regardless of the front-end layers as soon as it passes through the Conv2D? The model I am currently trying is the OcclusionAwareSPADEGenerator from <a href="https://github.com/Winfredy/SadTalker/blob/main/src/facerender/modules/generator.py" class="inline-onebox" rel="noopener nofollow ugc">SadTalker/generator.py at main · Winfredy/SadTalker · GitHub</a> . The error in the main text occurs when passing through the InstanceNorm2D on line 143, after going through the Conv2D on line 142.</p>
<p>Additionally, I am working with PyTorch 1.13.1+cu17.</p>

ptrblck: <aside class="quote no-group" data-username="MakeDirtyCode" data-post="3" data-topic="177670">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/makedirtycode/48/7310_2.png" class="avatar"> MakeDirtyCode:</div>
<blockquote>
<p>Of course, it worked well when I exported only those two layers.</p>
</blockquote>
</aside>
<p>That wasn’t clear from your description, so could you post a minimal and executable code snippet reproducing the issue, please?</p>

MakeDirtyCode: <p>Due to the complexity of the model, I couldn’t figure out where exactly the problem was occurring. See the picture below for the problematic part.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/f/5fa74d2774aaaf2be15ea3ccca5a5e5f498ad158.png" data-download-href="https://discuss.pytorch.org/uploads/default/5fa74d2774aaaf2be15ea3ccca5a5e5f498ad158" title="screenshot"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/f/5fa74d2774aaaf2be15ea3ccca5a5e5f498ad158_2_382x500.png" alt="screenshot" data-base62-sha1="dEbNG15mJv45zYTaaGmEKTXd81i" width="382" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/f/5fa74d2774aaaf2be15ea3ccca5a5e5f498ad158_2_382x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/f/5fa74d2774aaaf2be15ea3ccca5a5e5f498ad158_2_573x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/f/5fa74d2774aaaf2be15ea3ccca5a5e5f498ad158_2_764x1000.png 2x" data-dominant-color="404041"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">screenshot</span><span class="informations">1250×1634 155 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
In the middle part of the model, there is a part that goes through Conv2D and then InstanceNorm2D. (L142-L143 written in the comments above). I couldn’t understand the error like <code>torch.onnx.errors.SymbolicValueError: Unsupported: ONNX export of instance_norm for unknown channel size.</code> because both layers have their channels determined. Although there is a complicated process above, isn’t the channel determined the moment it goes through Conv2D?</p>

MakeDirtyCode: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/6/c6982db2f4dc17455002de5fa4e1f9c40625e0b5.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/c6982db2f4dc17455002de5fa4e1f9c40625e0b5" title="screenshot"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c6982db2f4dc17455002de5fa4e1f9c40625e0b5_2_690x407.jpeg" alt="screenshot" data-base62-sha1="skQzXVJQzreNYghVfIp8usdYcPX" width="690" height="407" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c6982db2f4dc17455002de5fa4e1f9c40625e0b5_2_690x407.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c6982db2f4dc17455002de5fa4e1f9c40625e0b5_2_1035x610.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c6982db2f4dc17455002de5fa4e1f9c40625e0b5_2_1380x814.jpeg 2x" data-dominant-color="3C3C3C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">screenshot</span><span class="informations">2764×1634 172 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
To be more clear, when the model is exported up to <a href="https://github.com/Winfredy/SadTalker/blob/9c36c32e64dc7beedae26790bb2795d729abbea7/src/facerender/modules/generator.py#L142" class="inline-onebox" rel="noopener nofollow ugc">SadTalker/src/facerender/modules/generator.py at 9c36c32e64dc7beedae26790bb2795d729abbea7 · OpenTalker/SadTalker · GitHub</a>, the size of the output is <code>*,512,*,*</code>. (The corresponding line is the Conv2D layer.) However, ifI execute <code>x = self.G_middle_0.norm_0(x, seg)</code> (InstanceNorm2D) in the next line, an error occurs.</p>

MakeDirtyCode: <p>As a result of a little more detailed analysis, it was a problem related to <a href="https://github.com/pytorch/pytorch/issues/92209" class="inline-onebox" rel="noopener nofollow ugc">ONNX Opset 16 GridSample Does Not Support 5D Volumetric Input Tensor · Issue #92209 · pytorch/pytorch · GitHub</a>. As a result of exporting with pytorch2.0, the PR was reflected and an error occurred that grid sample for 5D tensor is not supported.<br>
However, with torch&lt;2.0, instance_norm throws an error after passing the grid sample without problems. I think it should be a backport.</p>

huangxin168: <p>Any update on the sadtalker onnx model? I also try and got same error.</p>