Load an onnx model with ConvTranspose2d group > 1

sheh: <p>A pretrained model has the following layer</p>
<pre><code class="lang-auto">nn.ConvTranspose2d(64, 64, kernel_size=2, 
      stride=2, padding=0, output_padding=0, groups=64, bias=False)
</code></pre>
<p>the model was converted to onnx.</p>
<p>During compilation using command:</p>
<pre><code class="lang-auto">./build/bin/model-compiler -g -model model.onnx -emit-bundle ./bundle -backend=CPU
</code></pre>
<p>an assertion fails:</p>
<pre><code class="lang-auto">  assert(filterDims.n % group == 0 &amp;&amp; filterDims.h == kdim.height &amp;&amp;
         filterDims.w == kdim.width &amp;&amp; filterDims.c == idim.c / group &amp;&amp;
         "Invalid filter dims");
</code></pre>
<p>for my case:</p>
<pre><code class="lang-auto">idim = (1, 64, 15, 20)
filterDims = (64, 1, 2, 2)
group = 64
</code></pre>
<p>so <code>filterDims.n % group != 0</code> and <code>filterDims.c != idim.c / group</code>.</p>
<p>What wrong with the layer parameters and what the assertion checks?</p>

ptrblck: <p>What does <code>model-compiler</code> do and when is it used?<br>
Do you know, how <code>kdim</code> is defined and what it stands for?<br>
It seems that it’s another placeholder for the kernel and tries to compare the shapes with the <code>weight</code> tensor in the transposed convolution?</p>

sheh: <p>I’m testing glow compiler, <code>model-compiler</code> creates model <a href="https://github.com/pytorch/glow/blob/1e7d13a591abcdc4035363e11717febd38b0def3/docs/AOT.md#compile-a-bundle-for-a-floating-point-model" rel="nofollow noopener">bundle</a> and can profile model.<br>
<code>kdim</code> means “kernel dim”.</p>
<p>Sorry, I should have posted the whole <a href="https://github.com/pytorch/glow/blob/07caebd08dae4017a11c0460e769bc0aec31651f/lib/Graph/Graph.cpp#L691" rel="nofollow noopener">method code</a></p>
<pre><code class="lang-auto">static void assertConvTransposeDims(NodeValue input, NodeValue filter,
                                    NodeValue bias,
                                    llvm::ArrayRef&lt;unsigned_t&gt; kernels,
                                    llvm::ArrayRef&lt;unsigned_t&gt; strides,
                                    llvm::ArrayRef&lt;unsigned_t&gt; pads,
                                    unsigned_t group) {
  ShapeNHWC idim = ShapeNHWC(input.dims());
  (void)idim;
  ShapeHW kdim(kernels);
  (void)kdim;
  assert(idim.c % group == 0 &amp;&amp; "channels number must be divisible by groups");

  // NOTE: here the N in NHWC is abnormal because it is the number of filters
  // (and therefore the number of output channels of the conv) and not the
  // batch size. The rest of the dimensions are representative of the input
  // dimensions to the convolution.
  ShapeNHWC filterDims(filter.dims());
  (void)filterDims;

  assert(filterDims.n % group == 0 &amp;&amp; filterDims.h == kdim.height &amp;&amp;
         filterDims.w == kdim.width &amp;&amp; filterDims.c == idim.c / group &amp;&amp;
         "Invalid filter dims");

  assert(bias.getType()-&gt;size() == filterDims.n &amp;&amp; "Invalid bias size");
}
</code></pre>
<p>It looks like ConvTranspose2d must have <code>out_channels=in_channels*group</code> to pass the assertion.</p>

ptrblck: <p><code>in_channels</code> and <code>out_channels</code> should be both divisible by <code>groups</code>.<br>
If your code is working fine in PyTorch, I’m a bit confused if <code>glow</code> adds some other checks to the transposed convolutions.</p>
<p>Did you narrow it down to a <code>nn.ConvTranpose2d</code> layer, which creates the issue in the export?</p>

sheh: <blockquote>
<p>I’m a bit confused if  <code>glow</code>  adds some other checks to the transposed convolutions.</p>
</blockquote>
<p>Moreover the <a href="https://github.com/pytorch/glow/blob/07caebd08dae4017a11c0460e769bc0aec31651f/lib/Graph/Graph.cpp#L736" rel="noopener nofollow ugc">same check</a> works for Conv2d layers.</p>
<p>I managed to convert the model with <code>group=1</code>. Obviously, I have to retrain the model with new layer.</p>
<p>Anyway I wonder what’s wrong with original case in my initial post.</p>

ptrblck: <aside class="quote no-group" data-username="sheh" data-post="5" data-topic="80480">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/ba9def/48.png" class="avatar"> sheh:</div>
<blockquote>
<p>Moreover the <a href="https://github.com/pytorch/glow/blob/07caebd08dae4017a11c0460e769bc0aec31651f/lib/Graph/Graph.cpp#L736">same check </a> works for Conv2d layers.</p>
</blockquote>
</aside>
<p>That’s what I’m wondering. If your model works fine in your PyTorch code, I would assume that it should also pass all Glow checks.<br>
Are you seeing this error for all settings other than <code>groups=1</code>?</p>

sheh: <p>I’ve tested with <code>group=2</code> and <code>group=32</code> – it fails on the same assertion <code>assertConvTransposeDims</code>.<br>
If I comment the assertion it fails somewhere later.</p>
<p>Should I create a github issue?</p>

ptrblck: <p>Yes, please create an issue, as I’m not familiar enough with Glow’s internals.</p>

sheh: <p>Github issue<br>
</p><aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/glow/issues/4516" target="_blank" rel="nofollow noopener">github.com/pytorch/glow</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/glow/issues/4516" target="_blank" rel="nofollow noopener">Load an onnx model with ConvTranspose2d group &gt; 1</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-05-14" data-time="14:47:55" data-timezone="UTC">02:47PM - 14 May 20 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/sheh" target="_blank" rel="nofollow noopener">
          <img alt="sheh" src="https://avatars3.githubusercontent.com/u/1386851?v=4" class="onebox-avatar-inline" width="20" height="20">
          sheh
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">Original discussion
A pretrained model has the following layer
nn.ConvTranspose2d(64, 64, kernel_size=2, stride=2, 
 padding=0, output_padding=0, groups=64, bias=False)
the model was converted to onnx.
During...</p>
</div>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>