Resize_ cannot resize tensor that require grad

blackbirdbarber: <pre><code class="lang-auto">print(x.shape, x_skip.shape)        
x_skip.resize_(x_skip.shape[0], x_skip.shape[1], x.shape[2], x.shape[3])
x = torch.cat((x, x_skip), dim=1)
</code></pre>
<p>Out:</p>
<pre><code class="lang-auto">torch.Size([1, 512, 24, 24]) torch.Size([1, 256, 25, 25])
</code></pre>
<p>The problem is I wanted to <code>cat</code> two tensors but since they don’t match I don’t have the idea how to cut tensor <code>x_skip</code> to match the spatial dimensions of tensors <code>x</code>.</p>
<p>This is a part of solving the <a href="https://github.com/ptrblck/pytorch_misc/blob/master/unet_demo.py#L63" rel="noopener nofollow ugc">Unet</a> architecture that takes any input size and outputs the feature map of the input size.</p>

KFrank: <p>Hi Blackbird!</p>
<aside class="quote no-group" data-username="blackbirdbarber" data-post="1" data-topic="158532" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/6de8d8/48.png" class="avatar"> blackbirdbarber:</div>
<blockquote>
<p>Out:</p>
<pre><code class="lang-auto">torch.Size([1, 512, 24, 24]) torch.Size([1, 256, 25, 25])
</code></pre>
<p>The problem is I wanted to <code>cat</code> two tensors but since they don’t match I don’t have the idea how to cut tensor <code>x_skip</code> to match the spatial dimensions of tensors <code>x</code>.</p>
</blockquote>
</aside>
<p>To answer your specific question:  <code>resize_()</code> doesn’t do what you would<br>
want it to – it will mess up the two-dimensional structure of your image.<br>
You can just slice into <code>x_skip</code> to trim off the excess of one row and column<br>
of pixels:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.cat ((x, x_skip[:, :, :x.shape[2], :x.shape[3]), dim = 1)
</code></pre>
<p>(Of course, the <code>x</code> and trimmed <code>x_skip</code> might not align ideally.)</p>
<p>Two comments (in the context of the <a href="https://arxiv.org/abs/1505.04597" rel="noopener nofollow ugc">original U-Net paper</a>, rather than that<br>
of the link you cite):</p>
<p>When you perform U-Net’s “up-convolution,” you reduce the number of<br>
“features” (the dimension of your <code>x</code> tensor that is of size <code>512</code>) by a factor<br>
of two.  So your <code>x</code> and <code>x_skip</code> should have the <em>same</em> number of features,<br>
which I assume should be <code>256</code>.  Check where <code>x</code> is coming from and why<br>
it appears to have twice as many “features” as it should.</p>
<p>In my view, the cleanest way to get the height and width of <code>x</code> and <code>x_skip</code><br>
to match is to only input into the U-Net images whose height and width are<br>
such that as the “image” progresses through the U-Net the intermediate<br>
heights and widths are always evenly divisible by two when passed through<br>
each 2x2 max-pooling downsampling step.  This way, you’ll never have to<br>
pad or trim <code>x</code> or <code>x_skip</code> to get them to match in height and width (as they<br>
need to when you <code>cat()</code> them).</p>
<p>In order to input images of such “self-consistent” heights and widths, you<br>
can either choose tiles of such sizes, if you’re using a tiling strategy, or<br>
you can pad the input image up to such a self-consistent size.  (The U-Net<br>
paper recommends <em>reflection padding</em> which they call <em>mirroring.</em>)</p>
<p>Best.</p>
<p>K. Frank</p>

blackbirdbarber: <aside class="quote no-group" data-username="KFrank" data-post="2" data-topic="158532">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>trim off the excess</p>
</blockquote>
</aside>
<p>This seams like a good option.</p>
<p>Re the features, I am not highly opinionated at the moment, but it seams to me like not a problem to add any number of features as long as spatial dimensions match. What is the possible idea that equal number of features may be better?</p>
<p>Trimming strategy for U-Nets you just invented looks like a solution at the moment, self consistent size images are those where spatial dimension is divisible by 8. Can you clarify what is “choose tiles of such sizes” idea?</p>