How can I apply this convolution?

wayne1226: <p>I have a 6-dim tensor say [1,1,4012,6034,11,11], how can I apply the convolution along the last two dims? Excepting the unfold operation which will cause OOM</p>

ptrblck: <aside class="quote no-group" data-username="wayne1226" data-post="1" data-topic="159686">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wayne1226/48/15986_2.png" class="avatar"> wayne1226:</div>
<blockquote>
<p>apply the convolution along the last two dims</p>
</blockquote>
</aside>
<p>I assume this means you want to use a 2D convolution and treat the last two dims of your input as spatial sizes?<br>
If so, what would the other dimensions represent and how would you treat them?<br>
Usually an <code>nn.ConvXD</code> layer expects an input in <code>[batch_size, channels, *dims]</code> where <code>*dims</code> would correspond to 2 spatial dimensions (height and width), 3 volumetric dimensions (depth, height, width), etc.<br>
In your use case it seems the spatial dimensions are fixed while other (undefined) dimensions are added, so could you explain your use case a bit more?</p>

wayne1226: <p>Thank you for your reply. What I want to do is to split a tensor into sliding windows, then apply the window sum in these sliding windows.</p>
<p>For example, I unfold an [1,1,4022,6044] image as  [1,1,4012,6034,11,11], then in each 11x11 window, I want to apply summation on 3x3 window, although I can further unfold the last two dims, the OOM will occur.</p>

KFrank: <p>Hi Wayne!</p>
<aside class="quote no-group" data-username="wayne1226" data-post="3" data-topic="159686" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wayne1226/48/15986_2.png" class="avatar"> wayne1226:</div>
<blockquote>
<p>I unfold an [1,1,4022,6044] image as  [1,1,4012,6034,11,11], then in each 11x11 window, I want to apply summation on 3x3 window</p>
</blockquote>
</aside>
<p>If I understand your use case correctly, you want to apply a 3x3 convolution<br>
to each of the 11x11 patches spanned by your last two dimensions.</p>
<p>If this is right, you can use <code>view()</code> (or <code>reshape()</code>) to push your two large<br>
dimensions into the “batch” dimension (keeping a singleton “channels”<br>
dimension) and then apply an ordinary <code>Conv2d</code>:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.12.0'
&gt;&gt;&gt; _ = torch.manual_seed (2022)
&gt;&gt;&gt; conv = torch.nn.Conv2d (1, 1, 3, padding = 1)
&gt;&gt;&gt; t = torch.randn (1, 1, 412, 634, 11, 11)   # smaller tensor for this example
&gt;&gt;&gt; u = conv (t.view (261208, 1, 11, 11)).view (1, 1, 412, 634, 11, 11)
&gt;&gt;&gt; u.shape
torch.Size([1, 1, 412, 634, 11, 11])
</code></pre>
<p>I’m pretty sure that for large tensors <code>Conv2d</code> is smart enough to use a<br>
algorithm more memory-efficient than calling <code>unfold()</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

wayne1226: <p>Excellent!! This should work in my case, thank you</p>