Change the dimension of tensor

zahra: <p>Hi,</p>
<p>I have a tensor with dimension [1, 1, 4, 6] like this:</p>
<pre><code class="lang-auto">a = torch.tensor([[[ 1,  2,  3,  4,  5,  6],
                   [ 7,  8,  9, 10, 11, 12],
                   [13, 14, 15, 16, 17, 18],
                   [19, 20, 21, 22, 23, 24]]])
</code></pre>
<p>I want to change it to a tensor like this:</p>
<pre><code class="lang-auto">[[ [[1, 2],
    [7, 8]],

   [[3, 4],
    [9, 10]],

   [[5, 6],
    [11, 12]],

   [[13, 14],
    [19, 20]],

   [[15, 16],
    [21, 22]],

   [[17, 18],
    [23, 24]] ]]
   
</code></pre>
<p>Is it possible?</p>
<p>if we use this code:</p>
<pre><code class="lang-auto">print(a.view(1, 6, 2, 2))
</code></pre>
<p>the result would be as bellow that <strong>is not</strong> what I need:</p>
<pre><code class="lang-auto">tensor([[[[ 1,  2],
          [ 3,  4]],

         [[ 5,  6],
          [ 7,  8]],

         [[ 9, 10],
          [11, 12]],

         [[13, 14],
          [15, 16]],

         [[17, 18],
          [19, 20]],

         [[21, 22],
          [23, 24]]]])
</code></pre>
<p>Is there any idea to have such result?</p>
<p>Thanks</p>

Nikronic: <p>Hi,</p>
<p>It worked for me; I hope it helps:</p>
<pre><code class="lang-python">a = torch.tensor([[[ 1,  2,  3,  4,  5,  6],
                   [ 7,  8,  9, 10, 11, 12],
                   [13, 14, 15, 16, 17, 18],
                   [19, 20, 21, 22, 23, 24]]])
z= a.unsqueeze(0).unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(1, 6, 2, 2)

</code></pre>
<p>Bests</p>

zahra: <p>Thanks, Would you please explain about the code?</p>
<p>Best Regards</p>

Nikronic: <p>Yes, sure,</p>
<p>First, the tensor <code>a</code> your provided has size <code>[1, 4, 6]</code> so <code>unsqueeze(0)</code> will add a dimension to tensor so we have now <code>[1, 1, 4, 6]</code>.<br>
<code>.unfold(dim, size, stride)</code> will extract patches regarding the sizes. So first <code>unfold</code> will convert <code>a</code> to a tensor with size <code>[1, 1, 2, 6, 2]</code> and it means our <code>unfold</code> function extracted two 6x2 patches regarding the dimension with value <code>4</code>. Then we just discard first redundant dimension created by <code>unfold</code> using <code>[0]</code>. And finally, second <code>unfold</code> will extract 2x2 patches regarding the dimension with value <code>6</code>. So for sure it will create 3 patches out of a 6x2 tensor <code>(3x2x2=6x2)</code>.<br>
Now we have something like <code>[1, 2, 3, 2, 2]</code> and <code>.contiguous().view(1, 6, 2, 2)</code> will reshape our tensor to the desired one.</p>

Nikronic: <p>Could you please remove <code>autograd</code> tag from the question?</p>
<p><code>autograd</code> questions are about the autograd engine itself and it’s semantic not functions about tensors and manipulation of them.</p>
<p>Thanks</p>

zahra: <aside class="quote no-group quote-modified" data-username="Nikronic" data-post="3" data-topic="51459">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/48/57934_2.png" class="avatar"> Nikronic:</div>
<blockquote>
<p>Could you please remove  <code>autograd</code>  tag from the question?</p>
<p><code>autograd</code>  questions are about the autograd engine itself and it’s semantic not functions about tensors and manipulation of them.</p>
<p>Thanks</p>
</blockquote>
</aside>
<p>Yes, of course. Thanks for your reminder.</p>

zahra: <p>Thanks for comprehensive explanation.<br>
I want do the same task in batch mode; suppose I have a tensor like this:</p>
<pre><code class="lang-auto">a = torch.tensor([[[[ 1,  2,  3,  4,  5,  6],
                   [ 7,  8,  9, 10, 11, 12],
                   [13, 14, 15, 16, 17, 18],
                   [19, 20, 21, 22, 23, 24]]],
                  
                 [[[25,  26,  27, 28, 29, 30],
                   [31,  32,  33, 34, 35, 36],
                   [37, 38, 39, 37, 38, 39],
                   [40, 41, 42, 43, 44, 45]]]])
</code></pre>
<p>I want to have sth like this as a result:</p>
<pre><code class="lang-auto">tensor([[[[ 1,  2],
          [ 3,  4]],

         [[ 5,  6],
          [ 7,  8]],

         [[ 9, 10],
          [11, 12]],

         [[13, 14],
          [15, 16]],

         [[17, 18],
          [19, 20]],

         [[21, 22],
          [23, 24]]],
------------------------------
         [[[25, 26],
          [27, 28]],

         [[29, 30],
          [31, 32]],

         [[33, 34],
          [35, 36]],

         [[37, 38],
          [39, 37]],

         [[38, 39],
          [40, 41]],

         [[42, 43],
          [44, 45]]]])
</code></pre>
<p>when I use this code:</p>
<pre><code class="lang-auto">z= a.unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(1, 6, 2, 2)
</code></pre>
<p>Just the first matrix of tensor would be affected. Would you please guide me about it, too?</p>
<p>Thanks</p>

Nikronic: <p>Hi, you’re welcome.</p>
<p>The code is actually same, because for first answer, I considered your tensor as a batch of tensor which only have ONE tensor by using <code>squeeeze(0)</code>. So still same code will works:</p>
<pre><code class="lang-python">a.unsqueeze(0).unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(2, 6, 2, 2)
</code></pre>
<p>If I want to explain it in precise manner, extracting patches is all about playing with number of <code>unfold()</code> you consecutively use and the arguments of it which are <code>dim</code>, <code>size</code> and <code>stride</code>. The most important one is <code>dim</code> because it extract patches regarding that dimension.</p>
<p>PS.: I just changed first arg of view from <code>1</code> to <code>2</code>.</p>
<p>best regards<br>
Nik</p>

zahra: <p>Hi, thanks for your favor.</p>
<p>I tested what you said as below:</p>
<pre><code class="lang-auto">import torch
a = torch.tensor([[[[ 1,  2,  3,  4,  5,  6],
                   [ 7,  8,  9, 10, 11, 12],
                   [13, 14, 15, 16, 17, 18],
                   [19, 20, 21, 22, 23, 24]]],
                  
                 [[[25,  26,  27, 28, 29, 30],
                   [31,  32,  33, 34, 35, 36],
                   [37, 38, 39, 37, 38, 39],
                   [40, 41, 42, 43, 44, 45]]]])

z= a.unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(2, 6, 2, 2)
print(z)
</code></pre>
<p>but I received this error:</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-6-de88f8290113&gt; in &lt;module&gt;()
     10                    [40, 41, 42, 43, 44, 45]]]])
     11 
---&gt; 12 z= a.unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(2, 6, 2, 2)
     13 print(z)

RuntimeError: shape '[2, 6, 2, 2]' is invalid for input of size 24
</code></pre>
<p>Would you please look at it again?</p>
<p>Many thanks</p>

Nikronic: <aside class="quote no-group" data-username="Nikronic" data-post="8" data-topic="51459">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/48/71873_2.png" class="avatar"> Nikronic:</div>
<blockquote>
<p>a.unsqueeze(0).unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(2, 6, 2, 2)</p>
</blockquote>
</aside>
<p>add <code>unsqueeze(0)</code> at first. I have considered your tensors as images.</p>

zahra: <aside class="quote no-group" data-username="Nikronic" data-post="10" data-topic="51459">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/48/71873_2.png" class="avatar"> Nikronic:</div>
<blockquote>
<p>a.unsqueeze(0).unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(2, 6, 2, 2)</p>
</blockquote>
</aside>
<pre><code class="lang-auto">import torch
a = torch.tensor([[[[ 1,  2,  3,  4,  5,  6],
                   [ 7,  8,  9, 10, 11, 12],
                   [13, 14, 15, 16, 17, 18],
                   [19, 20, 21, 22, 23, 24]]],
                  
                 [[[25,  26,  27, 28, 29, 30],
                   [31,  32,  33, 34, 35, 36],
                   [37, 38, 39, 37, 38, 39],
                   [40, 41, 42, 43, 44, 45]]]])

z= a.unsqueeze(0).unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(2, 6, 2, 2)
print(z)

</code></pre>
<p>I received this error:</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-9-9e80a02d4148&gt; in &lt;module&gt;()
     10                    [40, 41, 42, 43, 44, 45]]]])
     11 
---&gt; 12 z= a.unsqueeze(0).unfold(2, 2, 2)[0].unfold(2, 2, 2).contiguous().view(2, 6, 2, 2)
     13 print(z)

RuntimeError: invalid argument 3: out of range at ..\aten\src\TH/generic/THTensor.cpp:392
</code></pre>
<p>Please attention that the dimension of tensor is torch.Size([2, 1, 4, 6]).<br>
I would appreciate it if you check it again.</p>
<p>Thanks a lot</p>

Nikronic: <aside class="quote no-group" data-username="zahra" data-post="1" data-topic="51459">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/aeb1de/48.png" class="avatar"> zahra:</div>
<blockquote>
<p>a = torch.tensor([[[ 1, 2, 3, 4, 5, 6], [ 7, 8, 9, 10, 11, 12], [13, 14, 15, 16, 17, 18], [19, 20, 21, 22, 23, 24]]])</p>
</blockquote>
</aside>
<p>Ow, sorry.<br>
I used the tensor in the first post.</p>
<pre data-code-wrap="python"><code class="lang-python">
a.unfold(2, 2,2).unfold(3, 2,2).contiguous().view(2, 6, 2, 2)
</code></pre>
<p>by the way, as I told you only need to work with <code>dim</code> and number of <code>unfold</code> functions. Easy does it.</p>
<p>Bests<br>
Nik</p>

zahra: <aside class="quote no-group" data-username="Nikronic" data-post="12" data-topic="51459">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/48/71873_2.png" class="avatar"> Nikronic:</div>
<blockquote>
<p>a.unfold(2, 2,2).unfold(3, 2,2).contiguous().view(2, 6, 2, 2)</p>
</blockquote>
</aside>
<p>Thanks Nik, it could help me a lot.<br>
Best Regards</p>

janina_alc: <p>Hi there, I was following up on the topic and could transform my tensor to [8, 1024, 169], however I’d need to transform it to [8, 1024, 13, 13].</p>
<p>Does anyone have an idea? Thank you very much in advance!</p>

ptrblck: <p>If your tensor already has the shape <code>[8, 1024, 169]</code>, you could use the <code>view</code> operations as: <code>x.view(8, 1024, 13, 13)</code> to create the desired shape.</p>

janina_alc: <p>oh thanks a lot for the solution!</p>
<p>Sorry for another question about chaging my tensor dimensions…I have another tensor, which I was able to transform to torch.Size([16, 1024, 9, 9]), but similar to the other I would need it to be transformed to torch.Size([16, 1024, 13, 13]). I tried some but haven’t found the solution yet <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> And thank you again!</p>

ptrblck: <p>You won’t be able to reshape/view the input tensor of <code>[16, 1024, 9, 9]</code> into the shape <code>[16, 1024, 13, 13]</code>, since the latter contains more elements.<br>
However, you could use an interpolation method via e.g. <code>nn.Upsample</code> or <code>F.interpolate</code> or alternatively e.g. transposed convolution layers.<br>
Let me know, if one of these approaches would work for you.</p>