How to correctly transform a 2-dimensional tensor into a 4 dimensional matrix

intern: <p>I have a 2-dimensional lookup/weight table of size vocab_size x channel size. I want to transform this tensor to be 4-dimensional (in order for it work with F.grid_sample).</p>
<p>I have done variations of reshape(), view(), unsqueezing &amp; permute()</p>
<pre><code class="lang-auto">import torch

tensor_2d = torch.rand(65000,128) #just a random lookup table has to be of this size

tensor_4d = tensor_2d.reshape(1,128,65000,1) # when I do this the channel dimension gets messed up 

tensor_4d = tensor_2d.unsqueeze(0).unsqueeze(-1).permute(0,2,1,3) #when I do this the channel dimension is filled with the values of the subsequent row at that index 

how can I make it so that the values within the rows of the original tensor stay the same? I


</code></pre>

KFrank: <p>Hi Intern!</p>
<aside class="quote no-group quote-modified" data-username="intern" data-post="1" data-topic="204397" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/intern/48/61191_2.png" class="avatar"> intern:</div>
<blockquote>
<p>I want to transform this tensor to be 4-dimensional (in order for it work with F.grid_sample).</p>
<pre><code class="lang-auto">tensor_4d = tensor_2d.unsqueeze(0).unsqueeze(-1).permute(0,2,1,3) #when I do this the channel dimension is filled with the values of the subsequent row at that index 
</code></pre>
</blockquote>
</aside>
<p>The <code>permute()</code> version of your 4d conversion works the way I would<br>
expect and would seem to work for what I imagine your use case to be:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'2.3.1'
&gt;&gt;&gt; L = 5
&gt;&gt;&gt; C = 2
&gt;&gt;&gt; tensor_2d = torch.arange (L * C).float().reshape (L, C)
&gt;&gt;&gt; tensor_2d
tensor([[0., 1.],
        [2., 3.],
        [4., 5.],
        [6., 7.],
        [8., 9.]])
&gt;&gt;&gt; tensor_4d = tensor_2d.unsqueeze (0).unsqueeze (-1).permute (0, 2, 1, 3)
&gt;&gt;&gt; tensor_4d
tensor([[[[0.],
          [2.],
          [4.],
          [6.],
          [8.]],

         [[1.],
          [3.],
          [5.],
          [7.],
          [9.]]]])
&gt;&gt;&gt; tensor_4d[0, :, :, 0]
tensor([[0., 2., 4., 6., 8.],
        [1., 3., 5., 7., 9.]])
</code></pre>
<p>If this isn’t what you want, could you explain what is wrong and illustrate<br>
your use case with a small, fully-self-contained, runnable script, together<br>
with the output you get when you run it?</p>
<p>Best.</p>
<p>K. Frank</p>

intern: <p>Hi K. Frank,</p>
<p>Thank you for your reply! I have attached a simple script. I am trying to get the e_output to match g_output. I have also attached a visual illustration of what I am trying to do with the matrix as well. Using permute, I end up bunching together the first value from</p>
<p>I appreciate the insight and help!</p>
<pre><code class="lang-auto">import torch 


L = 60000
C = 128


tensor_2D = torch.randn(L,C)


tensor_4D = tensor_2D.data.unsqueeze(0).unsqueeze(-1).permute(0,2,1,3)


tensor_2D, tensor_4D[0][0], tensor_2D[0]

Output:
(tensor([[-0.9203, -0.5730,  0.4688,  ..., -1.6398, -0.3495, -0.0658],
         [ 0.6133, -0.2353, -0.6844,  ...,  0.0541, -0.2642, -1.6251],
         [ 0.7498, -0.5778, -1.2933,  ...,  0.6337, -0.8241,  1.9411],
         ...,
         [-0.5282,  2.1966,  0.9805,  ..., -0.6725, -0.6689,  1.1485],
         [-0.7611,  0.0720,  0.1070,  ...,  1.4691,  0.5665, -0.4897],
         [ 0.4414,  0.0610, -1.5197,  ..., -1.4009, -0.4101,  0.1082]]),
 tensor([[-0.9203],
         [ 0.6133],
         [ 0.7498],
         ...,
         [-0.5282],
         [-0.7611],
         [ 0.4414]]),
 tensor([-0.9203, -0.5730,  0.4688,  0.7558, -0.0483, -0.7688, -0.0693,  1.0376,
          2.4042, -0.3007, -0.8850,  0.3835,  0.8781,  0.3174, -1.0734, -0.0287,
         -1.4459, -0.6875,  0.3105,  0.9651,  1.9093,  0.1718, -0.7249,  0.8602,
          1.1302,  0.4596, -0.1718, -0.4428, -0.1168, -1.0659,  0.3917,  0.0737,
          1.2247, -1.6025,  0.4712,  0.5392, -0.8835,  1.7168,  0.3307, -0.0836,
          1.7518,  0.5871, -0.0171,  0.1820, -0.0171, -0.3980, -0.3315, -0.6369,
          0.5448, -0.4139, -1.6431, -0.3533, -0.5043,  0.3944,  1.3845, -0.4939,
         -1.0925,  0.0763, -0.9511,  1.2899,  1.5805,  0.7202,  0.1146,  1.1339,
         -0.7482,  2.1890,  0.3551, -1.9820,  0.5496, -0.6619, -1.0239,  1.1724,
          1.1881, -1.3345,  0.0237,  1.9488, -1.5924, -0.3576,  0.2485,  0.4805,
          1.8664, -0.8312,  1.1447, -1.2070, -1.0071,  0.4795, -0.9977,  0.2653,
          0.8654, -0.5748,  0.9977, -0.5095, -0.8119, -0.5604,  0.9006,  0.3391,
          1.5522,  1.1773,  0.0244, -1.4265,  0.0526,  0.3777,  1.6553, -0.4786,
         -0.9769, -0.6977, -1.0240,  1.5423,  0.8922,  1.3467, -0.4278,  1.3876,
         -1.5790, -1.1220,  2.0624,  0.6103,  0.7713,  0.2780,  0.9061, -1.3642,
          0.7891, -2.5755, -0.4350,  0.0559,  0.7300, -1.6398, -0.3495, -0.0658]))
</code></pre>
<p>As you can see, instead of being a culmination of the rows, it is instead a culmination of the columns. I would like for the tensor_4D[0][0] to contain the same values as tensor_2D[0].</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/1/91b7613215f9895eb87f547c36816e0ddc43ce51.png" data-download-href="https://discuss.pytorch.org/uploads/default/91b7613215f9895eb87f547c36816e0ddc43ce51" title="Screenshot 2024-06-12 at 10.09.55 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/1/91b7613215f9895eb87f547c36816e0ddc43ce51_2_690x337.png" alt="Screenshot 2024-06-12 at 10.09.55 PM" data-base62-sha1="kN47AiKVf2rCbyRo2h0akMJBr0Z" width="690" height="337" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/1/91b7613215f9895eb87f547c36816e0ddc43ce51_2_690x337.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/1/91b7613215f9895eb87f547c36816e0ddc43ce51_2_1035x505.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/1/91b7613215f9895eb87f547c36816e0ddc43ce51_2_1380x674.png 2x" data-dominant-color="E1EBF3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2024-06-12 at 10.09.55 PM</span><span class="informations">1636×800 19.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

KFrank: <p>Hi Intern!</p>
<aside class="quote no-group quote-modified" data-username="intern" data-post="3" data-topic="204397" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/intern/48/61191_2.png" class="avatar"> intern:</div>
<blockquote>
<p>Using permute, I end up bunching together the first value from<br>
…<br>
I would like for the tensor_4D[0][0] to contain the same values as tensor_2D[0].</p>
</blockquote>
</aside>
<p>If you want <code>tensor_4D[0][0]</code> and <code>tensor_2D[0]</code> to contain the same<br>
values, simply leave out the <code>.permute()</code> (although I’m still not sure I<br>
understand what you want):</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'2.3.1'
&gt;&gt;&gt; L = 5
&gt;&gt;&gt; C = 2
&gt;&gt;&gt; tensor_2d = torch.arange (L * C).float().reshape (L, C)
&gt;&gt;&gt; tensor_2d
tensor([[0., 1.],
        [2., 3.],
        [4., 5.],
        [6., 7.],
        [8., 9.]])
&gt;&gt;&gt; tensor_4d = tensor_2d.unsqueeze (0).unsqueeze (-1)
&gt;&gt;&gt; tensor_4d
tensor([[[[0.],
          [1.]],

         [[2.],
          [3.]],

         [[4.],
          [5.]],

         [[6.],
          [7.]],

         [[8.],
          [9.]]]])
&gt;&gt;&gt; tensor_4d[0][0]
tensor([[0.],
        [1.]])
&gt;&gt;&gt; tensor_2d[0]
tensor([0., 1.])
</code></pre>
<p>To my mind, the question is whether you want <code>grid_sample()</code> to sample<br>
along the vocab_size dimension or along the channel_size dimension.<br>
That would seem to be what determines whether you want the <code>.permute()</code><br>
or not.</p>
<p>Best.</p>
<p>K. Frank</p>