Difference between 2 reshaping operations (reshape vs permute)

coyote: <p>Hi everyone,</p>
<p>Recently, I asked <a href="https://discuss.pytorch.org/t/how-to-change-shape-of-a-matrix-without-dispositioning-the-elements/30708">this</a> question. Though I got an answer for my original question, last <a href="https://discuss.pytorch.org/t/how-to-change-shape-of-a-matrix-without-dispositioning-the-elements/30708/4?u=coyote">comment</a> confused me a little bit.</p>
<p>I don’t understand the difference between these two cases:</p>
<ol>
<li>According to answers, this is a safe operation:</li>
</ol>
<pre><code class="lang-auto">bs,seq_len,input_size= 5,20,128
x = torch.rand(bs,seq_len,input_size)
torch.reshape(x,(x.shape[0],x.shape[1],1,x.shape[2])
</code></pre>
<ol start="2">
<li>However, if I want to change the shape <code>(bs, seq_len, input_size)</code> matrix <code>x</code> into <code>(seq_len, bs, input_size)</code>, it is said that I should use <code>x.permute(1, 0, 2)</code> rather than reshape.</li>
</ol>
<p>Why these two cases differs from each other ?</p>

JuanFMontesinos: <p>Im not sure about reshape.<br>
Permute is a multidimensional rotation saying somehow. It keeps the data ordering. View (which is another reshaping method) maps from one dimensionality to another sequentially reading data from the upper dimensions to the lower ones.</p>
<p>So if you want fuse two dimensions into one, you have to apply it over contiguous dimensions or u will modify the data ordering</p>

coyote: <aside class="quote no-group" data-username="JuanFMontesinos" data-post="2" data-topic="30749">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>So if you want fuse two dimensions into one, you have to apply it over contiguous dimensions or u will modify the data ordering</p>
</blockquote>
</aside>
<p>Thanks for the reply. How do I make sure that I am doing it over contiguous dimensions or not ? Could you, if possible, give me an example on a small tenser with a contiguous and non-contiguous dimensions ?</p>

JuanFMontesinos: <p>if u have a cube</p>
<pre><code class="lang-auto">c=torch.rand(3,4,5)
</code></pre>
<p>and you use permute</p>
<pre><code class="lang-auto">c=torch.rand(3,4,5)
rx  = c.permute(0,2,1)
ry  = c.permute(2,1,0)
rz = c.permute(1,0,2)
print(rx.size())
print(ry.size())
print(rz.size())
</code></pre>
<pre><code class="lang-auto">torch.Size([3, 5, 4])
torch.Size([5, 4, 3])
torch.Size([4, 3, 5])
</code></pre>
<p>you are just rotating the tensor, but order is preserved<br>
On the other hand, if you reshape you can see you are modifying the ordering because this is not rotating the cube but mapping in an ordered way from right to left. It takes numbers until it fills the dimensions.</p>
<pre><code class="lang-auto">sx = c.view(3,5,4)
rx - sx
</code></pre>
<p>That’s why this operation is different from 0</p>
<p>So an example about how to apply view could be the following one</p>
<p>if you have a tensor m= BxNxHxW<br>
and these tensor contains B batches of N images whose size is HxW and you want to make a montage of these images in a single one concatanating in the colums your outgoing dimension would be</p>
<p>B,H,WxN which is equivalent to B,H,NxW</p>
<p>So lets see what happens if you reshape vs permute + reshape vs permute without paying attention</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/f/fb7da18bfd13aca564475e012fdda6467b8192a2.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/fb7da18bfd13aca564475e012fdda6467b8192a2" title="cat"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/f/fb7da18bfd13aca564475e012fdda6467b8192a2_2_690x459.jpeg" alt="cat" data-base62-sha1="zSN0w5UNZwFsNHeKio3NfTnXy0y" width="690" height="459" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/f/fb7da18bfd13aca564475e012fdda6467b8192a2_2_690x459.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/f/fb7da18bfd13aca564475e012fdda6467b8192a2_2_1035x688.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/f/fb7da18bfd13aca564475e012fdda6467b8192a2_2_1380x918.jpeg 2x" data-dominant-color="968273"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">cat</span><span class="informations">1900×1266 124 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Im gonna mix this images<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/9/925e17010afd24fafcfcadda3eb2aa5cdf4781d7.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/925e17010afd24fafcfcadda3eb2aa5cdf4781d7" title="dog"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/9/925e17010afd24fafcfcadda3eb2aa5cdf4781d7_2_690x387.jpeg" alt="dog" data-base62-sha1="kSPiqKbSPkLqaD5il3eK684KUHZ" width="690" height="387" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/9/925e17010afd24fafcfcadda3eb2aa5cdf4781d7_2_690x387.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/9/925e17010afd24fafcfcadda3eb2aa5cdf4781d7_2_1035x580.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/9/925e17010afd24fafcfcadda3eb2aa5cdf4781d7.jpeg 2x" data-dominant-color="AEB7C5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">dog</span><span class="informations">1100×618 23.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>with this code</p>
<pre><code class="lang-auto">import imageio as skio
import matplotlib.pyplot as plt
import numpy as np
import torch
im1 = np.mean(skio.imread('/home/jfm/Downloads/dog.jpg'),axis=2)
im2 = np.mean(skio.imread('/home/jfm/Downloads/cat.jpg'),axis=2)[:618,:1100]

m = np.stack([im1,im2])
m = torch.from_numpy(np.stack([m,m,m]))
print(m.size())
#torch.Size([3, 2, 618, 1100])


m_reshape = m.view(3,618,1100*2).numpy()

m_permute_wrong = m.permute(0,2,3,1).contiguous().view(3,618,1100*2).numpy()
m_permute_right = m.permute(0,2,1,3).contiguous().view(3,618,1100*2).numpy()
</code></pre>
<p>I’m converting RGB images to gray and croping to have same size<br>
Then creating 3 batches of 2 images<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/0/0df047c473dcef3041c4172417f896953a238678.png" alt="image" data-base62-sha1="1Zj0kExvzCPS0x1BK2mSqaHmvjy" width="378" height="223"><br>
The cat cropped looks like that (that’s grayscale)<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/e/e7f635b69fe2bc3a37978419be1f2334b4c9ee59.png" alt="image" data-base62-sha1="x61NsEi9OSeXHXwK4gQ76dLWoHf" width="378" height="223"><br>
if you just reshape you get a wrong ordering<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/f/f5725a1f2ca067513fdc7804d6edd0d9748e9512.png" alt="image" data-base62-sha1="z1jYERpZIcQmDYeFhVIUdNt8yEG" width="378" height="129"><br>
What’s going on there? as you are reordering it’s getting the information in the original order which is, all colums of image 1, all rows of image 1, all colums of image 2, all rows of image 2 and so on. If u pay attention it 's resized to be fit in the desired shape</p>
<p>if you  permute and set dimensions before reshaping<br>
but you do it wrongly<br>
you get this<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/e/e24361e285ddba0c7656a47513e1c913c31012ae.png" alt="image" data-base62-sha1="whCh0FwIE6TpN0vKui9evbUo4eq" width="378" height="129"><br>
here you are filling taking the info of one image and then the other because u set N at the right. So it takes the information of the image1, colum 1, then image2, colum 1 and so on.<br>
However if u properly order the dimensions<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/6/6d56eb802f274dca9868cd2d59c3d30c1ed4422a.png" alt="image" data-base62-sha1="fBggHpJ1k285mGgr3MqTAHwiHc6" width="378" height="129"><br>
You achieve what you want which is all the colums of image 1, all the colums of image 2</p>
<p>Moral of this,</p>
<p>If you want to reshape the ordering only remains for contiguous dimensions. [1,2,3,4]<br>
Contiguous here mens 1-2, 2-3 even 1-2-3, but not 1-3 for example.</p>
<p>To do so use permute and the brain first</p>

jon: <p>Great answer and great examples! I have a question: can you explain why the m_reshape gives us the result we get? The original dimensions are</p>
<p>B x N x H x W</p>
<p>Now,  when we reshape, we want something like:<br>
B x H x W x N , so that the images are side-by-side/concatenated together along the x-axis. Here, N = 2, so we should have two images.</p>
<p>So why do we get 4 images with the reshape? Why don’t we get something like the top half of the dog on the left, then the bottom half of the dog on the right? Thanks!</p>

JuanFMontesinos: <p>Oh it’s actually very funny.<br>
<span class="hashtag">#torch</span>.Size([3, 2, 618, 1100])<br>
In this composition you have a BatchxImagesxrowsxcolums<br>
so when you reshape it takes pixels from dimensions at the right and places them until filling new shape dimensions</p>
<p>so this means it goes to<br>
m[0,0,0,0] takes that pixel and goes to m_reshape[0,0,0]<br>
so it is filling the row 0 of m_reshape with colum pixels from m. Those pixels are dog image in fact.<br>
The problem is that in m, in dimension 3 you have 1100 elements, meanwhile in dimension 2 of m_reshape you have 2200 elements, so it actually takes 2 rows from m to fill a row of m_reshape.</p>
<p>That’s why you can see that streching efect on the image. You are basically taking dogs[0::2] to create the image on the top left and then dogs[1::2] to create image in the top right.<br>
Obviously when dog image is finished, this is, when you have already used m[0,0,:,:] it takes moves to m[0,1,:,:] to keep taking pixels. That dimension corresponds to cat and basically has same effect. The interesting point is that as you are using 2 rows of m to fill a colum of m_reshape, you are later on filling those “missing” colums with cat info creating this strange composition.</p>
<p>I hope you could understand my messy explanation</p>

mkarthick271: <p>Hi John, what is happening is that for the dog image, there are 618 rows, and to fulfill the  new size of 1100<em>2 columns, the rows are alternatively placed in the same row. .For example, 1st row remains in the same place, the 2nd row is picked and placed adjacent to this 1st row. The rows of the dog image are placed alternatively in adjacent manner due to which there are 618/2 rows in the first dog image and the remaining 618/2 rows are in the 2nd dog image. so it is basically rescaled to 314</em>2200. The same explanation goes for the cat image. Hence, the images are not cut in half horizontally but rescaled.</p>