Permute elements of a tensor along a dimension

Nabarun_Goswami: <p>Hi,</p>
<p>I am trying to permute the feature map dimension in a tensor. As a very simplified case,</p>
<p>If I have a tensor of size (5, 4, 3, 6)<br>
I want to rearrange the above tensor along its dimension 1 (i.e. 4) from 0,1,2,3 to 0,2,1,3</p>
<p>One possible way I found was to do a index_select followed by cat. But, in case of a larger tensor, a lot of intermediate tensors will have to be created.</p>
<p>Is there a better way to achieve this?</p>
<p>Regards<br>
Nabarun</p>

apaszke: <p>I think this should do it: <code>input[:, perm]</code> where <code>perm</code> is a LongTensor that containes the permutation.</p>

Nabarun_Goswami: <p>OK, so this is what I tried:</p>
<pre><code>&gt;&gt;&gt; n = np.arange(480).reshape((5,4,4,6))
&gt;&gt;&gt; a = torch.from_numpy(n)
&gt;&gt;&gt; perm = torch.LongTensor([0,2,1,3])
&gt;&gt;&gt; a[:, perm, :, :]
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
TypeError: indexing a tensor with an object of type LongTensor. The only supported types are integers, slices, numpy scalars and torch.LongTensor or torch.ByteTensor as the only argument.
</code></pre>
<p>Shouldn’t perm be a torch.LongTensor? Am I missing something here?</p>

apaszke: <p>It will work, but you need to remove the last <code>:, :</code>. They don’t do anything, and for the moment we require LongTensors to appear as the last elements in the index.</p>

Nabarun_Goswami: <p>Getting the same error even when I remove the trailing :, :.</p>

Nabarun_Goswami: <p>I just realized the error states: <strong>torch.LongTensor or torch.ByteTensor as the only argument</strong> at the very end. So, I guess <code>input[:, perm]</code> will not work.</p>
<p>I can probably do a dimension permute to bring the required dimension to dim0 and do the permutation and dimension permute back to original order.</p>
<p>Seems like a hacky way to do it though.</p>

apaszke: <p>Ah yes, you’re right. We should have full indexing implemented soon.</p>

harpone: <p>Just a FYI: this is now working. However, I would be interested in a case where the permutation is <em>different</em> for each row (first axis)… any cool advanced indexing trick that would do it?</p>

tetratrio: <p>This answer is a bit late but I stumbled upon this thread while trying to create a permutation matrix from permutation indexes so I’ll just share what I found works.</p>
<pre><code class="lang-auto">permutation_indices = ...
size = permutation_indices.size(0)
permutation_matrix = torch.zeros(size, size)
permutation_matrix[torch.arange(size).long(), permutation_indices] = 1
</code></pre>

harpone: <p>Not quite what I (or the OP) was looking for but thanks for the effort!</p>
<p>Also, you can achieve the same you’re doing by <code>torch.eye(permutation_indices.size(0))[permutation_indices]</code></p>

tetratrio: <p>Oh yea thats an even simpler approach. Why didnt think of that. Thanks!</p>

Tim_Oleskiw: <p>Yeah as above, permutations can be achieved via matrix multiplication.  [a b] * [0 1; 1 0] =&gt; [b a], for example.</p>

cuixing158_1: <p>I do like this:</p>
<p>n = np.arange(480).reshape((5,4,4,6))<br>
a = torch.from_numpy(n)<br>
perm = a.permute((3,0,1,2))</p>

Daniel_Povey: <p>Regarding how to permute vectors in PyTorch: there seems to be a function pytorch.permute(), but I can’t find any documentation for it, and when I try it doesn’t seem to work as I might expect (seems to be a no-op).</p>

ptrblck: <p>The function is documented in <a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor.permute" rel="nofollow noopener">torch.Tensor.permute</a>. Could you post some code where you think this might be a no-op?<br>
Here is a small example using it:</p>
<pre><code class="lang-python">x = torch.randn(1, 2, 3)
print(x.shape)
&gt; torch.Size([1, 2, 3])
x = x.permute(1, 2, 0)
&gt; torch.Size([2, 3, 1])
</code></pre>
<p>print(x.shape)</p>

Daniel_Povey: <p>Oh, I see.  I guess i must have found the wrong version of the documentation when I searched online.  I was giving a second arg which was a LongTensor that I was expecting it to use for indexing, but which I now see was ignored.</p>
<p>x = torch.randn(3,3)</p>
<blockquote>
<blockquote>
<blockquote>
<p>x</p>
</blockquote>
</blockquote>
</blockquote>
<p>tensor([[-1.4109, -0.0597, -0.8855],</p>
<p>[-0.6355, -0.6556, -1.9610],</p>
<p>[ 1.0115,  1.5676,  0.8374]])</p>
<blockquote>
<blockquote>
<blockquote>
<p>x.permute((0,1), torch.LongTensor([2, 1, 0]))</p>
</blockquote>
</blockquote>
</blockquote>
<h1><a name="gives-the-same-as-x-1" class="anchor" href="#gives-the-same-as-x-1"></a>gives the same as x.</h1>
<p>I see now that what I was trying to do, you are supposed to do using indexing operators.</p>

Yuheng_Zhi: <p>Did you want something like this?</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; x = torch.randn(100, 57, 2)
&gt;&gt;&gt; idx = torch.randint(57, size=(100, 57))
&gt;&gt;&gt; permutated_x = x.gather(dim=1, index=idx.unsqueeze(-1).expand(x.shape))
&gt;&gt;&gt; torch.all(x[0][idx[0]] == permutated_x[0])
tensor(True)
&gt;&gt;&gt; torch.all(x[28][idx[28]] == permutated_x[28])
tensor(True)
</code></pre>