How to flatten a tensor in column-major order?

brudfors: <p>I would like to flatten a tensor into column-major order. What is the optimal way for achieving this in PyTorch? Row-major order seems to be the default in PyTorch’s flatten function, and I don’t think there is an order option like in Numpy’s flatten function. Thanks!</p>

donJuan: <p>Use tranpose method to reshape your tensor then flatten i.e.</p>
<pre><code>x = torch.tensor([[1,2,3],[4,5,6],[7,8,9]])
x.flatten()
tensor([1, 2, 3, 4, 5, 6, 7, 8, 9])
x.transpose(1, 0).flatten()
tensor([1, 4, 7, 2, 5, 8, 3, 6, 9])</code></pre>

brudfors: <p>Thank you for your reply!</p>
<p>That works well for matrices (where the transpose can be written as simply <code>x.t().flatten()</code>)</p>
<p>However, I am dealing with ndarrays, so that a matrix transpose is not applicable.</p>

brudfors: <p>I just found that the solution had already been posted on this forum, here:</p>
<aside class="quote" data-post="1" data-topic="17670">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spaceinvader/48/4137_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/pytorch-tensor-to-fortran-style-ordering/17670">PyTorch tensor to fortran-style ordering</a> 
  </div>
  <blockquote>
    I have already asked a similar question, but I’ll try to reformulate it. 
My question now is how to reorder PyTorch tensor data in Fortran-style order like np.reshape(order=‘F’) in numpy? 
I need column-major ordering to use CUDA functions from cusparse library.
  </blockquote>
</aside>

<p>The answer was:</p>
<pre><code class="lang-auto">t = torch.rand(3, 3, 3)

# convert to column-major order
t.set_(t.storage(), t.storage_offset(), t.size(), tuple(reversed(t.stride())))  
t.flatten()  # 1D array in column-major order
</code></pre>
<p>Note that if you just want a tensor’s 1D representation in column-major order, the above operation will change the ordering of the elements in tensor <code>t</code>. This function will pull out just the flattened array in column-major order:</p>
<pre><code class="lang-auto">def flatten_fortran_w_clone(t):    
    nt = t.clone()
    nt = nt.set_(nt.storage(), nt.storage_offset(), nt.size(), tuple(reversed(nt.stride())))    
    return nt.flatten()
</code></pre>
<p>But there might be a better way of doing it without having to create a copy…</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/brudfors">@brudfors</a>,</p>
<p>Your <code>flatten_fortran_w_clone</code> functiton doesn’t work for a 2D-array, for example,</p>
<pre><code class="lang-auto">def flatten_fortran_w_clone(t):    
  nt = t.clone()
  nt = nt.set_(nt.storage(), nt.storage_offset(), nt.size(), tuple(reversed(nt.stride())))    
  return nt.flatten()
 
x=torch.arange(6).reshape(3,2)
#returns tensor([[0, 1],
#                [2, 3],
#                [4, 5]])
flatten_fortran_w_clone(x) #returns tensor([0, 2, 1, 3, 2, 4]) #incorrect
x.transpose(-2,-1).flatten() #returns tensor([0, 2, 4, 1, 3, 5]) #correct
</code></pre>
<p>Also, is there a way to invert this process? For example, column-major flatten an array then reshape (via column-major formatting, instead of row-major formatting) to get back to the original array?</p>
<p>Also, for anyone reading this <code>nt.storage()</code> is now deprecated and should be replaced with <code>nt.untyped_storage()</code> instead, i.e.,</p>
<pre><code class="lang-auto">def flatten_fortran_w_clone(t):    
  nt = t.clone()
  nt = nt.set_(nt.untyped_storage(), nt.storage_offset(), nt.size(), tuple(reversed(nt.stride())))    
  return nt.flatten()
</code></pre>