How to do linear indexing and what is the formula?

username: Glerk
cooked: <p>I didn’t see any formula that explained how reshape or view works, I’m mostly interested in linearly indexing an existing tensor. So for example:</p>
<p>Suppose you have a k dimensional tensor A, with shape (n1,n2,n3,…nk) and you reshape it to a 1 dimensional tensor B, what is the formula for associating the linear index of B to the k dimensional index of A?</p>

username: JuanFMontesinos
cooked: <p>I don’t really know the formula but it basically works the following way:<br>
Assume that the number of elements of A and B are the same.<br>
so what id does is iterating through the dimensions of A starting by the outer one and filling with that B<br>
B[0]=A[0,0,0,0]<br>
B[1]=A[0,0,0,1]<br>
B[nk-1]=A[0,0,0,nk-1]<br>
B[nk]=A[0,0,1,0]<br>
B[nk+1]=A[0,0,1,1]<br>
and so on<br>
Note that nk-1 corresponds to the zero-index notation so that if an array has N elements the last index is N-1.</p>
<p>You can check numpy docs<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <a href="https://numpy.org/doc/stable/reference/generated/numpy.reshape.html" target="_blank" rel="noopener nofollow ugc">numpy.org</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail" width="" height="">

<h3><a href="https://numpy.org/doc/stable/reference/generated/numpy.reshape.html" target="_blank" rel="noopener nofollow ugc">numpy.reshape — NumPy v1.19 Manual</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
Since the logics are the same. There exist 2 types of ordering, fortran and c. In pytorch there is only C.<br>
So the diff is ordering from outer to inner or inner to outer.

username: Glerk
cooked: <p>Thanks, the numpy documentation is much better and I understand it.</p>
<p>Do you know if you there is a one to one correspondence between reshape and linear indexing? By this I mean suppose we reshape a tensor A to a 1d array B, and then reshape to another tensor C, does A reshape -&gt; B -&gt; reshape C, the same as A reshape -&gt; C? Assuming the shape of C is the same for both?</p>

username: JuanFMontesinos
cooked: <p>Yes, I mean, it’s something deterministic and ordered. You may want to have a look to this.</p><aside class="quote" data-post="2" data-topic="30749">
  <div class="title">
    <div class="quote-controls"></div>
    <img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/difference-between-2-reshaping-operations-reshape-vs-permute/30749/2">Difference between 2 reshaping operations (reshape vs permute)</a> <a class="badge-category__wrapper " href="/c/nlp/8"><span data-category-id="8" style="--category-badge-color: #AB9364; --category-badge-text-color: #FFFFFF;" data-drop-close="true" class="badge-category " title="Topics related to Natural Language Processing"><span class="badge-category__name">nlp</span></span></a>
  </div>
  <blockquote>
    Im not sure about reshape. 
Permute is a multidimensional rotation saying somehow. It keeps the data ordering. View (which is another reshaping method) maps from one dimensionality to another sequentially reading data from the upper dimensions to the lower ones. 
So if you want fuse two dimensions into one, you have to apply it over contiguous dimensions or u will modify the data ordering
  </blockquote>
</aside>
<p>
It’s really linked to how many dimensions does each dimension has.<br>
Think that it works in a “FIFO” way. the 1st element of B will be also the 1st element taken for C.<br>
So in the end A–&gt;C is same than A–&gt;B–&gt;C if you don’t modify the ordering of the elements in B.</p>
<p>Lastly think that, since it’s ordered, it respects spatial structure. What does this mean?<br>
Imagine you have a tensor<br>
A (2,5,7,7)<br>
You reshape it as<br>
A’(5<em>2,1,7,7)<br>
Then you apply a convolution and get<br>
B’ (5</em>2,1,5,5)<br>
if you reshape B back to<br>
B ( 5,2,5,5)<br>
The spatial structure of H and W remains since the ordering of the elements is equal to the ordering of A.</p>