Pytorch repeat 3rd dimension

aerinykim: <p>I’m following this example on <a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor.repeat" rel="nofollow noopener">doc</a></p>
<pre><code class="lang-auto">In [42]: x = torch.tensor([1,2,3])


In [45]: x.repeat(4,2) 
Out[45]: tensor([[1, 2, 3, 1, 2, 3],
        [1, 2, 3, 1, 2, 3],
        [1, 2, 3, 1, 2, 3],
        [1, 2, 3, 1, 2, 3]])

In [46]: x.repeat(4,2).shape 
Out[46]: torch.Size([4, 6])
</code></pre>
<p>So far, so good.</p>
<p>But why does repeating just 1 time on 3rd dimension expand 3rd dim to 3 (not 1)?</p>
<pre><code class="lang-auto">[On the doc]

&gt;&gt;&gt; x.repeat(4, 2, 1).size()
torch.Size([4, 2, 3])
</code></pre>
<p>Double checking.</p>
<pre><code class="lang-auto">In [43]: x.repeat(4,2,1)
Out[43]:
tensor([[[1, 2, 3],
         [1, 2, 3]],

        [[1, 2, 3],
         [1, 2, 3]],

        [[1, 2, 3],
         [1, 2, 3]],

        [[1, 2, 3],
         [1, 2, 3]]])
</code></pre>
<p>Why does it behave this way?</p>

vainaijr: <p>maybe, it consider unsqueezed shape, for example,</p>
<pre><code class="lang-auto">x = torch.tensor([1, 2, 3])
x.shape
</code></pre>
<p>give</p>
<pre><code class="lang-auto">torch.Size([3])
</code></pre>
<p>when we do,</p>
<pre><code class="lang-auto">x.repeat(4,2)
</code></pre>
<p>then,<br>
it consider x shape to be</p>
<pre><code class="lang-auto">torch.size([1, 3])
</code></pre>
<p>and repeat this tensor 4 times along the 1st dim, 2 times along the 2nd dim, resulting in output shape to be,</p>
<pre><code class="lang-auto">torch.size([4, 6])
</code></pre>
<p>when we do,</p>
<pre><code class="lang-auto">x.repeat(4, 2, 1)
</code></pre>
<p>then, the unsqueezed shape of x be,</p>
<pre><code class="lang-auto">torch.size([1, 1, 3])
</code></pre>
<p>and repeat this tensor 4 times along the 1st dim, 2 times along the 2nd dim, 1 time along the 3rd dim, resulting in output shape to be,</p>
<pre><code class="lang-auto">torch.size([4, 2, 3])
</code></pre>

aerinykim: <p>Thank you. Yes that’s my guess too. how should we confirm?</p>

vainaijr: <p>maybe implementation of torch.repeat is here,</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/zdevito/ATen/blob/2448a0d8b547e8c787bb948a23009c375c46272c/aten/src/ATen/native/TensorShape.cpp">
  <header class="source">

      <a href="https://github.com/zdevito/ATen/blob/2448a0d8b547e8c787bb948a23009c375c46272c/aten/src/ATen/native/TensorShape.cpp" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/zdevito/ATen/blob/2448a0d8b547e8c787bb948a23009c375c46272c/aten/src/ATen/native/TensorShape.cpp" target="_blank" rel="noopener nofollow ugc">zdevito/ATen/blob/2448a0d8b547e8c787bb948a23009c375c46272c/aten/src/ATen/native/TensorShape.cpp</a></h4>


      <pre><code class="lang-cpp">#include &lt;TH/THTensor.hpp&gt;
#include &lt;algorithm&gt;
#include &lt;vector&gt;
#include &lt;ATen/ATen.h&gt;
#include &lt;ATen/ExpandUtils.h&gt;
#include &lt;ATen/InferSize.h&gt;
#include &lt;ATen/NativeFunctions.h&gt;
#include &lt;ATen/WrapDimUtils.h&gt;
#include &lt;c10/util/Exception.h&gt;
#include &lt;c10/util/Optional.h&gt;
#include &lt;ATen/native/Resize.h&gt;
#include &lt;ATen/SparseTensorUtils.h&gt;
#include &lt;ATen/quantized/QTensorImpl.h&gt;
#include &lt;algorithm&gt;
#include &lt;vector&gt;
#include &lt;ATen/NamedTensorUtils.h&gt;
#include &lt;ATen/core/EnableNamedTensor.h&gt;

namespace at {
namespace native {
</code></pre>



  This file has been truncated. <a href="https://github.com/zdevito/ATen/blob/2448a0d8b547e8c787bb948a23009c375c46272c/aten/src/ATen/native/TensorShape.cpp" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>where they say, add new leading dimensions to the tensor if the number of target dimensions is larger than the number of source dimensions, that is this,<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/0/7090b99e78818c65789299f42c8923cbf1e5432b.png" data-download-href="https://discuss.pytorch.org/uploads/default/7090b99e78818c65789299f42c8923cbf1e5432b" title="Screenshot (652)"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/0/7090b99e78818c65789299f42c8923cbf1e5432b.png" alt="Screenshot (652)" data-base62-sha1="g3Ny1kcgyblDWmXGAqhodRODD4f" width="690" height="322" data-dominant-color="FAFAF8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (652)</span><span class="informations">1208×565 27.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>