Repeat examples along batch dimension

username: hanbit
cooked: <p>Hi, I’m trying to repeat tensors along the batch dimension.</p>
<p>Ex)<br>
We have a batch (8 x 3 x 224 x 224) where its size is 8<br>
and let’s say it is called as [a, b, c, d, e, f, g, h], and each alphabet denotes an example in the batch.</p>
<p>Then, I want to repeat each of them three times, resulting in EXACTLY FOLLOWING ORDER [a, a, a, b, b, b, c, c, c, d, d, d, …, h, h, h]. Same elements should come in succession.<br>
Thus, the final dimension is (24 x 3 x 224 x 224).</p>
<p>How can I make it possible?</p>

username: rasbt
cooked: <pre><code class="lang-auto">new_tensor = torch.cat(3*[orig_tensor])
</code></pre>
<p>would probably do the job.</p>

username: vmirly1
cooked: <p>An alternative way is to use <code>torch.repeat()</code>. So with <code>torch.repeat()</code>, you can specify the number of repeats for each dimension:</p>
<pre><code class="lang-python">&gt;&gt;&gt; a = torch.randn(8, 3, 224, 224)
&gt;&gt;&gt; b = a.repeat(3, 1, 1, 1)
&gt;&gt;&gt; b.shape
torch.Size([24, 3, 224, 224])
</code></pre>

username: hanbit
cooked: <p>Sorry for the confusion. I omit to mention that the same element should be in succession.</p>

username: vmirly1
cooked: <p>Oh, in that case, neither of these solutions work:</p>
<pre><code class="lang-python">&gt;&gt;&gt; t = torch.tensor([[1, 2, 3], [4, 4, 4]])                                                        
&gt;&gt;&gt; t
tensor([[1, 2, 3],
        [4, 4, 4]])
&gt;&gt;&gt; torch.cat(3*[t])
tensor([[1, 2, 3],
        [4, 4, 4],
        [1, 2, 3],
        [4, 4, 4],
        [1, 2, 3],
        [4, 4, 4]])
&gt;&gt;&gt; t.repeat(3, 1)
tensor([[1, 2, 3],
        [4, 4, 4],
        [1, 2, 3],
        [4, 4, 4],
        [1, 2, 3],
        [4, 4, 4]])
</code></pre>
<p>But based on the answer from <a href="https://discuss.pytorch.org/t/how-to-tile-a-tensor/13853/4" class="inline-onebox">How to tile a tensor?</a> we can do that so that the repeated elements are succession:</p>
<pre><code class="lang-python">def tile(a, dim, n_tile):
    init_dim = a.size(dim)
    repeat_idx = [1] * a.dim()
    repeat_idx[dim] = n_tile
    a = a.repeat(*(repeat_idx))
    order_index = torch.LongTensor(np.concatenate([init_dim * np.arange(n_tile) + i for i in range(init_dim)]))
    return torch.index_select(a, dim, order_index)
</code></pre>
<p>then, using this function, the repeated elements will be as follows:</p>
<pre><code class="lang-python">tile(t, 0, 3)
tensor([[1, 2, 3],
        [1, 2, 3],
        [1, 2, 3],
        [4, 4, 4],
        [4, 4, 4],
        [4, 4, 4]])
</code></pre>

username: hanbit
cooked: <p>Great! Thanks a lot.</p>

username: Imbesat_Rizvi
cooked: <p>Borrowing from my <a href="https://discuss.pytorch.org/t/how-to-tile-a-tensor/13853/13">answer</a>, for anyone new looking for this issue, an updated function has also been introduced in pytorch - <a href="https://pytorch.org/docs/stable/torch.html#torch.repeat_interleave" rel="nofollow noopener">torch.repeat_interleave() </a> to address this issue in a single operation.</p>
<p>So for <code>t = torch.tensor([[1, 2, 3], [4, 4, 4]])</code> one can use <code>torch.repeat_interleave(t, repeats=3, dim=0)</code> to obtain:</p>
<pre><code class="lang-auto">tensor([[1., 2., 3.],
        [1., 2., 3.],
        [1., 2., 3.],
        [4., 4., 4.],
        [4., 4., 4.],
        [4., 4., 4.]])
</code></pre>

username: Gaurav_Kumar
cooked: <blockquote>
<p>Suppose a tensor is of dimension (9,10), say it A, A.repeat(1,1) would produce same tensor as A.<br>
Calling A.repeat(1,1,10) produces tensor of dimension 1,9,100<br>
Again calling A.repeat(1,2,1) produces 1,18,10.<br>
It look likes that from right to left, element wise multiplication is happening from the input of repeat</p>
</blockquote>

username: arogozhnikov
cooked: <p><a href="https://github.com/arogozhnikov/einops" rel="nofollow noopener">Einops</a> recently got support for repeat-like patterns. Examples:</p>
<pre><code class="lang-auto"># np.repeat behavior (copies are in succession, like aaabbbcccddd as described by topic starter)
einops.repeat(x, 'b c h w -&gt; (b copy) c h w', copy=3)
# np.tile behavior (whole sequence is repeated 3 times like abcdabcdabcd)
einops.repeat(x, 'b c h w -&gt; (copy b) c h w', copy=3)
</code></pre>
<p>You can repeat/tile multiple axes independently within one operation.</p>

username: Dewei_Hu
cooked: <p>man, really helpful. I’d like to say a great thanks to you</p>