How to tile a tensor?

marcman411: <p>If I have a tensor like:</p>
<pre><code class="lang-auto">z = torch.FloatTensor([[1,2,3],[4,5,6]])

1 2 3
4 5 6
</code></pre>
<p>How might I turn it into a tensor like:</p>
<pre><code class="lang-auto">1 2 3
1 2 3
1 2 3
1 2 3
4 5 6
4 5 6
4 5 6
4 5 6
</code></pre>
<p>I imagine that <a href="http://pytorch.org/docs/master/tensors.html?highlight=repeat#torch.Tensor.repeat" rel="nofollow noopener">torch.repeat()</a> is somehow in play here.</p>
<p>The only solution I have come up with is to do:</p>
<pre><code class="lang-auto">z.repeat(1,4).view(-1, 3)
</code></pre>
<p>Is there one operation that collapses these two commands into one?</p>
<p>Moreover, if I have columnwise data I want to repeat, how can I do this without transposing the data back and forth? For example, going from</p>
<pre><code class="lang-auto">z = torch.FloatTensor([[1,2,3],[4,5,6],[7,8,9]])

1 2 3
4 5 6
7 8 9
</code></pre>
<p>to</p>
<pre><code class="lang-auto">1 1 1 2 2 2 3 3 3
4 4 4 5 5 5 6 6 6
7 7 7 8 8 8 9 9 9
</code></pre>
<p>without saying</p>
<p><code>z.transpose(0,1).repeat(1,3).view(-1, 3).transpose(0,1)</code></p>

richard: <p>For the second you can do:</p>
<pre><code class="lang-auto">z.view(-1, 1).repeat(1, 3).view(3, 9)
1 1 1 2 2 2 3 3 3
4 4 4 5 5 5 6 6 6
7 7 7 8 8 8 9 9 9
</code></pre>
<p>For the first, I don’t think there are operations that combine all of these together. Maxunpool does something similar but doesn’t have the repeat ability.</p>

jpeg729: <p>It may be worth mentioning that <code>.view</code> doesn’t change or copy the underlying data in any way, it just changes the strides of the view mechanism.</p>
<p>Hence, calling <code>.view</code> is an extremely fast operation.</p>

Edouard360: <p>For a general solution working on any dimension, I implemented <code>tile</code> based on the <code>.repeat</code> method of torch’s tensors:</p>
<pre><code class="lang-auto">def tile(a, dim, n_tile):
    init_dim = a.size(dim)
    repeat_idx = [1] * a.dim()
    repeat_idx[dim] = n_tile
    a = a.repeat(*(repeat_idx))
    order_index = torch.LongTensor(np.concatenate([init_dim * np.arange(n_tile) + i for i in range(init_dim)]))
    return torch.index_select(a, dim, order_index)
</code></pre>
<p><strong>Examples:</strong></p>
<pre><code class="lang-auto">t = torch.FloatTensor([[1,2,3],[4,5,6]])
Out[54]: 
tensor([[ 1.,  2.,  3.],
        [ 4.,  5.,  6.]])
</code></pre>
<ul>
<li>Across dim 0:</li>
</ul>
<pre><code class="lang-auto">tile(t,0,3)
Out[53]: 
tensor([[ 1.,  2.,  3.],
        [ 1.,  2.,  3.],
        [ 1.,  2.,  3.],
        [ 4.,  5.,  6.],
        [ 4.,  5.,  6.],
        [ 4.,  5.,  6.]])
</code></pre>
<ul>
<li>Across dim 1:</li>
</ul>
<pre><code class="lang-auto">tile(t,1,2)
Out[55]: 
tensor([[ 1.,  1.,  2.,  2.,  3.,  3.],
        [ 4.,  4.,  5.,  5.,  6.,  6.]])

</code></pre>
<p>No benchmarking performed, though <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Haydnspass: <p>Nice! What you wrote is I guess the equivalent to numpy.repeat (just the interface is with swapped arguments), however it would be nice to have it without numpy.</p>
<pre><code class="lang-auto">tile(torch.arange(5), dim=0, n_tile=2)
Out: tensor([0, 0, 1, 1, 2, 2, 3, 3, 4, 4])
</code></pre>
<p>and in numpy</p>
<pre><code class="lang-auto">np.repeat(np.arange(5), repeats=2, axis=0)
Out: array([0, 0, 1, 1, 2, 2, 3, 3, 4, 4])
</code></pre>

jaks19: <p>Agreed, especially if we wanted to carry the accumulated gradient if say it was a Variable.</p>

Haydnspass: <p>I guess it should then be</p>
<pre><code class="lang-auto">def repeat_np(a, repeats, dim):
    """
    Substitute for numpy's repeat function. Taken from https://discuss.pytorch.org/t/how-to-tile-a-tensor/13853/2
    torch.repeat([1,2,3], 2) --&gt; [1, 2, 3, 1, 2, 3]
    np.repeat([1,2,3], repeats=2, axis=0) --&gt; [1, 1, 2, 2, 3, 3]

    :param a: tensor
    :param repeats: number of repeats
    :param dim: dimension where to repeat
    :return: tensor with repitions
    """

    init_dim = a.size(dim)
    repeat_idx = [1] * a.dim()
    repeat_idx[dim] = repeats
    a = a.repeat(*(repeat_idx))
    if a.is_cuda:  # use cuda-device if input was on cuda device already
        order_index = torch.cuda.LongTensor(
            torch.cat([init_dim * torch.arange(repeats, device=a.device) + i for i in range(init_dim)]))
    else:
        order_index = torch.LongTensor(
            torch.cat([init_dim * torch.arange(repeats) + i for i in range(init_dim)]))

    return torch.index_select(a, dim, order_index)
</code></pre>

Yang_Kai: <p>it’s easy, just like</p>
<pre><code class="lang-python">z.unsqueeze(0).transpose(0,1).repeat(1,4,1).view(-1,3)
</code></pre>

Manu: <p>Or even more simply:</p>
<p><code>z.unsqueeze(1).repeat(1,4,1).view(-1,3)</code></p>

praf: <pre><code class="lang-auto"># Repeat along any dimension
import numpy as np
def repeat(x, n, dim):
    if dim == -1:
        dim = len(x.shape) - 1
    return x.view(int(np.prod(x.shape[:dim+1])), 1, int(np.prod(x.shape[dim+1:]))).repeat(1,n,1).view(*x.shape[:dim], n * x.shape[dim], *x.shape[dim+1:])
</code></pre>

obadia_yohan: <p>The best solution in my opinion! Would be great if pytorch implemented an equivalent to numpy <code>tile</code> method though.</p>

obadia_yohan: <p>A little function that builds on top of <a class="mention" href="/u/yang_kai">@Yang_Kai</a>’s answer and provides an easy way to implement a <code>tile</code> function for 2D  tensors:</p>
<pre><code>def torch_tile(tensor, dim, n):
    """Tile n times along the dim axis"""
    if dim == 0:
        return tensor.unsqueeze(0).transpose(0,1).repeat(1,n,1).view(-1,tensor.shape[1])
    else:
        return tensor.unsqueeze(0).transpose(0,1).repeat(1,1,n).view(tensor.shape[0], -1)</code></pre>

Imbesat_Rizvi: <p>For anyone new looking for this issue, an updated function has also been introduced in pytorch - <a href="https://pytorch.org/docs/stable/torch.html#torch.repeat_interleave" rel="nofollow noopener">torch.repeat_interleave()</a> to address this issue in a single operation.</p>
<p>So one can use <code>torch.repeat_interleave(z, repeats=3, dim=0)</code> to obtain:</p>
<pre><code class="lang-auto">tensor([[1., 2., 3.],
        [1., 2., 3.],
        [1., 2., 3.],
        [4., 5., 6.],
        [4., 5., 6.],
        [4., 5., 6.]])
</code></pre>
<p>and similarly can use <code>torch.repeat_interleave(z, repeats=3, dim=1)</code> to obtain:</p>
<pre><code class="lang-auto">tensor([[1., 1., 1., 2., 2., 2., 3., 3., 3.],
        [4., 4., 4., 5., 5., 5., 6., 6., 6.]])
</code></pre>

CarlosHernandezP: <p>Thank you very much, this is really what I look for in a forum.</p>

arogozhnikov: <p><a href="https://github.com/arogozhnikov/einops" rel="nofollow noopener">Einops</a> recently got support for various repeat-like patterns. Examples:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; x
tensor([[0, 1],
        [2, 3]])
</code></pre>
<p>Tile over the first axis:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; from einops import repeat
&gt;&gt;&gt; repeat(x, 'i j -&gt; (tile i) j', tile=2)
tensor([[0, 1],
        [2, 3],
        [0, 1],
        [2, 3]])
</code></pre>
<p>Tile over the second axis:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; repeat(x, 'i j -&gt; i (tile j)', tile=2)
tensor([[0, 1, 0, 1],
        [2, 3, 2, 3]])
</code></pre>
<p>Tiling over both axes:</p>
<pre><code class="lang-auto">repeat(x, 'i j -&gt; (tilei i) (tilej j)', tilei=2, tilej=3)
</code></pre>