Fast tensor appending operation

username: gilbertocunha
cooked: <p>Hi there!<br>
I am sort of a newbie with tensor manipulation, I have looked around for this, but I can’t seem to get anywhere, and I am looking for something using torch tensors that would allow me to do the following:</p>
<pre><code class="lang-auto">results = []
for xs in x:
    r = []
    for xss in xs:
        r.append(sift(xss))
    results.append(r)
</code></pre>
<p>Where x is a tensor, I am just going through its batch_size and feature_channels dimensions in these loops and applying a SIFT layer for each image patch group.<br>
I would also like to be able to run this as fast as possible<br>
Thanks in advance!</p>

username: JuanFMontesinos
cooked: <p>You can just do that as it is written, however, since xss comes from xs which comes from x you can run the whole xs in one run.</p>

username: gilbertocunha
cooked: <p>Sorry, how would I do it as it is written? This is implemented with python lists which are really slow. I want to make this as fast as possible to reduce training time.<br>
It probably should also be noted that this code is from the forward method of a costum layer, while running the model on a gpu.</p>

username: JuanFMontesinos
cooked: <p>Yep but I mean, you are running python. Thus, you use python lists.<br>
The way of avoiding lists is trying not to make use of loops and writting it as pytorch operations.</p>
<p>If you need to speed up anything and to avoid python lists the way to go is making a c++ implementation.<br>
<a href="https://pytorch.org/docs/master/cpp_index.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/master/cpp_index.html</a></p>
<p>But in general, it is “strange” to use lists in a forward. Because for almost all the official layers you can usually proccess everything at once with bit of rewriting.</p>
<p>The only case in which you must need to use for loops is for very customized operators or for dinamically generated networks (when you create N layers based on  a parameter and then you cannot manually call them all)</p>
<p>Another tool you is torchscript <a href="https://pytorch.org/docs/master/jit.html" rel="nofollow noopener">https://pytorch.org/docs/master/jit.html</a></p>
<p>But if you want the versatility and readability of lists, then you pay the price.</p>

username: gilbertocunha
cooked: <p>Basically what I need to do is I have a tensor of shape (b, c, h, w)<br>
I have to process each of the (h, w) in the tensor separately with a custom layer that only takes bidimensional input, so I just iterated over the b and c dimensions and appended each result.<br>
Isn’t there any other way to do this with a pytorch function?</p>

username: JuanFMontesinos
cooked: <p>You can try to extend the custom layer to process batches (all the layers in pytorch can process batches and most of the operations like matrix multiplication too)<br>
and then process everything as a (b*c,h,w) tensor without envolving loops</p>

username: gilbertocunha
cooked: <p>That is what I am trying to do, but by just wrapping it around another costum layer that already feeds it what is can process.<br>
I guess I will have to really change the inner structure of the costum layer, then.</p>

username: JuanFMontesinos
cooked: <p>as general advice I would recommend you to make all your custom layers to be able to process batches because that’s the standard way of working of pytorch.</p>

username: gilbertocunha
cooked: <p>Thanks for the advice. Properly noted, but unfortunately the costum layer isn’t mine, so I will probably just have to deal with it.<br>
Thank you so much for the help anyway!</p>