Help me to speed up nested loop?

username: sudddddd
cooked: <p>I am using a nested for loop to get outputs from my model for 1000 images compared with themselves. model takes 2 images and outputs a scaler. A is a tensor of size (1000,3,128,128) and B=A. But, it is taking a long time to get the output.</p>
<pre><code class="lang-auto">out=[]
for i in A:
        temp=[]
        for j in B:
            temp.append(model(i,j))
        out.append(temp)
</code></pre>
<p>Final shape of out is (1000,1000). Above code just gives the overall idea of my task and 1000 is arbitrary.<br>
I am actually using batch of size 8 instead of 1 (as above) to speed up the process. But, this is still taking a lot of time. Can anyone help me with this? I also want to know if there is a method to avoid duplicate comparisons as out matrix is symmetric.</p>

username: ptrblck
cooked: <p>Could you explain the comparison inside your model a bit?<br>
I.e. if you pass two batches of indices <code>[0, 1, 2] and </code>[0, 1, 2]<code>, will the model output the scalars for </code>0-0, 1-1, 2-2` or will it also compute all other combinations?</p>
<p>In the former case, you could increase the inner batch (<code>j</code>) by one while increasing the outer batch by <code>batch_size</code>.</p>
<aside class="quote no-group" data-username="sudddddd" data-post="1" data-topic="80584">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sudddddd/48/11257_2.png" class="avatar"> sudddddd:</div>
<blockquote>
<p>I also want to know if there is a method to avoid duplicate comparisons as out matrix is symmetric.</p>
</blockquote>
</aside>
<p>You could add a check for the indices as:</p>
<pre data-code-wrap="python"><code class="lang-python">for index_i, i in enumerate(A):
    temp = []
    for index_j, j in enumerate(B):
        if j &gt; i: # or j &gt;= i, if you don't want to compare the same indices
            continue
</code></pre>

username: sudddddd
cooked: <p>If you pass my model the batch ([0,1,2],[0,1,2]), then it will output a scalar for each of the case 0-0, 1-1 and 2-2, i.e. my output will be 3 scalars.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="80584">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>In the former case, you could increase the inner batch ( <code>j</code> ) by one while increasing the outer batch by <code>batch_size</code> .</p>
</blockquote>
</aside>
<p>As I have mentioned in my question, I am using batch size of 8. I replicate the i 8 times (for each element) and pass it to 8 elements of the inner batch, j. But, this is too slow. I want to optimise this part.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="80584">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>You could add a check for the indices as:</p>
<pre><code class="lang-auto">for index_i, i in enumerate(A):
    temp = []
    for index_j, j in enumerate(B):
        if j &gt; i: # or j &gt;= i, if you don't want to compare the same indices
            continue
</code></pre>
</blockquote>
</aside>
<p>Yes, I could do that, but, is there a way to combine this with the previous optimisation (if found) to further speed up the task.</p>

username: ptrblck
cooked: <p>I’m not sure if there is another way to optimize this.<br>
Assuming you are dealing with <code>N</code> images, your model would have to calculate<br>
<code>N + N-1 + N-2 + ... + 1 = (N+1) * N / 2</code> comparisons.<br>
You could speed it up using batches, but if I understand your explanation correctly, you are avoiding unnecessary calculations.</p>