Manual compute euclidean distance using 'one for loop'

username: snip3r77
cooked: <p>Please see the screenshot below<br>
                    <a href="https://i.imgur.com/UReSzFV.png" target="_blank" class="onebox" rel="nofollow noopener">
            <img src="https://i.imgur.com/UReSzFV.png" width="581" height="500">
          </a>

</p>
<p>or you can see the pastebin here<br>
<a href="https://pastebin.com/BFnLXveJ" class="onebox" target="_blank" rel="nofollow noopener">https://pastebin.com/BFnLXveJ</a></p>
<p>The related code is at line 31 and 32.</p>
<p>since x_train is (3, 16, 16 )<br>
and x_test is (10, 3, 16, 16 )</p>
<p>I need to reshape it to be (1, 3, 16, 16 ) such that broadcasting will be performed at the first dimension which is the number of observation.</p>
<p>Did I get this correct? Thank you.</p>
<pre><code class="lang-auto"> for train in range(num_train):
    dists[train]=((x_train[train].reshape(-1,C,H,W) - x_test)**2).sum().sqrt()
</code></pre>

username: albanD
cooked: <p>Hi,</p>
<p>I am not sure why you do the for loop here? And why do you compare each training sample with every test one.</p>
<p>Why not just replace the whole for loop by <code>(x_train - x_test).norm()</code> ? Note that if you want to keep the value for each sample, you can specify the <code>dim</code> on which to compute the norm in the <a href="https://pytorch.org/docs/stable/torch.html#torch.norm" rel="nofollow noopener"><code>torch.norm</code></a> function.</p>

username: snip3r77
cooked: <p>Thanks for the prompt reply.</p>
<p>The tutorial is trying to demonstrate the speed difference between .norm() and also the for loop and also in a way ‘teach’ us how to do vectorize it. The solution that you provided is the next step but I was wondering how do I do it with the for loop.</p>
<p>Thanks</p>

username: albanD
cooked: <p>Ok,<br>
In that case, I think the problem you have is that you should be indexing the x_test as well:</p>
<pre><code class="lang-auto">for train in range(num_train):
    dists[train] = ((x_train[train] - x_test[train])**2).sum().sqrt()
</code></pre>

username: snip3r77
cooked: <p>Yeah I think that’s for ‘two for loops’ whereby it’s the slowest of the bunch without any vectorization.</p>
<pre><code class="lang-auto">for train in range(num_train):
    for test in range(num_test):
      dists[train][test] = ((x_train[train]- x_test[test])**2).sum().sqrt()
</code></pre>

username: albanD
cooked: <p>Hi,</p>
<p>Do you actually want to compute the distance between every pair of train and test? Or just the first train with first test, second train with second test, etc?</p>

username: snip3r77
cooked: <p>With every pair as we have initialized it as accordingly.</p>
<pre><code class="lang-auto"># Initialize dists to be a tensor of shape (num_train, num_test) with the
  # same datatype and device as x_train
  num_train = x_train.shape[0]
  num_test = x_test.shape[0]
  dists = x_train.new_zeros(num_train, num_test)
</code></pre>

username: albanD
cooked: <p>Then your code sample will work as you want <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: snip3r77
cooked: <p>Sorry do you mean the one at post <span class="hashtag">#1</span>( single for loop ) will work and also the double for loops?</p>
<p>Thanks.</p>

username: albanD
cooked: <p>The one with the double for loop.<br>
The one in the first post won’t work because the <code>sum()</code> will sum the results for all targets for a single prediction. You could change that to <code>sum(dim=foo)</code> to keep the dimension but that would be a half vectorized solution, which might not be what you want.</p>

username: snip3r77
cooked: <p>Hi,</p>
<p>Thanks for your reply.</p>
<p>this works ok<br>
<code>dists[train]=((x_train[train].reshape(-1,C,H,W) - x_test)**2).sum().sqrt()</code></p>
<p>but I have errors with</p>
<p>                    <a href="https://i.imgur.com/B6KfEGn.png" target="_blank" class="onebox" rel="nofollow noopener">
            <img src="https://i.imgur.com/B6KfEGn.png" width="690" height="441">
          </a>

</p>
<pre><code class="lang-auto">dists[train]=((x_train[train].reshape(-1,C,H,W) - x_test)**2).sum(axis=1).sqrt()'

RuntimeError: expand(torch.DoubleTensor{[10, 16, 16]}, size=[10]): the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (3)
</code></pre>
<p>for axis=1, we’re collapsing along the rows rather than columns right?</p>
<p>Thank you once again.</p>

username: albanD
cooked: <p>Hi,</p>
<p>You can check the doc for sum <a href="https://pytorch.org/docs/stable/torch.html#torch.sum" rel="nofollow noopener">here</a> but the dim you specify is the one you sum over. In you case, you want to sum over the <code>C, H and W</code> dimensions. So either use 3 different sums or use a view to collapse these three into a single dimension then sum over this new dimension.</p>

username: snip3r77
cooked: <p>Thanks <a class="mention" href="/u/alband">@albanD</a>,</p>
<p>Managed to dial in with this code</p>
<pre><code class="lang-auto">  C = x_train[0].shape[0]
  H = x_train[0].shape[1]
  W = x_train[0].shape[2]
  flat = C * H * W
  for train in range(num_train):
    dists[train]=((x_train[train].view(-1,flat) - x_test.view(-1,flat))**2).sum(axis=1).sqrt()
</code></pre>
<p>Question:<br>
So when I reshape it with view (-1,flat ) I need to use axis=1 because I want to collapse via the row.<br>
If I were to use (flat, -1 ) I need to use axis=0 as I want to collapse via column, right?</p>
<p>Also, I equate both methods , double loop with single loop.it matches now.<br>
                        <a href="https://imgur.com/f3tBapV" target="_blank" class="onebox" rel="nofollow noopener">
              <img src="https://i.imgur.com/f3tBapV.png?fb" title="Imgur" alt="Imgur" height="315" width="600">
            </a>

</p>
<p>Thanks once again.</p>

username: albanD
cooked: <p>Hi,</p>
<p>You cannot do <code>(flat, -1)</code> ! View is not a transpose. It is just a different way to look at the same data.<br>
So if you do view with flat, -1  you will see very unexpected data.</p>
<p>I would recommend you play with this in a python shell. make a 2, 3 random tensor and try and view it as 3, 2. You’ll see that the result is fairly unintuitive (though expected).</p>

username: snip3r77
cooked: <p>Apologies. Yeah I understood regarding the .view which is actually the equivalent for .reshape<br>
So, in this case the correct way is to use np.newaxis instead?<br>
                    <a href="https://i.stack.imgur.com/zkMBy.png" target="_blank" class="onebox" rel="nofollow noopener">
            <img src="https://i.stack.imgur.com/zkMBy.png" width="500" height="500">
          </a>

</p>

username: albanD
cooked: <p>Hi,</p>
<p>You never have to use some numpy features in pytorch.<br>
If you want to add a new axis at position 0 for example, you can use <code>t.unsqueeze(0)</code> or <code>t.view(1, "your other dims")</code> or <code>t[None, :]</code> (not sure about the last one as I don’t use it personnaly).</p>