Distance between two tensors in batch mode

username: John_J_Watson
cooked: <p>I have two tensors in my <code>forward</code> function with sizes <code>torch.Size([64, 256]</code> where the 64 is the batch size. I would like to compute the L2 distance between the two tensors/vectors.</p>
<p>So, I have:</p>
<p>a=torch.randn(64,256)<br>
b=torch.randn(64,256)</p>
<p>I am able to do something like:</p>
<p>dist=torch.abs(b-a) # yields shape 64,256 which is the expected output</p>
<p>However, this seems too naive and I was wondering if there was a better distance measure I can use here which gives me back a 64,256 sized tensor (since I feed this into a linear layer).</p>
<p>I tried using <code>PairwiseDistance</code> but this gives me a back a tensor of size 256, instead of 64,256.</p>
<p>Any pointers would be great.</p>

username: Peter1998
cooked: <p>I don’t fully understand your question.</p>
<p>Some explenation about the L2 norm:<br>
The L2 norm reduces the dimension of a multi dimensional vector to 1, e.g. in physical space, which is a vector of shape 3, the L2 distance has shape 1 (which we use as the distance in every day live, if you measure the distance (shape 1), between two objects in your room, each with coordinates of shape 3).<br>
You can also calculate this for other cases than the 3 dimensional, but the output will always be of shape 1, e.g. for a vector of shape 256, the output will be 1d. You could also do this individually, i.e. see your 256 entries as 256 1d vectors, then the output shape would be 256. In the 1d case the L2 norm is the same as the absolute value.</p>
<p>Maybe you could specify: Do you want to know <em>how</em> to calculate the L2 distance? Or are you looking for another measure?</p>

username: Caruso
cooked: <aside class="quote no-group" data-username="John_J_Watson" data-post="1" data-topic="83493">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/john_j_watson/48/16421_2.png" class="avatar"> John_J_Watson:</div>
<blockquote>
<p>I tried using <code>PairwiseDistance</code> but this gives me a back a tensor of size 256, instead of 64,256.</p>
</blockquote>
</aside>
<p>This should normally return a [64] tensor or [64, 1] tensor if <code>keepdim=True</code></p>
<pre><code class="lang-auto">t1 = torch.randn(64, 256)
t2 = torch.randn(64, 256)

t_dist = F.pairwise_distance(t1, t2)

print(t_dist.shape)
</code></pre>

username: John_J_Watson
cooked: <p>Hello <a class="mention" href="/u/peter1998">@Peter1998</a>: firstly thank you for the explanation. It certainly helped. I guess what I really am trying to find out is what this line does:</p>
<p><code> dist=torch.abs(b-a)</code></p>
<p>Would you say this is the l2 norm since it is the absolute value? I am not sure how this translates to a distance measure <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"></p>

username: John_J_Watson
cooked: <p>Apologies <a class="mention" href="/u/caruso">@Caruso</a>: that is correct.</p>

username: Peter1998
cooked: <aside class="quote no-group" data-username="John_J_Watson" data-post="1" data-topic="83493">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/john_j_watson/48/16421_2.png" class="avatar"> John_J_Watson:</div>
<blockquote>
<p>dist=torch.abs(b-a)</p>
</blockquote>
</aside>
<p>For 1d coordinates the absolute value is the common L2 distance. If you see your second dimension as 256 seperate 1d coordinates, then your code computes you the 256 corresponding distances. But: It is not the L2 norm of a 256d vector.</p>