L2 normalisation via f.normalize dim variable

username: John_J_Watson
cooked: <p>I am quite new to pytorch and I am looking to apply L2 normalisation to two types of tensors, but I am npot totally sure what I am doing is correct:</p>
<p>[1]. type 1 (in the forward function) has shape <code>torch.Size([2, 128])</code> and I would like to normalise each tensor (L2 norm).</p>
<p>for this case, I do:<br>
<code>F.normalize(tensor_variable, p=2, dim=1)</code><br>
Is this the correct way to do it? is there any check I can perform to know that the vectors have been L2 normed?</p>
<p>[2] type2 has shape <code>([128])</code>: for this I just do:<br>
<code>F.normalize(tensor_variable, p=2, dim=0)</code><br>
Is this the correct way to go about it?</p>
<p>I am not able to find the doc link to this <code>F.normalize</code> function and I am having to take a guess at the dimension <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"></p>

username: KFrank
cooked: <p>Hi John!</p>
<aside class="quote no-group" data-username="John_J_Watson" data-post="1" data-topic="78192" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/john_j_watson/48/16421_2.png" class="avatar"> John_J_Watson:</div>
<blockquote>
<p>Is this the correct way to do it?</p>
</blockquote>
</aside>
<p>Yes, your use of <code>normalize()</code> is correct.</p>
<aside class="quote no-group">
<blockquote>
<p>is there any check I can perform to know that the vectors have been L2 normed?</p>
</blockquote>
</aside>
<p>Just compute the (square of the) norm:</p>
<pre data-code-wrap="python"><code class="lang-python">(torch.nn.functional.normalize (tensor_variable, p = 2, dim = 1)**2).sum (dim = 1)
</code></pre>
<p>and check that you get 1 for each row of <code>tensor_input</code>.</p>
<aside class="quote no-group">
<blockquote>
<p>I am not able to find the doc link to this <code>F.normalize</code> function</p>
</blockquote>
</aside>
<p>You can find the documentation here: <a href="https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.normalize" rel="noopener nofollow ugc">normalize</a>.</p>
<p>Best.</p>
<p>K. Frank</p>

username: John_J_Watson
cooked: <p>Hi Frank,</p>
<p>FIRSTLY, thank you very much for your reply. Your answers certainly helped me. I have one follow up question: <code>torch.nn.functional.normalize</code> and <code>F.Normalize</code> are presumably the same function?</p>
<p>thank you.</p>

username: KFrank
cooked: <p>Hi John!</p>
<aside class="quote no-group" data-username="John_J_Watson" data-post="3" data-topic="78192" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/john_j_watson/48/16421_2.png" class="avatar"> John_J_Watson:</div>
<blockquote>
<p><code>torch.nn.functional.normalize</code> and <code>F.Normalize</code> are presumably the same function?</p>
</blockquote>
</aside>
<p>First I think you have a minor typo – I think you mean <code>F.normalize</code><br>
with a lower-case <code>n</code>.</p>
<p>There is no <code>F.normalize</code>, strictly speaking, in pytorch.  Rather, it is<br>
common practice to:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch.nn.functional as F
</code></pre>
<p>so that you can save a little typing and write <code>F.normalize</code> instead<br>
of <code>torch.nn.functional.normalize</code>.  So, yes, they are the same<br>
function.</p>
<p>(There is a <code>torchvision.transforms.Normalize</code> – with an upper-case<br>
<code>N</code>, and a class, rather than a function – but that’s not what you’re talking about.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: John_J_Watson
cooked: <p>Thank you again Frank for the patient explanation. REALLY appreciate your time.</p>