Incorrect pow function

username: Mikhail_Karchevskiy
cooked: <p>If p = 2 , 4 or 6, the result of</p>
<pre><code class="lang-auto">p = 3
my_tensor.pow(p).pow(1/p)
</code></pre>
<p>is ok</p>
<p>if p=3, 5 and …<br>
the result with nan values, where negative numbers?</p>

username: albanD
cooked: <p>Hi,</p>
<p>This is because any power <code>1/p</code> is only defined for positive numbers.<br>
But negative number at the power 3,5, etc gives negative numbers.<br>
So you get nan.</p>
<p>PS: You should use backtick ` and not " to format code in a nicer way. I edited you post to make it look better.</p>

username: KFrank
cooked: <p>Hello Mikhail (and Alban)!</p>
<p>To give a little more context to Alban’s answer:</p>
<p>Yes, pytorch doesn’t want you to take a root of a negative<br>
number (e.g. <code>sqrt (-1)</code> which is <code>(-1)**(1/2)</code>, a fractional<br>
power), so it gives you a <code>nan</code>.</p>
<p>(I am using <code>x**y</code> to mean "raise <code>x</code> to the power <code>y</code>".)</p>
<p>When you raise a negative number to an <em>even</em> power you get<br>
a positive number, so there is no problem taking the root.   But<br>
when you raise a negative number to an <em>odd</em> power, you get a<br>
<em>negative</em> number, so pytorch gives you <code>nan</code> for the root.</p>
<p>However, if you use <em>complex</em> numbers (numbers that have a<br>
real and so-called <em>imaginary</em> part), roots of negative numbers<br>
make perfect sense.  But, for example, there is no square root<br>
of -1 that is purely real (that is, that has no imaginary part).<br>
So, if you stick to real numbers, the best you can do is <code>nan</code>.</p>
<p>Nonetheless, you <em>could</em> argue that the cube root of -1 (that<br>
is, <code>(-1)**(1/3)</code>) is perfectly well defined as the purely real<br>
number -1.  Indeed it is (because <code>(-1)**3 == -1)</code>. So why<br>
don’t we do this?</p>
<p>Now the fun begins:</p>
<p>A number (including a negative number) has k k-th roots.  That<br>
is <code>x**(1/k)</code> has k legitimate values, in that there are k distinct<br>
numbers, <code>z</code>, (most or all of which will be complex numbers) for<br>
which <code>z**k == x</code>.</p>
<p>To make the discussion a little simpler, let’s just look at <code>-1</code>.<br>
If we want to use just one value for <code>(-1)**(1/k)</code> (rather than<br>
k different values), which value should we use?  Mathematicians<br>
(for reasons that aren’t chiselled in stone but do make sense)<br>
prefer to use (of the k values) the value of <code>(-1)**(1/k)</code><br>
that is closest (in the complex plane) to 1.  This particular<br>
choice will always be complex (i.e., not purely real), so, if<br>
you want to stick to real numbers, the best you can do is <code>nan</code>.<br>
(You could have chosen <code>-1</code> – that would be perfectly legitimate,<br>
if less standard, but that’s not what pytorch does*.)</p>
<p>Just for fun, let’s compare pytorch’s <code>Tensor.pow()</code>, python’s <code>**</code><br>
operator, and python’s <code>math.pow()</code>:</p>
<pre><code class="lang-python">import torch
print (torch.__version__)

t = torch.Tensor ([1, -1])

for  p in [2, 3, 4, 5]:
    print ('p = ', p)
    print ('   pytorch-powpow = ', t.pow (p).pow (1 / p))


import math

for  p in [2, 3, 4, 5]:
    print ('p = ', p)
    try:
        print ('   math-powpow = ', math.pow (math.pow (-1, p), 1/p))
    except Exception as e:
        print ('Exception = ', e)
    
        
for  p in [2, 3, 4, 5]:
    print ('p = ', p)
    print ('   **-powpow = ', ((-1)**p)**(1/p))
</code></pre>
<p>Here is the output of running this script:</p>
<pre><code class="lang-nohighlight">0.3.0b0+591e73e
p =  2
   pytorch-powpow =
 1
 1
[torch.FloatTensor of size 2]

p =  3
   pytorch-powpow =
  1
nan
[torch.FloatTensor of size 2]

p =  4
   pytorch-powpow =
 1
 1
[torch.FloatTensor of size 2]

p =  5
   pytorch-powpow =
  1
nan
[torch.FloatTensor of size 2]

p =  2
   math-powpow =  1.0
p =  3
Exception =  math domain error
p =  4
   math-powpow =  1.0
p =  5
Exception =  math domain error
p =  2
   **-powpow =  1.0
p =  3
   **-powpow =  (0.5000000000000001+0.8660254037844386j)
p =  4
   **-powpow =  1.0
p =  5
   **-powpow =  (0.8090169943749475+0.5877852522924731j)
</code></pre>
<p>For an odd root of a negative number, pytorch and python’s<br>
<code>math.pow()</code> basically agree.  (Pytorch gives <code>nan</code>, and python<br>
throws an exception.)</p>
<p>But python’s <code>**</code> operator give you a complex number (and,<br>
indeed, of the k roots, the complex number that is closet to<br>
1), and (for the case of <code>(-1)**(1/k)</code>) chooses <em>not</em> to give<br>
you <code>-1</code>.</p>
<p>*) In order for pytorch to be smart enough to give you <code>-1</code> for,<br>
say, <code>torch.Tensor ([-1]).pow (1/3)</code>, it would have to know<br>
that the argument to <code>pow()</code> is, in fact one-third.  But all pytorch<br>
sees for this argument is a floating-point number that is very<br>
close to, but not exactly one-third.  So figuring out that you meant<br>
<em>precisely</em> one-third would be quite tricky.</p>
<p>Have fun!</p>
<p>K. Frank</p>

username: Mikhail_Karchevskiy
cooked: <p>Thanks a lot, now I understand why it happened.<br>
I decided to do as in LPPool1d class:</p>
<pre><code class="lang-auto"> p = 3
 torch.sign(my_tensor) * torch.abs(my_tensor.pow(p)).pow(1. / p)
</code></pre>
<p>maybe it will be useful for someone</p>