Numerical Problem with determinant of matrix and determinant of it's inverse matrix

username: nima_rafiee
cooked: <p>Hi. I’m trying to calculate the determinant of the following matrix and compare it with the determinant of its inverse</p>
<pre><code class="lang-auto">x = torch.tensor([[ 757.7089, -196.4800],
                   [-196.4800,   50.9489]])
</code></pre>
<pre><code class="lang-auto">x_inv = torch.inverse(x)
&gt;tensor([[ 1092.4257,  4212.8447],
        [ 4212.8447, 16246.4883]])
</code></pre>
<p>I use two ways to calculate the determinant of both matrix 1-product of eigenvalues 2-torch.det()<br>
and both ways have the problem that determinant of <code>x</code> is not equal to the inverse of the determinant of <code>x_inv</code></p>
<pre><code class="lang-auto">print(torch.det(x)) 
&gt;tensor(0.0466)
</code></pre>
<pre><code class="lang-auto">print(torch.det(x_inv))
&gt; tensor(19.6592) 
</code></pre>
<p>And</p>
<pre><code class="lang-auto">1/0.0466  == 19.6592
&gt; False 
</code></pre>
<p>which is wrong.</p>
<p>I even try to use</p>
<pre><code class="lang-auto">eig_val_x, _ = torch.symeig(x)
dete_x = torch.prod(eig_val_x)

eig_val_x_inv, _ = torch.symeig(x_inv)
dete_x_inv = torch.prod(eig_val_x_inv)
</code></pre>
<p>but still same problem exsit. and even two way of calculating determinant provide different values for the determinant.  Can some one help me with this issue.</p>

username: KFrank
cooked: <p>Hi Nima!</p>
<aside class="quote no-group quote-modified" data-username="nima_rafiee" data-post="1" data-topic="52977" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nima_rafiee/48/16289_2.png" class="avatar"> nima_rafiee:</div>
<blockquote>
<p>Hi. I’m trying to calculate the determinant of the following matrix and compare it with the determinant of its inverse</p>
<pre><code class="lang-auto">x = torch.tensor([[ 757.7089, -196.4800],
                   [-196.4800,   50.9489]])
</code></pre>
<p>…</p>
</blockquote>
</aside>
<p>Your matrix is ill-conditioned, and is therefore amplifying the<br>
round-off error in your single-precision (32-bit) calculations.</p>
<p>If such ill-conditioned matrices are part of your actual use case,<br>
you will need to pay the price of performing double-precision<br>
calculations.</p>
<p>A <code>torch.tensor</code> defaults to <code>float32</code> which has about 7 decimal<br>
digits of precision.  However the two rows of your matrix (when<br>
understood as vectors) are almost exactly anti-parallel, so your<br>
matrix is nearly degenerate.  (The angle between these two vectors<br>
is 179.9999839 degrees!)</p>
<p>Another way of seeing this is to calculate the <em>condition number</em><br>
of your matrix.  (This is the ratio between the matrix’s largest<br>
and smallest eigenvalues.)  Your condition number is 1.5 x 10^7.<br>
Very roughly speaking, the condition number tells you how much<br>
your round-off error will be amplified when doing things like<br>
inverting a matrix or solving a system of linear equations.</p>
<p>Your single-precision round-off error is about 10^-7, so – very<br>
roughly – it gets blown up to order 1.</p>
<p>If you use 64-bit double-precision numbers you will have about<br>
16 decimal digits of precision.  Even amplifying this round-off<br>
error by your condition number of 10^7, you will still have<br>
enough precision to get satisfactory results.</p>
<p>Redo your tests with a <code>tensor</code> of <code>dtype = torch.float64</code>,<br>
and see if that works for your use case.</p>
<p>This is illustrated by the following script (that uses numpy):</p>
<pre data-code-wrap="python"><code class="lang-python">import numpy as np

mmd = np.array([[757.7089, -196.4800], [-196.4800, 50.9489]])   # your matrix
mmd.dtype              # 64-bit double precision
np.linalg.cond (mmd)   # matrix is ill-conditioned

msin = np.linalg.det (mmd)       # det is proportional to sin (theta_vecs)
mcos = np.dot (mmd[0], mmd[1])   # dot is proportional to cos (theta_vecs)
np.degrees (np.math.atan2 (msin, mcos))   # angle between rows of matrix

mmdi = np.linalg.inv (mmd)
mmdi
np.matmul (mmd, mmdi)   # inverse is quite accurate

np.linalg.det (mmd)
1.0 / np.linalg.det (mmd)
np.linalg.det (mmdi)    # determinants match well


mms = np.float32 (mmd)   # convert to float
mms.dtype                # 32-bit single precision

mmsi = np.linalg.inv (mms)
mmsi
np.matmul (mms, mmsi)   # in single precision the inverse is quite inaccurate

np.linalg.det (mms)
1.0 / np.linalg.det (mms)
np.linalg.det (mmsi)    # calculated determinants differ significantly
</code></pre>
<p>Good luck.</p>
<p>K. Frank</p>

username: nima_rafiee
cooked: <p>tnx. with float64 they are now much similar</p>

username: BensonRen
cooked: <p>This is really helpful! Thanks!</p>