Torch.lstsq returns wrong results

kiranpurohit: <p>Hi, I was trying to run torch.lstsq to get the solution vector, but I am getting wrong results. My code and results are given below:</p>
<p>import torch<br>
A = torch.tensor([[1.0,1.0,1.0],[1.0,1.0,1.0]])<br>
B = torch.tensor([[5.0],[5.0]])<br>
lambda_id = torch.lstsq(A,B).solution<br>
print("A = ",A.shape)<br>
print("B = ",B.shape)<br>
print("lambda_id shape = ",lambda_id.shape)<br>
print("lambda_id = ",lambda_id)</p>
<p>Output:-<br>
A =  torch.Size([2, 3])<br>
B =  torch.Size([2, 1])<br>
lambda_id shape =  torch.Size([2, 3])<br>
lambda_id =  tensor([[ 2.0000e-01,  2.0000e-01,  2.0000e-01],<br>
[-5.2870e-08, -5.2870e-08, -5.2870e-08]])</p>
<p>Whereas when I run the same code with scipy.linalg.lstsq, I am getting correct results.</p>
<p>from scipy.linalg import lstsq<br>
A = torch.tensor([[1.0,1.0,1.0],[1.0,1.0,1.0]])<br>
B = torch.tensor([[5.0],[5.0]])<br>
lambda_id = lstsq(A,B)<br>
print("A = ",A.shape)<br>
print("B = ",B.shape)<br>
print("lambda_id shape = ",lambda_id[0].shape)<br>
print("lambda_id = ",lambda_id[0])</p>
<p>Output:-<br>
A =  torch.Size([2, 3])<br>
B =  torch.Size([2, 1])<br>
lambda_id shape =  (3, 1)<br>
lambda_id =  [[1.6666663]<br>
[1.6666663]<br>
[1.6666663]]</p>
<p>Please help me to figure it out.</p>

KFrank: <p>Hi Kiran!</p>
<aside class="quote no-group" data-username="kiranpurohit" data-post="1" data-topic="177579" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ba9def/48.png" class="avatar"> kiranpurohit:</div>
<blockquote>
<p>Hi, I was trying to run torch.lstsq to get the solution vector, but I am getting wrong results.</p>
</blockquote>
</aside>
<p>The short story:  Use <code>torch.linalg.lstsq()</code>.</p>
<p><code>torch.lstsq()</code> has been deprecated for some time now (and was<br>
removed, I believe, in pytorch version 1.13).</p>
<p>Quoting from the <a href="https://pytorch.org/docs/1.12/generated/torch.lstsq.html" rel="noopener nofollow ugc">1.12 documentation:</a></p>
<blockquote>
<p>Warning</p>
<p><a href="https://pytorch.org/docs/1.12/generated/torch.lstsq.html#torch.lstsq" rel="noopener nofollow ugc"><code>torch.lstsq()</code></a> is deprecated in favor of <a href="https://pytorch.org/docs/1.12/generated/torch.linalg.lstsq.html#torch.linalg.lstsq" rel="noopener nofollow ugc"><code>torch.linalg.lstsq()</code></a> and will be removed in a future PyTorch release. <a href="https://pytorch.org/docs/1.12/generated/torch.linalg.lstsq.html#torch.linalg.lstsq" rel="noopener nofollow ugc"><code>torch.linalg.lstsq()</code></a> has reversed arguments and does not return the QR decomposition in the returned tuple, (it returns other information about the problem). The returned solution in <a href="https://pytorch.org/docs/1.12/generated/torch.lstsq.html#torch.lstsq" rel="noopener nofollow ugc"><code>torch.lstsq()</code></a> stores the residuals of the solution in the last m - n columns in the case m &gt; n. In <a href="https://pytorch.org/docs/1.12/generated/torch.linalg.lstsq.html#torch.linalg.lstsq" rel="noopener nofollow ugc"><code>torch.linalg.lstsq()</code></a>, the residuals are in the field ‘residuals’ of the returned named tuple.</p>
<p>Unpacking the solution as <code>X = torch.lstsq(B, A).solution[:A.size(1)]</code> should be replaced with</p>
<p><code>X = torch.linalg.lstsq(A, B).solution</code></p>
</blockquote>
<aside class="quote no-group">
<blockquote>
<p>A = torch.tensor([[1.0,1.0,1.0],[1.0,1.0,1.0]])</p>
</blockquote>
</aside>
<p>As an aside, your tensor <code>A</code> has two identical rows so it is <em>rank-deficient</em>.<br>
The old <code>torch.lstsq()</code> was known not to support such cases, so even<br>
if you swap your arguments around, <code>torch.lstsq (B, A).solution</code>,<br>
<code>torch.lstsq()</code> probably still won’t work.</p>
<p>Best.</p>
<p>K. Frank</p>

kiranpurohit: <p>Thanks for such a detailed explanation!!</p>