`torch.linalg.lu_factor` with `pivot=False` doesn't return `LU` matrix?

username: mmitjans
cooked: <p>Hi!</p>
<p>I have a matrix <code>K</code> that I represent with an LU factorization, such that <code>K = L @ U</code>, and <code>L</code> and <code>U</code> are lower and upper triangular 2-D tensors respectively. I’m currently testing the functionality of <code>torch.linalg.lu_factor</code> with <code>pivot=False</code>, which enforces <code>P</code> (the permutation matrix) to be the identity.<br>
In my sample code I have the following:</p>
<pre><code class="lang-auto">L = torch.randn(3,3).tril()
L.diagonal()[:] = torch.ones(3) # To enforces 1's in the diagonal
U = torch.randn(3,3).triu()
K = L @ U
</code></pre>
<p>If then I do:<br>
<code>LU_factor, pivots = torch.linalg.lu_factor(K, pivot=False)</code><br>
I would expect <code>LU_factor</code> to be the same matrix as <code>K</code>, but it’s not. Moreover, if I do:<br>
<code>Pu, Lu, Uu, = torch.lu_unpack(LU_factor, pivots)</code><br>
Then <code>Lu</code> and <code>Uu</code> are now exactly <code>L</code> and <code>U</code> (as expected).</p>
<p>My question is, if by setting <code>pivot=False</code> in <code>lu_factor</code> I enforce <code>P</code> to be the identity, what other operations do <code>lu_factor()</code> and <code>lu_unpack()</code> perform under the hood that make <code>LU_factor</code> be different from <code>L @ U</code>?</p>
<p>Thanks beforehand!</p>

username: KFrank
cooked: <p>Hi Marc!</p>
<aside class="quote no-group" data-username="mmitjans" data-post="1" data-topic="193977" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/51bf81/48.png" class="avatar"> mmitjans:</div>
<blockquote>
<p><code>LU_factor, pivots = torch.linalg.lu_factor(K, pivot=False)</code><br>
I would expect <code>LU_factor</code> to be the same matrix as <code>K</code>, but it’s not.</p>
</blockquote>
</aside>
<p>Even when you turn off pivoting, <code>lu_factor()</code> still performs the non-trivial<br>
LU factorization and returns the “packed” factorization in <code>LU_factor</code>.  So<br>
<code>LU_factor</code> is not the original, unfactored <code>K</code>.  (Why would it be?)</p>
<p>Here is a script that illustrates this by constructing the “packed” factorization<br>
from your original <code>L</code> and <code>U</code> matrices:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)

_ = torch.manual_seed (2023)

L = torch.randn (3, 3, device = 'cuda').tril()
L.diagonal()[:] = torch.ones (3, device = 'cuda')   # To enforces 1's in the diagonal
U = torch.randn (3, 3, device = 'cuda').triu()
K = L @ U

LU_factor, pivots = torch.linalg.lu_factor (K, pivot=False)

tind = torch.triu_indices (3, 3)                    # "pack" factorization into a single square matrix
LU_pack = L.clone()
LU_pack[tind[0], tind[1]] = U[tind[0], tind[1]]

print ('LU_factor = ...')
print (LU_factor)
print ('LU_pack = ...')
print (LU_pack)
print ('allclose:', torch.allclose (LU_factor, LU_pack))
</code></pre>
<p>And here is its output:</p>
<pre><code class="lang-plaintext">2.1.1
LU_factor = ...
tensor([[-0.4913,  0.5382, -3.1042],
        [ 0.7900, -0.2999,  0.2209],
        [ 1.1008, -0.1755,  0.5342]], device='cuda:0')
LU_pack = ...
tensor([[-0.4913,  0.5382, -3.1042],
        [ 0.7900, -0.2999,  0.2209],
        [ 1.1008, -0.1755,  0.5342]], device='cuda:0')
allclose: True
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

username: mmitjans
cooked: <p>Oh my bad! I thought <code>LU_factor</code> would be <code>L@U</code> and not <code>L</code> and <code>U</code> packed together (which makes a lot of sense now that you mention it, for many reasons). Thanks for the very quick reply.</p>