Rank factorization of a matrix

username: nazareem
cooked: <p>Hi,</p>
<p>I have a tall rectangular matrix <code>A</code> of shape <code>m</code> by <code>n</code>, where <code>m</code> &gt;&gt; <code>n</code>, and it has a rank <code>r</code> such that <code>r</code> &lt;= <code>n</code>.</p>
<p>I want to extract the set of  <code>r</code> columns from <code>A</code> that are linearly independent, is there an implemented function in Pytorch that does this?</p>
<p>One way to do this is to perform a <a href="https://en.wikipedia.org/wiki/Rank_factorization" rel="noopener nofollow ugc">rank factorization algorithm</a>, and I want to know if such an algorithm is implemented in Pytorch, because it will be more efficient than implementing it by hand, and support back-propagation.</p>
<p>Thank you in advance for your help!</p>

username: AlphaBetaGamma96
cooked: <p>Hi <a class="mention" href="/u/nazareem">@nazareem</a>,</p>
<p>Perhaps <a href="https://pytorch.org/docs/stable/generated/torch.svd_lowrank.html" rel="noopener nofollow ugc">torch.svd_lowrank</a> might be what you’re looking for?</p>

username: nazareem
cooked: <p>Hi <a class="mention" href="/u/alphabetagamma96">@AlphaBetaGamma96</a></p>
<p>Thank you for your response!<br>
What does the <a href="https://pytorch.org/docs/stable/generated/torch.svd_lowrank.html" rel="noopener nofollow ugc">torch.svd_lowrank</a> exactly do? how it is different from the normal <a href="https://pytorch.org/docs/stable/generated/torch.linalg.svd.html#torch.linalg.svd" rel="noopener nofollow ugc">torch.linalg.svd</a>?</p>
<p>If I understand correctly, the resulting matrix will have the same number of columns as our input matrix <code>A</code>, no?</p>

username: AlphaBetaGamma96
cooked: <p>Well, it does a low-rank approximation (of a specified rank) to a singular value decomposition, and given you’re looking for a rank factorization (of a specified rank) it might be what you’re looking for. The documentation explains a bit more in detail but the full detail can be found in Algorithm 5.1 of <a href="https://arxiv.org/abs/0909.4061" class="inline-onebox" rel="noopener nofollow ugc">[0909.4061] Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions</a></p>
<p>In your original link of <a href="https://en.wikipedia.org/wiki/Rank_factorization" rel="noopener nofollow ugc">rank factorization algorthim</a>, <code>torch.low_svdrank</code> would be the SVD approach shown <a href="https://en.wikipedia.org/wiki/Rank_factorization#Singular_value_decomposition" rel="noopener nofollow ugc">there</a>.</p>

username: nazareem
cooked: <p>Hi <a class="mention" href="/u/alphabetagamma96">@AlphaBetaGamma96</a></p>
<p>Thank you for your help</p>
<p>I tried using it, it still returns a matrix that has the same dimension as my input matrix <code>A</code>, but that has rank <code>q</code>, specified as the parameter to the function.</p>
<p>This is not what I’m looking for, I’m looking for removing the linearly dependant column from matrix <code>A</code>, so I end up with a matrix that has the dimension <code>m</code> by <code>r</code>, that is full rank!</p>
<p>Basically, I’m looking for matrix C in <a href="https://en.wikipedia.org/wiki/Rank_factorization#Existence" rel="noopener nofollow ugc">this paragraph</a>, I hope this made my question clear!</p>