Lack of implementation in current CUDA driver for sparse matrix multiplication (NYI CUDA sspadmm Runtime Error)

username: slobodaapl
cooked: <p>I’m trying to multiply a sparse matrix with a dense matrix, but I’ve encountered an error I can’t find mentioned anywhere on the forums or GitHub and I’m suspicious it is related to the current CUDA toolkit that I’m using, or perhaps the torch version, where the function is simply not implemented, but I’m not sure:</p>
<pre><code class="lang-auto">def ptr_loss(real: torch.Tensor, fake: torch.Tensor, laplacian_real: torch.Tensor, laplacian_fake: torch.Tensor):
    return real.smm(laplacian_fake).mm(real.T) + fake.smm(laplacian_real).mm(fake.T)
</code></pre>
<p>Here, <code>real</code> and <code>fake</code> are both a <code>[1,66536]</code> size tensor, and <code>laplacian_fake</code> and <code>laplacian_real</code> are a size <code>[65536,65536]</code> sparse tensor (coo). I’m attempting to use <code>torch.Tensor.smm</code> function to multiply the dense and sparse matrix, but when I reach this point in code, I get a Runtime error:</p>
<p><code>RuntimeError: NYI: CUDA sspaddmm is not implemented</code></p>
<p>I am using Torch 1.12.1 with CUDA 11.7 on driver version 516.94, on a RTX 3090. The Torch was installed from Conda, and running on Windows 10.</p>
<p>Any ideas how to fix this issue or what is the actual cause?</p>

username: KFrank
cooked: <p>Hi Tibor!</p>
<aside class="quote no-group" data-username="slobodaapl" data-post="1" data-topic="166729" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/slobodaapl/48/54612_2.png" class="avatar"> slobodaapl:</div>
<blockquote>
<p><code>RuntimeError: NYI: CUDA sspaddmm is not implemented</code></p>
<p>I am using Torch 1.12.1 with CUDA 11.7</p>
</blockquote>
</aside>
<p>I can reproduce your issue on both pytorch 1.12.0 / cuda 11.6 and pytorch<br>
1.14.0.dev20221014 / cuda 11.7.</p>
<p>This <a href="https://github.com/pytorch/pytorch/issues/76644" rel="noopener nofollow ugc">github issue</a> suggests using <code>torch.sparse.mm()</code> as a replacement.<br>
Note that <code>torch.sparse.mm()</code> returns a dense tensor (on both the cpu and<br>
gpu), while torch.smm() returns a sparse tensor (on the cpu and fails on the<br>
gpu).  (For your use case I doubt that returning a dense tensor matters, as<br>
you’ve already contracted over the large dimension.)</p>
<p>Not that it changes the core issue, but your attempted use of <code>.smm()</code> is<br>
incorrect in that <code>real.smm (laplacian_fake)</code> is attempting to left-multiply<br>
a dense matrix onto a sparse matrix while <code>.smm()</code> is intended to left-multiply<br>
a sparse matrix onto a dense matrix.</p>
<p>Why pytorch doesn’t have cuda support for <code>.smm()</code> I don’t know.  However,<br>
pytorch’s support for sparse tensors is in general incomplete and buggy.</p>
<p>Best.</p>
<p>K. Frank</p>

username: slobodaapl
cooked: <p>Hello Frank, and thank you for the insight on the issue!</p>
<p>I have tried using <code>torch.sparse.mm</code> and I can confirm it has fixed the issue.</p>
<p>I want to point out to others in case they encounter this that <code>torch.sparse.mm</code> requires for the first input to be the sparse matrix, and the output is a dense matrix. You may need to take advantage of matrix multiplication associativity to reorder your operations to use the function.</p>