Autograd error for sparse tensor, Pytorch 1.6

username: Nanco-L
cooked: <p>Hello, I found that the autograd of the multiplication of sparse and dense tensor not work properly.<br>
Sample code is listed below.</p>
<pre><code class="lang-python">import torch                                                                                                                                                         
                                                                                                                                                                      
aa = torch.rand([4, 3], requires_grad=True)                                                                                                                          
ind = torch.tensor(                                                                                                                                                  
    [                                                                                                                                                                
        [0, 0, 1, 2],                                                                                                                                                
        [1, 2, 0, 0]                                                                                                                                                 
    ]                                                                                                                                                                
).long()                                                                                                                                                             
                                                                                                                                                                     
yy = torch.sparse_coo_tensor(ind, aa).to_dense().sum(1)                                                                                                              
bb = torch.rand(3, 4)                                                                                                                                                
                                                                                                                                                                     
yy = torch.einsum('ij,jk-&gt;ik', yy, bb) // Without this line, no error occur                                                                                                                   
                                                                                                                                                                     
dx = torch.autograd.grad(yy.sum(), aa)[0]                                                                                                                            
print(dx)
</code></pre>
<p>With this code, I face the error message like below:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
File "coo_tensor_deriv_test.py", line 16, in &lt;module&gt;                                                                                                                  
    dx = torch.autograd.grad(yy.sum(), aa)[0]
File "/home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/autograd/__init__.py", line 192, in grad
    inputs, allow_unused)                                                                                                                                                RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead. Exception raised from view at /pytorch/aten/src/ATen/native/TensorShape.cpp:1568 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7fca078141e2 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: at::native::view(at::Tensor const&amp;, c10::ArrayRef&lt;long&gt;) + 0x25c (0x7fca3f7079dc in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)                                                                           
frame #2: &lt;unknown function&gt; + 0x10f9579 (0x7fca3f962579 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #3: &lt;unknown function&gt; + 0x1114253 (0x7fca3f97d253 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #4: at::Tensor::view(c10::ArrayRef&lt;long&gt;) const + 0xdf (0x7fca3fbb137f in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #5: &lt;unknown function&gt; + 0xf5a67d (0x7fca3f7c367d in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #6: at::native::sparse_mask_cpu(at::Tensor const&amp;, at::Tensor const&amp;) + 0x73 (0x7fca3f7c44d3 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #7: &lt;unknown function&gt; + 0x1284419 (0x7fca3faed419 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #8: &lt;unknown function&gt; + 0xa56530 (0x7fca3f2bf530 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #9: at::Tensor c10::Dispatcher::call&lt;at::Tensor, at::Tensor const&amp;, at::Tensor const&amp;&gt;(c10::TypedOperatorHandle&lt;at::Tensor (at::Tensor const&amp;, at::Tensor const&amp;)&gt; const&amp;, at::Tensor const&amp;, at::Tensor const&amp;) const + 0xbc (0x7fca3faa781c in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #10: at::Tensor::sparse_mask(at::Tensor const&amp;) const + 0x4b (0x7fca3fb8e36b in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #11: &lt;unknown function&gt; + 0x2e9c6be (0x7fca417056be in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #12: &lt;unknown function&gt; + 0xa56530 (0x7fca3f2bf530 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #13: at::Tensor c10::Dispatcher::call&lt;at::Tensor, at::Tensor const&amp;, at::Tensor const&amp;&gt;(c10::TypedOperatorHandle&lt;at::Tensor (at::Tensor const&amp;, at::Tensor const&amp;)&gt; const&amp;, at::Tensor const&amp;, at::Tensor const&amp;) const + 0xbc (0x7fca3faa781c in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #14: at::Tensor::sparse_mask(at::Tensor const&amp;) const + 0x4b (0x7fca3fb8e36b in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #15: at::native::to_dense_backward(at::Tensor const&amp;, at::Tensor const&amp;) + 0x90 (0x7fca3f6ca6f0 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #16: &lt;unknown function&gt; + 0x129df90 (0x7fca3fb06f90 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #17: &lt;unknown function&gt; + 0x2de5b29 (0x7fca4164eb29 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #18: &lt;unknown function&gt; + 0xa56530 (0x7fca3f2bf530 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #19: at::Tensor c10::Dispatcher::call&lt;at::Tensor, at::Tensor const&amp;, at::Tensor const&amp;&gt;(c10::TypedOperatorHandle&lt;at::Tensor (at::Tensor const&amp;, at::Tensor const&amp;)&gt; const&amp;, at::Tensor const&amp;, at::Tensor const&amp;) const + 0xbc (0x7fca3faa781c in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #20: at::to_dense_backward(at::Tensor const&amp;, at::Tensor const&amp;) + 0x4b (0x7fca3f9f8e2b in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #21: torch::autograd::generated::ToDenseBackward::apply(std::vector&lt;at::Tensor, std::allocator&lt;at::Tensor&gt; &gt;&amp;&amp;) + 0x165 (0x7fca415a3f25 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #22: &lt;unknown function&gt; + 0x3375bb7 (0x7fca41bdebb7 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #23: torch::autograd::Engine::evaluate_function(std::shared_ptr&lt;torch::autograd::GraphTask&gt;&amp;, torch::autograd::Node*, torch::autograd::InputBuffer&amp;, std::shared_ptr&lt;torch::autograd::ReadyQueue&gt; const&amp;) + 0x1400 (0x7fca41bda400 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #24: torch::autograd::Engine::thread_main(std::shared_ptr&lt;torch::autograd::GraphTask&gt; const&amp;) + 0x451 (0x7fca41bdafa1 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #25: torch::autograd::Engine::execute_with_graph_task(std::shared_ptr&lt;torch::autograd::GraphTask&gt; const&amp;, std::shared_ptr&lt;torch::autograd::Node&gt;) + 0x37c (0x7fca41bd86bc in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #26: torch::autograd::python::PythonEngine::execute_with_graph_task(std::shared_ptr&lt;torch::autograd::GraphTask&gt; const&amp;, std::shared_ptr&lt;torch::autograd::Node&gt;) + 0x3c (0x7fca4f37376c in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #27: torch::autograd::Engine::execute(std::vector&lt;torch::autograd::Edge, std::allocator&lt;torch::autograd::Edge&gt; &gt; const&amp;, std::vector&lt;at::Tensor, std::allocator&lt;at::Tensor&gt; &gt; const&amp;, bool, bool, std::vector&lt;torch::autograd::Edge, std::allocator&lt;torch::autograd::Edge&gt; &gt; const&amp;) + 0x803 (0x7fca41bd79f3 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #28: torch::autograd::python::PythonEngine::execute(std::vector&lt;torch::autograd::Edge, std::allocator&lt;torch::autograd::Edge&gt; &gt; const&amp;, std::vector&lt;at::Tensor, std::allocator&lt;at::Tensor&gt; &gt; const&amp;, bool, bool, std::vector&lt;torch::autograd::Edge, std::allocator&lt;torch::autograd::Edge&gt; &gt; const&amp;) + 0x4e (0x7fca4f37356e in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #29: THPEngine_run_backward(THPEngine*, _object*, _object*) + 0xa54 (0x7fca4f374254 in /home/sr6/kyuhyun9.lee/env_ML/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
&lt;omitting python frames&gt;
frame #43: __libc_start_main + 0xf5 (0x7fca58637c05 in /lib64/libc.so.6)
frame #44: python() [0x400bda]
</code></pre>
<p>I cannot figure out the source of error. What is the source of this error?</p>

username: albanD
cooked: <p>Hi,</p>
<p>I am afraid the error comes from the fact that <code>einsum</code> does not support autograd for sparse inputs.<br>
You can open a feature request on github if you want this, but this is most likely going to be quite a bit of work.</p>

username: Nanco-L
cooked: <p>Thank you, albanD.</p>
<p>Actually, I use this kind of tensor (sparsetensor.to_dense().sum()) with <code>torch.nn.Linear</code>.<br>
I’m not sure whether torch.nn.Linear use <code>einsum</code> but error with <code>torch.nn.Linear</code> might be the same problem (lack of support for sparse tensor).<br>
I’ll open a feature request soon.</p>

username: albanD
cooked: <p>Ho actually the stack trace points to <code>ToDenseBackward</code> indeed. So the <code>.to_dense()</code> might be the issue here.<br>
But as mentioned above, the backward for sparse Tensor is not well supported I’m afraid.<br>
You can try to do <code>to_dense().clone().sum(1)   </code> ?</p>

username: Nanco-L
cooked: <p>Unfortunately, <code>to_dense().clone().sum()</code> does not work. <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"><br>
But thank you for your kind suggestion!</p>
<p>p.s. I open a feature request for this problem.</p>