Issue 1:
🐛 Bug

Multiplication of sparse with dense tensor on GPU breaks

To Reproduce

I have the following implementation of page rank on pytorch using sparse tensors:

i = torch.LongTensor(idx)
values = torch.FloatTensor([1] * len(idx))
M = torch.sparse.FloatTensor(i.t(), values, torch.Size([4847571, 4847571]))
N = M.shape[1]
v = torch.rand(N, 1).float()
values = torch.FloatTensor([(1 - d)/N] * len(indices))
temp = torch.sparse.FloatTensor(i.t(), values, torch.Size([4847571,
4847571]))
if torch.cuda.is_available():
v = v.cuda()
M = M.cuda()
temp = temp.cuda()

v = v / torch.norm(v, 1)
M_hat = self.d * M + temp
for i in range(num_iter):
v = torch.mm(M_hat, v)

The indices are gained by parsing this graph.

On CPU everything runs fine. On GPU however I am receiving the following error:

v = torch.mm(M_hat, v)
RuntimeError: sub_iter.strides(0)[0] == 0 INTERNAL ASSERT FAILED at /pytorch/aten/src/ATen/native/cuda/Reduce.cuh:706, please report a bug to PyTorch.

Environment

Collecting environment information...
PyTorch version: 1.5.0
Is debug build: No
CUDA used to build PyTorch: 10.2
OS: Ubuntu 18.04.4 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.10.2
Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: Tesla P100-PCIE-16GB
Nvidia driver version: 440.82
cuDNN version: Could not collect
Versions of relevant libraries:
[pip3] numpy==1.18.3
[pip3] torch==1.5.0

If it helps more my machine is the first one from here.

Issue 2:
🐛 Bug

Calling .sum() on large cuda tensors throws the error.

To Reproduce

Steps to reproduce the behavior:

import torch
t = torch.zeros(5, 14400, 14400).cuda()
!nvidia-smi
t.sum(dim=0)

shows the following nvidia info

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.64.00 Driver Version: 418.67 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |
| N/A 40C P0 32W / 250W | 4663MiB / 16280MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+

and the traceback is

RuntimeError Traceback (most recent call last)
<ipython-input-1-bbe24183337c> in <module>()
2 t = torch.zeros(5, 14400, 14400).cuda()
3 get_ipython().system('nvidia-smi')
----> 4 t.sum(dim=0)

RuntimeError: sub_iter.strides(0)[0] == 0 INTERNAL ASSERT FAILED at /pytorch/aten/src/ATen/native/cuda/Reduce.cuh:706, please report a bug to PyTorch.

This behavior is reproduced with any (n, 14400, 14400) tensor for n>=5.

Note. I can easily create a (6, 14400, 14400) tensor - memory usage, in this case, is 5455MiB / 16280MiB.

Expected behavior

Returns a (14400, 14400) tensor

Environment

PyTorch version: 1.5.0+cu101

Is debug build: No

CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.3 LTS

GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

CMake version: version 3.12.0

Python version: 3.6

Is CUDA available: Yes

CUDA runtime version: 10.1.243

GPU models and configuration: GPU 0: Tesla P100-PCIE-16GB

Nvidia driver version: 418.67

cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5

Versions of relevant libraries:

[pip3] numpy==1.18.3

[pip3] pytorch-mighty==0.1.0

[pip3] torch==1.5.0+cu101

[pip3] torchfile==0.1.0

[pip3] torchsummary==1.5.1

[pip3] torchtext==0.3.1

[pip3] torchvision==0.6.0+cu101

[conda] Could not collect

Additional context

Google colab, GPU instance.

cc @ezyang @gchanan @zou3519 @bdhirsh @jbschlosser @anjali411 @ngimel @heitorschueroff