Repository: pytorch
Issue: Tensor to_sparse fails on large matrices
Labels: module: sparse,module: cuda,triaged

### 🐛 Describe the bug

```
>>> import torch
>>> torch.randn(46000, 46000, device='cuda:3').to_sparse()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
```

### Versions

```
Collecting environment information...                                                                                                                                                                              
PyTorch version: 1.13.0a0+936e930                                                                                                                                                                            
Is debug build: False                                                                                                                                                                                              
CUDA used to build PyTorch: 11.8                                                                                                                                                                                   
ROCM used to build PyTorch: N/A                                                                                                                                                                                    
                                                                                                                                                                                                                   
OS: Ubuntu 20.04.5 LTS (x86_64)                                                                                                                                                                                    
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0                                                                                                                                                                 
Clang version: Could not collect                                                                                                                                                                                   
CMake version: version 3.24.1      
Libc version: glibc-2.31                                                                                 
                                                    
Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.111-14-x86_64-with-glibc2.29
Is CUDA available: True               
CUDA runtime version: 11.8.89           
CUDA_MODULE_LOADING set to: LAZY        
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.7.0  
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.7.0                                               
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.7.0                                                
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.7.0                                                                                                                                                              
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:                                                                                                                                                                                                               
Architecture:                    x86_64                                                                                                                                                                            
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   48 bits physical, 48 bits virtual
CPU(s):                          54
On-line CPU(s) list:             0-53
Thread(s) per core:              1
Core(s) per socket:              1
Socket(s):                       54
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC 7662 64-Core Processor
Stepping:                        0
CPU MHz:                         1996.014
BogoMIPS:                        3992.02
Virtualization:                  AMD-V
L1d cache:                       3.4 MiB
L1i cache:                       3.4 MiB
L2 cache:                        27 MiB
L3 cache:                        864 MiB
NUMA node0 CPU(s):               0-53
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, IBRS_FW, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_
tsc cpuid extd_apicid pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch 
osvw perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch
_capabilities

Versions of relevant libraries:
[pip3] functorch==1.13.0a0+936e930
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.22.2
[pip3] pytorch-quantization==2.1.2
[pip3] torch==1.13.0a0+936e930
[pip3] torch-tensorrt==1.3.0a0
[pip3] torchtext==0.13.0a0+fae8e8c
[pip3] torchvision==0.15.0a0
[conda] Could not collect
```

cc @alexsamardzic @nikitaved @pearu @cpuhrsch @amjames @bhosmer @ptrblck