Repository: pytorch
Issue: The CUDA download should include cusparseLt.dll since xformers now wants to use it in the context of torch
Labels: module: binaries,module: windows,triaged

### 🚀 The feature, motivation and pitch

cusparseLt.dll is only a 15MB library, which is bizarrely not included in the default CUDA toolkit install and available for download here  (perhaps the same reason it was left out of torch?)
xformers looks for and loads 
        Path(torch._C.__file__).parent / "lib" / "libcusparseLt*.so.0"
in its sp24.py for 2:4 sparsity model training features implemented in the last release.  Making that feature OS agnostic (for operating systems supporting CUDA at least) appears to be as simple as adding to a check for the OS being Windows and returning the string "cusparseLt.dll" instead since the function just spits out a lib name, and of course cusparseLt.dll actually existing in torch/lib.   Since I was able to download it from their developer site, put the dll in my local torch/lib install, and modify my local xformers install and get it saying everything ok including:

`sp24._cslt_sparse_mm@0.5.2:                        available
`
Looks like a good sign but I can't do much more testing at the moment.  Hopefully this is premature and the lib is getting added to enable the backend in sparse/semi_structured.py anyway, and just didn't exist in 2.2 since it's being worked on heavily.  Building xformers isn't too difficult but I don't have the time to pull the nightlies and check for the file right now sadly. 

There shouldn't be any impact on torch itself unless something decides to use it and it has a functionality-breaking bug of some sort but I'm pretty sure it's the same library from linux.  

### Alternatives

Documenting which features don't work on Windows in "what's new" based on things that aren't ported, although in this case it's easier to just include the file.  

### Additional context
Will be filing a feature request over at xformers to enable the x-platform check there, too.  

```
C:\Programs\ComfyUI>python -m xformers.info
ModuleNotFoundError: No module named 'triton'
Unable to find python bindings at /usr/local/dcgm/bindings/python3. No data will be captured.
xFormers 0.0.24
memory_efficient_attention.cutlassF:               available
memory_efficient_attention.cutlassB:               available
memory_efficient_attention.decoderF:               available
memory_efficient_attention.flshattF@v2.3.6:        available
memory_efficient_attention.flshattB@v2.3.6:        available
memory_efficient_attention.smallkF:                available
memory_efficient_attention.smallkB:                available
memory_efficient_attention.tritonflashattF:        unavailable
memory_efficient_attention.tritonflashattB:        unavailable
memory_efficient_attention.triton_splitKF:         unavailable
indexing.scaled_index_addF:                        unavailable
indexing.scaled_index_addB:                        unavailable
indexing.index_select:                             unavailable
sequence_parallel_fused.write_values:              unavailable
sequence_parallel_fused.wait_values:               unavailable
sequence_parallel_fused.cuda_memset_32b_async:     unavailable
sp24.sparse24_sparsify_both_ways:                  available
sp24.sparse24_apply:                               available
sp24.sparse24_apply_dense_output:                  available
sp24._sparse24_gemm:                               available
sp24._cslt_sparse_mm@0.5.2:                        available
swiglu.dual_gemm_silu:                             available
swiglu.gemm_fused_operand_sum:                     available
swiglu.fused.p.cpp:                                available
is_triton_available:                               False
pytorch.version:                                   2.2.0+cu121
pytorch.cuda:                                      available
gpu.compute_capability:                            8.9
gpu.name:                                          NVIDIA GeForce RTX 4090
dcgm_profiler:                                     unavailable
build.info:                                        available
build.cuda_version:                                1201
build.python_version:                              3.11.7
build.torch_version:                               2.2.0+cu121
build.env.TORCH_CUDA_ARCH_LIST:                    5.0+PTX 6.0 6.1 7.0 7.5 8.0+PTX 9.0
build.env.XFORMERS_BUILD_TYPE:                     Release
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              None
build.env.XFORMERS_PACKAGE_FROM:                   wheel-v0.0.24
build.nvcc_version:                                12.1.66
source.privacy:                                    open source
```

cc @seemethere @malfet @osalpekar @atalman @peterjc123 @mszhanyi @skyline75489 @nbcsm @vladimir-aubrecht @iremyux @Blackhex @cristianPanaite