ROCm + torch + xformers

username: rrunner77
cooked: <p>Hi,<br>
I am trying to get the xformers running on AMD GPU with the A1111.<br>
I have overcome many issues but now I am stuck on this one:</p>
<pre><code class="lang-auto">File "/home/rrunner/ai/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/_ops.py", line 1156, in __call__
        return self._op(*args, **(kwargs or {}))
    NotImplementedError: Could not run 'xformers::efficient_attention_forward_ck' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'xformers::efficient_attention_forward_ck' is only available for these backends: [Meta, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradMPS, AutogradXPU, AutogradHPU, AutogradLazy, AutogradMTIA, AutogradMeta, Tracer, AutocastCPU, AutocastMTIA, AutocastXPU, AutocastMPS, AutocastCUDA, FuncTorchBatched, BatchedNestedTensor, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PreDispatch, PythonDispatcher].
</code></pre>
<p>When I test the xformers I get this:</p>
<pre><code class="lang-auto">python -m xformers.info
xFormers 0.0.30+4cfab36.d20250207
memory_efficient_attention.ckF:                    available
memory_efficient_attention.ckB:                    available
memory_efficient_attention.ck_decoderF:            available
memory_efficient_attention.ck_splitKF:             available
memory_efficient_attention.cutlassF-pt:            available
memory_efficient_attention.cutlassB-pt:            available
memory_efficient_attention.fa2F@0.0.0:             unavailable
memory_efficient_attention.fa2B@0.0.0:             unavailable
memory_efficient_attention.fa3F@0.0.0:             unavailable
memory_efficient_attention.fa3B@0.0.0:             unavailable
memory_efficient_attention.triton_splitKF:         available
indexing.scaled_index_addF:                        available
indexing.scaled_index_addB:                        available
indexing.index_select:                             available
sequence_parallel_fused.write_values:              available
sequence_parallel_fused.wait_values:               available
sequence_parallel_fused.cuda_memset_32b_async:     available
sp24.sparse24_sparsify_both_ways:                  available
sp24.sparse24_apply:                               available
sp24.sparse24_apply_dense_output:                  available
sp24._sparse24_gemm:                               available
sp24._cslt_sparse_mm_search@0.0.0:                 available
sp24._cslt_sparse_mm@0.0.0:                        available
swiglu.dual_gemm_silu:                             available
swiglu.gemm_fused_operand_sum:                     available
swiglu.fused.p.cpp:                                available
is_triton_available:                               True
pytorch.version:                                   2.7.0.dev20250206+rocm6.3
pytorch.cuda:                                      available
gpu.compute_capability:                            11.0
gpu.name:                                          AMD Radeon RX 7900 XTX
dcgm_profiler:                                     unavailable
build.info:                                        available
build.cuda_version:                                None
build.hip_version:                                 None
build.python_version:                              3.10.16
build.torch_version:                               2.7.0.dev20250206+rocm6.3
build.env.TORCH_CUDA_ARCH_LIST:                    None
build.env.PYTORCH_ROCM_ARCH:                       None
build.env.XFORMERS_BUILD_TYPE:                     None
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              None
build.env.XFORMERS_PACKAGE_FROM:                   None
source.privacy:                                    open source
</code></pre>
<p>I am not sure why torch is reporting that this function is not available or that it was not compiled with the CUDA support. There is no CUDA support but the xformers should work also with ROCm if I got it right.</p>
<pre><code class="lang-auto">python3.10 -m torch.utils.collect_env
Collecting environment information...
PyTorch version: 2.7.0.dev20250206+rocm6.3
Is debug build: False
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: 6.3.42131-fa1d09cbd

OS: Ubuntu 24.04.1 LTS (x86_64)
GCC version: (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
Clang version: Could not collect
CMake version: version 3.28.3
Libc version: glibc-2.39

Python version: 3.10.16 (main, Dec  4 2024, 08:53:38) [GCC 13.2.0] (64-bit runtime)
Python platform: Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: AMD Radeon RX 7900 XTX (gfx1100)
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: 6.3.42131
MIOpen runtime version: 3.3.0
Is XNNPACK available: True
</code></pre>
<p>Does anyone have a clue why ti is not working. I know that if I would by NVIDIA I would have no issue.</p>
<p>or anyone has a idea how to test the xformers with torch. I assume that the error is that this part was probably left out during the compilation of the torch.</p>
<p>M.</p>

username: Eric_D
cooked: <p>Hello! rx 7800xt with the same issue. Have you found a solution?<br>
I’ve tons of combinations of Pytorch and ROCm and xformers, with no solution. Interner doesent help much either… most people seem as clueless as I do to find a solution…</p>

username: rrunner77
cooked: <p>No I started to use the --opt-sdp-attention.<br>
I had already a discussion with the ROCm guys on thier git and it seems that the composable kernels do not support the backend functions. The issue is that by the page it should be supported.</p>
<p>Anyway with the sdp attention you get better performance on a WLS.<br>
On linux I did not seen any difference.</p>