ROCm6.2 build of Pytroch is faling for Llama3.2 vision model

username: Syed_Hussain
cooked: <p>Hi,<br>
I found that ROCM6.2 version of PyTorch using nightly build is failing for llama3.2-11B and MiniCPM-6.2 version. I’m using a system with AMD GPU Radeon RRO W7900 in Ubuntu 22.4 OS.</p>
<h2><a name="p-457269-the-issue-is-1" class="anchor" href="#p-457269-the-issue-is-1"></a>The issue is:</h2>
<h2><a name="p-457269-runtimeerror-attempting-to-use-hipblaslt-on-a-unsupported-architecture-2" class="anchor" href="#p-457269-runtimeerror-attempting-to-use-hipblaslt-on-a-unsupported-architecture-2"></a><strong>RuntimeError: Attempting to use hipBLASLt on a unsupported architecture!</strong></h2>
<p>This issue has been around for more than 36 hours. I hope someone will fix it ASAP.</p>
<p>Thanks,<br>
Syed Hussain</p>

username: en_zo
cooked: <p>same for me but on many transformers models, not just llama models. It is making me crazy. I can’t even use pytorch with my rx 7900xtx</p>

username: Syed_Hussain
cooked: <p>Make a new environent with latest PyTorch Stable version 2.5 for RCOM6.2, which is now available.<br>
That should solve the problem.</p>
<p>~Syed</p>

username: en_zo
cooked: <p>I’m already on the latest version of pytorch. I found this: <a href="https://github.com/pytorch/pytorch/issues/138067" class="inline-onebox" rel="noopener nofollow ugc">Attempting to use hipBLASLt on a unsupported architecture! · Issue #138067 · pytorch/pytorch · GitHub</a> which means my gpu architecture is failing with hipblastlt. They say to downgrade the pytorch version to an older version where hipblastlt didn’t cause any problem until amd fixes the problem which means i have to try with different versions of pytorch and rocm until i don’t have this problem. Here are the pytorch version I’m going to try with rocm 6.2.2:<br>
2.3, 2.2, 2.1, 2.0, 1.13</p>
<p>as a conclusion, pytorch 2.5/rocm6.2.2 is unusable for gpu architectures like gfx1100. You and amd should update your site because it took me 5 days to realize it. I really thought i had a useless 1000$ gpu</p>

username: Syed_Hussain
cooked: <p>Hi,</p>
<p>I’ve conda environment with Python3.11 and installed:</p>
<p>pip3 install torch torchvision torchaudio --index-url <a href="https://download.pytorch.org/whl/rocm6.2" rel="noopener nofollow ugc">https://download.pytorch.org/whl/rocm6.2</a><br>
pip install --upgrade-strategy eager optimum[amd]</p>
<p>This will install the latest PyTorch2.5 stable version for ROCm6.2<br>
I tested Llama3.2-11B a few minutes back, it is working properly.</p>
<p>I had the same error using the earlier nightly PyTorch version for ROCm6.2.<br>
Using the latest stable release of PyTorch2.5 I found this issue is no more there.</p>
<p>You can try with a fresh conda environment.</p>
<p>~Syed</p>

username: en_zo
cooked: <p>thank you the command “pip3 install torch torchvision torchaudio --index-url <a href="https://download.pytorch.org/whl/rocm6.2" rel="noopener nofollow ugc">https://download.pytorch.org/whl/rocm6.2</a>” works but i still have the warning: UserWarning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas.</p>
<p>I guess i’ll just wait that you and amd work together to enable hipBLASLt on gfx1100 gpus</p>