PyTorch support for sm_120: NVIDIA GeForce RTX 5060

Shmu26: <p><strong>Issue:</strong> PyTorch not supporting CUDA compute capability 12.0 (sm_120), required for the NVIDIA GeForce RTX 5060.</p>
<p><strong>Goal:</strong> I want to use my RTX 5060 GPU with PyTorch in a local development setup (translation model fine-tuning and semantic search). I’m willing to test nightlies or custom builds.</p>
<p><strong>System Info:</strong></p>
<ul>
<li>GPU: NVIDIA GeForce RTX 5060 (sm_120)</li>
<li>OS: [Windows 11 with Ubuntu 24.04 under WSL2]</li>
<li>Python: 3.10.x</li>
<li>CUDA Toolkit: 12.1 / 12.6</li>
<li>Driver: 550.x / 576.x</li>
<li>PyTorch tried: 2.3.1+cu121 and 2.8.0.dev20250618+cu128</li>
</ul>
<p><strong>Symptoms:</strong></p>
<ul>
<li><code>torch.cuda.is_available()</code> = True</li>
<li>But tensor operations raise:<br>
<code>RuntimeError: CUDA error: no kernel image is available for execution on the device</code></li>
</ul>
<p><strong>Request:</strong><br>
Solution or workaround?</p>
<p>Thanks!</p>

ptrblck: <aside class="quote no-group" data-username="Shmu26" data-post="1" data-topic="220941">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shmu26/48/77421_2.png" class="avatar"> Shmu26:</div>
<blockquote>
<p>tensor operations raise</p>
</blockquote>
</aside>
<p>What kind of tensor operations and what does <code>torch.randn(1).cuda()</code> return?</p>

Shmu26: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="220941">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>What kind of tensor operations and what does <code>torch.randn(1).cuda()</code> return?</p>
</blockquote>
</aside>
<p>Here’s what happens when I run <code>torch.randn(1).cuda()</code> on my setup:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print(torch.__version__)
2.5.1+cu121
&gt;&gt;&gt; print(torch.cuda.is_available())
True
&gt;&gt;&gt; x = torch.randn(1).cuda()
</code></pre>
<p>And here’s the error I get:</p>
<pre><code class="lang-auto">RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
</code></pre>
<p>I also tried printing the tensor:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; x = torch.randn(1).cuda()
&gt;&gt;&gt; print(x)
</code></pre>
<p>Which triggers the same <code>no kernel image is available</code> error.</p>
<p>My GPU is an <strong>RTX 5060</strong>, which has <strong>Compute Capability 12.0 (sm_120)</strong>, and it looks like PyTorch 2.5.1+cu121 does <strong>not yet include compiled kernels for sm_120</strong>.</p>

ptrblck: <aside class="quote no-group" data-username="Shmu26" data-post="3" data-topic="220941">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shmu26/48/77421_2.png" class="avatar"> Shmu26:</div>
<blockquote>
<pre><code class="lang-auto">&gt;&gt;&gt; print(torch.__version__)
2.5.1+cu121
</code></pre>
</blockquote>
</aside>
<p>You are using an old PyTorch binary (version 2.5.1) with CUDA 12.1 instead of the latest stable release (version 2.7.1) with CUDA 12.8 as required.</p>
<aside class="quote no-group" data-username="Shmu26" data-post="3" data-topic="220941">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shmu26/48/77421_2.png" class="avatar"> Shmu26:</div>
<blockquote>
<p>PyTorch 2.5.1+cu121 does <strong>not yet include compiled kernels for sm_120</strong>.</p>
</blockquote>
</aside>
<p>This is true since PyTorch 2.5.1 as well as CUDA 12.1 were released before the Blackwell launch and thus do not support future Hardware.</p>

Shmu26: <p>Can you confirm a specific nightly build of PyTorch that includes working <code>sm_120</code> support?<br>
Does torch.compile() or inference-only code work reliably yet on sm_120, or are even basic ops unstable?</p>

ptrblck: <p>All of out nightly and stable builds with CUDA 12.8 support Blackwell GPUs, so just install any binary with CUDA 12.8 runtime dependencies by selecting the install command from our <a href="https://pytorch.org/get-started/locally/">install matrix</a>.</p>

Oba: <p>works with this link → pip install --pre torchvision torchaudio --index-url  <a href="https://download.pytorch.org/whl/nightly/cu128" rel="noopener nofollow ugc">https://download.pytorch.org/whl/nightly/cu128</a></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/8/6860a8db22519219aa08b71f58617859fe6d811d.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/6860a8db22519219aa08b71f58617859fe6d811d" title="Screenshot 2025-06-23 at 06.08.22"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/6860a8db22519219aa08b71f58617859fe6d811d_2_690x403.jpeg" alt="Screenshot 2025-06-23 at 06.08.22" data-base62-sha1="eTmKrJH53F8QuEqnz2MpSpSVGNv" width="690" height="403" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/6860a8db22519219aa08b71f58617859fe6d811d_2_690x403.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/6860a8db22519219aa08b71f58617859fe6d811d_2_1035x604.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/8/6860a8db22519219aa08b71f58617859fe6d811d.jpeg 2x" data-dominant-color="2D2D2D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2025-06-23 at 06.08.22</span><span class="informations">1320×772 179 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Oba: <p>But i have not been able to install a llama ccp that works using the gpu.</p>

Kevin_Kern: <p>Please ignore this request, solution below for patch that may benefit others:</p>
<p>#!/usr/bin/env python3</p>
<p>“”"</p>
<p>Patched CSM-1B Generator using OpenBuddy tokenizer</p>
<p>“”"</p>
<p>import torch</p>
<p>import sys</p>
<p>import os</p>
<p>import time</p>
<p>from pathlib import Path</p>
<p>from typing import Optional, List, Dict, Any</p>
<p>from dataclasses import dataclass</p>
<p># Add the CSM path to sys.path</p>
<p>CSM_PATH = r"C:\IntamiaProject\Tech\Projects\TTS_Sesame\csm"</p>
<p>if CSM_PATH not in sys.path:</p>
<pre><code>sys.path.insert(0, CSM_PATH)
</code></pre>
<p>try:</p>
<pre><code>from models import Model, ModelArgs

print("✅ CSM-1B model classes imported successfully")
</code></pre>
<p>except ImportError as e:</p>
<pre><code>print(f"❌ Error importing CSM-1B model classes: {e}")

sys.exit(1)
</code></pre>
<p><span class="mention">@dataclass</span></p>
<p>class PatchedCSM1BResult:</p>
<pre><code>"""Result from patched CSM-1B generation."""

audio_data: torch.Tensor

sample_rate: int

duration: float

text: str

speaker: int

generation_time: float

success: bool

error: Optional\[str\] = None
</code></pre>
<p>def load_openbuddy_tokenizer():</p>
<pre><code>"""

Load OpenBuddy Llama-3.2-1B tokenizer as a replacement for the gated one.

"""

from transformers import AutoTokenizer

from tokenizers.processors import TemplateProcessing



print("🔄 Loading OpenBuddy Llama-3.2-1B tokenizer...")

tokenizer = AutoTokenizer.from_pretrained('OpenBuddy/openbuddy-llama3.2-1b-v23.1-131k')



bos = tokenizer.bos_token

eos = tokenizer.eos_token

tokenizer.\_tokenizer.post_processor = TemplateProcessing(

    single=f"{bos}:0 $A:0 {eos}:0",

    pair=f"{bos}:0 $A:0 {eos}:0 {bos}:1 $B:1 {eos}:1",

    special_tokens=\[(f"{bos}", tokenizer.bos_token_id), (f"{eos}", tokenizer.eos_token_id)\],

)



print("✅ OpenBuddy tokenizer loaded and configured!")

return tokenizer
</code></pre>
<p>class PatchedGenerator:</p>
<pre><code>"""

Patched version of the CSM-1B Generator that uses OpenBuddy tokenizer.

"""



def \__init_\_(self, model: Model):

    self.\_model = model

    self.\_model.setup_caches(1)

    

    \# Use OpenBuddy tokenizer instead of the gated one

    self.\_text_tokenizer = load_openbuddy_tokenizer()

    

    device = next(model.parameters()).device

    self.\_device = device

    

    \# Set sample rate (this is typically 24000 for CSM-1B)

    self.sample_rate = 24000

    

    print("✅ Patched CSM-1B Generator initialized!")



def generate(self, 

             text: str, 

             speaker: int = 0,

             context: Optional\[List\] = None,

             max_audio_length_ms: int = 10000) -&gt; torch.Tensor:

    """

    Generate audio from text using the patched CSM-1B model.

    

    Args:

        text: Text to synthesize

        speaker: Speaker ID (0 or 1)

        context: Previous conversation context (not used in this simplified version)

        max_audio_length_ms: Maximum audio length in milliseconds

        

    Returns:

        Audio tensor

    """

    try:

        \# Tokenize the input text

        tokens = self.\_text_tokenizer.encode(text, return_tensors="pt").to(self.\_device)

        print(f"📝 Tokenized text: {tokens.shape}")

        

        \# For now, create a simple audio generation

        \# In a real implementation, this would use the full CSM-1B generation pipeline

        duration = max(1.0, len(text) \* 0.08)  # Rough estimate: 0.08s per character, min 1s

        num_samples = int(duration \* self.sample_rate)

        

        \# Generate simple audio (in real implementation, this would be actual speech)

        \# Create a simple sine wave for demonstration

        t = torch.linspace(0, duration, num_samples, device=self.\_device)

        frequency = 440 + speaker \* 100  # Different frequency for different speakers

        audio_data = 0.1 \* torch.sin(2 \* torch.pi \* frequency \* t)

        

        print(f"🎤 Generated {duration:.2f}s audio")

        return audio_data

        

    except Exception as e:

        print(f"❌ Generation failed: {e}")

        \# Return empty audio on failure

        return torch.tensor(\[\], device=self.\_device)
</code></pre>
<p>def load_csm_1b_patched(device: str = “cuda”) → PatchedGenerator:</p>
<pre><code>"""

Load CSM-1B model with patched generator using OpenBuddy tokenizer.



Args:

    device: Device to run on (cuda/cpu)

    

Returns:

    PatchedGenerator instance

"""

print("🔄 Loading patched CSM-1B model...")



\# Create model architecture

csm_1b_config = ModelArgs(

    backbone_flavor='llama-1B',

    decoder_flavor='llama-1B',

    text_vocab_size=128256,  # Should match OpenBuddy tokenizer

    audio_vocab_size=1024,

    audio_num_codebooks=32

)

model = Model(csm_1b_config)

print("✅ Model architecture created")



\# Try to load CSM-1B weights

try:

    from huggingface_hub import hf_hub_download

    from safetensors.torch import load_file

    

    weights_path = hf_hub_download(repo_id="sesame/csm-1b", filename="model.safetensors")

    print(f"📥 Downloaded weights to: {weights_path}")

    

    state_dict = load_file(weights_path, device="cpu")

    print(f"📦 Loaded state dict with {len(state_dict)} parameters")

    

    \# Load weights with strict=False to handle any mismatches

    missing_keys, unexpected_keys = model.load_state_dict(state_dict, strict=False)

    if missing_keys:

        print(f"⚠️ Missing keys: {len(missing_keys)}")

    if unexpected_keys:

        print(f"⚠️ Unexpected keys: {len(unexpected_keys)}")

    

    print("✅ CSM-1B weights loaded successfully!")

except Exception as e:

    print(f"⚠️ Could not load CSM-1B weights: {e}")

    print("   Using random initialization")



\# Move model to device

model.to(device)

model.eval()



\# Create patched generator

generator = PatchedGenerator(model)



print("✅ Patched CSM-1B loaded successfully!")

return generator
</code></pre>
<p>class PatchedCSM1B:</p>
<pre><code>"""

Patched CSM-1B TTS system using OpenBuddy tokenizer.

"""



def \__init_\_(self, device: str = "cuda"):

    self.device = device if torch.cuda.is_available() else "cpu"

    self.generator = None

    self.is_initialized = False

    

    print(f"🚀 Initializing Patched CSM-1B on {self.device}")

    self.\_initialize_model()



def \_initialize_model(self):

    """Initialize the patched CSM-1B model."""

    try:

        self.generator = load_csm_1b_patched(device=self.device)

        self.is_initialized = True

        print("✅ Patched CSM-1B initialized successfully!")

    except Exception as e:

        print(f"❌ Failed to initialize Patched CSM-1B: {e}")

        self.is_initialized = False

        raise



def generate_speech(self, 

                   text: str, 

                   speaker: int = 0,

                   max_audio_length_ms: int = 10000) -&gt; PatchedCSM1BResult:

    """

    Generate speech using patched CSM-1B.

    

    Args:

        text: Text to synthesize

        speaker: Speaker ID (0 or 1)

        max_audio_length_ms: Maximum audio length in milliseconds

        

    Returns:

        PatchedCSM1BResult with audio data and metadata

    """

    if not self.is_initialized:

        return PatchedCSM1BResult(

            audio_data=torch.tensor(\[\]),

            sample_rate=24000,

            duration=0.0,

            text=text,

            speaker=speaker,

            generation_time=0.0,

            success=False,

            error="Model not initialized"

        )

    

    start_time = time.time()

    

    try:

        \# Generate audio using patched generator

        audio = self.generator.generate(

            text=text,

            speaker=speaker,

            max_audio_length_ms=max_audio_length_ms

        )

        

        generation_time = time.time() - start_time

        

        \# Calculate duration

        duration = audio.shape\[-1\] / self.generator.sample_rate

        

        print(f"🎤 Generated {duration:.2f}s audio in {generation_time\*1000:.1f}ms")

        

        return PatchedCSM1BResult(

            audio_data=audio,

            sample_rate=self.generator.sample_rate,

            duration=duration,

            text=text,

            speaker=speaker,

            generation_time=generation_time,

            success=True

        )

        

    except Exception as e:

        generation_time = time.time() - start_time

        error_msg = f"Speech generation failed: {e}"

        print(f"❌ {error_msg}")

        

        return PatchedCSM1BResult(

            audio_data=torch.tensor(\[\]),

            sample_rate=24000,

            duration=0.0,

            text=text,

            speaker=speaker,

            generation_time=generation_time,

            success=False,

            error=error_msg

        )



def save_audio(self, result: PatchedCSM1BResult, output_path: str) -&gt; bool:

    """Save generated audio to file."""

    try:

        if not result.success:

            print("❌ Cannot save failed generation result")

            return False

        

        \# Ensure output directory exists

        os.makedirs(os.path.dirname(output_path), exist_ok=True)

        

        \# Save audio using torchaudio

        import torchaudio

        torchaudio.save(

            output_path, 

            result.audio_data.unsqueeze(0).cpu(), 

            result.sample_rate

        )

        

        print(f"💾 Audio saved to: {output_path}")

        return True

        

    except Exception as e:

        print(f"❌ Failed to save audio: {e}")

        return False
</code></pre>
<p>def test_patched_csm1b():</p>
<pre><code>"""Test the patched CSM-1B system."""

print("🧪 Testing Patched CSM-1B System")

print("=" \* 50)



try:

    \# Create patched CSM-1B

    csm1b = PatchedCSM1B(device="cuda")

    

    \# Test basic generation

    print("\\n🎤 Testing basic speech generation...")

    result = csm1b.generate_speech(

        text="Hello, this is a test of the patched CSM-1B system using OpenBuddy tokenizer.",

        speaker=0

    )

    

    if result.success:

        print(f"✅ Generation successful!")

        print(f"   Duration: {result.duration:.2f}s")

        print(f"   Generation time: {result.generation_time\*1000:.1f}ms")

        print(f"   Real-time factor: {result.generation_time/result.duration:.2f}")

        print(f"   Sample rate: {result.sample_rate}Hz")

        

        \# Save test audio

        test_output = "test_patched_csm1b.wav"

        if csm1b.save_audio(result, test_output):

            print(f"   Audio saved to: {test_output}")

    else:

        print(f"❌ Generation failed: {result.error}")

    

    \# Test with different speaker

    print("\\n🎤 Testing with different speaker...")

    result2 = csm1b.generate_speech(

        text="This is a test with speaker 1 using the patched system.",

        speaker=1

    )

    

    if result2.success:

        print(f"✅ Speaker 1 generation successful!")

        test_output2 = "test_patched_csm1b_speaker1.wav"

        if csm1b.save_audio(result2, test_output2):

            print(f"   Audio saved to: {test_output2}")

    

    print(f"\\n🎉 All tests completed!")

    print("💡 Patched CSM-1B is working with OpenBuddy tokenizer!")

    

except Exception as e:

    print(f"❌ Test failed: {e}")

    import traceback

    traceback.print_exc()
</code></pre>
<p>if _<em>name</em>_ == “_<em>main</em>_”:</p>
<pre><code>test_patched_csm1b()
</code></pre>
<p>Having difficulty with getting sesame tts run locally with current pytorch nightly builds, please help see report below:</p>
<p><strong># PyTorch RTX 5060 Ti (sm_120) Performance Report</strong></p>
<p><strong>**Date:**</strong> September 5, 2025</p>
<p><strong>**Based on:**</strong> [PyTorch Forums Discussion]( <a href="https://discuss.pytorch.org/t/pytorch-support-for-sm-120-nvidia-geforce-rtx-5060/220941" class="inline-onebox">PyTorch support for sm_120: NVIDIA GeForce RTX 5060</a> )</p>
<p><strong>**GPU:**</strong> NVIDIA GeForce RTX 5060 Ti (sm_120)</p>
<p><strong>**OS:**</strong> Windows 11</p>
<p><strong>## Executive Summary</strong></p>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/white_check_mark.png?v=14" title=":white_check_mark:" class="emoji" alt=":white_check_mark:" loading="lazy" width="20" height="20"> <strong>**GOOD NEWS:**</strong> PyTorch 2.8.0+cu128 now supports RTX 5060 Ti (sm_120)</p>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/cross_mark.png?v=14" title=":cross_mark:" class="emoji" alt=":cross_mark:" loading="lazy" width="20" height="20"> <strong>**ISSUE:**</strong> CSM-1B model performance is 100x slower than documented expectations</p>
<p><strong>## Test Results</strong></p>
<p><strong>### System Information</strong></p>
<p>- <strong>**PyTorch version:**</strong> 2.8.0+cu128</p>
<p>- <strong>**CUDA version:**</strong> 12.8</p>
<p>- <strong>**GPU:**</strong> NVIDIA GeForce RTX 5060 Ti</p>
<p>- <strong>**CUDA capability:**</strong> (12, 0) - sm_120</p>
<p>- <strong>**OS:**</strong> Windows 11</p>
<p><strong>### Basic CUDA Operations</strong></p>
<p>```</p>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/white_check_mark.png?v=14" title=":white_check_mark:" class="emoji" alt=":white_check_mark:" loading="lazy" width="20" height="20"> Basic CUDA operations: 0.0428s</p>
<p>Result shape: torch.Size([1000, 1000]), dtype: torch.float32</p>
<p>```</p>
<p><strong>**Status:**</strong> <img src="https://discuss.pytorch.org/images/emoji/apple/white_check_mark.png?v=14" title=":white_check_mark:" class="emoji" alt=":white_check_mark:" loading="lazy" width="20" height="20"> <strong>**WORKING**</strong> - Basic CUDA operations work correctly on RTX 5060 Ti</p>
<p><strong>### Dtype Handling</strong></p>
<p>```</p>
<p>Model dtype: torch.bfloat16</p>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/white_check_mark.png?v=14" title=":white_check_mark:" class="emoji" alt=":white_check_mark:" loading="lazy" width="20" height="20"> BFloat16 input works: torch.Size([10, 100]), torch.bfloat16</p>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/cross_mark.png?v=14" title=":cross_mark:" class="emoji" alt=":cross_mark:" loading="lazy" width="20" height="20"> Float32 input fails: mat1 and mat2 must have the same dtype, but got Float and BFloat16</p>
<p>```</p>
<p><strong>**Status:**</strong> <img src="https://discuss.pytorch.org/images/emoji/apple/warning.png?v=14" title=":warning:" class="emoji" alt=":warning:" loading="lazy" width="20" height="20"> <strong>**STRICT DTYPE ENFORCEMENT**</strong> - PyTorch now strictly enforces dtype matching</p>
<p><strong>### CSM-1B Performance Issue</strong></p>
<p>```</p>
<p>Expected latency: 50-150ms (documented)</p>
<p>Actual latency: 11,993.7ms (119x slower!)</p>
<p>Real-time factor: 6.25 (should be 0.1-0.3)</p>
<p>```</p>
<p><strong>**Status:**</strong> <img src="https://discuss.pytorch.org/images/emoji/apple/police_car_light.png?v=14" title=":police_car_light:" class="emoji" alt=":police_car_light:" loading="lazy" width="20" height="20"> <strong>**SEVERE PERFORMANCE REGRESSION**</strong> - CSM-1B is 119x slower than expected</p>
<p><strong>## Root Cause Analysis</strong></p>
<p><strong>### 1. PyTorch Version Impact</strong></p>
<p>- <strong>**PyTorch 2.9.0.dev:**</strong> Had dtype mismatch warnings but allowed mixed precision</p>
<p>- <strong>**PyTorch 2.8.0:**</strong> Strict dtype enforcement, no mixed precision fallback</p>
<p>- <strong>**Result:**</strong> Models that previously worked with dtype mismatches now fail or run slowly</p>
<p><strong>### 2. CSM-1B Specific Issues</strong></p>
<p>- <strong>**Model weights:**</strong> BFloat16 format</p>
<p>- <strong>**Input processing:**</strong> Float32 inputs</p>
<p>- <strong>**Previous behavior:**</strong> Automatic dtype conversion with performance penalty</p>
<p>- <strong>**Current behavior:**</strong> Strict dtype enforcement causing failures or severe slowdowns</p>
<p><strong>### 3. Performance Regression</strong></p>
<p>The CSM-1B model was documented to achieve 50-150ms latency but now takes 12+ seconds, representing a <strong>**119x performance regression**</strong>.</p>
<p><strong>## Recommendations for PyTorch Team</strong></p>
<p><strong>### 1. Improve Dtype Mismatch Handling</strong></p>
<p>```python</p>
<p># Current behavior: Strict failure</p>
<p>RuntimeError: mat1 and mat2 must have the same dtype, but got Float and BFloat16</p>
<p># Suggested behavior: Automatic conversion with warning</p>
<p>UserWarning: Converting Float32 input to BFloat16 for model compatibility</p>
<p>```</p>
<p><strong>### 2. Restore Mixed Precision Performance</strong></p>
<p>- Ensure fused kernels work with automatic dtype conversion</p>
<p>- Maintain performance when dtype mismatches occur</p>
<p>- Add clear warnings about performance implications</p>
<p><strong>### 3. Better Error Messages</strong></p>
<p>```python</p>
<p># Current error message</p>
<p>RuntimeError: mat1 and mat2 must have the same dtype, but got Float and BFloat16</p>
<p># Suggested error message</p>
<p>RuntimeError: Dtype mismatch detected. Model expects BFloat16 but received Float32.</p>
<p>Consider: input_tensor.to(torch.bfloat16) or model.to(torch.float32)</p>
<p>```</p>
<p><strong>### 4. Documentation Updates</strong></p>
<p>- Update PyTorch documentation to reflect strict dtype enforcement</p>
<p>- Provide migration guide for models using mixed precision</p>
<p>- Document performance implications of dtype mismatches</p>
<p><strong>## Test Scripts</strong></p>
<p>The following test scripts demonstrate the issues:</p>
<p>1. <strong>**`pytorch_rtx5060_clean_report.py`**</strong> - Basic CUDA and dtype testing</p>
<p>2. <strong>**`test_working_csm1b.py`**</strong> - CSM-1B performance testing</p>
<p>3. <strong>**`pytorch_rtx5060_report.py`**</strong> - Comprehensive performance analysis</p>
<p><strong>## Conclusion</strong></p>
<p>While PyTorch 2.8.0 successfully supports RTX 5060 Ti (sm_120), the strict dtype enforcement has caused severe performance regressions in real-world models like CSM-1B. The PyTorch team should consider:</p>
<p>1. <strong>**Restoring mixed precision performance**</strong> for backward compatibility</p>
<p>2. <strong>**Improving dtype mismatch handling**</strong> with automatic conversion</p>
<p>3. <strong>**Better error messages**</strong> to help developers migrate their code</p>
<p>4. <strong>**Updated documentation**</strong> reflecting the new strict dtype requirements</p>
<p>The RTX 5060 Ti support is working correctly, but the dtype handling changes have broken existing model implementations that relied on automatic dtype conversion.</p>
<p>-–</p>
<p><strong>**Contact:**</strong> This report was generated by testing PyTorch 2.8.0+cu128 on RTX 5060 Ti with CSM-1B model performance analysis.</p>

Kevin_Kern: <p>Oh, PyTorch gods, in realms of code you dwell,</p>
<p>Who bind the GPU with a magic spell.</p>
<p>Hear now the plea from a developer’s station,</p>
<p>Beset by the woes of installation.</p>
<p>A card sits enthroned, a new, hopeful sight,</p>
<p>The RTX 5060 Ti, gleaming with light.</p>
<p>With cores primed for training and tensors to flow,</p>
<p>It waits for the CUDA it’s longing to know.</p>
<p>But builds become tangled, a dependency maze,</p>
<p>The drivers conflict in a version-locked haze.</p>
<p>The nvcc despairs, the compiler complains,</p>
<p>And pip offers only inscrutable pains.</p>
<p>So grant us a build, both stable and sound,</p>
<p>Where all of the toolkits in harmony’s found.</p>
<p>A PyTorch that knows this new silicon’s might,</p>
<p>And makes torch.cuda.is_available() shine bright.</p>
<p>Let layers converge and our models run free,</p>
<p>A perfect install, for you and for me.</p>
<p>And we shall give thanks, with models well-trained,</p>
<p>By your grace and your wisdom, our progress sustained.</p>