Better deepgemm errors
Author: IlyasMoutawwakil
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47154)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47154)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Okay so I fully went down the rabbit hole of DeepGEMM requirements, and what can and can't be done with the Hub kernel build.

TLDR; Statically verifying that the environment can compile DeepGEMM kernels before loading them lets us avoid downloading the kernel files entirely (there are too many, and it even causes rate limits 🥲).

DeepGEMM doesn't ship compiled kernels — it JIT-compiles them at runtime with the system CUDA toolkit. So the only thing that determines whether it'll work is whether that toolkit can compile, which we can read straight off disk (nvcc presence + version from version.json/version.txt/cuda.h), with no GPU, no probe compile, and no side effects.

Checking this up front is better on three counts:

- Correctness: we gate on the compiler that actually builds the kernels (nvcc under CUDA_HOME), not the CUDA runtime (the old signal), which can be a different, stray, or mismatched version and silently disable DeepGEMM on a perfectly capable box. Version floors follow the DeepGEMM README: SM90 needs CUDA ≥ 12.3, SM100 needs ≥ 12.9.
- Speed: the DeepGEMM kernel is a large tree of many small files, so downloading it is slow. Validating the env before the load short-circuits that download entirely when the toolchain can't use it — instead of paying minutes to fetch it only to fail.
- Predictable failure: if the env claims to support DeepGEMM it takes the fast path; otherwise it fails early with an actionable message and auto-falls back to Triton — never a cryptic crash deep in the first forward.

On NVRTC: I initially added an NVRTC fallback (compile via torch's bundled libnvrtc when nvcc is missing/too old), but the Hub deep-gemm build ignores DG_JIT_USE_NVRTC and always uses nvcc ("DG_JIT_USE_NVRTC is ignored in Kernel Hub builds; using NVCC", baked into the compiled .so). So there's no real NVRTC path with the Hub build — a resolvable CUDA_HOME must contain an nvcc of the required version, and anything short of that (unset CUDA_HOME, runtime-only install with headers but no nvcc, or a too-old nvcc) is rejected up front and routed to Triton.

Verified on B200 (SM100) across four CUDA_HOME settings — unset, cuda-12.8 (too old), cuda-12.9 (works), and a headers-only pip CUDA-13 dir (no nvcc): the three unusable configs fall back to Triton cleanly (previously the headers-only one crashed mid-forward), and 12.9 runs through DeepGEMM.

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result, 
**we ask that new users do not submit pure code agent PRs** at this time. 
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously. 

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result, 
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [ ] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://huggingface.co/docs/transformers/contributing) and the
      [Pull Request](https://huggingface.co/docs/transformers/pr_checks) checks?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes according to the [guidelines](https://github.com/huggingface/transformers/tree/main/docs)?
- [ ] Did you write any new necessary [tests](https://huggingface.co/docs/transformers/testing)?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez @vasqu
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @Cyrilvallez (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @vasqu @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47154). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> I have a few nits to address, including the verbosity of the comments 😅 probably the code agent went a little too far, maybe if we trim them down it will be more readable.

I actually asked it to avoid anyone adding nvrtc-related nudges or not understanding the torch compile opaque fx node 😅

> And an additional concern: this checks if the components of the toolkits are recent enough, but can they be recent and yet not match? Something like CUDA 12.9 at one part of the toolkit and 13.1 at some other part. Like a ragged toolkit situation: is that an issue? Im far from an expert so asking for your opinion here. Thanks a lot!

ngl i've never heard of "ragged toolkit" before now 🥲 will research it, try to create one and see

I think it's fair to say that ragged toolkits are out of scope here, they are not smth we'd wanna consider (kernels doesn't). normal installers (apt/dnf, runfile, conda, the tarballs) ship nvcc + headers + version files as one matched set, so raggedness only comes from a corrupted or hand-assembled install, same bucket as pointing CUDA_HOME at a wrong path

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47154)
**Latest run:** [28921430449:1](https://github.com/huggingface/transformers/actions/runs/28921430449)
**Result:** `success` | **Jobs:** 15 | **Tests:** 172,467 | **Failures:** 0 | **Duration:** 16h 44m

<!-- ci-dashboard-recap:end -->

Ok! Read all of the above, answered one last comment, whenever it is fine for you you can merge. Thanks!