Triton version check compatibility on windows
Author: Tsumugii24
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.55.0
- Platform: Windows-11-10.0.26100-SP0
- Python version: 3.12.11
- Huggingface_hub version: 0.34.3
- Safetensors version: 0.6.1
- Accelerate version: 1.9.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0+cu129 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA GeForce RTX 5090

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

## Introduction

The bug is discovered as I am trying to deploy MXFP4 quantization of [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) on Windows, utilizing a NVIDIA GeForce RTX 5090 GPU. More system Information can be seen above.

## Description

When running the official code snippet provided in [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) that use `transformers` library. The following issue occurs on Windows:
```
MXFP4 quantization requires triton >= 3.4.0 and triton_kernels installed, we will default to dequantizing the model to bf16
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 3/3 [00:10<00:00,  3.55s/it]
Some parameters are on the meta device because they were offloaded to the cpu.
Device set to use cuda:0
```
The response can be generated in the end, but it takes a long time because of cpu offloading. 

## Reproduction

### code
code is officially provided by openai on [huggingface](https://huggingface.co/openai/gpt-oss-20b)
```
# inference.py
from transformers import pipeline
import torch

model_id = "openai/gpt-oss-20b"

pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Explain quantum mechanics clearly and concisely."},
]

outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])
```

### environment
```
Python 3.12.11
```

```
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 576.80                 Driver Version: 576.80         CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5090      WDDM  |   00000000:01:00.0 Off |                  N/A |
|  0%   39C    P8             16W /  600W |       0MiB /  32607MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
```

```
# Name                     Version          Build            Channel
accelerate                 1.9.0            pypi_0           pypi
addict                     2.4.0            pypi_0           pypi
aiohappyeyeballs           2.6.1            pypi_0           pypi
aiohttp                    3.12.15          pypi_0           pypi
aiosignal                  1.4.0            pypi_0           pypi
attrs                      25.3.0           pypi_0           pypi
bzip2                      1.0.8            h2bbff1b_6
ca-certificates            2025.2.25        haa95532_0
certifi                    2025.8.3         pypi_0           pypi
charset-normalizer         3.4.2            pypi_0           pypi
colorama                   0.4.6            pypi_0           pypi
datasets                   4.0.0            pypi_0           pypi
dill                       0.3.8            pypi_0           pypi
expat                      2.7.1            h8ddb27b_0
filelock                   3.13.1           pypi_0           pypi
frozenlist                 1.7.0            pypi_0           pypi
fsspec                     2024.6.1         pypi_0           pypi
huggingface-hub            0.34.3           pypi_0           pypi
idna                       3.10             pypi_0           pypi
iniconfig                  2.1.0            pypi_0           pypi
jinja2                     3.1.4            pypi_0           pypi
kernels                    0.9.0            pypi_0           pypi
libffi                     3.4.4            hd77b12b_1
markupsafe                 2.1.5            pypi_0           pypi
modelscope                 1.28.2           pypi_0           pypi
mpmath                     1.3.0            pypi_0           pypi
multidict                  6.6.3            pypi_0           pypi
multiprocess               0.70.16          pypi_0           pypi
networkx                   3.3              pypi_0           pypi
numpy                      2.3.2            pypi_0           pypi
openssl                    3.0.17           h35632f6_0
packaging                  25.0             pypi_0           pypi
pandas                     2.3.1            pypi_0           pypi
pillow                     11.3.0           pypi_0           pypi
pip                        25.1             pyhc872135_2
pluggy                     1.6.0            pypi_0           pypi
propcache                  0.3.2            pypi_0           pypi
psutil                     7.0.0            pypi_0           pypi
pyarrow                    21.0.0           pypi_0           pypi
pygments                   2.19.2           pypi_0           pypi
pytest                     8.4.1            pypi_0           pypi
python                     3.12.11          h716150d_0
python-dateutil            2.9.0.post0      pypi_0           pypi
pytz                       2025.2           pypi_0           pypi
pyyaml                     6.0.2            pypi_0           pypi
regex                      2025.7.34        pypi_0           pypi
requests                   2.32.4           pypi_0           pypi
safetensors                0.6.1            pypi_0           pypi
setuptools                 78.1.1           py312haa95532_0
six                        1.17.0           pypi_0           pypi
sqlite                     3.50.2           hda9a48d_1
sympy                      1.13.3           pypi_0           pypi
tk                         8.6.14           h5e9d12e_1
tokenizers                 0.21.4           pypi_0           pypi
torch                      2.8.0+cu129      pypi_0           pypi
tqdm                       4.67.1           pypi_0           pypi
transformers               4.55.0           pypi_0           pypi
triton-kernels             1.0.0            pypi_0           pypi
triton-windows             3.4.0.post20     pypi_0           pypi
typing-extensions          4.12.2           pypi_0           pypi
tzdata                     2025.2           pypi_0           pypi
ucrt                       10.0.22621.0     haa95532_0
urllib3                    2.5.0            pypi_0           pypi
vc                         14.3             h2df5915_10
vc14_runtime               14.44.35208      h4927774_10
vs2015_runtime             14.44.35208      ha6b5a95_10
wheel                      0.45.1           py312haa95532_0
xxhash                     3.5.0            pypi_0           pypi
xz                         5.6.4            h4754444_1
yarl                       1.20.1           pypi_0           pypi
zlib                       1.2.13           h8cc25b3_1
```

### run
```
python inference.py
```

## Pinpoint

As official repo of [triton](https://github.com/triton-lang/triton) does not support windows installation, for windows users we use a forked version of triton — [triton-windows](https://github.com/woct0rdho/triton-windows) that is compiled and maintained by @woct0rdho . And triton-windows is installed by the following command described in [triton-windows-releases](https://github.com/woct0rdho/triton-windows/releases):
```
pip install -U "triton-windows<3.5"
```
The only difference of `triton-windows` is that it cannot be directly grepped by `pip show triton` (should use `pip show triton-windows` instead). However all the usage and import is the same as `triton`. The test script and result of `triton-windows` is illustrated below:
```
# test.py
import torch
import triton
import triton.language as tl
import importlib.metadata

# triton language test
@triton.jit
def add_kernel(x_ptr, y_ptr, output_ptr, n_elements, BLOCK_SIZE: tl.constexpr):
    pid = tl.program_id(axis=0)
    block_start = pid * BLOCK_SIZE
    offsets = block_start + tl.arange(0, BLOCK_SIZE)
    mask = offsets < n_elements
    x = tl.load(x_ptr + offsets, mask=mask)
    y = tl.load(y_ptr + offsets, mask=mask)
    output = x + y
    tl.store(output_ptr + offsets, output, mask=mask)

def add(x: torch.Tensor, y: torch.Tensor):
    output = torch.empty_like(x)
    n_elements = output.numel()
    grid = lambda meta: (triton.cdiv(n_elements, meta["BLOCK_SIZE"]),)
    add_kernel[grid](x, y, output, n_elements, BLOCK_SIZE=1024)
    return output

a = torch.rand(3, device="cuda")
b = a + a
b_compiled = add(a, a)
print(b_compiled - b)
print("If you see tensor([0., 0., 0.], device='cuda:0'), then it works")

# triton version
package = importlib.import_module("triton")
package_version = getattr(package, "__version__", "N/A")
print(package_version)

# linux
# print(importlib.metadata.version("triton"))
# windows
print(importlib.metadata.version("triton-windows"))
```

```
python test.py
tensor([0., 0., 0.], device='cuda:0')
If you see tensor([0., 0., 0.], device='cuda:0'), then it works
3.4.0
3.4.0.post20
```

The issue lies in this block of transformers source code:
https://github.com/huggingface/transformers/blob/555cbf59178134d1b713a7022129aaddfe6e70cf/src/transformers/utils/import_utils.py#L45-L91
The version check of triton is not compatible with windows because of:
https://github.com/huggingface/transformers/blob/555cbf59178134d1b713a7022129aaddfe6e70cf/src/transformers/utils/import_utils.py#L79-L83

## Conclusion
Transformers fails to detect triton on windows and automatically dequantizes to bf16, leading to CPU offloading and very slow inference — despite having `triton-windows >= 3.4.0` properly installed.
The bug is caused by incompatibility of triton version check on windows, however with appropriate checking logic, `triton-windows` can be detected and models like [gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) can be loaded in MXFP4 quantization smoothly.


### Expected behavior

MXFP4 quantization should work as intended on Windows if `triton-windows` is installed and functioning, instead of defaulting to bf16. This PR https://github.com/huggingface/transformers/pull/39986 may resolve this bug.

Comments:
I found this too on my win11 4090

https://huggingface.co/openai/gpt-oss-20b/discussions/53

> I found this too on my win11 4090
> 
> https://huggingface.co/openai/gpt-oss-20b/discussions/53

Hi! I've opened a PR https://github.com/huggingface/transformers/pull/39986 to support `triton` on Windows so that MXFP4 quantization doesn't fall back unnecessarily. Hope this helps!

Works for me

`pip uninstall transformers`
`pip install git+https://github.com/Tsumugii24/transformers`

Now runs fast, takes 16gb vram on 4090

Woot!

You might want to remove the trailing space, it is breaking the build

https://github.com/huggingface/transformers/pull/39986#issuecomment-3164014879