Errors using TinyLlama-1.1B-Chat-v1.0 and DirectML
Author: anakatt
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.52.3
- Platform: Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39
- Python version: 3.12.9
- Huggingface_hub version: 0.32.0
- Safetensors version: 0.5.3
- Accelerate version: 1.7.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.4.1+cu121 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>



### Who can help?

@SunMarc @zach-huggingface @ivarflakstad

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. Use torch_directml

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
import platform 
import os       
from datetime import datetime, timezone 
import torch_directml
model_dtype = torch.float32

 if used_directml:
            print(f"Loading model to CPU first (with dtype={model_dtype}), then moving to DirectML device: {device}")
            model = AutoModelForCausalLM.from_pretrained(
                MODEL_NAME,
                torch_dtype=model_dtype # Use the determined dtype
            )
            print(f"Model loaded initially (device: {model.device if hasattr(model, 'device') else 'N/A'}). Now moving to {device}...")
            model = model.to(device)
            print(f"Model moved to DirectML device: {device} (model.device: {model.device if hasattr(model, 'device') else 'N/A'})")
```

Final selected device for model loading: privateuseone:0, target model dtype: torch.float32
Loading model to CPU first (with dtype=torch.float32), then moving to DirectML device: privateuseone:0
Detailed error during LLM initialization: TypeError: argument of type 'NoneType' is not iterable
Traceback (most recent call last):
  File "/home/afonso/skillspark/llmservice/utils/llm_integration.py", line 167, in initialize_llm
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/afonso/miniconda3/envs/directml/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 571, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/afonso/miniconda3/envs/directml/lib/python3.12/site-packages/transformers/modeling_utils.py", line 309, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/afonso/miniconda3/envs/directml/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4507, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/afonso/miniconda3/envs/directml/lib/python3.12/site-packages/transformers/models/llama/modeling_llama.py", line 618, in __init__
    self.model = LlamaModel(config)
                 ^^^^^^^^^^^^^^^^^^
  File "/home/afonso/miniconda3/envs/directml/lib/python3.12/site-packages/transformers/models/llama/modeling_llama.py", line 379, in __init__
    self.post_init()
  File "/home/afonso/miniconda3/envs/directml/lib/python3.12/site-packages/transformers/modeling_utils.py", line 1968, in post_init
    if v not in ALL_PARALLEL_STYLES:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable

### Expected behavior

Model loads successfully, with no errors

Comments:
same for `Qwen/Qwen3-32B`, `transformers==4.52.3`

fixed by switching to ```microsoft/DialoGPT-small```, ```transformers>=4.30.0```

same for Qwen2.5-VL transformers==4.52.3

Llama3.3-70b-instruct (Hugging Face model, model-type = llama) has the same issue.
Transformers version: 4.52.3
I checked transformers/modeling_utils.py and found the following code:
```
pythonif self._tp_plan is not None and is_torch_greater_or_equal("2.3"):
    for *, v in self.*tp_plan.items():
        if v not in ALL_PARALLEL_STYLES:
            raise ValueError(
                f"Unsupported tensor parallel style {v}. Supported styles are {ALL_PARALLEL_STYLES}"
            )
```
My PyTorch version is 2.7, so I think the problem is not the transformers version, but rather the PyTorch version.
Has anyone solved this problem?

I found a way to solve this error.
You can avoid the error by pasting the code below before starting the training.
```python
from transformers import modeling_utils
if not hasattr(modeling_utils, "ALL_PARALLEL_STYLES") or modeling_utils.ALL_PARALLEL_STYLES is None:
    modeling_utils.ALL_PARALLEL_STYLES = ["tp", "none","colwise",'rowwise']
```
I looked through the transformers code, but it seems that there is no place where the ALL_PARALLEL_STYLES variable is used importantly. So if you put the code below before loading the model, the error disappears.
However, in my case, OOM occurs. LoL

same for `meta-llama/Llama-3.1-8B-Instruct`, `yentinglin/Llama-3-Taiwan-8B-Instruct`. `transformers==4.52.3`

> same for Qwen2.5-VL transformers==4.52.3

I solved this problem by pip install transformers==4.51.3

For the latest version, I tried this https://github.com/huggingface/transformers/issues/38340#issuecomment-2907735645 and it helped. While it is a quick fix, I think rolling back to the previous may be my preference since `ALL_PARALLEL_STYLES` has not been documented yet, and what its purpose/usage is.

> Llama3.3-70b-instruct (Hugging Face model, model-type = llama) has the same issue. Transformers version: 4.52.3 I checked transformers/modeling_utils.py and found the following code:
> 
> ```
> pythonif self._tp_plan is not None and is_torch_greater_or_equal("2.3"):
>     for *, v in self.*tp_plan.items():
>         if v not in ALL_PARALLEL_STYLES:
>             raise ValueError(
>                 f"Unsupported tensor parallel style {v}. Supported styles are {ALL_PARALLEL_STYLES}"
>             )
> ```
> 
> My PyTorch version is 2.7, so I think the problem is not the transformers version, but rather the PyTorch version. Has anyone solved this problem?


I downgraded my torch to 2.2.0 and solved the problem

(with Qwen3-0.6B and transformers==4.52.3)

> ip install transformers==4.51.3

thanks it also work for me

Sorry for that, there were indeed an issue with a torch check. This is should partly be fixed in main + with this PR https://github.com/huggingface/transformers/pull/38328. We will do a patch for these soon 

> > same for Qwen2.5-VL transformers==4.52.3
> 
> I solved this problem by pip install transformers==4.51.3

this works for me !

> > same for Qwen2.5-VL transformers==4.52.3
> 
> I solved this problem by pip install transformers==4.51.3

works for me

> > same for Qwen2.5-VL transformers==4.52.3
> 
> I solved this problem by pip install transformers==4.51.3

Thanks！

> > same for Qwen2.5-VL transformers==4.52.3
> 
> I solved this problem by pip install transformers==4.51.3

same issue for `Qwen/Qwen3-0.6B`. `transformers==4.52.4`, it also work for me

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Since transformers 4.53 was released, this should be fixed ! Hence, i'm closing this