transformers not compatible with lower versions of torch e.g. 1.11
Author: amyeroberts
Pull request: no
Labels: name: Should Fix

name: bug

### System Info

One breaking env:

```
- `transformers` version: 4.38.2
- Platform: Linux-6.1.58+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.20.3
- Safetensors version: 0.4.2
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 1.11.0+cu102 (True)
- Tensorflow version (GPU?): 2.15.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.8.2 (gpu)
- Jax version: 0.4.23
- JaxLib version: 0.4.23
```

### Who can help?

@ydshieh 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run `import transformers` on lower versions of pytorch. 

Will return the error: 
```
>>> import transformers
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/xxx/lib/python3.9/site-packages/transformers/__init__.py", line 26, in <module>
  from . import dependency_versions_check
 File "/xxx/lib/python3.9/site-packages/transformers/dependency_versions_check.py", line 16, in <module>
  from .utils.versions import require_version, require_version_core
 File "/xxx/lib/python3.9/site-packages/transformers/utils/__init__.py", line 33, in <module>
  from .generic import (
 File "/xxx/lib/python3.9/site-packages/transformers/utils/generic.py", line 455, in <module>
  _torch_pytree.register_pytree_node(
AttributeError: module 'torch.utils._pytree' has no attribute 'register_pytree_node'
```

Also report on torch 1.13

c.f. this PR: https://github.com/huggingface/transformers/pull/29364

### Expected behavior

Officially supported torch versions work

Comments:
@dangkhoasdc @Keneyr @stevhliu Thanks for reporting on the PR. I've opened this issue to keep track of this. 

The behaviour is very weird, as the offending line is guarded by the pytorch version, so shouldn't be run at all.

@dangkhoasdc @Keneyr Could you share the transformers version you're running and what device you're running on e.g. GPU? 

I'm not able to reproduce running just on my mac:
```
- `transformers` version: 4.38.2
- Platform: macOS-14.2.1-arm64-arm-64bit
- Python version: 3.10.9
- Huggingface_hub version: 0.21.3
- Safetensors version: 0.4.2
- Accelerate version: 0.28.0
- Accelerate config:    not found
- PyTorch version (GPU?): 1.11.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (cpu)
- Jax version: 0.4.12
- JaxLib version: 0.4.12
```

I am not able to reproduce within several combination of envs (but I am only in python 3.8 and tf version is only up to 2.13). @amyeroberts is the env. in the description reproduces the issue?


@ydshieh Env in the issue's description is from @stevhliu cf [this comment](https://github.com/huggingface/transformers/pull/29364#issuecomment-2010170048) which replicated the issue. 

Env in my comment [here](https://github.com/huggingface/transformers/issues/29763#issuecomment-2010499680) is what I ran, but didn't replicate the issue.  

> Hi, unfortunately I'm unable to reproduce your issue with torch==1.11.0  Here's the install environment I used:

IIRC, @stevhliu is unable to replicate the issue with the env posted 

Let's wait the PR author's response.

@ydshieh D'oh, that's my reading comprehension, sorry, I misread as "I'm able to reproduce". I'd say let's just close the issue then - it's not something we're seeing reported elsewhere and not something we can replicate. 


Hi I have the same issue with on Oracle Linux 9.3 with python3 compiled from source. We do have python 3.8-3.12 installed in envs modules if you need anything tested. 
* python: 3.10.13
* torch==1.13.1
* transformers==4.38.2

MWE: 
```
python3 -c 'import transformers'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/app/tmbed/1.0.0-3.10.13/transformers/__init__.py", line 26, in <module>
    from . import dependency_versions_check
  File "/app/tmbed/1.0.0-3.10.13/transformers/dependency_versions_check.py", line 16, in <module>
    from .utils.versions import require_version, require_version_core
  File "/app/tmbed/1.0.0-3.10.13/transformers/utils/__init__.py", line 33, in <module>
    from .generic import (
  File "/app/tmbed/1.0.0-3.10.13/transformers/utils/generic.py", line 455, in <module>
    _torch_pytree.register_pytree_node(
AttributeError: module 'torch.utils._pytree' has no attribute 'register_pytree_node'. Did you mean: '_register_pytree_node'?
```

EDIT: I tried importing `transformers` with the same python and pytorch versions and different transformers versions and got these results:
* 4.37.0 works
* 4.37.1 works
* 4.37.2 works 
* 4.38.0 worksn't  
* 4.38.1 worksn't 
* 4.38.2 worksn't 

!conda update -n base conda -y
!conda update python -y
!pip3 install --upgrade pip setuptools wheel packaging
!pip3 install --upgrade tensorflow torch pandas matplotlib nltk faiss-gpu ipywidgets einops
!pip3 install --upgrade accelerate scipy langchain langchain-community datasets PyMuPDF 
!pip3 install --upgrade attention-sinks tiktoken sentence_transformers optimum auto-gptq
!pip3 install transformers==4.40.0

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[5], line 13
     11 import scipy
     12 import tiktoken
---> 13 from sentence_transformers import SentenceTransformer
     14 from scipy.stats import fisher_exact
     15 from torch import nn

File /opt/conda/lib/python3.10/site-packages/sentence_transformers/__init__.py:7
      4 import importlib
      5 import os
----> 7 from sentence_transformers.cross_encoder.CrossEncoder import CrossEncoder
      8 from sentence_transformers.datasets import ParallelSentencesDataset, SentencesDataset
      9 from sentence_transformers.LoggingHandler import LoggingHandler

File /opt/conda/lib/python3.10/site-packages/sentence_transformers/cross_encoder/__init__.py:1
----> 1 from .CrossEncoder import CrossEncoder
      3 __all__ = ["CrossEncoder"]

File /opt/conda/lib/python3.10/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:12
     10 from torch.utils.data import DataLoader
     11 from tqdm.autonotebook import tqdm, trange
---> 12 from transformers import AutoConfig, AutoModelForSequenceClassification, AutoTokenizer, is_torch_npu_available
     13 from transformers.tokenization_utils_base import BatchEncoding
     14 from transformers.utils import PushToHubMixin

File /opt/conda/lib/python3.10/site-packages/transformers/__init__.py:26
     23 from typing import TYPE_CHECKING
     25 # Check the dependencies satisfy the minimal versions required.
---> 26 from . import dependency_versions_check
     27 from .utils import (
     28     OptionalDependencyNotAvailable,
     29     _LazyModule,
   (...)
     48     logging,
     49 )
     52 logger = logging.get_logger(__name__)  # pylint: disable=invalid-name

File /opt/conda/lib/python3.10/site-packages/transformers/dependency_versions_check.py:16
      1 # Copyright 2020 The HuggingFace Team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
     15 from .dependency_versions_table import deps
---> 16 from .utils.versions import require_version, require_version_core
     19 # define which module versions we always want to check at run time
     20 # (usually the ones defined in install_requires in setup.py)
     21 #
     22 # order specific notes:
     23 # - tqdm must be checked before tokenizers
     25 pkgs_to_check_at_runtime = [
     26     "python",
     27     "tqdm",
   (...)
     37     "pyyaml",
     38 ]

File /opt/conda/lib/python3.10/site-packages/transformers/utils/__init__.py:33
     24 from .constants import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD, IMAGENET_STANDARD_MEAN, IMAGENET_STANDARD_STD
     25 from .doc import (
     26     add_code_sample_docstrings,
     27     add_end_docstrings,
   (...)
     31     replace_return_docstrings,
     32 )
---> 33 from .generic import (
     34     ContextManagers,
     35     ExplicitEnum,
     36     ModelOutput,
     37     PaddingStrategy,
     38     TensorType,
     39     add_model_info_to_auto_map,
     40     cached_property,
     41     can_return_loss,
     42     expand_dims,
     43     find_labels,
     44     flatten_dict,
     45     infer_framework,
     46     is_jax_tensor,
     47     is_numpy_array,
     48     is_tensor,
     49     is_tf_symbolic_tensor,
     50     is_tf_tensor,
     51     is_torch_device,
     52     is_torch_dtype,
     53     is_torch_tensor,
     54     reshape,
     55     squeeze,
     56     strtobool,
     57     tensor_size,
     58     to_numpy,
     59     to_py_obj,
     60     transpose,
     61     working_or_temp_dir,
     62 )
     63 from .hub import (
     64     CLOUDFRONT_DISTRIB_PREFIX,
     65     HF_MODULES_CACHE,
   (...)
     91     try_to_load_from_cache,
     92 )
     93 from .import_utils import (
     94     ACCELERATE_MIN_VERSION,
     95     ENV_VARS_TRUE_AND_AUTO_VALUES,
   (...)
    207     torch_only_method,
    208 )

File /opt/conda/lib/python3.10/site-packages/transformers/utils/generic.py:478
    475     return output_type(**dict(zip(context, values)))
    477 if version.parse(get_torch_version()) >= version.parse("2.2"):
--> 478     _torch_pytree.register_pytree_node(
    479         ModelOutput,
    480         _model_output_flatten,
    481         partial(_model_output_unflatten, output_type=ModelOutput),
    482         serialized_type_name=f"{ModelOutput.__module__}.{ModelOutput.__name__}",
    483     )
    484 else:
    485     _torch_pytree._register_pytree_node(
    486         ModelOutput,
    487         _model_output_flatten,
    488         partial(_model_output_unflatten, output_type=ModelOutput),
    489     )

AttributeError: module 'torch.utils._pytree' has no attribute 'register_pytree_node'



@Kuchiriel Could you please provide a reproducible code snippet and your running environment: run `transformers-cli env` in the terminal and copy-paste the output? 

Hmm, that line is guarded with `if version.parse(get_torch_version()) >= version.parse("2.2")`.
Not sure if the specified version is incorrect. Let's wait @Kuchiriel reply.

confirmed that torch==1.13.2 and transformers==4.38.2 still has the above problem.

reproduce steps:

```
""" It is recommmended NOT to change codes in this cell """
print(torch.__version__)
# create the output directory you specify
os.makedirs(output_dir, exist_ok = True)
os.makedirs(ckpt_dir, exist_ok = True)

# 根據 from_ckpt 標誌，從 checkpoint 載入模型權重
if from_ckpt:
    model = PeftModel.from_pretrained(model, ckpt_name)

# 將模型準備好以使用 INT8 訓練
model = prepare_model_for_int8_training(model)

# 使用 LoraConfig 配置 LORA 模型
config = LoraConfig(
    r=LORA_R,
    lora_alpha=LORA_ALPHA,
    target_modules=TARGET_MODULES,
    lora_dropout=LORA_DROPOUT,
    bias="none",
    task_type="CAUSAL_LM",
)
model = get_peft_model(model, config)

# 將 tokenizer 的 padding token 設定為 0
tokenizer.pad_token_id = 0

# 載入並處理訓練數據
with open(dataset_dir, "r", encoding = "utf-8") as f:
    data_json = json.load(f)
with open("tmp_dataset.json", "w", encoding = "utf-8") as f:
    json.dump(data_json[:num_train_data], f, indent = 2, ensure_ascii = False)

data = load_dataset('json', data_files="tmp_dataset.json", download_mode="force_redownload")

# 將訓練數據分為訓練集和驗證集（若 VAL_SET_SIZE 大於 0）
if VAL_SET_SIZE > 0:
    train_val = data["train"].train_test_split(
        test_size=VAL_SET_SIZE, shuffle=True, seed=42
    )
    train_data = train_val["train"].shuffle().map(generate_training_data)
    val_data = train_val["test"].shuffle().map(generate_training_data)
else:
    train_data = data['train'].shuffle().map(generate_training_data)
    val_data = None

# 使用 Transformers Trainer 進行模型訓練
trainer = transformers.Trainer(
    model=model,
    train_dataset=train_data,
    eval_dataset=val_data,
    args=transformers.TrainingArguments(
        per_device_train_batch_size=MICRO_BATCH_SIZE,
        gradient_accumulation_steps=GRADIENT_ACCUMULATION_STEPS,
        warmup_steps=50,
        num_train_epochs=num_epoch,
        learning_rate=LEARNING_RATE,
        fp16=True,  # 使用混合精度訓練
        logging_steps=logging_steps,
        save_strategy="steps",
        save_steps=save_steps,
        output_dir=ckpt_dir,
        save_total_limit=save_total_limit,
        ddp_find_unused_parameters=False if ddp else None,  # 是否使用 DDP，控制梯度更新策略
        report_to=report_to
    ),
    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),
    
)

# 禁用模型的 cache 功能
model.config.use_cache = False

# 若使用 PyTorch 2.0 版本以上且非 Windows 系統，進行模型編譯
if torch.__version__ >= "2" and sys.platform != 'win32':
    model = torch.compile(model)

# 開始模型訓練
trainer.train()

# 將訓練完的模型保存到指定的目錄中
model.save_pretrained(ckpt_dir)

# 印出訓練過程中可能的缺失權重的警告信息
print("\n If there's a warning about missing keys above, please disregard :)")

```

The error is,

```

This code block generates error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[29], line 78
     75     model = torch.compile(model)
     77 # 開始模型訓練
---> 78 trainer.train()
     80 # 將訓練完的模型保存到指定的目錄中
     81 model.save_pretrained(ckpt_dir)

File ~/miniconda3/envs/ML-base/lib/python3.8/site-packages/transformers/trainer.py:1624, in Trainer.train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1622         hf_hub_utils.enable_progress_bars()
   1623 else:
-> 1624     return inner_training_loop(
   1625         args=args,
   1626         resume_from_checkpoint=resume_from_checkpoint,
   1627         trial=trial,
   1628         ignore_keys_for_eval=ignore_keys_for_eval,
   1629     )

File ~/miniconda3/envs/ML-base/lib/python3.8/site-packages/transformers/trainer.py:1653, in Trainer._inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   1651 logger.debug(f"Currently training with a batch size of: {self._train_batch_size}")
   1652 # Data loader and number of training steps
-> 1653 train_dataloader = self.get_train_dataloader()
   1654 if self.is_fsdp_xla_v2_enabled:
   1655     train_dataloader = tpu_spmd_dataloader(train_dataloader)

File ~/miniconda3/envs/ML-base/lib/python3.8/site-packages/transformers/trainer.py:852, in Trainer.get_train_dataloader(self)
    849     dataloader_params["worker_init_fn"] = seed_worker
    850     dataloader_params["prefetch_factor"] = self.args.dataloader_prefetch_factor
--> 852 return self.accelerator.prepare(DataLoader(train_dataset, **dataloader_params))

File ~/miniconda3/envs/ML-base/lib/python3.8/site-packages/torch/utils/data/dataloader.py:241, in DataLoader.__init__(self, dataset, batch_size, shuffle, sampler, batch_sampler, num_workers, collate_fn, pin_memory, drop_last, timeout, worker_init_fn, multiprocessing_context, generator, prefetch_factor, persistent_workers, pin_memory_device)
    238     raise ValueError('timeout option should be non-negative')
    240 if num_workers == 0 and prefetch_factor != 2:
--> 241     raise ValueError('prefetch_factor option could only be specified in multiprocessing.'
    242                      'let num_workers > 0 to enable multiprocessing.')
    243 assert prefetch_factor > 0
    245 if persistent_workers and num_workers == 0:

ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing.

Find the root cause of the error, look into version mismatch between libraries or others.
```


Hi @Oklahomawhore 

This is not the same issue as the one opened initially.

But from the traceback you provided, it seems you need to specify `dataloader_num_workers` (> 0) in `TrainingArguments`, or provide `dataloader_prefetch_factor=None`.

TBH, I am not sure why the default is set to

> "Default is 2 for PyTorch < 2.0.0 and otherwise None."