ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' 
Author: congchan
Pull request: no

### System Info

transformer version 4.34
accelerate 0.20.3

Actually I never call peft and use npu, not sure why this import is neccecssary.

cc trainer: @muellerzr and @pacman100

```
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
    return importlib.import_module("." + module_name, self.__name__)    
    return importlib.import_module("." + module_name, self.__name__)return importlib.import_module("." + module_name, self.__name__)
  File "/usr/local/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module

      File "/usr/local/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module
return importlib.import_module("." + module_name, self.__name__)  File "/usr/local/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module

  File "/usr/local/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
Traceback (most recent call last):
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/local/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/trainer.py", line 200, in <module>
    from peft import PeftModel
  File "/usr/local/conda/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
    from .auto import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/usr/local/conda/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    from .other import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/other.py", line 24, in <module>
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
    from accelerate.utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workdir/train/train.py", line 28, in <module>
    from transformers import Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
Traceback (most recent call last):
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
Traceback (most recent call last):
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
Traceback (most recent call last):
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/trainer.py", line 200, in <module>
    from peft import PeftModel
  File "/usr/local/conda/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    return importlib.import_module("." + module_name, self.__name__)    
from .auto import (
  File "/usr/local/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module
  File "/usr/local/conda/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/usr/local/conda/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
    return importlib.import_module("." + module_name, self.__name__)
      File "/usr/local/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/local/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module
    from .other import (
      File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/other.py", line 24, in <module>
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
    from accelerate.utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workdir/train/train.py", line 28, in <module>
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
    from transformers import Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/trainer.py", line 200, in <module>
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)
    from peft import PeftModel
  File "/usr/local/conda/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/usr/local/conda/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    from .other import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/other.py", line 24, in <module>
    from accelerate.utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workdir/train/train.py", line 28, in <module>
    from transformers import Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/trainer.py", line 200, in <module>
    from peft import PeftModel
  File "/usr/local/conda/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/usr/local/conda/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    from .other import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/other.py", line 24, in <module>
    from accelerate.utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workdir/train/train.py", line 28, in <module>
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
    from transformers import Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/trainer.py", line 200, in <module>
    from peft import PeftModel
  File "/usr/local/conda/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/usr/local/conda/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    from .other import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/other.py", line 24, in <module>
    from accelerate.utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workdir/train/train.py", line 28, in <module>
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/trainer.py", line 200, in <module>
    from transformers import Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
    from peft import PeftModel
  File "/usr/local/conda/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/usr/local/conda/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    from .other import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/other.py", line 24, in <module>
    from accelerate.utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workdir/train/train.py", line 28, in <module>
    from transformers import Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/trainer.py", line 200, in <module>
    from peft import PeftModel
  File "/usr/local/conda/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/usr/local/conda/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    from .other import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/other.py", line 24, in <module>
    from accelerate.utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workdir/train/train.py", line 28, in <module>
    from transformers import Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/trainer.py", line 200, in <module>
    from peft import PeftModel
  File "/usr/local/conda/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/usr/local/conda/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    from .other import (
  File "/usr/local/conda/lib/python3.9/site-packages/peft/utils/other.py", line 24, in <module>
    from accelerate.utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workdir/train/train.py", line 28, in <module>
    from transformers import Trainer, TrainerCallback
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'is_npu_available' from 'accelerate.utils' (/usr/local/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py)
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 3161) of binary: /usr/bin/python3
Traceback (most recent call last):
  File "/usr/local/conda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/conda/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/conda/lib/python3.9/site-packages/torch/distributed/run.py", line 766, in <module>
    main()
  File "/usr/local/conda/lib/python3.9/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/conda/lib/python3.9/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/usr/local/conda/lib/python3.9/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/usr/local/conda/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/usr/local/conda/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
train/train.py FAILED
------------------------------------------------------------
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

A causal language modeling task with trainer and FSDP

### Expected behavior

Don't call `is_npu_available`

Comments:
Hi, peft requires a higher accelerate version, please do `pip install accelerate -U`

> Hi, peft requires a higher accelerate version, please do `pip install accelerate -U`

Hi, I need this version because higher accelerate raise another error `ValueError: FSDP requires PyTorch >= 2.0.1`. 
Yet for hardware compatibility reasons, I cannot easily change torch version right now. 
And I actually don't need peft at all. 
Any other solutions?

You can try fully uninstalling peft then from the ecosystem. 

Thanks