Cannot import Trainer class from transformers
Author: Mark531
Pull request: no
Labels: name: bug

### System Info

transformers 4.46.0
tensorflow 2.10.1
keras 2.10.0
torch 2.2.1+cu118
python 3.10.13
Windows 10

### Who can help?

@muellerzr @SunMarc

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Step to reproduce:
`from transformers import Trainer`

Output:

> 2024-11-04 13:36:14.203152: E tensorflow/core/lib/monitoring/collection_registry.cc:81] Cannot register 2 metrics with the same name: /tensorflow/api/enable_tensor_equality
> Traceback (most recent call last):
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1778, in _get_module
>     return importlib.import_module("." + module_name, self.__name__)
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\importlib\__init__.py", line 126, in import_module
>     return _bootstrap._gcd_import(name[level:], package, level)
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
>   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\modeling_tf_utils.py", line 38, in <module>
>     from .activations_tf import get_tf_activation
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\activations_tf.py", line 22, in <module>
>     import tf_keras as keras
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\__init__.py", line 3, in <module>
>     from tf_keras import __internal__
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\__internal__\__init__.py", line 3, in <module>
>     from tf_keras.__internal__ import backend
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\__internal__\backend\__init__.py", line 3, in <module>
>     from tf_keras.src.backend import _initialize_variables as initialize_variables
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\__init__.py", line 21, in <module>
>     from tf_keras.src import applications
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\applications\__init__.py", line 18, in <module>
>     from tf_keras.src.applications.convnext import ConvNeXtBase
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\applications\convnext.py", line 28, in <module>
>     from tf_keras.src import backend
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\backend.py", line 35, in <module>
>     from tf_keras.src.engine import keras_tensor
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\engine\keras_tensor.py", line 19, in <module>
>     from tf_keras.src.utils import object_identity
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\utils\__init__.py", line 53, in <module>
>     from tf_keras.src.utils.feature_space import FeatureSpace
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\utils\feature_space.py", line 20, in <module>
>     from tf_keras.src.engine import base_layer
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\engine\base_layer.py", line 35, in <module>
>     from tf_keras.src.dtensor import lazy_variable
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tf_keras\src\dtensor\lazy_variable.py", line 23, in <module>
>     from tensorflow.python.framework import tensor
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tensorflow\python\framework\tensor.py", line 50, in <module>
>     _tensor_equality_api_usage_gauge = monitoring.BoolGauge(
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tensorflow\python\eager\monitoring.py", line 356, in __init__
>     super(BoolGauge, self).__init__('BoolGauge', _bool_gauge_methods,
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\tensorflow\python\eager\monitoring.py", line 131, in __init__
>     self._metric = self._metric_methods[self._label_length].create(*args)
> tensorflow.python.framework.errors_impl.AlreadyExistsError: Another metric with the same name already exists.
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1778, in _get_module
>     return importlib.import_module("." + module_name, self.__name__)
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\importlib\__init__.py", line 126, in import_module
>     return _bootstrap._gcd_import(name[level:], package, level)
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
>   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\integrations\integration_utils.py", line 36, in <module>
>     from .. import PreTrainedModel, TFPreTrainedModel
>   File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1766, in __getattr__
>     module = self._get_module(self._class_to_module[name])
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1780, in _get_module
>     raise RuntimeError(
> RuntimeError: Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
> Another metric with the same name already exists.
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1778, in _get_module
>     return importlib.import_module("." + module_name, self.__name__)
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\importlib\__init__.py", line 126, in import_module
>     return _bootstrap._gcd_import(name[level:], package, level)
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
>   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\trainer.py", line 42, in <module>
>     from .integrations import (
>   File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1766, in __getattr__
>     module = self._get_module(self._class_to_module[name])
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1780, in _get_module
>     raise RuntimeError(
> RuntimeError: Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
> Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
> Another metric with the same name already exists.
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1766, in __getattr__
>     module = self._get_module(self._class_to_module[name])
>   File "C:\Users\kevguima\AppData\Local\anaconda3\envs\genai\lib\site-packages\transformers\utils\import_utils.py", line 1780, in _get_module
>     raise RuntimeError(
> RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
> Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
> Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
> Another metric with the same name already exists.


### Expected behavior

The above line should work. Besides, the documentation says that the Trainer class uses pytorch rather than tensorflow, so I'm confused...

Comments:
Hey @Mark531, thanks for the report ! Could you try to update your version of tf and tf-keras ? I was able to load it without issues with the following versions: tensorflow:2.18.0, tf_keras:2.18.0. Another quick solution would be to uninstall tf and tf_keras. 

First, on Windows, the GPU version of tensorflow is limited to the 2.10 version. But ok, I installed tensorflow 2.18 (keras got updated to 3.6) and I ran into another error when calling TFAutoModel.from_pretrained():

> RuntimeError: Failed to import transformers.models.bert.modeling_tf_bert because of the following error (look up to see its traceback):
> No module named 'tensorflow.python.trackable'

So this time the trainer import worked but  `TFAutoModel.from_pretrained()` failed ? cc @Rocketknight1 

No, the Trainer also fails, but with these errors:

> RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
> Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
> Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
> Failed to import transformers.data.data_collator because of the following error (look up to see its traceback):
> module 'tensorflow' has no attribute 'data'

I've spent the whole day struggling to install the transformers library properly...

And you want to use it with tensorflow ? 

Well, preferably since I found snippets with Tensorflow/Keras models.

Hi @mark531, there's a lot of stuff to unpack here! TensorFlow compatibility has been a real problem for us since the launch of Keras 3, which broke backwards compatibility with Keras 2. If you really want to use TF/Keras, then I would advise installing the `tf-keras` compatibility package, and seeing if that helps.

Also, note that `Trainer` is actually a PyTorch-only class, and for TF models we use the model methods like `fit()` directly, see [here](https://huggingface.co/blog/tensorflow-philosophy) for more. In general, mixing PyTorch and TensorFlow tends to cause other issues, so I only recommend it for advanced users.

I think your options are:

1) `pip uninstall tensorflow` entirely and just make a clean PyTorch environment, then use `Trainer`
2) `pip install tf-keras` and try to get a working TensorFlow environment
3) Use WSL and get a Linux environment on your system, which will generally work better than Python on pure Windows, then try either 1) or 2) again.

Beyond that, though, we can't really debug these kinds of OS/environment problems in GitHub issues, because the problem isn't really anything we can fix in Transformers! I'm closing this for now, but feel free to take this to Discord or the forums if you need more assistance.