Repository: huggingface/evaluate
Issue: Issue using evaluate with transformers 4.24.0
Root cause: Environment Incompatibility

I tried updating transformers today (`pip install transformers --upgrade`) and saw a strange error related to evaluate, transformers, and perhaps my version of CUDA:

```
    import evaluate
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/evaluate/__init__.py", line 29, in <module>
    from .evaluator import (
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/evaluate/evaluator/__init__.py", line 27, in <module>
    from .base import Evaluator
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/evaluate/evaluator/base.py", line 34, in <module>
    from transformers import Pipeline, pipeline
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1066, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1078, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback):
/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cpp.so: undefined symbol: cudaGraphRetainUserObject, version libcudart.so.11.0
```

I fixed it for now by downgrading to transformers 4.23.1, but I want to upgrade again eventually.

I just tried this with the latest `evaluate` version and `transformers==4.24.0` and can't reproduce it. Can you run `from transformers import Pipeline, pipeline`? If not then this is a `transformers` issue rather than an `evaluate` issue.

Tried updating again. I got a different issue on a different machine:
```python
torch ❯ python
Python 3.9.7 (default, Sep 16 2021, 13:09:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import evaluate
2022-11-11 11:06:55.815827: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-11-11 11:06:55.970929: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2022-11-11 11:06:56.008781: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2022-11-11 11:06:57.507879: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2022-11-11 11:06:57.507961: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2022-11-11 11:06:57.507972: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/evaluate/__init__.py", line 29, in <module>
    from .evaluator import (
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/evaluate/evaluator/__init__.py", line 27, in <module>
    from .base import Evaluator
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/evaluate/evaluator/base.py", line 20, in <module>
    from datasets import Dataset, load_dataset
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/datasets/__init__.py", line 43, in <module>
    from .arrow_dataset import Dataset
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 54, in <module>
    import pandas as pd
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/__init__.py", line 48, in <module>
    from pandas.core.api import (
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/core/api.py", line 48, in <module>
    from pandas.core.groupby import (
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import (
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/core/groupby/generic.py", line 70, in <module>
    from pandas.core.frame import DataFrame
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/core/frame.py", line 157, in <module>
    from pandas.core.generic import NDFrame
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/core/generic.py", line 152, in <module>
    from pandas.core.window import (
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/core/window/__init__.py", line 1, in <module>
    from pandas.core.window.ewm import (  # noqa:F401
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/core/window/ewm.py", line 12, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/pandas/_libs/window/aggregations.cpython-39-x86_64-linux-gnu.so)
```

I try running your command and see the same issue from above:

```
>>> from transformers import Pipeline, pipeline
Traceback (most recent call last):
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1076, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/pipelines/__init__.py", line 47, in <module>
    from .audio_classification import AudioClassificationPipeline
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/pipelines/audio_classification.py", line 20, in <module>
    from .base import PIPELINE_INIT_ARGS, Pipeline
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/pipelines/base.py", line 34, in <module>
    from ..modelcard import ModelCard
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/modelcard.py", line 47, in <module>
    from .training_args import ParallelMode
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/training_args.py", line 28, in <module>
    from .debug_utils import DebugOption
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/debug_utils.py", line 21, in <module>
    import torch
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/torch/__init__.py", line 197, in <module>
    from torch._C import *  # noqa: F403
ImportError: /home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cpp.so: undefined symbol: cudaGraphRetainUserObject, version libcudart.so.11.0

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1066, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1078, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback):
/home/jxm3/.conda/envs/torch/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cpp.so: undefined symbol: cudaGraphRetainUserObject, version libcudart.so.11.0
```

This doesn't seem like an `evaluate` issue but an issue with your environment in general: in the first case the error comes from importing `pandas` and in the second case from `transformers`. How are you setting up your environment? Are you using `conda` or `virtualenv`?

I'm using conda. The issue is caused by upgrading transformers. It works when I downgrade. how could that be an issue with my environment?


It's just weird that it also breaks `pandas`.  In any case it seems to be not an issue with `evaluate` but rather `transformers`. Do you mind opening an issue there?

ok