Repository: huggingface/evaluate
Issue: ImportError: To be able to use evaluate-metric/glue, you need to install the following dependencies['sklearn'] using 'pip install sklearn' for instance'
Root cause: Validation Gap

 I was following the tutorial in the [Transformers course at Huggingface](https://huggingface.co/course/chapter3/3?fw=pt):

The error appears when I using evaluate library as follows:
```py
import evaluate

metric = evaluate.load("glue", "mrpc")
metric.compute(predictions=preds, references=predictions.label_ids)

```

The error I have faced as follows: (**the Traceback is below**)

```py
ImportError                               Traceback (most recent call last)
/home/think3/Desktop/Barham/transformers-course/Fine_tunning.py/Fine_tunning_with_Trainer_API.py in line 5
      64 # %%
      66 import evaluate
----> 68 metric = evaluate.load("glue", "mrpc")
      69 metric.compute(predictions=preds, references=predictions.label_ids)

File ~/Desktop/Barham/transformers-course/.env/lib/python3.8/site-packages/evaluate/loading.py:731, in load(path, config_name, module_type, process_id, num_process, cache_dir, experiment_id, keep_in_memory, download_config, download_mode, revision, **init_kwargs)
    703 """Load a `evaluate.EvaluationModule`.
    704 
    705 Args:
   (...)
    728     `evaluate.EvaluationModule`
    729 """
    730 download_mode = DownloadMode(download_mode or DownloadMode.REUSE_DATASET_IF_EXISTS)
--> 731 evaluation_module = evaluation_module_factory(
    732     path, module_type=module_type, revision=revision, download_config=download_config, download_mode=download_mode
    733 )
    734 evaluation_cls = import_main_class(evaluation_module.module_path)
    735 evaluation_instance = evaluation_cls(
    736     config_name=config_name,
    737     process_id=process_id,
   (...)
    743     **init_kwargs,
    744 )

File ~/Desktop/Barham/transformers-course/.env/lib/python3.8/site-packages/evaluate/loading.py:680, in evaluation_module_factory(path, module_type, revision, download_config, download_mode, force_local_path, dynamic_modules_path, **download_kwargs)
    678                 pass
    679         if not isinstance(e1, (ConnectionError, FileNotFoundError)):
--> 680             raise e1 from None
    681         raise FileNotFoundError(
    682             f"Couldn't find a module script at {relative_to_absolute_path(combined_path)}. "
    683             f"Module '{path}' doesn't exist on the Hugging Face Hub either."
    684         ) from None
    685 else:

File ~/Desktop/Barham/transformers-course/.env/lib/python3.8/site-packages/evaluate/loading.py:633, in evaluation_module_factory(path, module_type, revision, download_config, download_mode, force_local_path, dynamic_modules_path, **download_kwargs)
    631 for current_type in ["metric", "comparison", "measurement"]:
    632     try:
--> 633         return HubEvaluationModuleFactory(
    634             f"evaluate-{current_type}/{path}",
    635             revision=revision,
    636             download_config=download_config,
    637             download_mode=download_mode,
    638             dynamic_modules_path=dynamic_modules_path,
    639         ).get_module()
    640     except ConnectionError:
    641         pass

File ~/Desktop/Barham/transformers-course/.env/lib/python3.8/site-packages/evaluate/loading.py:489, in HubEvaluationModuleFactory.get_module(self)
    486         raise err
    488 imports = get_imports(local_path)
--> 489 local_imports = _download_additional_modules(
    490     name=self.name,
    491     base_path=hf_hub_url(path=self.name, name="", revision=revision),
    492     imports=imports,
    493     download_config=self.download_config,
    494 )
    495 # copy the script and the files in an importable directory
    496 dynamic_modules_path = self.dynamic_modules_path if self.dynamic_modules_path else init_dynamic_modules()

File ~/Desktop/Barham/transformers-course/.env/lib/python3.8/site-packages/evaluate/loading.py:265, in _download_additional_modules(name, base_path, imports, download_config)
    263         needs_to_be_installed.add((library_import_name, library_import_path))
    264 if needs_to_be_installed:
--> 265     raise ImportError(
    266         f"To be able to use {name}, you need to install the following dependencies"
    267         f"{[lib_name for lib_name, lib_path in needs_to_be_installed]} using 'pip install "
    268         f"{' '.join([lib_path for lib_name, lib_path in needs_to_be_installed])}' for instance'"
    269     )
    270 return local_imports

ImportError: To be able to use evaluate-metric/glue, you need to install the following dependencies['sklearn'] using 'pip install sklearn' for instance'
```



The ImportError raised guided me to install sklearn using `pip install sklearn` but it didn't worked for me and the error still appearing.

**However, I solved the problem by using `pip install scikit-learn` and it works now.**

**Is there any error from my side? or is there anything should be changed on the evaluate library for this point?**

Note: it seems that sklearn is deprecated, [here](https://pypi.org/project/sklearn/)

**The libraries after installing sklearn:**

```bash
Package            Version       
------------------ --------------
aiohttp            3.8.3         
aiosignal          1.3.1         
asttokens          2.2.1         
async-timeout      4.0.2         
attrs              22.2.0        
backcall           0.2.0         
certifi            2022.12.7     
charset-normalizer 2.1.1         
comm               0.1.2         
datasets           2.8.0         
debugpy            1.6.4         
decorator          5.1.1         
dill               0.3.6         
entrypoints        0.4           
evaluate           0.4.0         
executing          1.2.0         
filelock           3.8.2         
frozenlist         1.3.3         
fsspec             2022.11.0     
huggingface-hub    0.11.1        
idna               3.4           
ipykernel          6.19.4        
ipython            8.7.0         
jedi               0.18.2        
joblib             1.2.0         
jupyter-client     7.4.8         
jupyter-core       5.1.1         
matplotlib-inline  0.1.6         
multidict          6.0.4         
multiprocess       0.70.14       
nest-asyncio       1.5.6         
numpy              1.24.1        
packaging          22.0          
pandas             1.5.2         
parso              0.8.3         
pexpect            4.8.0         
pickleshare        0.7.5         
Pillow             9.3.0         
pip                20.0.2        
pkg-resources      0.0.0         
platformdirs       2.6.2         
prompt-toolkit     3.0.36        
protobuf           3.20.2        
psutil             5.9.4         
ptyprocess         0.7.0         
pure-eval          0.2.2         
pyarrow            10.0.1        
Pygments           2.13.0        
python-dateutil    2.8.2         
pytz               2022.7        
PyYAML             6.0           
pyzmq              24.0.1        
regex              2022.10.31    
requests           2.28.1        
responses          0.18.0        
scipy              1.9.3         
sentencepiece      0.1.97        
setuptools         44.0.0        
six                1.16.0        
sklearn            0.0.post1     
stack-data         0.6.2         
threadpoolctl      3.1.0         
tokenizers         0.13.2        
torch              1.10.0+cu102  
torchaudio         0.10.0+rocm4.1
torchvision        0.11.0+cu102  
tornado            6.2           
tqdm               4.64.1        
traitlets          5.8.0         
transformers       4.25.1        
typing-extensions  4.4.0         
urllib3            1.26.13       
wcwidth            0.2.5         
xxhash             3.2.0         
yarl               1.8.2  

```

**The libraries after installing scikit-learn:**


```bash

Package            Version       
------------------ --------------
aiohttp            3.8.3         
aiosignal          1.3.1         
asttokens          2.2.1         
async-timeout      4.0.2         
attrs              22.2.0        
backcall           0.2.0         
certifi            2022.12.7     
charset-normalizer 2.1.1         
comm               0.1.2         
datasets           2.8.0         
debugpy            1.6.4         
decorator          5.1.1         
dill               0.3.6         
entrypoints        0.4           
evaluate           0.4.0         
executing          1.2.0         
filelock           3.8.2         
frozenlist         1.3.3         
fsspec             2022.11.0     
huggingface-hub    0.11.1        
idna               3.4           
ipykernel          6.19.4        
ipython            8.7.0         
jedi               0.18.2        
joblib             1.2.0         
jupyter-client     7.4.8         
jupyter-core       5.1.1         
matplotlib-inline  0.1.6         
multidict          6.0.4         
multiprocess       0.70.14       
nest-asyncio       1.5.6         
numpy              1.24.1        
packaging          22.0          
pandas             1.5.2         
parso              0.8.3         
pexpect            4.8.0         
pickleshare        0.7.5         
Pillow             9.3.0         
pip                20.0.2        
pkg-resources      0.0.0         
platformdirs       2.6.2         
prompt-toolkit     3.0.36        
protobuf           3.20.2        
psutil             5.9.4         
ptyprocess         0.7.0         
pure-eval          0.2.2         
pyarrow            10.0.1        
Pygments           2.13.0        
python-dateutil    2.8.2         
pytz               2022.7        
PyYAML             6.0           
pyzmq              24.0.1        
regex              2022.10.31    
requests           2.28.1        
responses          0.18.0        
scikit-learn       1.2.0         
scipy              1.9.3         
sentencepiece      0.1.97        
setuptools         44.0.0        
six                1.16.0        
sklearn            0.0.post1     
stack-data         0.6.2         
threadpoolctl      3.1.0         
tokenizers         0.13.2        
torch              1.10.0+cu102  
torchaudio         0.10.0+rocm4.1
torchvision        0.11.0+cu102  
tornado            6.2           
tqdm               4.64.1        
traitlets          5.8.0         
transformers       4.25.1        
typing-extensions  4.4.0         
urllib3            1.26.13       
wcwidth            0.2.5         
xxhash             3.2.0         
yarl               1.8.2         
```

Were there any updates on this? I am seeing the same error from following the same tutorial. 

My issue was a mismatch in virtual environments -- I had `evaluate` installed system-wide and `sklearn` installed in my virtualenv. Hope this helps.

> The ImportError raised guided me to install sklearn using `pip install sklearn` but it didn't worked for me and the error still appearing.
> 
> **However, I solved the problem by using `pip install scikit-learn` and it works now.**
> 
> Note: it seems that sklearn is deprecated, [here](https://pypi.org/project/sklearn/)

I was having the exact same issue and this fixed it for me. Thank you!

Still not working for me after installing `pip install scikit-learn`



@JuanuMusic 

Add more information, please


> Still not working for me after installing `pip install scikit-learn`

Had similar issue, try restarting the kernel, it worked for me

similar issue. worked by "pip install scikit-learn".   The error msg needs to be updated: "you need to install the following dependencies['sklearn'] using 'pip install sklearn'"


same issue and solve with pip install scikit-learn
is there a procedure to amend the hugging face notebook to reflect that so at the beggining it specify this as a library to install ?

So glad I looked this up.  I ran into the same problem and installing `scikit-learn` also fixed the problem.

So glad I looked this up.  I ran into the same problem and installing `scikit-learn` also fixed the problem.

I hit this too, resolved by installing scikit-learn


Had the same problem while trying the tutorial here https://huggingface.co/docs/transformers/tasks/sequence_classification
Installing 'pip install scikit-learn' did the trick for me too; with sklearn it didn't work

+1 to `pip install scikit-learn` doing the job

`pip install scikit-learn` fix this problem