.add_faiss_index and .add_elasticsearch_index returns ImportError at Google Colab
Opened by MapleBloom (open)

### Describe the bug

At Google Colab
```!pip install faiss-cpu``` works
```import faiss``` no error
but
```embeddings_dataset.add_faiss_index(column='embeddings')```
returns
```
[/usr/local/lib/python3.11/dist-packages/datasets/search.py](https://localhost:8080/#) in init(self, device, string_factory, metric_type, custom_index)
247 self.faiss_index = custom_index
248 if not _has_faiss:
--> 249 raise ImportError(
250 "You must install Faiss to use FaissIndex. To do so you can run conda install -c pytorch faiss-cpu or conda install -c pytorch faiss-gpu. "
251 "A community supported package is also available on pypi: pip install faiss-cpu or pip install faiss-gpu. "
```

because
```_has_faiss = importlib.util.find_spec("faiss") is not None```    at the beginning of ```datasets/search.py``` returns ```False```
when
the same code at colab notebook returns
```ModuleSpec(name='faiss', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7b7851449f50>, origin='/usr/local/lib/python3.11/dist-packages/faiss/init.py', submodule_search_locations=['/usr/local/lib/python3.11/dist-packages/faiss'])```

But
```
import datasets
datasets.search._has_faiss
```
at ```colab notebook``` also returns ```False```

The same story with   ```_has_elasticsearch```

### Steps to reproduce the bug

1. Follow https://huggingface.co/learn/nlp-course/chapter5/6?fw=pt at Google Colab
2. till ```embeddings_dataset.add_faiss_index(column='embeddings')```
3. ```embeddings_dataset.add_elasticsearch_index(column='embeddings')```
4. https://colab.research.google.com/drive/1h2cjuiClblqzbNQgrcoLYOC8zBqTLLcv#scrollTo=3ddzRp72auOF

### Expected behavior

I've only started Tutorial and don't know exactly. But something tells me that ```embeddings_dataset.add_faiss_index(column='embeddings')```
should work without ```Import Error```

### Environment info

Google Colab notebook with default config

Comments:
I can fix this.
It's mainly because faiss-gpu requires python<=3.10 but the default python version in colab is 3.11. We just have to downgrade the CPython version down to 3.10 and it should work fine.


I think I just had no chance to meet with faiss-cpu.
It could be import problem? 
_has_faiss gets its value at the beginning of datasets/search.
I tried to call object before import faiss, so _has_faiss took False. And never updated later. 

Yes you can't meet the requirements because faiss-cpu runs only on
python3.10 and lower but the default version for colab is python3.11 which
results in pip not being able to find wheels for faiss-cpu with python3.11.

On Mon, 17 Mar, 2025, 3:56 pm MapleBloom, ***@***.***> wrote:

> I think I just had no chance to meet with faiss-cpu.
> It could be import problem?
> _has_faiss gets its value at the beginning of datasets/search.
> I tried to call object before import faiss, so _has_faiss took False. And
> never updated later.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/7456#issuecomment-2728975672>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AVUSZMBVD7LEDDUGALOTVN32U2PMBAVCNFSM6AAAAABZDBA426VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRYHE3TKNRXGI>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
> [image: MapleBloom]*MapleBloom* left a comment (huggingface/datasets#7456)
> <https://github.com/huggingface/datasets/issues/7456#issuecomment-2728975672>
>
> I think I just had no chance to meet with faiss-cpu.
> It could be import problem?
> _has_faiss gets its value at the beginning of datasets/search.
> I tried to call object before import faiss, so _has_faiss took False. And
> never updated later.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/7456#issuecomment-2728975672>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AVUSZMBVD7LEDDUGALOTVN32U2PMBAVCNFSM6AAAAABZDBA426VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRYHE3TKNRXGI>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>


> you can't meet the requirements

It is not the case (or I didn't reach this point) because the same code in notebook
```importlib.util.find_spec("faiss")```
finds faiss. I've mention it.
I think the problem is in the very moment when _has_faiss takes its value and never try again. 
(or it couldn't find the path that was easily found when started from my code)

When you run the first cell containing pip install faiss-cpu does it
install it?

On Mon, 17 Mar, 2025, 8:01 pm MapleBloom, ***@***.***> wrote:

> you can't meet the requirements
>
> It is not the case (or I didn't reach this point) because the same code in
> notebook
> importlib.util.find_spec("faiss")
> finds faiss. I've mention it.
> I think the problem is in the very moment when _has_faiss takes its value
> and never try again.
> (or it couldn't find the path that was easily found when started from my
> code)
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/7456#issuecomment-2729737414>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AVUSZMCCE6BPZCOVAWXKIY32U3MFVAVCNFSM6AAAAABZDBA426VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRZG4ZTONBRGQ>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
> [image: MapleBloom]*MapleBloom* left a comment (huggingface/datasets#7456)
> <https://github.com/huggingface/datasets/issues/7456#issuecomment-2729737414>
>
> you can't meet the requirements
>
> It is not the case (or I didn't reach this point) because the same code in
> notebook
> importlib.util.find_spec("faiss")
> finds faiss. I've mention it.
> I think the problem is in the very moment when _has_faiss takes its value
> and never try again.
> (or it couldn't find the path that was easily found when started from my
> code)
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/7456#issuecomment-2729737414>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AVUSZMCCE6BPZCOVAWXKIY32U3MFVAVCNFSM6AAAAABZDBA426VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRZG4ZTONBRGQ>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>


> When you run the first cell containing pip install faiss-cpu does it
> install it?
> […](#)

Yes. It was installed succesfully. 
Methods of datasets library that depends on _has_faiss constant didn't start to work.