RagRetriever.from_pretrained doesn't get another cache_dir. 
Author: josemlopez
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.3.1
- Platform: Linux-4.19
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0
- Tensorflow version (GPU?): No
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@VictorSanh

## Information

Model I am using RAG:

The problem arises when using:
* [x] the official example scripts: (give details below)


## To reproduce

Steps to reproduce the behavior:

1. Open notebook 
2. Run the example code changing the 'TRANSFORMERS_CACHE' path to place the dataset in another place than the default one

```
import os
os.environ['TRANSFORMERS_CACHE'] = '/workspace/notebooks/POCs/cache'

from transformers import RagTokenizer, RagRetriever, RagTokenForGeneration

tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-nq") # Here the data is placed in the expected path /workspace...

retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", use_dummy_dataset=False) # The dataset is placed in the default place /root/.cache/huggingface/datasets/wiki_dpr/psgs_w100.nq.no_index/0.0.0/
```


## Expected behavior
`RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", use_dummy_dataset=False)` should place the data in the expected patch  '/workspace/notebooks/POCs/cache'

I tried with as well with: ` retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", chache_dir='/workspace/notebooks/POCs/cache' use_dummy_dataset=False)` but it doesn't work neither. 



Comments:
tagging @patrickvonplaten who will be more suitable to help

Hey @josemlopez  - thanks for the issue. @lhoestq - I think we should add an argument to the `RagRetriever.from_pretrained(...)` that passes the cache dir to the `load_dataset` function, no? What do you think? 

Thanks for your work guys.

BTW, in case this can be helpful. 
I've move my things so I can have enough room for the dataset in "/root/.cache/huggingface/datasets/".

Doing that, I've suffered this error. I can't say if it is related or not:

```
---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    459             try:
--> 460                 return pickle.load(fid, **pickle_kwargs)
    461             except Exception:

UnpicklingError: pickle data was truncated

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    552                 # Prepare split will record examples associated to the split
--> 553                 self._prepare_split(split_generator, **prepare_split_kwargs)
    554             except OSError:

/opt/conda/lib/python3.7/site-packages/datasets/builder.py in _prepare_split(self, split_generator)
    840             for key, record in utils.tqdm(
--> 841                 generator, unit=" examples", total=split_info.num_examples, leave=False, disable=not_verbose
    842             ):

/opt/conda/lib/python3.7/site-packages/tqdm/notebook.py in __iter__(self, *args, **kwargs)
    217         try:
--> 218             for obj in super(tqdm_notebook, self).__iter__(*args, **kwargs):
    219                 # return super(tqdm...) will not catch exception

/opt/conda/lib/python3.7/site-packages/tqdm/std.py in __iter__(self)
   1128         try:
-> 1129             for obj in iterable:
   1130                 yield obj

~/.cache/huggingface/modules/datasets_modules/datasets/wiki_dpr/14b973bf2a456087ff69c0fd34526684eed22e48e0dfce4338f9a22b965ce7c2/wiki_dpr.py in _generate_examples(self, data_file, vectors_files)
    131                         break
--> 132                     vecs = np.load(open(vectors_files.pop(0), "rb"), allow_pickle=True)
    133                     vec_idx = 0

/opt/conda/lib/python3.7/site-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    462                 raise IOError(
--> 463                     "Failed to interpret file %s as a pickle" % repr(file))
    464     finally:

OSError: Failed to interpret file <_io.BufferedReader name='/root/.cache/huggingface/datasets/downloads/f34d5f091294259b4ca90e813631e69a6ded660d71b6cbedf89ddba50df94448'> as a pickle

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-6-f28df370ac47> in <module>
      1 # ln -s /workspace/notebooks/POCs/cache /root/.cache/huggingface/datasets
----> 2 retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", use_dummy_dataset=False)

/opt/conda/lib/python3.7/site-packages/transformers/retrieval_rag.py in from_pretrained(cls, retriever_name_or_path, **kwargs)
    307         generator_tokenizer = rag_tokenizer.generator
    308         return cls(
--> 309             config, question_encoder_tokenizer=question_encoder_tokenizer, generator_tokenizer=generator_tokenizer
    310         )
    311 

/opt/conda/lib/python3.7/site-packages/transformers/retrieval_rag.py in __init__(self, config, question_encoder_tokenizer, generator_tokenizer)
    298         self.config = config
    299         if self._init_retrieval:
--> 300             self.init_retrieval()
    301 
    302     @classmethod

/opt/conda/lib/python3.7/site-packages/transformers/retrieval_rag.py in init_retrieval(self)
    324 
    325         logger.info("initializing retrieval")
--> 326         self.index.init_index()
    327 
    328     def postprocess_docs(self, docs, input_strings, prefix, n_docs, return_tensors=None):

/opt/conda/lib/python3.7/site-packages/transformers/retrieval_rag.py in init_index(self)
    238                 split=self.dataset_split,
    239                 index_name=self.index_name,
--> 240                 dummy=self.use_dummy_dataset,
    241             )
    242             self.dataset.set_format("numpy", columns=["embeddings"], output_all_columns=True)

/opt/conda/lib/python3.7/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    609         download_config=download_config,
    610         download_mode=download_mode,
--> 611         ignore_verifications=ignore_verifications,
    612     )
    613 

/opt/conda/lib/python3.7/site-packages/datasets/builder.py in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, **download_and_prepare_kwargs)
    474                     if not downloaded_from_gcs:
    475                         self._download_and_prepare(
--> 476                             dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    477                         )
    478                     # Sync info

/opt/conda/lib/python3.7/site-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    553                 self._prepare_split(split_generator, **prepare_split_kwargs)
    554             except OSError:
--> 555                 raise OSError("Cannot find data file. " + (self.manual_download_instructions or ""))
    556 
    557         if verify_infos:

OSError: Cannot find data file. 
```

When running this: 

`retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", use_dummy_dataset=False)`

> Hey @josemlopez - thanks for the issue. @lhoestq - I think we should add an argument to the `RagRetriever.from_pretrained(...)` that passes the cache dir to the `load_dataset` function, no? What do you think?

Sure we can add `cache_dir=...` to `RagRetriever.from_pretrained`.
In the meantime you can specify `HF_DATASETS_CACHE` to tell where to store the dataset used by RAG for retrieval

> Thanks for your work guys.
> 
> BTW, in case this can be helpful.
> I've move my things so I can have enough room for the dataset in "/root/.cache/huggingface/datasets/".
> 
> Doing that, I've suffered this error. I can't say if it is related or not:
> ...
> When running this:
> 
> `retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", use_dummy_dataset=False)`

Could you create an issue on the `datasets` repo ? this seems unrelated 

Hi @lhoestq , 

>In the meantime you can specify HF_DATASETS_CACHE to tell where to store the dataset used by RAG for retrieval

HF_DATASETS_CACHE works fine: 

```
retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", use_dummy_dataset=False)

Using custom data configuration psgs_w100.nq.no_index
Reusing dataset wiki_dpr (/my_cache/cache/wiki_dpr/psgs_w100.nq.no_index/0.0.0/14b973bf2a456087ff69c0fd34526684eed22e48e0dfce4338f9a22b965ce7c2)
Using custom data configuration psgs_w100.nq.exact

Downloading and preparing dataset wiki_dpr/psgs_w100.nq.exact (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to /my_cache/cache/wiki_dpr/psgs_w100.nq.exact/0.0.0/14b973bf2a456087ff69c0fd34526684eed22e48e0dfce4338f9a22b965ce7c2...
```

>Could you create an issue on the datasets repo ? this seems unrelated

sure, I'll post the other issue in the datasets repo.

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.