Title: Use passed --cache_dir for modules cache
Author: albertvillanova (MEMBER)
State: open
Created: 2020-12-09T10:59:59Z

When passed `--cache_dir` arg:
```shell
python datasets-cli test datasets/<my-dataset-folder> --save_infos --all_configs --cache_dir <my-cache-dir>
```
it is not used for caching the modules, which are cached in the default location at `.cache/huggingface/modules`.

With this fix, the modules will be cached at `<my-cache-dir>/modules`.

Comments:
I have a question: why not using a tmp dir instead, like the DummyDataGeneratorDownloadManager does?

Hi @lhoestq, I am trying to understand better the logic...

Why do we have a `dynamic_module_path` besides the modules cache path?
```python
DYNAMIC_MODULES_PATH = os.path.join(HF_MODULES_CACHE, "datasets_modules")
```
Moreover, 2 subdirectories (for datasets and for metrics) were created inside it:
```python
DATASETS_PATH = os.path.join(DYNAMIC_MODULES_PATH, "datasets")
METRICS_PATH = os.path.join(DYNAMIC_MODULES_PATH, "metrics")
```

Hi :) 
The modules cache path is the path added to `sys.path`.
Therefore inside we need to have a folder that is going to be a package: `datasets_modules`.
This package will contain dynamic modules, i.e. datasets and metrics modules added on-the-fly.
Then we have two sub-modules `datasets_modules.datasets` and `datasets_modules.metrics`.

Maybe we can make things more explicit in the code with some comments explaining the structure, and maybe better variable naming as well..

Also I wanted to say that I started to work on offline loading of modules in #1726 and actually it lead to do similar changes to what you did to control the path where modules are stored.

Hi @lhoestq, I see...

Indeed I was also creating a draft for test_load, to clarify the expected behavior... ;)

So, for the command line:
```sh
python datasets-cli test datasets/<my-dataset-folder> --save_infos --all_configs --cache_dir <my-cache-dir>
```
the `cache_dir` argument refers to dataset cache dir. We do not have control over the modules cache dir, but we would like to have. And if I understand well, you suggest adding another argument `dynamic_module_path`. Am I right?

> So, for the command line:
> 
> ```shell
> python datasets-cli test datasets/<my-dataset-folder> --save_infos --all_configs --cache_dir <my-cache-dir>
> ```
> 
> the `cache_dir` argument refers to dataset cache dir. We do not have control over the modules cache dir, but we would like to have. And if I understand well, you suggest adding another argument `dynamic_module_path`. Am I right?

Yes the cache_dir is used to download files and also so save the dataset arrow files.
This is indeed different from the path for dynamic modules.

I suggested to have `dynamic_module_path` as a parameter but actually this is the parent directory `hf_modules_cache` that we would need (it's the one that is passed to `init_dynamic_modules ` that we need to add to `sys.path`).

Currently it's already possible to override it using the env variable `HF_MODULES_CACHE` but we can imagine having it as a parameter as well.

This way the user controls both the `cache_dir` and the `hf_modules_cache` which are the two places used by the library to read/write stuff.



I think #1726 is going to be merged pretty soon. Maybe can work on this as soon as it's merged to avoid doing the same things twice and to avoid conflicts ?

I agree. Indeed I took some of your code in one of my last commit, to try to implement the logic you described.