Add env variable for MAX_IN_MEMORY_DATASET_SIZE_IN_BYTES
Thank you for clarifying the precedence, @albertvillanova 

Isn't it typically the case where env vars have the highest precedence? 

In my understanding the point of env vars is to be able to override software w/o needing to touch the code. 

Please correct me if this is not so in the general case. Hi @stas00, 

Well, I'm not an expert on this topic, but the precedence hierarchy I have normally found is from higher to lower:
- command line parameters
- env vars
- config files
So yes, normally env vars have precedence over configuration files.

Anyway, for Datasets, there are no configuration files. The _in-memory_ config is set from default values or env vars (which have precedence over default values). But this is done at import.

However, once the library is imported, the user can modify the in-memory config, and this will have precedence over the rest of mechanisms (which take place only at import). In my limited experience env vars are typically above cmd line args, so that one can override scripts with cmd lines using env vars, but usually one then uses env vars inside cmd line args, so it's loud and clear.

For example specifying a specific gpu number on a command line will depend on `CUDA_VISIBLE_DEVICES` so gpu0 will be different if someone sets `CUDA_VISIBLE_DEVICES=2,3` vs  `CUDA_VISIBLE_DEVICES=0,1`.

> However, once the library is imported, the user can modify the in-memory config, and this will have precedence over the rest of mechanisms (which take place only at import).

And this is exactly the problem we are trying to solve here. For a good reason HF examples don't want to use `keep_in_memory=False`, and they may choose to now set `datasets.config.MAX_IN_MEMORY_DATASET_SIZE_IN_BYTES` and which means we again can't override it via env var.

 oops, sorry, didn't think earlier - do we need to prefix this with `HF_DATASETS` or `HF_` like all the other env vars? or is it long enough already to be unique - it's just not telling the user in the config file what projet this variable is for... You're right, I just opened https://github.com/huggingface/datasets/pull/2409
Add env variable for `MAX_IN_MEMORY_DATASET_SIZE_IN_BYTES`.

This will allow to turn off default behavior: loading in memory (and not caching) small datasets.

Fix #2387.