Datasets performance slow? - 6.4x slower than in memory dataset 
 I've been very excited about this amazing datasets project. However, I've noticed that the performance can be substantially slower than using an in-memory dataset.

Now, this is expected I guess, due to memory mapping data using arrow files, and you don't get anything for free. But I was surprised at how much slower.

For example, in the `yelp_polarity` dataset (560000 datapoints, or 17500 batches of 32), it was taking me 3:31 to just get process the data and get it on the GPU (no model involved). Whereas, the equivalent in-memory dataset would finish in just 0:33.

Is this expected? Given that one of the goals of this project is also accelerate dataset processing, this seems a bit slower than I would expect. I understand the advantages of being able to work on datasets that exceed memory, and that's very exciting to me, but thought I'd open this issue to discuss.

For reference I'm running a AMD Ryzen Threadripper 1900X 8-Core Processor CPU, with 128 GB of RAM and an NVME SSD Samsung 960 EVO. I'm running with an RTX Titan 24GB GPU.

I can see with `iotop` that the dataset gets quickly loaded into the system read buffers, and thus doesn't incur any additional IO reads. Thus in theory, all the data *should* be in RAM, but in my benchmark code below it's still 6.4 times slower.

What am I doing wrong? And is there a way to force the datasets to completely load into memory instead of being memory mapped in cases where you want maximum performance?

At 3:31 for 17500 batches, that's 12ms per batch. Does this 12ms just become insignificant as a proportion of forward and backward passes in practice, and thus it's not worth worrying about this in practice?

In any case, here's my code `benchmark.py`. If you run it with an argument of `memory` it will copy the data into memory before executing the same test.

``` py
import sys
from datasets import load_dataset
from transformers import DataCollatorWithPadding, BertTokenizerFast
from torch.utils.data import DataLoader
from tqdm import tqdm

if __name__ == '__main__':
    tokenizer = BertTokenizerFast.from_pretrained('bert-base-cased')
    collate_fn = DataCollatorWithPadding(tokenizer, padding=True)

    ds = load_dataset('yelp_polarity')

    def do_tokenize(x):
        return tokenizer(x['text'], truncation=True)

    ds = ds.map(do_tokenize, batched=True)
    ds.set_format('torch', ['input_ids', 'token_type_ids', 'attention_mask'])

    if len(sys.argv) == 2 and sys.argv[1] == 'memory':
        # copy to memory - probably a faster way to do this - but demonstrates the point
        # approximately 530 batches per second - 17500 batches in 0:33
        print('using memory')
        _ds = [data for data in tqdm(ds['train'])]
    else:
        # approximately 83 batches per second - 17500 batches in 3:31
        print('using datasets')
        _ds = ds['train']

    dl = DataLoader(_ds, shuffle=True, collate_fn=collate_fn, batch_size=32, num_workers=4)

    for data in tqdm(dl):
        for k, v in data.items():
            data[k] = v.to('cuda')
```

For reference, my conda environment is [here](https://gist.github.com/05b6101518ff70ed42a858b302a0405d)

Once again, I'm very excited about this library, and how easy it is to load datasets, and to do so without worrying about system memory constraints.

Thanks for all your great work.
 
 Great! Thanks a lot.

I did a test using `map(..., keep_in_memory=True)` and also a test using in-memory only data.

```python
features = dataset.map(tokenize, batched=True, remove_columns=dataset['train'].column_names)
features.set_format(type='torch', columns=['input_ids', 'token_type_ids', 'attention_mask'])

features_in_memory = dataset.map(tokenize, batched=True, keep_in_memory=True, remove_columns=dataset['train'].column_names)
features_in_memory.set_format(type='torch', columns=['input_ids', 'token_type_ids', 'attention_mask'])

in_memory = [features['train'][i] for i in range(len(features['train']))]
```

For using the features without any tweak, I got **1min17s** for copying the entire DataLoader to CUDA:

```
%%time

for i, batch in enumerate(DataLoader(features['train'], batch_size=16, num_workers=4)):
    batch['input_ids'].to(device)
```

For using the features mapped with `keep_in_memory=True`, I also got **1min17s** for copying the entire DataLoader to CUDA:

```
%%time

for i, batch in enumerate(DataLoader(features_in_memory['train'], batch_size=16, num_workers=4)):
    batch['input_ids'].to(device)
```

And for the case using every element in memory, converted from the original dataset, I got **12.5s**:

```
%%time

for i, batch in enumerate(DataLoader(in_memory, batch_size=16, num_workers=4)):
    batch['input_ids'].to(device)
```

Taking a closer look in my SQuAD code, using a profiler, I see a lot of calls to `posix read` api. It seems that it is really reliying on disk, which results in a very high train time.