Title: dataset.shuffle() and select() resets format. Intended?
Author: vegarab (CONTRIBUTOR)
State: closed
Created: 2020-08-18T13:46:01Z

Calling `dataset.shuffle()` or `dataset.select()` on a dataset resets its format set by `dataset.set_format()`. Is this intended or an oversight?

When working on quite large datasets that require a lot of preprocessing I find it convenient to save the processed dataset to file using `torch.save("dataset.pt")`. Later loading the dataset object using `torch.load("dataset.pt")`, which conserves the defined format before saving. 
I do shuffling and selecting (for controlling dataset size) after loading the data from .pt-file, as it's convenient whenever you train multiple models with varying sizes of the same dataset. 

The obvious workaround for this is to set the format again after using `dataset.select()` or `dataset.shuffle()`.

_I guess this is more of a discussion on the design philosophy of the functions. Please let me know if this is not the right channel for these kinds of discussions or if they are not wanted at all!_

####  How to reproduce:

```python
import nlp
from transformers import T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("t5-base")
def create_features(batch):
    context_encoding = tokenizer.batch_encode_plus(batch["context"])
    return {"input_ids": context_encoding["input_ids"]}

dataset = nlp.load_dataset("cosmos_qa", split="train")
dataset = dataset.map(create_features, batched=True)
dataset.set_format(type="torch", columns=["input_ids"])
dataset[0]
# {'input_ids': tensor([ 1804,  3525,  1602,  ...   0,     0])}

dataset = dataset.shuffle()
dataset[0]
# {'id': '3Q9(...)20', 'context': "Good Old War an (...) play ?', 'answer0': 'None of the above choices .', 'answer1': 'This person likes music and likes to see the show , they will see other bands play .', (...) 'input_ids': [1804, 3525, 1602, ... , 0, 0]}

```

Comments:
Hi @vegarab yes feel free to open a discussion here.

This design choice was not very much thought about.

Since `dataset.select()` (like all the method without a trailing underscore) is non-destructive and returns a new dataset it has most of its properties initialized from scratch (except the table and infos).

Thinking about it I don't see a strong reason against transmitting the format from the parent dataset to its newly created child. It's probably what's expected by the user in most cases. What do you think @lhoestq?

By the way, I've been working today on a refactoring of all the samples re-ordering/selection methods (`select`, `sort`, `shuffle`, `shard`, `train_test_split`). The idea is to speed them up by a lot (like, really a lot) by working as much as possible with an indices mapping table instead of doing a deep copy of the full dataset as we've been doing currently. You can give it a look and try it here: https://github.com/huggingface/nlp/pull/513
Feedbacks are very much welcome

I think it's ok to keep the format.
If we want to have this behavior for `.map` too we just have to make sure it doesn't keep a column that's been removed.

Shall we have this in the coming release by the way @lhoestq ?

Yes sure !

Since datasets 1.0.0 the format is not reset anymore.
Closing this one, but feel free to re-open if you have other questions