Dataset.shuffle(seed=None) gives fixed row permutation
State: closed | Author: elisno (NONE)
Labels: name: bug

## Describe the bug
Repeated attempts to `shuffle` a dataset without specifying a seed give the same results.

## Steps to reproduce the bug
```python
import datasets

# Some toy example
data = datasets.Dataset.from_dict(
    {"feature": [1, 2, 3, 4, 5], "label": ["a", "b", "c", "d", "e"]}
)

# Doesn't work as expected
print("Shuffle dataset")
for _ in range(3):
    print(data.shuffle(seed=None)[:])

# This seems to work with pandas
print("\nShuffle via pandas")
for _ in range(3):
    df = data.to_pandas().sample(frac=1.0)
    print(datasets.Dataset.from_pandas(df, preserve_index=False)[:])

```

## Expected results
I assumed that the default setting would initialize a new/random state of a `np.random.BitGenerator` (see [docs](https://huggingface.co/docs/datasets/package_reference/main_classes.html?highlight=shuffle#datasets.Dataset.shuffle)).

Wouldn't that reshuffle the rows each time I call `data.shuffle()`?

## Actual results

```bash
Shuffle dataset
{'feature': [5, 1, 3, 2, 4], 'label': ['e', 'a', 'c', 'b', 'd']}
{'feature': [5, 1, 3, 2, 4], 'label': ['e', 'a', 'c', 'b', 'd']}
{'feature': [5, 1, 3, 2, 4], 'label': ['e', 'a', 'c', 'b', 'd']}

Shuffle via pandas
{'feature': [4, 2, 3, 1, 5], 'label': ['d', 'b', 'c', 'a', 'e']}
{'feature': [2, 5, 3, 4, 1], 'label': ['b', 'e', 'c', 'd', 'a']}
{'feature': [5, 2, 3, 1, 4], 'label': ['e', 'b', 'c', 'a', 'd']}
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.0
- Platform: Linux-5.13.0-27-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- PyArrow version: 6.0.1


Comments: I'm not sure if this is expected behavior.

Am I supposed to work with a copy of the dataset, i.e. `shuffled_dataset = data.shuffle(seed=None)`?

```diff
import datasets

# Some toy example
data = datasets.Dataset.from_dict(
    {"feature": [1, 2, 3, 4, 5], "label": ["a", "b", "c", "d", "e"]}
)

+shuffled_data = data.shuffle(seed=None)

# Doesn't work as expected
print("Shuffle dataset")
for _ in range(3):
+    shuffled_data = shuffled_data.shuffle(seed=None)
+    print(shuffled_data[:])
-    print(data.shuffle(seed=None)[:])

# This seems to work with pandas
print("\nShuffle via pandas")
for _ in range(3):
    df = data.to_pandas().sample(frac=1.0)
    print(datasets.Dataset.from_pandas(df, preserve_index=False)[:])

```

or provide a `generator` instead?

```diff
import datasets
+from numpy.random import default_rng

# Some toy example
data = datasets.Dataset.from_dict(
    {"feature": [1, 2, 3, 4, 5], "label": ["a", "b", "c", "d", "e"]}
)

+rng = default_rng()

# Doesn't work as expected
print("Shuffle dataset")
for _ in range(3):
+   print(data.shuffle(generator=rng)[:])
-   print(data.shuffle(seed=None)[:])

# This seems to work with pandas
print("\nShuffle via pandas")
for _ in range(3):
    df = data.to_pandas().sample(frac=1.0)
    print(datasets.Dataset.from_pandas(df, preserve_index=False)[:])

```

Hi! Thanks for reporting! Yes, this is not expected behavior. I've opened a PR with the fix.