Title: dataset.filter overwriting previously set dataset._indices values, resulting in the wrong elements being selected.
Author: mmajurski (NONE)
State: closed
Created: 2021-12-20T16:16:49Z
Labels: bug

## Describe the bug
When using dataset.select to select a subset of a dataset, dataset._indices are set to indicate which elements are now considered in the dataset.
The same thing happens when you shuffle the dataset; dataset._indices are set to indicate what the new order of the data is.
However, if you then use a dataset.filter, that filter interacts with those dataset._indices values in a non-intuitive manner.
https://huggingface.co/docs/datasets/_modules/datasets/arrow_dataset.html#Dataset.filter

Effectively, it looks like the original set of _indices were discared and overwritten by the set created during the filter operation.

I think this is actually an issue with how the map function handles dataset._indices. Ideally it should use the _indices it gets passed, and then return an updated _indices which reflect the map transformation applied to the starting _indices.

## Steps to reproduce the bug
```python
dataset = load_dataset('imdb', split='train', keep_in_memory=True)

dataset = dataset.shuffle(keep_in_memory=True)

dataset = dataset.select(range(0, 10), keep_in_memory=True)
print("initial 10 elements")
print(dataset['label'])   # -> [1, 1, 0, 1, 0, 0, 0, 1, 0, 0]

dataset = dataset.filter(lambda x: x['label'] == 0, keep_in_memory=True)
print("filtered 10 elements looking for label 0")
print(dataset['label'])  # -> [1, 1, 1, 1, 1, 1]
```

## Actual results
```
$ python indices_bug.py
initial 10 elements
[1, 1, 0, 1, 0, 0, 0, 1, 0, 0]
filtered 10 elements looking for label 0
[1, 1, 1, 1, 1, 1]
```

This code block first shuffles the dataset (to get a mix of label 0 and label 1).
Then it selects just the first 10 elements (the number of elements does not matter, 10 is just easy to visualize). The important part is that you select some subset of the dataset. 
Finally, a filter is applied to pull out just the elements with `label == 0`.

The bug is that you cannot combine any dataset operation which sets the dataset._indices with filter.
In this case I have 2, shuffle and subset.

If you just use a single dataset._indices operation (in this case shuffle) the bug still shows up.

The shuffle sets the dataset._indices and then filter uses those indices in the map, then overwrites dataset._indices with the filter results.
```python
dataset = load_dataset('imdb', split='train', keep_in_memory=True)

dataset = dataset.shuffle(keep_in_memory=True)

dataset = dataset.filter(lambda x: x['label'] == 0, keep_in_memory=True)

dataset = dataset.select(range(0, 10), keep_in_memory=True)
print(dataset['label']) # -> [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
```

## Expected results
In an ideal world, the dataset filter would respect any dataset._indices values which had previously been set.

If you use dataset.filter with the base dataset (where dataset._indices has not been set) then the filter command works as expected.

## Environment info
Here are the commands required to rebuild the conda environment from scratch.
```
# create a virtual environment
conda create -n dataset_indices python=3.8 -y

# activate the virtual environment
conda activate dataset_indices

# install huggingface datasets
conda install datasets
```

<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Linux-5.11.0-41-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- PyArrow version: 3.0.0


### Full Conda Environment
```
$ conda env export
name: dasaset_indices
channels:
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=4.5=1_gnu
  - abseil-cpp=20210324.2=h2531618_0
  - aiohttp=3.8.1=py38h7f8727e_0
  - aiosignal=1.2.0=pyhd3eb1b0_0
  - arrow-cpp=3.0.0=py38h6b21186_4
  - attrs=21.2.0=pyhd3eb1b0_0
  - aws-c-common=0.4.57=he6710b0_1
  - aws-c-event-stream=0.1.6=h2531618_5
  - aws-checksums=0.1.9=he6710b0_0
  - aws-sdk-cpp=1.8.185=hce553d0_0
  - bcj-cffi=0.5.1=py38h295c915_0
  - blas=1.0=mkl
  - boost-cpp=1.73.0=h27cfd23_11
  - bottleneck=1.3.2=py38heb32a55_1
  - brotli=1.0.9=he6710b0_2
  - brotli-python=1.0.9=py38heb0550a_2
  - brotlicffi=1.0.9.2=py38h295c915_0
  - brotlipy=0.7.0=py38h27cfd23_1003
  - bzip2=1.0.8=h7b6447c_0
  - c-ares=1.17.1=h27cfd23_0
  - ca-certificates=2021.10.26=h06a4308_2
  - certifi=2021.10.8=py38h06a4308_0
  - cffi=1.14.6=py38h400218f_0
  - conllu=4.4.1=pyhd3eb1b0_0
  - cryptography=36.0.0=py38h9ce1e76_0
  - dataclasses=0.8=pyh6d0b6a4_7
  - dill=0.3.4=pyhd3eb1b0_0
  - double-conversion=3.1.5=he6710b0_1
  - et_xmlfile=1.1.0=py38h06a4308_0
  - filelock=3.4.0=pyhd3eb1b0_0
  - frozenlist=1.2.0=py38h7f8727e_0
  - gflags=2.2.2=he6710b0_0
  - glog=0.5.0=h2531618_0
  - gmp=6.2.1=h2531618_2
  - grpc-cpp=1.39.0=hae934f6_5
  - huggingface_hub=0.0.17=pyhd3eb1b0_0
  - icu=58.2=he6710b0_3
  - idna=3.3=pyhd3eb1b0_0
  - importlib-metadata=4.8.2=py38h06a4308_0
  - importlib_metadata=4.8.2=hd3eb1b0_0
  - intel-openmp=2021.4.0=h06a4308_3561
  - krb5=1.19.2=hac12032_0
  - ld_impl_linux-64=2.35.1=h7274673_9
  - libboost=1.73.0=h3ff78a5_11
  - libcurl=7.80.0=h0b77cf5_0
  - libedit=3.1.20210910=h7f8727e_0
  - libev=4.33=h7f8727e_1
  - libevent=2.1.8=h1ba5d50_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.3.0=h5101ec6_17
  - libgomp=9.3.0=h5101ec6_17
  - libnghttp2=1.46.0=hce63b2e_0
  - libprotobuf=3.17.2=h4ff587b_1
  - libssh2=1.9.0=h1ba5d50_1
  - libstdcxx-ng=9.3.0=hd4cf53a_17
  - libthrift=0.14.2=hcc01f38_0
  - libxml2=2.9.12=h03d6c58_0
  - libxslt=1.1.34=hc22bd24_0
  - lxml=4.6.3=py38h9120a33_0
  - lz4-c=1.9.3=h295c915_1
  - mkl=2021.4.0=h06a4308_640
  - mkl-service=2.4.0=py38h7f8727e_0
  - mkl_fft=1.3.1=py38hd3c417c_0
  - mkl_random=1.2.2=py38h51133e4_0
  - multiprocess=0.70.12.2=py38h7f8727e_0
  - multivolumefile=0.2.3=pyhd3eb1b0_0
  - ncurses=6.3=h7f8727e_2
  - numexpr=2.7.3=py38h22e1b3c_1
  - numpy=1.21.2=py38h20f2e39_0
  - numpy-base=1.21.2=py38h79a1101_0
  - openpyxl=3.0.9=pyhd3eb1b0_0
  - openssl=1.1.1l=h7f8727e_0
  - orc=1.6.9=ha97a36c_3
  - packaging=21.3=pyhd3eb1b0_0
  - pip=21.2.4=py38h06a4308_0
  - py7zr=0.16.1=pyhd3eb1b0_1
  - pycparser=2.21=pyhd3eb1b0_0
  - pycryptodomex=3.10.1=py38h27cfd23_1
  - pyopenssl=21.0.0=pyhd3eb1b0_1
  - pyparsing=3.0.4=pyhd3eb1b0_0
  - pyppmd=0.16.1=py38h295c915_0
  - pysocks=1.7.1=py38h06a4308_0
  - python=3.8.12=h12debd9_0
  - python-dateutil=2.8.2=pyhd3eb1b0_0
  - python-xxhash=2.0.2=py38h7f8727e_0
  - pyzstd=0.14.4=py38h7f8727e_3
  - re2=2020.11.01=h2531618_1
  - readline=8.1=h27cfd23_0
  - requests=2.26.0=pyhd3eb1b0_0
  - setuptools=58.0.4=py38h06a4308_0
  - six=1.16.0=pyhd3eb1b0_0
  - snappy=1.1.8=he6710b0_0
  - sqlite=3.36.0=hc218d9a_0
  - texttable=1.6.4=pyhd3eb1b0_0
  - tk=8.6.11=h1ccaba5_0
  - typing_extensions=3.10.0.2=pyh06a4308_0
  - uriparser=0.9.3=he6710b0_1
  - utf8proc=2.6.1=h27cfd23_0
  - wheel=0.37.0=pyhd3eb1b0_1
  - xxhash=0.8.0=h7f8727e_3
  - xz=5.2.5=h7b6447c_0
  - zipp=3.6.0=pyhd3eb1b0_0
  - zlib=1.2.11=h7f8727e_4
  - zstd=1.4.9=haebb681_0
  - pip:
    - async-timeout==4.0.2
    - charset-normalizer==2.0.9
    - datasets==1.16.1
    - fsspec==2021.11.1
    - huggingface-hub==0.2.1
    - multidict==5.2.0
    - pandas==1.3.5
    - pyarrow==6.0.1
    - pytz==2021.3
    - pyyaml==6.0
    - tqdm==4.62.3
    - typing-extensions==4.0.1
    - urllib3==1.26.7
    - yarl==1.7.2

```

Comments:
I think this is a duplicate of [#3190](https://github.com/huggingface/datasets/issues/3190)?

Upgrading the datasets version as per #3190 fixes this bug. 
I'm Marking as closed.