Repository: dask/dask

Issue:
_iLocIndexer / _LocIndexer deterministic hashing
Hi,

dask.dataframe.indexing._iLocIndexer and _LocIndexer should implement dask deterministic hashing:
https://docs.dask.org/en/latest/custom-collections.html#deterministic-hashing

The current situation is that dask create new tokens even if loc/iloc instructions are exactly the same:

```python
import pandas as pd
from dask.base import tokenize
from dask.dataframe import from_pandas
import dask

df = pd.DataFrame([[1, 2]], columns=['foo', 'bar'])
dask_df = from_pandas(df, npartitions=1)

foo_df = dask_df.loc[:, ['foo']]
foo_df_2 = dask_df.loc[:, ['foo']]

assert tokenize(foo_df) == tokenize(foo_df_2)
```

Suggested implementation:

```python
@normalize_token.register(_LocIndexer)
def normalize_loc_indexer(x):
    return type(x).__name__, x.obj
```

If you are ok with this implementation, I can add this code to dask.base and raise a PR.

Thanks for you help,

**Environment**:

- Dask version: 2022.05.0
- Python version: 3.9.12
- Operating System: CentOS Linux release 7.6.1810 (Core)
- Install method pip (pypi)


Discussion:
@faulaire Thank you for reporting and for your interest in opening a PR! I can reproduce this and it does look like a bug. 

@jrbourbeau could you please help confirm?
Thanks for reporting @faulaire -- agreed that having identical slices produce the same token hash would be useful 

> Suggested implementation:

Since we control the `_iLocIndexer` / `_LocIndexer` classes, I'd prefer to just add a `__dask_tokenize__` method directly to that class as we do in other place (e.g. [here](https://github.com/dask/dask/blob/6b62cc1215ba05ba6f89b226db0fe8a01a792306/dask/array/ufunc.py#L42-L43) and [here](https://github.com/dask/dask/blob/137206dc04eb62424617a068405545b26db99a6f/dask/dataframe/io/parquet/core.py#L132-L142)). That will help keep tokenization logic co-located with the relevant class

Resolving patch:
diff --git a/dask/dataframe/indexing.py b/dask/dataframe/indexing.py
--- a/dask/dataframe/indexing.py
+++ b/dask/dataframe/indexing.py
@@ -36,6 +36,9 @@ def _make_meta(self, iindexer, cindexer):
         else:
             return self._meta_indexer[:, cindexer]
 
+    def __dask_tokenize__(self):
+        return type(self).__name__, tokenize(self.obj)
+
 
 class _iLocIndexer(_IndexerBase):
     @property


Test patch:
diff --git a/dask/dataframe/tests/test_indexing.py b/dask/dataframe/tests/test_indexing.py
--- a/dask/dataframe/tests/test_indexing.py
+++ b/dask/dataframe/tests/test_indexing.py
@@ -4,6 +4,7 @@
 
 import dask
 import dask.dataframe as dd
+from dask.base import tokenize
 from dask.dataframe._compat import PANDAS_GT_110, PANDAS_GT_120, tm
 from dask.dataframe.indexing import _coerce_loc_index
 from dask.dataframe.utils import assert_eq, make_meta
@@ -684,3 +685,42 @@ def test_pandas_nullable_boolean_data_type():
 
     assert_eq(ddf1[ddf2], s1[s2])
     assert_eq(ddf1.loc[ddf2], s1.loc[s2])
+
+
+def test_deterministic_hashing_series():
+    obj = pd.Series([0, 1, 2])
+
+    dask_df = dd.from_pandas(obj, npartitions=1)
+
+    ddf1 = dask_df.loc[0:1]
+    ddf2 = dask_df.loc[0:1]
+
+    assert tokenize(ddf1) == tokenize(ddf2)
+
+    ddf2 = dask_df.loc[0:2]
+    assert tokenize(ddf1) != tokenize(ddf2)
+
+
+def test_deterministic_hashing_dataframe():
+    # Add duplicate column names in order to use _iLocIndexer._iloc path
+    obj = pd.DataFrame([[0, 1, 2, 3], [4, 5, 6, 7]], columns=["a", "b", "c", "c"])
+
+    dask_df = dd.from_pandas(obj, npartitions=1)
+
+    ddf1 = dask_df.loc[0:1, ["a", "c"]]
+    ddf2 = dask_df.loc[0:1, ["a", "c"]]
+
+    assert tokenize(ddf1) == tokenize(ddf2)
+
+    ddf1 = dask_df.loc[0:1, "c"]
+    ddf2 = dask_df.loc[0:1, "c"]
+
+    assert tokenize(ddf1) == tokenize(ddf2)
+
+    ddf1 = dask_df.iloc[:, [0, 1]]
+    ddf2 = dask_df.iloc[:, [0, 1]]
+
+    assert tokenize(ddf1) == tokenize(ddf2)
+
+    ddf2 = dask_df.iloc[:, [0, 2]]
+    assert tokenize(ddf1) != tokenize(ddf2)