Title: Failure to hash (and cache) a `.map(...)` (almost always) - using this method can produce incorrect results
Author: DrMatters (NONE)
State: closed
Created: 2022-06-15T17:11:31Z
Labels: bug

## Describe the bug
Sometimes I get messages about not being able to hash a method:
`Parameter 'function'=<function StupidDataModule._separate_speaker_id_from_dialogue at 0x7f1b27180d30> of the transform datasets.arrow_dataset.Dataset.
_map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.`
Whilst the function looks like this:
```python
@staticmethod
def _separate_speaker_id_from_dialogue(example: arrow_dataset.Example):
    speaker_id, dialogue = tuple(zip(*(example["dialogue"])))
    example["speaker_id"] = speaker_id
    example["dialogue"] = dialogue
    return example
```
This is the first step in my preprocessing pipeline, but sometimes the message about failure to hash is not appearing on the first step, but then appears on a later step.
This error is sometimes causing a failure to use cached data, instead of re-running all steps again.

## Steps to reproduce the bug
```python
import copy
import datasets
from datasets import arrow_dataset

def main():
    dataset = datasets.load_dataset("blended_skill_talk")
    res = dataset.map(method)
    print(res)

def method(example: arrow_dataset.Example):
    example['previous_utterance_copy'] = copy.deepcopy(example['previous_utterance'])
    return example

if __name__ == '__main__':
    main()
```
Run with:
```
python -m reproduce_error
```

## Expected results
Dataset is mapped and cached correctly.

## Actual results
The code outputs this at some point:
`Parameter 'function'=<function method at 0x7faa83d2a160> of the transform datasets.arrow_dataset.Dataset._map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.`

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform: Ubuntu 20.04.3
- Python version: 3.9.12
- PyArrow version: 8.0.0
- Datasets version: 2.3.1

Comments:
Important info:

As hashes are generated randomly for functions, it leads to **false identifying some results as already hashed** (mapping function is not executed after a method update) when there's a `pytorch_lightning.seed_everything(123)`

@lhoestq
seems like quite critical stuff for me, if I'm not making a mistake

Hi ! Thanks for reporting. This bug seems to appear in python 3.9 using dill 3.5.1

As a workaround you can use an older version of dill:
```
pip install "dill<0.3.5"
```

installing `dill<0.3.5` after installing `datasets` by pip results in dependency conflict with the version required for `multiprocess`. It can be solved by installing `pip install datasets "dill<0.3.5"` (simultaneously) on a clean environment

This has been fixed in https://github.com/huggingface/datasets/pull/4516, we will do a new release soon to include the fix :)