Failure to hash function when using .map() 

### Describe the bug

_Parameter 'function'=<function process at 0x7f1ec4388af0> of the transform datasets.arrow_dataset.Dataset.\_map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed._

This issue with `.map()` happens for me consistently, as also described in closed issue #4506

Dataset indices can be individually serialized using dill and pickle without any errors.  I'm using tiktoken to encode in the function passed to map(). Similarly, indices can be individually encoded without error.

### Steps to reproduce the bug

```py
from datasets import load_dataset
import tiktoken

dataset = load_dataset("stas/openwebtext-10k")

enc = tiktoken.get_encoding("gpt2")

tokenized = dataset.map(
    process,
    remove_columns=['text'],
    desc="tokenizing the OWT splits",
)

def process(example):
        ids = enc.encode(example['text'])
        ids.append(enc.eot_token)
        out = {'ids': ids, 'len': len(ids)}
        return out
```

### Expected behavior

Should encode simple text objects.

### Environment info


Python versions tried: both 3.8 and 3.10.10
`PYTHONUTF8=1` as env variable

Datasets tried: 
- stas/openwebtext-10k
- rotten_tomatoes
- local text file

OS: Ubuntu Linux 20.04

Package versions:
- torch 1.13.1
- dill 0.3.4 (if using 0.3.6 - same issue)
- datasets 2.9.0
- tiktoken 0.2.0



Comments:
Hi ! `enc` is not hashable:
```python
import tiktoken
from datasets.fingerprint import Hasher

enc = tiktoken.get_encoding("gpt2")
Hasher.hash(enc)
# raises TypeError: cannot pickle 'builtins.CoreBPE' object
```
It happens because it's not picklable, and because of that it's not possible to cache the result of `map`, hence the warning message.

You can find more details about caching here: https://huggingface.co/docs/datasets/about_cache

You can also provide your own unique hash in `map` if you want, with the `new_fingerprint` argument.
Or disable caching using
```python
import datasets
datasets.disable_caching()
```

@lhoestq Thank you for the explanation and advice. Will relay all of this to the repo where this (non)issue arose. 

Great job with huggingface! 

We made tiktoken tokenizers hashable in #5552, which is included in today's release `datasets==2.10.0`

Just a heads up that when I'm trying to use TikToken along with the a given Dataset `.map()` method, I am still met with the following error :

```
  File "/opt/conda/lib/python3.8/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/conda/lib/python3.8/pickle.py", line 578, in save
    rv = reduce(self.proto)
TypeError: cannot pickle 'builtins.CoreBPE' object
```

My current environment is running datasets v2.10.0.

cc @mariosasko 

@lhoestq @edhenry I am also seeing this, do you have any suggested solution?

With which `datasets` version ? Can you try to udpate ?

@lhoestq @edhenry I am on datasets version `'2.12.0'. I see the same `TypeError: cannot pickle 'builtins.CoreBPE' object` that others are seeing.