Allow ignoring kwargs inside fn_kwargs during dataset.map's fingerprinting
Opened by falcaopetri (open)

### Feature request

`dataset.map` accepts a `fn_kwargs` that is passed to `fn`. Currently, the whole `fn_kwargs` is used by `fingerprint_transform` to calculate the new fingerprint.

I'd like to be able to inform `fingerprint_transform` which `fn_kwargs` shoud/shouldn't be taken into account during hashing.

Of course, users should be aware to properly use this new feature, just like the internal usages of `fingerprint_transform` [does](https://github.com/huggingface/datasets/blob/2699593b33ee63d17aad2a2bfddedd38a8df57b8/src/datasets/arrow_dataset.py#L2700).

### Motivation

This is originally motivated by https://github.com/huggingface/transformers/pull/18351#issuecomment-1263588680. 

Nonetheless, consider a more general processing function that accepts a kwarg that does not influence it's output:
```python
def fn(example, verbose=False):
    ...
```

Then `dataset.map(fn, verbose=True)` would not benefit from dataset caching.

I'm not sure if other methods in the `Dataset` API could benefit from this feature.

### Your contribution

Based on `fingerprint_transform `'s `wrapper` function [here](https://github.com/huggingface/datasets/blob/c59cc34fcd2a369d27b77cc678017f5976a926a9/src/datasets/fingerprint.py#L443), it seems to me that it should be possible to make `.map`/`._map_single` accept something like `fn_use_fingerprint_kwargs`/`fn_ignore_fingerprint_kwargs` (probably another arg name). This would then be used by `fingerprint_transform.wrapper` to better/more flexibly hash the transformation.

I could contribute with a PR if this feature and approach look good to you.

Comments:
Hi ! In the `transformers` issue the object to not hash is a `Pool` - I think you can instantiate it inside your function instead of passing it as a parameter. It's good practice that your function and all its fn_kwargs are picklable, in case you want to parallelize `map` using `num_proc>1`

For the other case `def fn(example, verbose=False):` however, I agree it would be nice to let the user specify that "verbose" needs to be ignored.

Do you think providing a decorator could help ? Maybe
```python
@datasets.hashing.register(ignore_kwargs=["verbose"])
def func(example, verbose=False):
    ...
```

Hi @lhoestq! Thanks for your response.

A `Pool` shouldn't be instantiated within the function, because there's a huge overhead in doing so. The main idea is that the same `Pool` should be used across all function calls. Parallel `map` is not helpful/desired in that specific scenario, because the heavy parallel computation is done by another lib (`pyctcdecode`, called within `transformer`'s model inference code).

But yes, it makes sense to be able to leverage parallel processing by just doing `num_proc>1` when possible.

Your decorator suggestions seems like a pretty clean API to me. I didn't find a `datasets.hashing` module though. Would it be created for this specific purpose? Any downsides in just using `datasets.fingerprint`?

And would `datasets.hashing.register` just add some metadata to `func` in your approach (so it could be inspected from `fingerprint_transform`)?

And looking to the `datasets.Dataset` API, `.filter` would also benefited from this.

> Would it be created for this specific purpose? Any downsides in just using datasets.fingerprint?

This can also go in datasets.fingerprint indeed - but maybe datasets.hashing tells more about what the register function does (i.e. register this function to have a custom hashing) ?

> And would datasets.hashing.register just add some metadata to func in your approach (so it could be inspected from fingerprint_transform)?

Yup that's the idea :)

> And looking to the datasets.Dataset API, .filter would also benefited from this.

Indeed !

-----

If you would like to contribute this you can assign yourself to this issue by posting #self-assign
And of course if you have questions or if I can help, feel free to ping me !

> This can also go in datasets.fingerprint indeed - but maybe datasets.hashing tells more about what the register function does (i.e. register this function to have a custom hashing) ?

Sure, it makes sense.

---

I don't plan to work on it right now, so I'll let it unassigned in case somebody wants to join. I'll get back at it as soon as possible though.