"no-token" is passed to `huggingface_hub` when token is `None`
Opened by Wauplin (closed)

## Describe the bug

In the 2 lines listed below, a token is passed to `huggingface_hub` to get information from a dataset. If no token is provided, a "no-token" string is passed. What is the purpose of it ? If no real, I would prefer if the `None` value could be sent directly to be handle by `huggingface_hub`. I feel that here it is working because we assume the token will never be validated.

https://github.com/huggingface/datasets/blob/5b23f58535f14cc4dd7649485bce1ccc836e7bca/src/datasets/load.py#L753
https://github.com/huggingface/datasets/blob/5b23f58535f14cc4dd7649485bce1ccc836e7bca/src/datasets/load.py#L1121

## Expected results
Pass `token=None` to `huggingface_hub`.

## Actual results
`token="no-token"` is passed.

## Environment info
`huggingface_hub v0.10.0dev`

Comments:
Hi @Wauplin, thanks for raising this potential issue.

The choice of passing `"no-token"` instead of `None` was made in this PR:
- #4536 

According to the PR description, the reason why it is passed is to avoid that `HfApi.dataset_info` uses the local token when no token should be used.

Hi @albertvillanova , thanks for finding the original issue :+1: 

As of next release of `huggingface_hub`, the `token` argument will be deprecated in favor of the `use_auth_token` argument in `dataset_info` method. This change as been done by @SBrandeis in https://github.com/huggingface/huggingface_hub/pull/928. `use_auth_token` is a bit different and allow the case "don't sent the cached token by default".

If you want to strictly avoid sending the cached token from `datasets`, you can use:
```py
# token=token if token else "no-token", <- will fail because token is not valid

use_auth_token=token if token else False,  # using the new `use_auth_token` parameter
```

And as a note, I am currently updating the "don't send the cached token by default"-rule to "don't send the cached token on public repos by default but use it in private ones" in https://github.com/huggingface/huggingface_hub/pull/1064. This will not change the fact that `use_auth_token=False` doesn't send the token at all.


What is current strategy in term of updating `huggingface_hub` version in `datasets` ? I don't want to break stuff in the next release so let's find a proper solution :) 

As soon as `token` is deprecated and hfh has a new release, we'll update `datasets` to use the new argument instead. Does it sound good to you ?

Perfect :ok_hand: 

Hi @Wauplin, thanks for the warning about the deprecation of `token` in favor of `use_auth_token`.

Indeed, in datasets we use internally `use_auth_token`, which in this case was transformed to `token` to call `HfApi.dataset_info`:
https://github.com/huggingface/datasets/blob/1a9385d7cc8a3241b44015145ef56a230fdadc51/src/datasets/load.py#L747

Therefore, for the new hfh release, the fix will be trivial: we will pass directly `use_auth_token`.

As discussed during our meeting yesterday, due to the fact that at datasets we support multiple hfh versions, I think we should handle passing `token` or `use_auth_token` depending on the hfh version.