Pyright reportPrivateImportUsage when `from datasets import load_dataset` 
State: closed
Labels: description: Something isn't working
name: bug

## Describe the bug
Pyright complains about module not exported. 

## Steps to reproduce the bug
Use an editor/IDE with Pyright Language server with default configuration:
```python
from datasets import load_dataset
```

## Expected results
No complain from Pyright

## Actual results
Pyright complain below:
```
`load_dataset` is not exported from module "datasets"
 Import from "datasets.load" instead [reportPrivateImportUsage] 
```
Importing from `datasets.load` does indeed solves the problem but I believe importing directly from top level `datasets` is the intended usage per the documentation.

## Environment info
- `datasets` version: 1.18.3
- Platform: macOS-12.2.1-arm64-arm-64bit
- Python version: 3.9.10
- PyArrow version: 7.0.0


Comments:
Hi! 

This issue stems from `datasets` having `py.typed` defined (see https://github.com/microsoft/pyright/discussions/3764#discussioncomment-3282142) - to avoid it, we would either have to remove `py.typed` (added to be compliant with PEP-561) or export the names with `__all__`/`from .submodule import name as name`.

Transformers is fine as it no longer has `py.typed` (removed in https://github.com/huggingface/transformers/pull/18485)

WDYT @lhoestq @albertvillanova @polinaeterna 

@sgugger's point makes sense - we should either be "properly typed" (have py.typed + mypy tests) or drop `py.typed` as Transformers did (I like this option better).

(cc @Wauplin since `huggingface_hub` has the same issue.)

I'm fine with dropping it, but autotrain people won't be happy @SBrandeis 

> (cc @Wauplin since huggingface_hub has the same issue.)

Hmm maybe we have the same issue but I haven't been able to reproduce something similar to `"load_dataset" is not exported from module "datasets"`  message (using VSCode+Pylance -that is powered by Pyright). `huggingface_hub` contains a `py.typed` file but the package itself is actually typed. We are running `mypy` in our CI tests since ~3 months and so far it seems to be ok. But happy to change if it causes some issues with linters.

Also the top-level [`__init__.py`](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/__init__.py) is quite different in `hfh` than `datasets` (at first glance). We have a section at the bottom to import all high level methods/classes in a `if TYPE_CHECKING` block.

@Wauplin I only get the error if I use Pyright's CLI tool or the Pyright extension (not sure why, but Pylance also doesn't report this issue on my machine)

> Also the top-level [`__init__.py`](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/__init__.py) is quite different in `hfh` than `datasets` (at first glance). We have a section at the bottom to import all high level methods/classes in a `if TYPE_CHECKING` block.

I tried to fix the issue with `TYPE_CHECKING`, but it still fails if `py.typed` is present.

@mariosasko thank for the tip. I have been able to reproduce the issue as well. I would be up for including a (huge) static `__all__` variable in the `__init__.py` (since the file is already generated automatically in `hfh`) but honestly I don't think it's worth the hassle. 

I'll delete the `py.typed` file in `huggingface_hub` to be consistent between HF libraries. I opened a PR here: https://github.com/huggingface/huggingface_hub/pull/1329

I am getting this error in google colab today:

![image](https://user-images.githubusercontent.com/3464445/219883967-c7193a23-0388-4ba3-b00c-a53883fb6512.png)

The code runs just fine too.