Improve torch version check
Author: Rocketknight1
Pull request: yes

`import_utils.py` has a lot of functions for checking when packages are available, and also what their version is. However, some problems can occur because `_is_package_available` uses `importlib.metadata` to check versions, but this checks the version of the [distribution package, not the import package](https://packaging.python.org/en/latest/discussions/distribution-package-vs-import-package/). Therefore, it fails when the distribution package cannot be correctly located, even if `torch` is available.

I reworked the test in the following ways:

1) Stop reporting any package as unavailable just because we can't find its version, unless the version was specifically requested
2) Modify the `torch` check specifically to import torch and check its version directly, in the case when `torch` is importable but the distribution package version cannot be determined.

Fixes #29861

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29884). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

The user reported the issue seems use an installed torch 2.2.1 ~~(I guess this is `distribution package`?)~~. Do you know which line the check fails and cause `_is_package_available` to return `False`?

@ydshieh I think the check fails when you can `import torch`, which means `importlib.util.find_spec("torch") is not None` but `importlib.metadata.version("torch")` raises an error. In other words, the check fails when the **import package** is present but the **distribution package** is not.

I have some doubts because: it seems the issue author has a normal torch installation (`2.2.1+cpu`) which is the same as our CircleCI and yet they get the error however our CircleCI is fine.

It's unclear (to me) that `importlib.util.find_spec` and `importlib.metadata.version` would behave differently between their env. and our CircleCI env.

I think it would be best if we can reproduce (in some env.) before we make the changes (IMO).

I'll see what I can do! I suspect it's something to do with the specific package manager in use.

@robertdecaire can you give us some information about how exactly Python and Torch are installed on your system? Which distribution of Python are you using, and did you install `torch` with their package manager, or with another package manager like `conda` or `pip`?

> I'll see what I can do! I suspect it's something to do with the specific package manager in use.
> 
> @robertdecaire can you give us some information about how exactly Python and Torch are installed on your system? Which distribution of Python are you using, and did you install `torch` with their package manager, or with another package manager like `conda` or `pip`?

In particular, you mentioned you are running on  Jupyter Lab. Are you able to reproduce with the same issue with the command line directly (especially with the same python environment)?

@Rocketknight1 Let's wait until the author of the original opened issue provide some more information regarding the env. to reproduce.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Tempted not to merge this - we haven't seen other reports of the problem since I made the PR,  so I'm not sure we want to complicate the import machinery with a heuristic. It may only be a problem with very idiosyncratic environments. I'll close it for now, but we can reopen it if needed!