Some of test/utils tests fail being invalidated by tests/utils/test_import_utils.py::test_clear_import_cache
Author: dvrogozh
Pull request: no

With:
* https://github.com/huggingface/transformers/commit/547911e727fffa06052fcd35776c1df115ec32ed

On:
* Nvidia A10
* Intel Data Center GPU Max (PVC)

Test defined in [tests/utils/test_import_utils.py](https://github.com/huggingface/transformers/blob/main/tests/utils/test_import_utils.py) seems to invalidate the state of the test engine (unloads loaded modules) which causes failures for some of the next tests, but not all. Overall running `pytest tests/utils` will result in 19 failing tests which otherwise would pass with `--ignore=tests/utils/test_import_utils.py`. Below is the minimal reproducer:

```
# python3 -m pytest tests/utils/test_import_utils.py tests/utils/test_logging.py
...
tests/utils/test_import_utils.py::test_clear_import_cache PASSED                                               [ 14%]
tests/utils/test_logging.py::HfArgumentParserTest::test_advisory_warnings PASSED                               [ 28%]
tests/utils/test_logging.py::HfArgumentParserTest::test_env_invalid_override FAILED                            [ 42%]
tests/utils/test_logging.py::HfArgumentParserTest::test_env_override FAILED                                    [ 57%]
tests/utils/test_logging.py::HfArgumentParserTest::test_integration PASSED                                     [ 71%]
tests/utils/test_logging.py::HfArgumentParserTest::test_set_level PASSED                                       [ 85%]
tests/utils/test_logging.py::test_set_progress_bar_enabled PASSED                                              [100%]
...
E       AssertionError: 'Unknown option TRANSFORMERS_VERBOSITY=super-error' not found in ''
...
E       AssertionError: 40 != 30 : TRANSFORMERS_VERBOSITY=error/40, but internal verbosity is 30
```

Compare with running without `tests/utils/test_import_utils.py`:
```
# python3 -m pytest tests/utils/test_logging.py
...
tests/utils/test_logging.py::HfArgumentParserTest::test_advisory_warnings PASSED
tests/utils/test_logging.py::HfArgumentParserTest::test_env_invalid_override
--------------------------------------------------- live log call ----------------------------------
WARNING  root:logging.py:66 Unknown option TRANSFORMERS_VERBOSITY=super-error, has to be one of: detwarning, error, critical
PASSED
tests/utils/test_logging.py::HfArgumentParserTest::test_env_override PASSED
tests/utils/test_logging.py::HfArgumentParserTest::test_integration PASSED
tests/utils/test_logging.py::HfArgumentParserTest::test_set_level PASSED
tests/utils/test_logging.py::test_set_progress_bar_enabled PASSED
```

The `tests/utils/test_import_utils.py` was introduced by:
* https://github.com/huggingface/transformers/pull/35858

Can test in `tests/utils/test_import_utils.py` be modified to reset the state of the test engine so next tests will run without being impacted? or test be isolated to avoid invalidating the test engine (executed in separate process)?

CC: @sambhavnoobcoder @ydshieh

Comments:
thank you for pointing it out . #36345 PR solves this issue . Hopefully that gets merged quickly and solves this issue . 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

ig we can close this issue now since this is fixed in #36345  . cc: @ydshieh 

Ah yes :-)