Repository: huggingface/transformers
Pull request: Fix test for model with dotted name and relative imports

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

The test `test_model_with_dotted_name_and_relative_imports` added in #40745 is not working as expected because `model_id = "hf-internal-testing/remote_code_model_with_dots"` does not include `.` in the name.

If this PR change looks good, could you create this new remote model? (I didn't create it myself because relying on an external user's resource is risky, as you mentioned in https://github.com/huggingface/transformers/pull/40745#pullrequestreview-3206353873).

Below is a detailed explanation of why `test_model_with_dotted_name_and_relative_imports` is not working. The explanation is a bit long, so feel free to skip it if you understand the problem from the above explanation.

---

If we remove the `_sanitize_module_name` process simply by changing the method like below:

```py
def _sanitize_module_name(name: str) -> str:
    r"""
    Tries to sanitize a module name so that it can be used as a Python module.

    The following transformations are applied:

    1. Replace `.` in module names with `_dot_`.
    2. Replace `-` in module names with `_hyphen_`.
    3. If the module name starts with a digit, prepend it with `_`.
    4. Warn if the sanitized name is a Python reserved keyword or not a valid identifier.

    If the input name is already a valid identifier, it is returned unchanged.
    """
    # Not replacing `\W` characters with `_` to avoid collisions, because `_` is very common
    return name
```

The test should raise an error, but it actually passes:

```sh
pytest tests/models/auto/test_modeling_auto.py::AutoModelTest::test_model_with_dotted_name_and_relative_imports
================================================================================== test session starts ===================================================================================
platform linux -- Python 3.12.11, pytest-8.4.1, pluggy-1.6.0
rootdir: /home/shutotakahashi/projects/transformers-uv/transformers
configfile: pyproject.toml
plugins: anyio-4.11.0, xdist-3.8.0
collected 1 item                                                                                                                                                                         

tests/models/auto/test_modeling_auto.py::AutoModelTest::test_model_with_dotted_name_and_relative_imports PASSED                                                                    [100%]
```

The reason the test passes is that the current `model_id = "hf-internal-testing/remote_code_model_with_dots"` does not include `.` in the name, as reported in the original issue https://github.com/huggingface/transformers/issues/40496.

If we replace `model_id` with[ `August4293/test-model_v1.0`](https://huggingface.co/August4293/test-model_v1.0/tree/main), which includes `.` in the name (this model was uploaded by the original PR contributor in https://github.com/huggingface/transformers/pull/40745#issuecomment-3270735455), the test raises the expected error:

```py
    def test_model_with_dotted_name_and_relative_imports(self):
        """
        Test for issue #40496: AutoModel.from_pretrained() doesn't work for models with '.' in their name
        when there's a relative import.

        Without the fix, this raises: ModuleNotFoundError:
        No module named 'transformers_modules.August4293.test-model_v1'
        """
        model_id = "August4293/test-model_v1.0"

        model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
        self.assertIsNotNone(model)
```

```sh
pytest tests/models/auto/test_modeling_auto.py::AutoModelTest::test_model_with_dotted_name_and_relative_imports
================================================================================== test session starts ===================================================================================
platform linux -- Python 3.12.11, pytest-8.4.1, pluggy-1.6.0
rootdir: /home/shutotakahashi/projects/transformers-uv/transformers
configfile: pyproject.toml
plugins: anyio-4.11.0, xdist-3.8.0
collected 1 item                                                                                                                                                                         

tests/models/auto/test_modeling_auto.py::AutoModelTest::test_model_with_dotted_name_and_relative_imports FAILED                                                                    [100%]

======================================================================================== FAILURES ========================================================================================
_____________________________________________________________ AutoModelTest.test_model_with_dotted_name_and_relative_imports _____________________________________________________________

self = <tests.models.auto.test_modeling_auto.AutoModelTest testMethod=test_model_with_dotted_name_and_relative_imports>

    def test_model_with_dotted_name_and_relative_imports(self):
        """
        Test for issue #40496: AutoModel.from_pretrained() doesn't work for models with '.' in their name
        when there's a relative import.
    
        Without the fix, this raises: ModuleNotFoundError:
        No module named 'transformers_modules.August4293.test-model_v1'
        """
        model_id = "August4293/test-model_v1.0"
    
>       model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

tests/models/auto/test_modeling_auto.py:587: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/transformers/models/auto/auto_factory.py:367: in from_pretrained
    model_class = get_class_from_dynamic_module(
src/transformers/dynamic_module_utils.py:606: in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module, force_reload=force_download)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
src/transformers/dynamic_module_utils.py:312: in get_class_in_module
    module_spec.loader.exec_module(module)
<frozen importlib._bootstrap_external>:999: in exec_module
    ???
<frozen importlib._bootstrap>:488: in _call_with_frames_removed
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    """
    Custom model with relative import to demonstrate the bug.
    """
    from transformers import PreTrainedModel
    from transformers.modeling_outputs import BaseModelOutput
    import torch
    import torch.nn as nn
    
    # This relative import should cause the bug when the folder has a dot in the name
>   from .another_module import custom_function
E   ModuleNotFoundError: No module named 'transformers_modules.August4293.test-model_v1'
```

Fixes #40496 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Rocketknight1


Changed files:
filename: tests/models/auto/test_modeling_auto.py
patch: @@ -579,9 +579,10 @@ def test_model_with_dotted_name_and_relative_imports(self):
         Test for issue #40496: AutoModel.from_pretrained() doesn't work for models with '.' in their name
         when there's a relative import.
 
-        Without the fix, this raises: ModuleNotFoundError: No module named 'transformers_modules.test-model_v1'
+        Without the fix, this raises: ModuleNotFoundError:
+        No module named 'transformers_modules.hf-internal-testing.remote_code_model_with_dots_v1'
         """
-        model_id = "hf-internal-testing/remote_code_model_with_dots"
+        model_id = "hf-internal-testing/remote_code_model_with_dots_v1.0"
 
         model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
         self.assertIsNotNone(model)