Repository: huggingface/huggingface_hub
Pull request #2079 (2024-03-01 17:19:09)

Correctly inject config in `PytorchModelHubMixin`
Solves https://github.com/huggingface/huggingface_hub/pull/2079.

With this PR, users don't have to rely on a  `config` parameter to configure their model. We read default+passed values automatically to build the config file ourselves. This way, we are able to save a correct `config.json` file so that reloaded model have exact same config. Only "jsonable" fields are saved in config.json.

And ofc it's backward compatible so users that prefer to pass `config.json` directly are still supported.

PR is inspired by Inspired by https://github.com/facebookresearch/hiera/pull/26. cc @dbolya with this change, you won't need the `@has_config` decorator anymore :hugs: 

Here is an example to showcase it:

```python
>>> import torch
>>> import torch.nn as nn
>>> from huggingface_hub import PyTorchModelHubMixin

>>> class MyModel(nn.Module, PyTorchModelHubMixin):
...     def __init__(self, hidden_size: int = 512, vocab_size: int = 30000, output_size: int = 4):
...         super().__init__()
...         self.param = nn.Parameter(torch.rand(hidden_size, vocab_size))
...         self.linear = nn.Linear(output_size, vocab_size)

...     def forward(self, x):
...         return self.linear(x + self.param)
>>> model = MyModel(hidden_size=256)

# Save model weights to local directory
>>> model.save_pretrained("my-awesome-model")

# Push model weights to the Hub
>>> model.push_to_hub("my-awesome-model")

# Download and initialize weights from the Hub
>>> model = MyModel.from_pretrained("username/my-awesome-model")
>>> model.hidden_size
256
```

cc @mfuntowicz  as well. This shouldn't impact you except that the generated config.json files should be more complete.

HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_2079). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
NielsRogge: That looks great. However, in your PR description, do you mean `self.hidden_size` instead of `config.hidden_size`?
```diff
self.param = nn.Parameter(torch.rand(self.hidden_size, config.vocab_size))
self.linear = nn.Linear(self.output_size, config.vocab_size)
Wauplin: > That looks great. However, in your PR description, do you mean self.hidden_size instead of config.hidden_size etc. in order to access the attributes?

Yes sorry, I've updated the example.
codecov-commenter: ## [Codecov](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2079?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
Attention: Patch coverage is `89.18919%` with `4 lines` in your changes are missing coverage. Please review.
> Project coverage is 82.87%. Comparing base [(`05d715f`)](https://app.codecov.io/gh/huggingface/huggingface_hub/commit/05d715f7a49e770ec53a7f2fcb7e028fb12d6992?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) to head [(`4330b03`)](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2079?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).
> Report is 1 commits behind head on main.

> :exclamation: Current head 4330b03 differs from pull request most recent head 0c900cd. Consider uploading reports for the commit 0c900cd to get more accurate results

| [Files](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2079?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | Patch % | Lines |
|---|---|---|
| [src/huggingface\_hub/hub\_mixin.py](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2079?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9odWJfbWl4aW4ucHk=) | 89.18% | [4 Missing :warning: ](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2079?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #2079      +/-   ##
==========================================
- Coverage   82.90%   82.87%   -0.03%     
==========================================
  Files         102      102              
  Lines        9480     9499      +19     
==========================================
+ Hits         7859     7872      +13     
- Misses       1621     1627       +6     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2079?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).

Wauplin: Thanks for the review @LysandreJik :hugs: Yes I also expect it to make integrations much easier now :)

Patch:
diff --git a/docs/source/en/guides/integrations.md b/docs/source/en/guides/integrations.md
index 045b8c43..e64bd0b8 100644
--- a/docs/source/en/guides/integrations.md
+++ b/docs/source/en/guides/integrations.md
@@ -148,7 +148,7 @@ are ready to go. You don't need to worry about stuff like repo creation, commits
 of this is handled by the mixin and is available to your users. The Mixin also ensures that public methods are well
 documented and type annotated.
 
-As a bonus, [`ModelHubMixin`] handles the model configuration for you. In some cases, you have a `config` input parameter when initializing your class (dictionary or dataclass containing high-level settings). In such cases, the `config` value is automatically serialized into a `config.json` dictionary for you. When re-loading the model from the Hub, the configuration is correctly deserialized. Make sure to use type annotation if you want to deserialize it as a dataclass. The big advantage of having a `config.json` file in your model repository is that it automatically enables the analytics on the Hub (e.g. the "downloads" count).
+As a bonus, [`ModelHubMixin`] handles the model configuration for you. If your `__init__` method expects a `config` input, it will be automatically saved in the repo when calling `save_pretrained` and reloaded correctly by `load_pretrained`. Moreover, if the `config` input parameter is annotated with dataclass type (e.g. `config: Optional[MyConfigClass] = None`), then the `config` value will be correctly deserialized for you. Finally, all jsonable values passed at initialization will be also stored in the config file. This means you don't necessarily have to expect a `config` input to benefit from it. The big advantage of having a `config.json` file in your model repository is that it automatically enables the analytics on the Hub (e.g. the "downloads" count).
 
 ### A concrete example: PyTorch
 
@@ -159,29 +159,27 @@ A good example of what we saw above is [`PyTorchModelHubMixin`], our integration
 Here is how any user can load/save a PyTorch model from/to the Hub:
 
 ```python
->>> from dataclasses import dataclass
 >>> import torch
 >>> import torch.nn as nn
 >>> from huggingface_hub import PyTorchModelHubMixin
 
-# 0. (optional) define a config class
->>> @dataclass
-... class Config:
-...     hidden_size: int = 512
-...     vocab_size: int = 30000
-...     output_size: int = 4
 
-# 1. Define your Pytorch model exactly the same way you are used to
+# Define your Pytorch model exactly the same way you are used to
 >>> class MyModel(nn.Module, PyTorchModelHubMixin): # multiple inheritance
-...     def __init__(self, config: Config):
+...     def __init__(self, hidden_size: int = 512, vocab_size: int = 30000, output_size: int = 4):
 ...         super().__init__()
-...         self.param = nn.Parameter(torch.rand(config.hidden_size, config.vocab_size))
-...         self.linear = nn.Linear(config.output_size, config.vocab_size)
+...         self.param = nn.Parameter(torch.rand(hidden_size, vocab_size))
+...         self.linear = nn.Linear(output_size, vocab_size)
 
 ...     def forward(self, x):
 ...         return self.linear(x + self.param)
 
->>> model = MyModel(Config(hidden_size=128))
+# 1. Create model
+>>> model = MyModel(hidden_size=128)
+
+# Config is automatically created based on input + default values
+>>> model.config
+{"hidden_size": 128, "vocab_size": 30000, "output_size": 4}
 
 # 2. (optional) Save model to local directory
 >>> model.save_pretrained("path/to/my-awesome-model")
@@ -189,10 +187,10 @@ Here is how any user can load/save a PyTorch model from/to the Hub:
 # 3. Push model weights to the Hub
 >>> model.push_to_hub("my-awesome-model")
 
-# 4. Initialize model from the Hub
+# 4. Initialize model from the Hub => config has been preserved
 >>> model = MyModel.from_pretrained("username/my-awesome-model")
 >>> model.config
-Config(hidden_size=128, vocab_size=30000, output_size=4)
+{"hidden_size": 128, "vocab_size": 30000, "output_size": 4}
 ```
 
 #### Implementation
@@ -211,25 +209,15 @@ class PyTorchModelHubMixin(ModelHubMixin):
 2. Implement the `_save_pretrained` method:
 
 ```py
-from huggingface_hub import ModelCard, ModelCardData
+from huggingface_hub import ModelHubMixin
 
 class PyTorchModelHubMixin(ModelHubMixin):
    (...)
 
-   def _save_pretrained(self, save_directory: Path):
-      """Generate Model Card and save weights from a Pytorch model to a local directory."""
-      model_card = ModelCard.from_template(
-         card_data=ModelCardData(
-            license='mit',
-            library_name="pytorch",
-            ...
-         ),
-         model_summary=...,
-         model_type=...,
-         ...
-      )
-      (save_directory / "README.md").write_text(str(model))
-      torch.save(obj=self.module.state_dict(), f=save_directory / "pytorch_model.bin")
+    def _save_pretrained(self, save_directory: Path) -> None:
+        """Save weights from a Pytorch model to a local directory."""
+        save_model_as_safetensor(self.module, str(save_directory / SAFETENSORS_SINGLE_FILE))
+
 ```
 
 3. Implement the `_from_pretrained` method:
@@ -255,13 +243,15 @@ class PyTorchModelHubMixin(ModelHubMixin):
       **model_kwargs,
    ):
       """Load Pytorch pretrained weights and return the loaded model."""
-      if os.path.isdir(model_id): # Can either be a local directory
-         print("Loading weights from local directory")
-         model_file = os.path.join(model_id, "pytorch_model.bin")
-      else: # Or a model on the Hub
-         model_file = hf_hub_download( # Download from the hub, passing same input args
+        model = cls(**model_kwargs)
+        if os.path.isdir(model_id):
+            print("Loading weights from local directory")
+            model_file = os.path.join(model_id, SAFETENSORS_SINGLE_FILE)
+            return cls._load_as_safetensor(model, model_file, map_location, strict)
+
+         model_file = hf_hub_download(
             repo_id=model_id,
-            filename="pytorch_model.bin",
+            filename=SAFETENSORS_SINGLE_FILE,
             revision=revision,
             cache_dir=cache_dir,
             force_download=force_download,
@@ -269,14 +259,8 @@ class PyTorchModelHubMixin(ModelHubMixin):
             resume_download=resume_download,
             token=token,
             local_files_only=local_files_only,
-         )
-
-      # Load model and return - custom logic depending on your framework
-      model = cls(**model_kwargs)
-      state_dict = torch.load(model_file, map_location=torch.device(map_location))
-      model.load_state_dict(state_dict, strict=strict)
-      model.eval()
-      return model
+            )
+         return cls._load_as_safetensor(model, model_file, map_location, strict)
 ```
 
 And that's it! Your library now enables users to upload and download files to and from the Hub.
diff --git a/src/huggingface_hub/hub_mixin.py b/src/huggingface_hub/hub_mixin.py
index 124c689a..4ce164c8 100644
--- a/src/huggingface_hub/hub_mixin.py
+++ b/src/huggingface_hub/hub_mixin.py
@@ -3,7 +3,7 @@ import json
 import os
 from dataclasses import asdict, is_dataclass
 from pathlib import Path
-from typing import TYPE_CHECKING, Dict, List, Optional, Type, TypeVar, Union, get_args
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Type, TypeVar, Union, get_args
 
 from .constants import CONFIG_NAME, PYTORCH_WEIGHTS_NAME, SAFETENSORS_SINGLE_FILE
 from .file_download import hf_hub_download
@@ -12,6 +12,7 @@ from .utils import (
     EntryNotFoundError,
     HfHubHTTPError,
     SoftTemporaryDirectory,
+    is_jsonable,
     is_safetensors_available,
     is_torch_available,
     logging,
@@ -48,18 +49,11 @@ class ModelHubMixin:
     Example:
 
     ```python
-    >>> from dataclasses import dataclass
     >>> from huggingface_hub import ModelHubMixin
 
-    # Define your model configuration (optional)
-    >>> @dataclass
-    ... class Config:
-    ...     foo: int = 512
-    ...     bar: str = "cpu"
-
     # Inherit from ModelHubMixin (and optionally from your framework's model class)
     >>> class MyCustomModel(ModelHubMixin):
-    ...     def __init__(self, config: Config):
+    ...     def __init__(self, size: int = 512, device: str = "cpu"):
     ...         # define how to initialize your model
     ...         super().__init__()
     ...         ...
@@ -85,7 +79,7 @@ class ModelHubMixin:
     ...         # define how to deserialize your model
     ...         ...
 
-    >>> model = MyCustomModel(config=Config(foo=256, bar="gpu"))
+    >>> model = MyCustomModel(size=256, device="gpu")
 
     # Save model weights to local directory
     >>> model.save_pretrained("my-awesome-model")
@@ -96,27 +90,75 @@ class ModelHubMixin:
     # Download and initialize weights from the Hub
     >>> reloaded_model = MyCustomModel.from_pretrained("username/my-awesome-model")
     >>> reloaded_model.config
-    Config(foo=256, bar="gpu")
+    {"size": 256, "device": "gpu"}
     ```
     """
 
     config: Optional[Union[dict, "DataclassInstance"]] = None
     # ^ optional config attribute automatically set in `from_pretrained` (if not already set by the subclass)
 
+    _init_parameters: Dict[str, inspect.Parameter]
+    _jsonable_default_values: Dict[str, Any]
+
+    def __init_subclass__(cls) -> None:
+        """Inspect __init__ signature only once when subclassing."""
+        super().__init_subclass__()
+        cls._init_parameters = dict(inspect.signature(cls.__init__).parameters)
+        cls._jsonable_default_values = {
+            param.name: param.default
+            for param in cls._init_parameters.values()
+            if param.default is not inspect.Parameter.empty and is_jsonable(param.default)
+        }
+
     def __new__(cls, *args, **kwargs) -> "ModelHubMixin":
+        """Create a new instance of the class and handle config.
+
+        3 cases:
+        - If `self.config` is already set, do nothing.
+        - If `config` is passed as a dataclass, set it as `self.config`.
+        - Otherwise, build `self.config` from default values and passed values.
+        """
         instance = super().__new__(cls)
 
-        # Set `config` attribute if not already set by the subclass
-        if instance.config is None:
-            if "config" in kwargs:
-                instance.config = kwargs["config"]
-            elif len(args) > 0:
-                sig = inspect.signature(cls.__init__)
-                parameters = list(sig.parameters)[1:]  # remove `self`
-                for key, value in zip(parameters, args):
-                    if key == "config":
-                        instance.config = value
-                        break
+        # If `config` is already set, return early
+        if instance.config is not None:
+            return instance
+
+        # Infer passed values
+        passed_values = {
+            **{
+                key: value
+                for key, value in zip(
+                    # Skip `self` and `config` parameters
+                    list(cls._init_parameters)[1:],
+                    args,
+                )
+            },
+            **kwargs,
+        }
+
+        # If config passed as dataclass => set it and return early
+        if is_dataclass(passed_values.get("config")):
+            instance.config = passed_values["config"]
+            return instance
+
+        # Otherwise, build config from default + passed values
+        init_config = {
+            # default values
+            **cls._jsonable_default_values,
+            # passed values
+            **{key: value for key, value in passed_values.items() if is_jsonable(value)},
+        }
+        init_config.pop("config", {})
+
+        # Populate `init_config` with provided config
+        provided_config = passed_values.get("config")
+        if isinstance(provided_config, dict):
+            init_config.update(provided_config)
+
+        # Set `config` attribute and return
+        if init_config != {}:
+            instance.config = init_config
         return instance
 
     def save_pretrained(
@@ -246,32 +288,37 @@ class ModelHubMixin:
             except HfHubHTTPError as e:
                 logger.info(f"{CONFIG_NAME} not found on the HuggingFace Hub: {str(e)}")
 
+        # Read config
         config = None
         if config_file is not None:
-            # Read config
             with open(config_file, "r", encoding="utf-8") as f:
                 config = json.load(f)
 
-            # Check if class expect a `config` argument
-            init_parameters = inspect.signature(cls.__init__).parameters
-            if "config" in init_parameters:
+            # Populate model_kwargs from config
+            for param in cls._init_parameters.values():
+                if param.name not in model_kwargs and param.name in config:
+                    model_kwargs[param.name] = config[param.name]
+
+            # Check if `config` argument was passed at init
+            if "config" in cls._init_parameters:
                 # Check if `config` argument is a dataclass
-                config_annotation = init_parameters["config"].annotation
+                config_annotation = cls._init_parameters["config"].annotation
                 if config_annotation is inspect.Parameter.empty:
                     pass  # no annotation
                 elif is_dataclass(config_annotation):
-                    config = config_annotation(**config)  # expect a dataclass
+                    config = _load_dataclass(config_annotation, config)
                 else:
                     # if Optional/Union annotation => check if a dataclass is in the Union
                     for _sub_annotation in get_args(config_annotation):
                         if is_dataclass(_sub_annotation):
-                            config = _sub_annotation(**config)
+                            config = _load_dataclass(_sub_annotation, config)
                             break
 
                 # Forward config to model initialization
                 model_kwargs["config"] = config
-            elif any(param.kind == inspect.Parameter.VAR_KEYWORD for param in init_parameters.values()):
-                # If __init__ accepts **kwargs, let's forward the config as well (as a dict)
+
+            elif any(param.kind == inspect.Parameter.VAR_KEYWORD for param in cls._init_parameters.values()):
+                # 2. If __init__ accepts **kwargs, let's forward the config as well (as a dict)
                 model_kwargs["config"] = config
 
         instance = cls._from_pretrained(
@@ -288,7 +335,7 @@ class ModelHubMixin:
 
         # Implicitly set the config as instance attribute if not already set by the class
         # This way `config` will be available when calling `save_pretrained` or `push_to_hub`.
-        if config is not None and instance.config is None:
+        if config is not None and (instance.config is None or instance.config == {}):
             instance.config = config
 
         return instance
@@ -428,26 +475,19 @@ class PyTorchModelHubMixin(ModelHubMixin):
     Example:
 
     ```python
-    >>> from dataclasses import dataclass
     >>> import torch
     >>> import torch.nn as nn
     >>> from huggingface_hub import PyTorchModelHubMixin
 
-    >>> @dataclass
-    ... class Config:
-    ...     hidden_size: int = 512
-    ...     vocab_size: int = 30000
-    ...     output_size: int = 4
-
     >>> class MyModel(nn.Module, PyTorchModelHubMixin):
-    ...     def __init__(self, config: Config):
+    ...     def __init__(self, hidden_size: int = 512, vocab_size: int = 30000, output_size: int = 4):
     ...         super().__init__()
-    ...         self.param = nn.Parameter(torch.rand(config.hidden_size, config.vocab_size))
-    ...         self.linear = nn.Linear(config.output_size, config.vocab_size)
+    ...         self.param = nn.Parameter(torch.rand(hidden_size, vocab_size))
+    ...         self.linear = nn.Linear(output_size, vocab_size)
 
     ...     def forward(self, x):
     ...         return self.linear(x + self.param)
-    >>> model = MyModel()
+    >>> model = MyModel(hidden_size=256)
 
     # Save model weights to local directory
     >>> model.save_pretrained("my-awesome-model")
@@ -457,6 +497,8 @@ class PyTorchModelHubMixin(ModelHubMixin):
 
     # Download and initialize weights from the Hub
     >>> model = MyModel.from_pretrained("username/my-awesome-model")
+    >>> model.hidden_size
+    256
     ```
     """
 
@@ -536,3 +578,11 @@ class PyTorchModelHubMixin(ModelHubMixin):
             )
             model.to(map_location)  # type: ignore [attr-defined]
         return model
+
+
+def _load_dataclass(datacls: Type["DataclassInstance"], data: dict) -> "DataclassInstance":
+    """Load a dataclass instance from a dictionary.
+
+    Fields not expected by the dataclass are ignored.
+    """
+    return datacls(**{k: v for k, v in data.items() if k in datacls.__dataclass_fields__})
diff --git a/src/huggingface_hub/utils/__init__.py b/src/huggingface_hub/utils/__init__.py
index f56cdfd4..14473e22 100644
--- a/src/huggingface_hub/utils/__init__.py
+++ b/src/huggingface_hub/utils/__init__.py
@@ -97,6 +97,7 @@ from ._safetensors import (
 from ._subprocess import capture_output, run_interactive_subprocess, run_subprocess
 from ._telemetry import send_telemetry
 from ._token import get_token
+from ._typing import is_jsonable
 from ._validators import (
     HFValidationError,
     smoothly_deprecate_use_auth_token,
diff --git a/src/huggingface_hub/utils/_typing.py b/src/huggingface_hub/utils/_typing.py
index b45d88dd..ae502b82 100644
--- a/src/huggingface_hub/utils/_typing.py
+++ b/src/huggingface_hub/utils/_typing.py
@@ -14,10 +14,37 @@
 # limitations under the License.
 """Handle typing imports based on system compatibility."""
 
-from typing import Callable, Literal, TypeVar
+from typing import Any, Callable, Literal, TypeVar
 
 
 HTTP_METHOD_T = Literal["GET", "OPTIONS", "HEAD", "POST", "PUT", "PATCH", "DELETE"]
 
 # type hint meaning "function signature not changed by decorator"
 CallableT = TypeVar("CallableT", bound=Callable)
+
+_JSON_SERIALIZABLE_TYPES = (int, float, str, bool, type(None))
+
+
+def is_jsonable(obj: Any) -> bool:
+    """Check if an object is JSON serializable.
+
+    This is a weak check, as it does not check for the actual JSON serialization, but only for the types of the object.
+    It works correctly for basic use cases but do not guarantee an exhaustive check.
+
+    Object is considered to be recursively json serializable if:
+    - it is an instance of int, float, str, bool, or NoneType
+    - it is a list or tuple and all its items are json serializable
+    - it is a dict and all its keys are strings and all its values are json serializable
+    """
+    try:
+        if isinstance(obj, _JSON_SERIALIZABLE_TYPES):
+            return True
+        if isinstance(obj, (list, tuple)):
+            return all(is_jsonable(item) for item in obj)
+        if isinstance(obj, dict):
+            return all(isinstance(key, str) and is_jsonable(value) for key, value in obj.items())
+        if hasattr(obj, "__json__"):
+            return True
+        return False
+    except RecursionError:
+        return False


Test patch:
diff --git a/tests/test_hub_mixin_pytorch.py b/tests/test_hub_mixin_pytorch.py
index bff22bae..9fe623bc 100644
--- a/tests/test_hub_mixin_pytorch.py
+++ b/tests/test_hub_mixin_pytorch.py
@@ -3,7 +3,7 @@ import os
 import struct
 import unittest
 from pathlib import Path
-from typing import TypeVar
+from typing import Any, TypeVar
 from unittest.mock import Mock, patch
 
 import pytest
@@ -17,6 +17,8 @@ from .testing_constants import ENDPOINT_STAGING, TOKEN, USER
 from .testing_utils import repo_name, requires
 
 
+DUMMY_OBJECT = object()
+
 if is_torch_available():
     import torch
     import torch.nn as nn
@@ -32,8 +34,21 @@ if is_torch_available():
         def forward(self, x):
             return self.l1(x)
 
+    class DummyModelNoConfig(nn.Module, PyTorchModelHubMixin):
+        def __init__(
+            self,
+            num_classes: int = 42,
+            state: str = "layernorm",
+            not_jsonable: Any = DUMMY_OBJECT,
+        ):
+            super().__init__()
+            self.num_classes = num_classes
+            self.state = state
+            self.not_jsonable = not_jsonable
+
 else:
     DummyModel = None
+    DummyModelNoConfig = None
 
 
 @requires("torch")
@@ -235,6 +250,55 @@ class PytorchHubMixinTest(unittest.TestCase):
         # Delete repo
         self._api.delete_repo(repo_id=repo_id)
 
+    def test_load_no_config(self):
+        config_file = self.cache_dir / "config.json"
+
+        # Test creating model => auto-generated config
+        model = DummyModelNoConfig(num_classes=50)
+        assert model.config == {"num_classes": 50, "state": "layernorm"}
+
+        # Test saving model => auto-generated config is saved
+        model.save_pretrained(self.cache_dir)
+        assert config_file.exists()
+        assert json.loads(config_file.read_text()) == {"num_classes": 50, "state": "layernorm"}
+
+        # Reload model => config is reloaded
+        reloaded = DummyModelNoConfig.from_pretrained(self.cache_dir)
+        assert reloaded.num_classes == 50
+        assert reloaded.state == "layernorm"
+        assert reloaded.config == {"num_classes": 50, "state": "layernorm"}
+
+        # Reload model with custom config => custom config is used
+        reloaded_with_default = DummyModelNoConfig.from_pretrained(self.cache_dir, state="other")
+        assert reloaded_with_default.num_classes == 50
+        assert reloaded_with_default.state == "other"
+        assert reloaded_with_default.config == {"num_classes": 50, "state": "other"}
+
+        reloaded_with_default.save_pretrained(self.cache_dir)
+        assert json.loads(config_file.read_text()) == {"num_classes": 50, "state": "other"}
+
+    def test_save_with_non_jsonable_config(self):
+        # Save with a non-jsonable value
+        my_object = object()
+        model = DummyModelNoConfig(not_jsonable=my_object)
+        assert model.not_jsonable is my_object
+        assert "not_jsonable" not in model.config
+
+        # Reload with default value
+        model.save_pretrained(self.cache_dir)
+        reloaded_model = DummyModelNoConfig.from_pretrained(self.cache_dir)
+        assert reloaded_model.not_jsonable is DUMMY_OBJECT
+        assert "not_jsonable" not in model.config
+
+        # If jsonable value passed by user, it's saved in the config
+        new_model = DummyModelNoConfig(not_jsonable=123)
+        new_model.save_pretrained(self.cache_dir)
+        assert new_model.config["not_jsonable"] == 123
+
+        reloaded_new_model = DummyModelNoConfig.from_pretrained(self.cache_dir)
+        assert reloaded_new_model.not_jsonable == 123
+        assert reloaded_new_model.config["not_jsonable"] == 123
+
     def test_save_model_with_shared_tensors(self):
         """
         Regression test for #2086. Shared tensors should be saved correctly.
diff --git a/tests/test_utils_typing.py b/tests/test_utils_typing.py
new file mode 100644
index 00000000..afc14884
--- /dev/null
+++ b/tests/test_utils_typing.py
@@ -0,0 +1,49 @@
+import json
+
+import pytest
+
+from huggingface_hub.utils._typing import is_jsonable
+
+
+class NotSerializableClass:
+    pass
+
+
+OBJ_WITH_CIRCULAR_REF = {"hello": "world"}
+OBJ_WITH_CIRCULAR_REF["recursive"] = OBJ_WITH_CIRCULAR_REF
+
+
+@pytest.mark.parametrize(
+    "data",
+    [
+        123,  #
+        3.14,
+        "Hello, world!",
+        True,
+        None,
+        [],
+        [1, 2, 3],
+        [(1, 2.0, "string"), True],
+        {},
+        {"name": "Alice", "age": 30},
+    ],
+)
+def test_is_jsonable_success(data):
+    assert is_jsonable(data)
+    json.dumps(data)
+
+
+@pytest.mark.parametrize(
+    "data",
+    [
+        set([1, 2, 3]),
+        lambda x: x + 1,
+        NotSerializableClass(),
+        {"obj": NotSerializableClass()},
+        OBJ_WITH_CIRCULAR_REF,
+    ],
+)
+def test_is_jsonable_failure(data):
+    assert not is_jsonable(data)
+    with pytest.raises((TypeError, ValueError)):
+        json.dumps(data)