[Validation] First implementation of `@strict` from `huggingface_hub`
Author: gante
Pull request: yes

# What does this PR do?

Testbed for https://github.com/huggingface/huggingface_hub/pull/2895, released recently.

Core ideas:
- All config arguments are individually validated at `__init__` and assignment time, using strict type checks and custom value validators;
- Class-level validation happens after `__init__` through `dataclass`'s `__post_init__`, or manually through `.validate()` (⚠️ when mutating the config after initializing, `.validate()` must be called to validate at a class-level). With this, we can check e.g. if a special token is within the vocabulary range;
- Arbitrary validators can be added as `validate_xxx` methods: `@strict` adds them to its validation functions.

____________________________________
Minimal test script:
```py
from transformers import AlbertConfig

# These should work
config = AlbertConfig()
config = AlbertConfig(eos_token_id=5)
config = AlbertConfig(foo="bar")  # Ensures backward compatibility

# Manual specification, traveling through an invalid config, should be allowed
config.hidden_size = 65 # the config is impossible here
config.num_attention_heads = 5
config.validate()

# These will raise an exception
config = AlbertConfig(vocab_size=10.0)  # vocab_size is an int
config = AlbertConfig(num_hidden_layers=None)  # num_hidden_layers is an int
config = AlbertConfig(position_embedding_type="foo")  # position_embedding_type is a Literal, foo is not one of the options
config = AlbertConfig(hidden_size=65)  # hidden_size % num_attention_heads must be 0
```

Comments:
@Wauplin notwithstanding the syntax for validation in discussion above, the current commit is a nearly working version. If we pass standard arguments, everything happens as expected 🙌 

However, I've hit one limitation, and I have one further request:
1. limitation: our config classes accept arbitrary kwargs 😢 These are stored in the config as attributes. E.g. on `main` we can do this:
```py
config = AlbertConfig(foo="bar")
print(config.foo)  # should print "bar"
```
I could keep the original `__init__` method in `AlbertConfig`, which accepts `**kwargs`. However, that would add a lot of boilerplate to each config class, resulting in less readable code than without `@strict_dataclass`. Do you think it would be possible for `strict_dataclass` to overload the `__init__` creating method in `dataclass`, so as to have an option to accept arbitrary kwargs? On each class, we could then do `@strict_dataclass(accept_kwargs=True)`, storing the `**kwargs` on `self.kwargs`, and on `__post_init__` we would handle them on our side.

2. wishlist: can we pass `dataclass` kwargs into `strict_dataclass`? I would like to keep the original `__repr__` here for now, i.e. `@strict_dataclass(repr=False)` 🤗 

> Do you think it would be possible for strict_dataclass to overload the __init__ creating method in dataclass, so as to have an option to accept arbitrary kwargs?

Definitely possible yes!

> we could then do `@strict_dataclass(accept_kwargs=True)`, storing the `**kwargs` on `self.kwargs`

So in the example above, you would have

```py
config = AlbertConfig(foo="bar")
print(config.kwargs["foo"])  # should print "bar"
```
?
(just want to be sure we are aligned since it's not exactly the example you've mentioned. Both solutions are ok to me).

> wishlist: can we pass dataclass kwargs into strict_dataclass? I would like to keep the original __repr__ here for now, i.e. @strict_dataclass(repr=False) 🤗

Feature request accepted :hugs: 

@Wauplin 

> So in the example above, you would have
config = AlbertConfig(foo="bar")
print(config.kwargs["foo"])  # should print "bar"
?
(just want to be sure we are aligned since it's not exactly the example you've mentioned. Both solutions are ok to me).

To be fully BC, `config.foo` (!= `config.kwargs["foo"]`) would have to be valid in this case 😢 

What I had in mind, to avoid specifying an `__init__` in each class to ensure BC:
- decorate with `@strict_dataclass(accept_kwargs=True)`

and either (option A)
- `strict_dataclass` took care of mapping kwargs into a `self.kwargs` attribute
- `__post_init__` in each class took care of setting attributes from `self.kwargs`

or (option B)
- `strict_dataclass` would set the extra kwargs as instance attributes without validation, e.g. with a loop over `kwargs` with `setattr`

@Wauplin: Reviving this project. The current issue is that `transformers` needs to accept (unvalidated) kwargs for BC. WDYT of having the hub class as-is, and we in `transformers` expanding the hub class to meet our unorthodox needs?

hub would then hold a technically sound class for everyone to use 🤗 

Potentially related: https://github.com/vllm-project/vllm/issues/14764 -> add shape checkers

@gante I reviewed and updated my previous PR. The main change is that instead of the previous `@strict_dataclass` you must use both `@strict` and `@dataclass`. Without this I couldn't have perfect type hinting + autocompletion in IDEs which I think is super useful to have

> The current issue is that transformers needs to accept (unvalidated) kwargs for BC. WDYT of having the hub class as-is, and we in transformers expanding the hub class to meet our unorthodox needs?

This is now doable yes!

```py
@strict(accept_kwargs=True)
@dataclass
class ConfigWithKwargs:
    model_type: str
    vocab_size: int = 16

config = ConfigWithKwargs(model_type="bert", vocab_size=30000, extra_field="extra_value")
print(config)  # ConfigWithKwargs(model_type='bert', vocab_size=30000, *extra_field='extra_value')
```

### Check out documentation on https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_2895/en/package_reference/dataclasses

@Wauplin PR updated with the latest syntax 🙌  Moving `__post_init__` to the hub-side would be a nice addition :)

@ArthurZucker syntax is much shorter now 🤗 

> PR updated with the latest syntax 🙌 

:tada: 

> Moving __post_init__ to the hub-side would be a nice addition :)

https://github.com/huggingface/huggingface_hub/pull/2895/commits/27ca2d712cca8ced44fad6b2ad70fb98ca9be74d :heart_hands:  with docs in https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_2895/en/package_reference/dataclasses#class-validators



With class validators, you can now do that:

```py
@strict(accept_kwargs=True)
@dataclass
class AlbertConfig(PretrainedConfig):
    vocab_size: int = interval(min=1)(default=30000)
    embedding_size: int = interval(min=1)(default=128)
    hidden_size: int = interval(min=1)(default=4096)
    num_hidden_layers: int = interval(min=1)(default=12)
    num_hidden_groups: int = interval(min=1)(default=1)
    num_attention_heads: int = interval(min=0)(default=64)
    intermediate_size: int = interval(min=1)(default=16384)
    inner_group_num: int = interval(min=0)(default=1)
    hidden_act: str = activation_fn_key(default="gelu_new")
    hidden_dropout_prob: float = probability(default=0.0)
    attention_probs_dropout_prob: float = probability(default=0.0)
    max_position_embeddings: int = interval(min=0)(default=512)
    type_vocab_size: int = interval(min=1)(default=2)
    initializer_range: float = interval(min=0.0)(default=0.02)
    layer_norm_eps: float = interval(min=0.0)(default=1e-12)
    classifier_dropout_prob: float = probability(default=0.1)
    position_embedding_type: Literal["absolute", "relative_key", "relative_key_query"] = "absolute"
    pad_token_id: Optional[int] = token(default=0)
    bos_token_id: Optional[int] = token(default=2)
    eos_token_id: Optional[int] = token(default=3)

    # Not part of __init__
    model_type = "albert"

    def validate_architecture(self):
        """Validates the architecture of the model."""
        # Check if the number of attention heads is a divisor of the hidden size
        if self.hidden_size % self.num_attention_heads != 0:
            raise ValueError(
                f"The hidden size ({self.hidden_size}) must be divisible by the number of attention "
                f"heads ({self.num_attention_heads})."
            )
```

=> `validate_architecture` is automatically considered as a class validator. Class validators are executed once after `__post_init__` and can be re-executed anytime by calling `config.validate()` explicitly. Note that `.validate()` is not ran after each field attribution as it could block some use cases (e.g. if 2 related fields have to be updated at once)

**Documentation:** https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_2895/en/package_reference/dataclasses#class-validators

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36534). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Note to myself before I go on holidays

_______________________________

Now that `strict` is released, we can easily run the full CI to detect issues. The PR is nearly in a compatible state (see CI status), but it wouldn't be wise to merge even if CI was green.

Let's ignore the validation logic itself, which is working well, and focus on the pressing issue: handling inheritance of `PretrainedConfig`. There are a few implementation details of the class that make inheritance challenging, which I'll try to isolate.

### Issue: Base class inheritance

We are defining a model-level `dataclass` that inherits another class (non-`dataclass`). This makes attribute inheritance weird, especially given the base class init. I think it can be solved with: 
1. converting the base class to a `dataclass` (with `@strict`, to accept kwargs) 
2. adding a `__post_init__` to add defaults
3. attributes that are not stored under the same name will need a getter and a setter (e.g. attn_implementation, see latest commit)



This is excellent work, it will be amazing to have :rocket: super looking forward! 

(If someone stumbles upon this in the future: @zucchini-nlp is working on a better `@strict` usage :) )