Limit Pydantic to V1 in dependencies
Author: lig
Pull request: yes

Pydantic is about to release V2 release which will break a lot of things. This change prevents `transformers` to be used with Pydantic V2 to avoid breaking things.

Also, see #24597 


Comments:
Hi @lig, thanks for opening this PR! 

Could you provide some more information about the kind of issues / breakages expected? I can only see `pydantic` used in one place in the library [here](https://github.com/huggingface/transformers/blob/78a2b19fc84ed55c65f4bf20a901edb7ceb73c5f/src/transformers/commands/serving.py#L26), so thankfully impact is limited. 

For the quality checks, you'll need to run `make style` at the top level of the repo and push any changes made. 

cc @ydshieh 

This issue about `pydantic` is real. We get errors when trying to build docker image in the push CI triggered by commit [299aafe](https://github.com/huggingface/transformers/commit/299aafe55ff03c565c059682c6fd312e4b89bc2f).

@lig Thank you for this PR, it helps us a lot before the issue! I also add one more change quickly (for our CI).

@amyeroberts I am going to merge once @sgugger approves.


```bash
2023-06-30T20:07:31.9883431Z  > [19/19] RUN python3 -c "from deepspeed.launcher.runner import main":
2023-06-30T20:07:31.9883916Z 1.621     from deepspeed.runtime.zero.config import DeepSpeedZeroConfig
2023-06-30T20:07:31.9884613Z 1.621   File "/usr/local/lib/python3.8/dist-packages/deepspeed/runtime/zero/config.py", line 76, in <module>
2023-06-30T20:07:31.9885116Z 1.621     class DeepSpeedZeroConfig(DeepSpeedConfigModel):
2023-06-30T20:07:31.9885814Z 1.621   File "/usr/local/lib/python3.8/dist-packages/pydantic/_internal/_model_construction.py", line 171, in __new__
2023-06-30T20:07:31.9886256Z 1.621     set_model_fields(cls, bases, config_wrapper, types_namespace)
2023-06-30T20:07:31.9886812Z 1.621   File "/usr/local/lib/python3.8/dist-packages/pydantic/_internal/_model_construction.py", line 361, in set_model_fields
2023-06-30T20:07:31.9887329Z 1.621     fields, class_vars = collect_model_fields(cls, bases, config_wrapper, types_namespace, typevars_map=typevars_map)
2023-06-30T20:07:31.9888039Z 1.621   File "/usr/local/lib/python3.8/dist-packages/pydantic/_internal/_fields.py", line 112, in collect_model_fields
2023-06-30T20:07:31.9888950Z 1.621     raise NameError(f'Field "{ann_name}" has conflict with protected namespace "{protected_namespace}"')
2023-06-30T20:07:31.9889546Z 1.621 NameError: Field "model_persistence_threshold" has conflict with protected namespace "model_"
```

_The documentation is not available anymore as the PR was closed or merged._

@amyeroberts answering your question. I've had a quick look and I can say that this https://github.com/huggingface/transformers/blob/78a2b19fc84ed55c65f4bf20a901edb7ceb73c5f/src/transformers/commands/serving.py#L73C1-L73C36 will break.

Instead of
```py
    tokens_ids: Optional[List[int]]
```
it should read 
```py
    tokens_ids: Optional[List[int]] = None
```
There is no implicit default None in Pydantic V2 here.

Thankfully, `bump-pydantic` helps with that https://github.com/pydantic/bump-pydantic/#bp001-add-default-none-to-optionalt-uniont-none-and-any-fields