Add DINOv2 with registers
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds DINOv2 with registers as proposed in [Vision Transformers Need Registers](https://arxiv.org/abs/2309.16588).

Fixes #27379 

To do:

- [x] fix a weird issue where `make fix-copies` does not do what I want

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31832). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Gently pinging @amyeroberts 

Wouldn't just patching in registers to the existing DINOv2 classes be easier and backwards compatible?

The code changes for my idea are visible [here](https://github.com/huggingface/transformers/compare/main...audioXD:transformers:dinov2-checkout).

I don't know if I changed everthing needed, and only tests I did was importing the model with registers, and `RUN_SLOW=1 pytest tests/models/dinov2/`.

Yes @audioXD I know that that is the minimal way of adding DINOv2 with Registers.

However by following the philosophy https://huggingface.co/blog/transformers-design-philosophy I decided to copy the model and view it as an independent addition

Can dinov2 models or at least maybe this model be supported by Optimum or faster inferencing?

So I've been trying to use the diff method. Here's my branch: https://github.com/NielsRogge/transformers/blob/add_dinov2_with_registers_diff/src/transformers/models/dinov2_with_registers/diff_dinov2_with_registers.py.

However when running 
```bash
python utils/diff_model_converter.py --files_to_parse "(...)/transformers/src/transformers/models/dinov2_with_registers/diff_dinov2_with_registers.py"
```
I get:
```
  File "/Users/nielsrogge/Documents/python_projecten/transformers/env/lib/python3.9/site-packages/libcst/_nodes/expression.py", line 1410, in _visit_and_replace_children
    attr=visit_required(self, "attr", self.attr, visitor),
  File "/Users/nielsrogge/Documents/python_projecten/transformers/env/lib/python3.9/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/Users/nielsrogge/Documents/python_projecten/transformers/env/lib/python3.9/site-packages/libcst/_nodes/base.py", line 233, in visit
    visitor.on_leave(self)
  File "/Users/nielsrogge/Documents/python_projecten/transformers/env/lib/python3.9/site-packages/libcst/_visitors.py", line 137, in on_leave
    leave_func(original_node)
  File "/Users/nielsrogge/Documents/python_projecten/transformers/utils/diff_model_converter.py", line 126, in leave_Name
    parent = self.get_metadata(cst.metadata.ScopeProvider, node)
  File "/Users/nielsrogge/Documents/python_projecten/transformers/env/lib/python3.9/site-packages/libcst/_metadata_dependent.py", line 136, in get_metadata
    value = self.metadata[key][node]
KeyError: Name(
    value='drop_path',
    lpar=[],
    rpar=[],
)
```
Perhaps @ArthurZucker has some insights here. It's also unclear to me whether this method allows for a new `model_type` or whether it should keep using the same `model_type` of the model you're taking the diff from

Friendly pinging @ArthurZucker 

Hi @NielsRogge, the command for diff method seems to work fine if the `drop_path` function (copied from beit) in Dinov2 is changed to a method of `Dinov2DropPath` class, removing the "Copied from" statements.

Side Note:
Since the Beit drop_path implementation is directly used in many other models, similar issues may arise in the future unless the one in Beit is modified. Incidentally, FlaxBeitDropPath is coded correctly, in a single class, and this class is directly copied in FlaxDinov2.

````
class Dinov2DropPath(nn.Module):
    """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks)."""

    def __init__(self, drop_prob: Optional[float] = None) -> None:
        super().__init__()
        self.drop_prob = drop_prob

    def drop_path(self, input: torch.Tensor, drop_prob: float = 0.0, training: bool = False) -> torch.Tensor:
        """
        Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

        Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks,
        however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper...
        See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the
        layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the
        argument.
        """
        if drop_prob == 0.0 or not training:
            return input
        keep_prob = 1 - drop_prob
        shape = (input.shape[0],) + (1,) * (input.ndim - 1)  # work with diff dim tensors, not just 2D ConvNets
        random_tensor = keep_prob + torch.rand(shape, dtype=input.dtype, device=input.device)
        random_tensor.floor_()  # binarize
        output = input.div(keep_prob) * random_tensor
        return output

    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
        return self.drop_path(hidden_states, self.drop_prob, self.training)

    def extra_repr(self) -> str:
        return "p={}".format(self.drop_prob)
```

Thanks a lot @MHRDYN7, I'll close this one in favor of the follow-up PR.