Add AutoBackbone + ResNetBackbone
Author: NielsRogge
Pull request: yes

# What does this PR do?

As #20204 is a big PR, this PR adds a part of it as a standalone PR. This PR adds the AutoBackbone class, along with one example class that it supports, namely ResNetBackbone.

## Usage

Usage is as follows:

```
from transformers import AutoImageProcessor, AutoBackbone
import torch
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

processor = AutoImageProcessor.from_pretrained("microsoft/resnet-50")
model = AutoBackbone.from_pretrained("microsoft/resnet-50", out_features=["stage1", "stage2", "stage3", "stage4"])

inputs = processor(image, return_tensors="pt")

outputs = model(**inputs)

for k,v in zip(outputs.stage_names, outputs.hidden_states):
    print(k, v.shape)
```
which prints:
```
stage1 torch.Size([1, 256, 56, 56])
stage2 torch.Size([1, 512, 28, 28])
stage3 torch.Size([1, 1024, 14, 14])
stage4 torch.Size([1, 2048, 7, 7])
```

Besides this, one can also obtain information about the channel dimension and stride for each of the requested stages, like so:

```
print(model.channels)
print(model.strides)
```

This is handy as frameworks (like MaskFormer) need to know this information at initialization time.

## To do's/questions

- [ ] We don't want `xxxBackbone` classes to be tested by all tests defined in `test_modeling_common.py`(i.e. it should probably not be part of `all_model_classes`), hence I added the class to IGNORE_NON_TESTED, and added a separate test for it. Let me know if this is ok.
- [ ] It would probably be best to not have our backbones included in the documentation from the start. For now they are just an internal part of models like DETR and MaskFormer. Could we not include them in the docs for now? Currently I'm getting:
```
Exception: The following objects are in the public init so should be documented:
 - AutoBackbone
 - ResNetBackbone
```
An alternative option could be to add backbones to the list of PRIVATE_MODELS in utils/check_repo.py for now.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20229). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20229). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20229). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20229). All of your documentation changes will be reflected on that endpoint.

> My main question is about how the backbone is loaded and saved alongside our current models. For example, if the backbone has non-default settings - where is this information saved? Is it part of the model e.g. DETR config?

A backbone can be loaded and saved just like any other model in the library (due to the inheritance of `PreTrainedModel`), either using a config to initialize the backbone with randomly initialized weights or using the `from_pretrained` method to load pre-trained weights.

> Can we assume the backbone is always frozen, or are there cases when people might want to fine-tune it? In this case, how would the backbone weights be saved out?

The backbone is always meant to be fine-tuned together with the backbone, I've not seen a case where the backbone is kept frozen for now, but we could definitely add a `freeze` method in case people want to do that.

@sgugger the remaining CI issue is about the fact that AutoBackbone and ResNetBackbone are not documented, however I'd like to actually keep them away from the docs for now. However I need to keep ResNetBackbone in the main init for it to work with the Auto API.

@NielsRogge Thanks for your answers. The backbone can definitely be saved with `save_pretrained`, my question is really about how that's bundled together with a model. For example, if I wanted a new model, with a new-non-standard backbone - is this how I would create it? 

```
from transformers import AutoConfig, AutoBackbone, AutoModelForXXX

# Modify the default backbone configuration
backbone_config = AutoConfig(backbone_repo)
backbone_config.param = new_value_0

# Modify the default model configuration
model_config = AutoConfig(model_repo)
model_config.param = new_value_1

model = AutoModelForXXX(model_config, backbone_config)
```

Or is the backbone configuration part of the model configuration?

```
from transformers import AutoConfig

model_config = AutoConfig(model_repo)
model_config["backbone"]["param"] = new_value_0
model_config.param = new_value_1

model = AutoModel(model_config)
```

or something else?

@NielsRogge You can add them to [this list](https://github.com/huggingface/transformers/blob/0d0d77693f79c7f7d39bba6921cc9741f00de988/utils/check_repo.py#L665) for now.

The way frameworks like DETR and MaskFormer can use a backbone can be as follows. In their configuration, e.g. `MaskFormerConfig`, they should have a `backbone_config` attribute, which can be set to ResNetConfig for instance, like so:

```
from transformers import ResNetConfig, MaskFormerConfig

backbone_config = ResNetConfig(hidden_sizes=...)
config = MaskFormerConfig(backbone_config=backbone_config)
````
So yes, the backbone configuration is part of the model configuration.

Then, inside modeling_maskformer.py, one can instantiate the backbone like so:

```
from transformers import AutoBackbone

self.backbone = AutoBackbone.from_config(config.backbone_config)
```

@NielsRogge Great - thanks for clarifying :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20229). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20229). All of your documentation changes will be reflected on that endpoint.

@sgugger @amyeroberts feel free to approve :) 

This has been fixed in https://github.com/NielsRogge/transformers/commit/1863e8ff5dcaf29333d5bd08f266a562d36a8ee6, have marked the conversation as resolved

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20229). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20229). All of your documentation changes will be reflected on that endpoint.

Failing tests are unrelated (there seems to be an issue with the cache of MarkupLM and mBART50 on the CI), merging.