Calling DetrFeatureExtractor will modify its inputs
Author: kongzii
Pull request: no
Labels: name: bug

name: Feature extractors

### System Info

Python 3.9.1, Transformers 4.21.3, Ubuntu 18.04 (inside Docker) but I think it will be the case in all versions.

### Who can help?

@NielsRogge

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
from PIL import Image
from transformers import DetrForObjectDetection, DetrFeatureExtractor

feature_extractor = DetrFeatureExtractor.from_pretrained("facebook/detr-resnet-50")

images = [Image.open('test.jpg')] 
annotations = [{                                                                        
    'image_id': 7,                                                       
    'annotations': [                                                     
    {                                                                
        'name': '_universal_',                                       
        'bbox': [                                                    
            2.0971755981445312,                                      
            194.83389282226562,                                      
            74.80730438232422,                                       
            77.12481689453125                                        
        ],                                                           
        'bbox_original': {                                           
            'xmin': 2.0971755981445312,                              
            'ymin': 194.83389282226562,                              
            'xmax': 76.90447998046875,                               
            'ymax': 271.9587097167969                                
        },                                                           
        'area': 5769.4996528602205,                                  
        'category_id': 0                                             
    }                                                                
    ]                                                                    
}]

features = feature_extractor(images, annotations, return_tensors='pt')
```


### Expected behavior

I would expect that `features` will contain encoded `images` and `annotations`, but `images` and `annotations` themselves would remain untouched. Right now, they are both changed:

```
>>> annotations
[{'boxes': array([[0.03857503, 0.34272584, 0.07305401, 0.1132523 ]], dtype=float32), 'class_labels': array([0]), 'image_id': array([7]), 'area': array([7955.8306], dtype=float32), 'iscrowd': array([0]), 'orig_size': array([ 681, 1024]), 'size': array([ 800, 1202])}]
>>> images
[array([[[-1.5014129 , -1.7754089 , -1.8610327 , ..., -1.7925336 , ...
```

which was very surprising for me and it took me some time to figure out where is the problem in my code. I think it can happen to others as well.

This is happening because lists are passed by reference and inside` __call__` the list is being directly modified, for example [here](https://github.com/huggingface/transformers/blob/v4.21.3/src/transformers/models/detr/feature_extraction_detr.py#L565).

I can think of two possible ways how to fix this and I am happy to do PR if you tell me which one you prefer (or if you suggest something else):

1. At the beginning of the function, simply do a copy of `images` and `annotations`.
2. Instead of reassigning to the original list, change the algorithms to create a new list and append new things to it.

And I think the same problem will happen in `YolosFeatureExtractor`.

Comments:
Thanks for reporting. I've noticed this myself and we'll fix this.

cc @amyeroberts @alaradirik 

Yes, thanks for reporting! This will be fixed shortly.

Closing this issue as the fix PR is merged.