noamaanMulla-03/transformers-issues huggingface dataset:
Difficult to understand warning
Author: patrickvonplaten
Pull request: no

### System Info

- `transformers` version: 4.25.0.dev0
- Platform: Linux-5.18.10-76051810-generic-x86_64-with-glibc2.34
- Python version: 3.9.7
- Huggingface_hub version: 0.11.0.dev0
- PyTorch version (GPU?): 1.11.0+cpu (False)
- Tensorflow version (GPU?): 2.9.1 (False)
- Flax version (CPU?/GPU?/TPU?): 0.6.0 (cpu)
- Jax version: 0.3.16
- JaxLib version: 0.3.15
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@amyeroberts @sgugger 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import CLIPImageProcessor

processor = CLIPImageProcessor.from_pretrained("openai/clip-vit-large-patch14")
```

=> this throws the following warning:

```
The size parameter should be a dictionary with keys ('height', 'width'), ('shortest_edge', 'longest_edge') or ('shortest_edge',) got 224. Setting as {'shortest_edge': 224}.
The size parameter should be a dictionary with keys ('height', 'width'), ('shortest_edge', 'longest_edge') or ('shortest_edge',) got 224. Setting as {'height': 224, 'width': 224}.
```

### Expected behavior

I'm wondering whether it would be better to for now downgrade the warning here: https://github.com/huggingface/transformers/blob/6cc06d17394f5715cdf2d13a1ef7680bedaee9e2/src/transformers/image_processing_utils.py#L500 to just `logger.info(...)` 

The `openai/clip-vit-large-patch14` model is used **a lot** in downstream applications (CompVis, diffusers) and the thrown warning here is maybe a bit too much and scares the user? 

Or should we aggressively update all the configs of CLIP models on the Hub in order to make the warning go away we would have to update the config online of a lot of clip models..



Comments:
Updating the config online would make them incompatible with previous versions of Transformers so that's not a possibility. I'm fine with downgrading the warnings to infos.

I wonder why that warning is being printed twice, one time for "shortest_edge" and one time for "height, width"


I'll open a PR to downgrade the warnings. I agree the message isn't very clear either - I'll open up another PR to tidy this up. 

@NielsRogge This is likely because there's two parameters `size` and `crop_size` that are being converted to a dict - makes the point about the warnings being unclear quite well!
[docs] index page has overflowing image
Author: wonhyeongseo
Pull request: no

The current index page has an overflowing image for huggingface.co/support as shown below.
![image](https://user-images.githubusercontent.com/29195190/201600669-f5bf8b40-ffc2-4c93-aada-27ba50209671.png)

This can be fixed easily by adding `width=100%;` to the inline style.
Note: all current languages are affected.

Comments:
[docs] set overflowing image width to auto-scale
Author: wonhyeongseo
Pull request: yes

# What does this PR do?

The image shown below overflowed on small screens. A simple inline-css to set the same width as its div solved the problem.
before | after
------- | -------
![Screenshot_20221115_032312_Chrome.jpg](https://user-images.githubusercontent.com/29195190/201737154-59876bf8-1d71-494f-a25f-015a0636d382.jpg) | ![Screenshot_20221115_032216_Chrome.jpg](https://user-images.githubusercontent.com/29195190/201737194-a23a28a8-71f7-4042-94c3-076c32288de8.jpg)

Fixes #20196 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
Hello @sgugger, may you please review this simple PR?


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20197). All of your documentation changes will be reflected on that endpoint.

Thanks for fixing this, it looks good to me!
@LysandreJik Do you mind double-checking?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20197). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20197). All of your documentation changes will be reflected on that endpoint.
ImportError: cannot import name 'PegasusTokenizer' from 'transformers' 
Author: Sonali234
Pull request: no

@patrickvonplaten 
Tried:
from transformers import PegasusTokenizer
import torch
Output:
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/tmp/ipykernel_7943/3816365261.py in <cell line: 1>()
----> 1 from transformers import PegasusTokenizer
      2 import torch

ImportError: cannot import name 'PegasusTokenizer' from 'transformers' 


Comments:
Can you check transformers version and update it, if too old?

It's work fine for me:
```
In [1]: from transformers import PegasusTokenizer

In [2]: import transformers

In [3]: transformers.__version__
Out[3]: '4.23.1'
```

Gently pinging @ArthurZucker here

Hey, as mentioned you are probably using an old version of transformers. I am not able to reproduce this bug. You should use `transformers>=3.1` as `Pegasus` was first introduced in the [release 3.1](https://github.com/huggingface/transformers/releases?q=PegasusForConditionalGeneration&expanded=true)

Did you find any solution for this problem?


Yes, as mentioned in my previous answer, updating `transformers` 🤗 

> Yes, as mentioned in my previous answer, updating `transformers` hugs

This worked for me. Thanks!

Thank you. It seems to work for me too!
Fix `ImageSegmentationPipelineTests`
Author: ydshieh
Pull request: yes

# What does this PR do?

- If the concept is approved, I can apply the same changes to other places)

- On `CircleCI`, we get `0.9469921875 not greater than or equal to 0.99`. Maybe I should lower the threshold ..?
-------

Fix `ImageSegmentationPipelineTests.test_small_model_pt`.

Compare hash of the output/expected masks is too flaky. This PR:
- get the current output masks, upload them to Hub
- use the uploaded masks as the new expected values
- only compare if the output/expected masks match with 99% or above

It should be safe to use the current output masks as the new expected masks, as the current output/expected masks seems match close:

```python
            [
                {
                    "label": "LABEL_88",
                    "mask": {"hash": "4e2da4b9a4", "shape": (480, 640), "white_pixels": 11},
                    "score": None,
                },
                {
                    "label": "LABEL_101",
                    "mask": {"hash": "9ec7310913", "shape": (480, 640), "white_pixels": 8946},
                    "score": None,
                },
                {
                    "label": "LABEL_215",
                    "mask": {"hash": "21dcfdc10d", "shape": (480, 640), "white_pixels": 298243},
                    "score": None,
                },
            ],
```


current expected values (before this PR):
```python
            [
                {
                    "label": "LABEL_88",
                    "mask": {"hash": "7f0bf661a4", "shape": (480, 640), "white_pixels": 3},
                    "score": None,
                },
                {
                    "label": "LABEL_101",
                    "mask": {"hash": "10ab738dc9", "shape": (480, 640), "white_pixels": 8948},
                    "score": None,
                },
                {
                    "label": "LABEL_215",
                    "mask": {"hash": "b431e0946c", "shape": (480, 640), "white_pixels": 298249},
                    "score": None,
                },
            ],
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Mark as draft in order to debug this super flaky test!

@sgugger  @Narsil @alaradirik  and others: ready for your review

@Narsil 

- no zip now
- I decided to simply use (for now) `.../blob/...` instead of `.../resolve/...` to link to the pages (where we can visualize the images, although not the full size).
  ```
  https://huggingface.co/datasets/hf-internal-testing/mask-for-image-segmentation-tests/blob/main/mask_0.png
  ```
  I don't like much the usage of  `datasets-server`:
    - too long
    - link strings not corresponding to file names

I am going to merge if you are OK 🙏
update relative positional embedding
Author: ArthurZucker
Pull request: yes

# What does this PR do?

The current `relative_key` positional embedding is incorrect when `use_cache=True`. This was first highlighted in #19045.

Reproducing script : 
```pyton 
import torch
from transformers import BertTokenizer, BertLMHeadModel, set_seed
tokenizer = BertTokenizer.from_pretrained("zhiheng-huang/bert-base-uncased-embedding-relative-key")
model = BertLMHeadModel.from_pretrained("zhiheng-huang/bert-base-uncased-embedding-relative-key", is_decoder = True)
inputs = tokenizer("No I'm not missing the ", return_tensors="pt")
input_ids = inputs.input_ids[:,:-1]
attention_mask = inputs.attention_mask[:,:-1]

with torch.no_grad():
    model.config.use_cache = False
    set_seed(0)
    output = model(input_ids, attention_mask = attention_mask, use_cache =False)
    print(output.logits[:,-1,:])

    model.config.use_cache = True
    output_1 = model(input_ids[:,:-1], use_cache = True, attention_mask = attention_mask[:,:-1])
    pkv = output_1.past_key_values
    output_2 = model(input_ids[:,-1:], past_key_values = pkv , use_cache = True)
    print(output_2.logits[:,-1,:])
```

```
tensor([[-5.4971, -6.4888, -8.3359,  ..., -7.3612, -5.5480, -0.9784]])
tensor([[ -7.2693,  -7.7799, -10.0905,  ...,  -7.5183,  -7.4255,  -4.6804]])
```

Lets also make sure that this feature is tested using `create_and_check_decoder_model_past_large_inputs`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20203). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20203). All of your documentation changes will be reflected on that endpoint.

cc @sgugger just FYI, will ping you once I know that all the tests pass. 

( before the modification the added test did not pass, now they do) 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20203). All of your documentation changes will be reflected on that endpoint.
[WHISPER] Update modeling tests
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes the test after the update of the tokenizer, which added sufix and prefix tokens. 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.

I agree, it is indeed better to just use `add_special_tokens=False`! Slipped my mind 😉 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.

Ping me again once when you think it's ready :) 

Looks like we still need to add `add_special_tokens=False` to the tf test @ArthurZucker!

Yeah on it 🤗

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.

Ah there's small bug with this, the kwargs is passed to `self.pad` in the feature extractor. Gonna fix that

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.
Add `accelerate` support for `ViT` family
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR adds `accelerate` support for ViT models, therefore these models can be loaded in 8-bit as follows:
```
# pip install accelerate bitsandbytes
from transformers import ViTFeatureExtractor, ViTForImageClassification
from PIL import Image
import requests

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

model_8bit = ViTForImageClassification.from_pretrained('google/vit-large-patch32-384', device_map="auto", load_in_8bit=True)
outputs_8bit = model_8bit(**inputs)
logits = outputs_8bit.logits
# model predicts one of the 1000 ImageNet classes
predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", model_8bit.config.id2label[predicted_class_idx])
``` 
This PR introduces the first 8-bit compatible vision model.
The same script works for `deit` too

Putting the PR as a draft as I have few questions!

cc @NielsRogge @sgugger 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20174). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20174). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20174). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20174). All of your documentation changes will be reflected on that endpoint.
Add Switch transformers
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR attempts to add Switch Transformers from t5x with @ArthurZucker & @thomwolf 

The architecture seems to be similar to a t5 architecture (the architecture is copied from T5), where the FF layer is slightly modified, introducing the first Mixture of Experts (MoE) architecture inside `transformers` library.

paper: https://arxiv.org/abs/2101.03961
weights: https://github.com/google-research/t5x/blob/eb42c2524bf65c8a46624f1a9b9e034d9bc65b14/docs/models.md#converted-mesh-tensorflow-checkpoints
original modeling code: https://github.com/google/flaxformer/tree/b725bd2a51d70e866d819c92de166fbf24425e6a/flaxformer/architectures/moe


# TODOs:

- [x] Make the forward pass run
- [x] Convert the weights in Pytorch format and upload them on the Hub
- [x] Match the logits between the original implementation and ours

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19323). All of your documentation changes will be reflected on that endpoint.

Thanks a lot @sgugger for your comments! 
Would love to have another round of review as I added some modification for `accelerate` and `bnb` compatibility 🙏 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19323). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19323). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19323). All of your documentation changes will be reflected on that endpoint.

Failing tests seems to be unrelated to this PR, merging!
Add object detection + segmentation transforms
Author: amyeroberts
Pull request: yes

# What does this PR do?

Adds logic for processing bounding boxes and some additional transforms (`rgb_to_id`, `id_to_rgb`) needed for DETR. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20003). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20003). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20003). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20003). All of your documentation changes will be reflected on that endpoint.
Remove `authorized_missing_keys`in favor of _keys_to_ignore_on_load_missing
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Just cleans up a bug/typo discoverd. `_keys_to_ignore_on_load_missing` is supported but not `authorized_missing_keys`. 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20228). All of your documentation changes will be reflected on that endpoint.
Spelling Error in Testing Documentation - "checkt" -> "check"
Author: kasmith11
Pull request: no

### System Info

- `transformers` version: 4.24.0.dev0
- Platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.17
- Python version: 3.8.13
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.12.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sgugger @stevhliu 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Under `Run documentation tests` section of [Testing documentation](https://huggingface.co/docs/transformers/testing#run-documentation-tests) it says: `In order to test whether the documentation examples are correct, you should checkt that the doctests are passing.`

### Expected behavior

The line can be changed to: `In order to test whether the documentation examples are correct, you should check that the doctests are passing.`

Comments:
Thanks for reporting, would you like to open a PR to fix this typo?

@sgugger sure I can open a PR to fix this type.
Fix `run_clip.py`
Author: ydshieh
Pull request: yes

# What does this PR do?

The type annotation for `Transform.forward` (in `run_clip.py`) never work when I tried several torch/pillow versions.
This forward is compiled by `torch.jit`.

With the current type annotation `x: Image`, we get error
```bash
RuntimeError: 
Unknown type name 'Image':
```

With `x: Image.Image`, error is
```bash
NotSupportedError: Compiled functions can't take variable number of arguments or use keyword-only arguments with defaults:
  File "/home/huggingface/miniconda3/envs/py38/lib/python3.8/site-packages/PIL/Image.py", line 550
    def __exit__(self, *args):
                       ~~~~~ <--- HERE
        if hasattr(self, "fp") and getattr(self, "_exclusive_fp", False):
            if getattr(self, "_fp", False):
'__torch__.PIL.Image.Image' is being compiled since it was called from 'Transform.forward'

```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix docstring of CLIPTokenizer(Fast)
Author: TilmannR
Pull request: yes

# What does this PR do?

The docstrings of `CLIPTokenizer` and `CLIPTokenizerFast` had the wrong default value for `bos_token` (`
` instead of `
`).

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

- Documentation: @sgugger

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20233). All of your documentation changes will be reflected on that endpoint.
Fix MaskformerFeatureExtractor
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR fixes MaskFormer's feature extractor. 

PR #18997 introduced a bug which made the feature extractor create the same binary mask for all segments/instances in an image, hence making it impossible to fine-tune the model. This PR fixes it and makes sure the model can be properly fine-tuned on instance, semantic and panoptic segmentation datasets.

To do:

- [x] add tests

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20100). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20100). All of your documentation changes will be reflected on that endpoint.

@sgugger could you approve this PR, so that I can merge this critical fix to MaskFormerFeatureExtractor?

I'll open a separate PR for improving the docs around image segmentation.