noamaanMulla-03/transformers-issues huggingface dataset:
There is a type annotation error
Author: PVMPATCH
Pull request: no

### System Info

Transformer 4.22.0
Ubuntu 20.04
Python 3.10.6

### Who can help?

@sgugger, @patil-suraj

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The type annotation bug is [here](https://github.com/huggingface/transformers/blob/main/examples/research_projects/rag/finetune_rag.py#:~:text=model%3A%20GenerativeQAModule%20%3D%20GenerativeQAModule(args))
Trying to annotate `model` as type  `GenerativeQAModule` at line 586 violates the rules of type annotation, 
because `model` had been firstly initialized as `None` or other value at line 536. 


```
#transformers/examples/research_projects/rag/finetune_rag.py #Line 536

def main(args=None, model=None) -> GenerativeQAModule:
     ...
    if model is None:
        model: GenerativeQAModule = GenerativeQAModule(args)
```


I found this defect by using the tool called [Pyre](https://pyre-check.org/docs/getting-started/).
`pyre init`
`pyre`

### Expected behavior

I expected no defect reported



Comments:
Thanks for reporting. Note that we do not maintain the examples lying in the research project folder, and we only use type annotations for documentation purpose, not for type-checkers.

@sgugger Will you have any comming project related to vision transformer that I can contribute to. 

Can i get this issue?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Support Bloom models in distillation example
Author: mapmeld
Pull request: yes

# What does this PR do?

This updates the model distillation example to support more models on the Hub, including Bloom models.

I use this code to create smaller, monolingual generative models based on mGPT or Bloom. Demo notebook: https://colab.research.google.com/drive/1DJLNA4TcW45HYzuSDAVb8t6n7OTU1NyI?usp=sharing

- Support Bloom model and tokenizer in distillation example scripts
- Handles Hub model and tokenizer names with a slash, such as 'sberbank-ai/mGPT'
- Allow models without `max_position_embeddings` and tokenizers without set `max_model_input_sizes`
- Add `max_model_input_size` as a CLI param to make it easier to train on one GPU and support tokenizers without `max_model_input_sizes`
- remove `git log` call and `git-python` dependency

----

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Did you make sure to update the documentation with your changes?
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20178). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20178). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20178). All of your documentation changes will be reflected on that endpoint.
Special Language Token for PLBART needs to be updated
Author: wasiahmad
Pull request: no

### System Info

The `FAIRSEQ_LANGUAGE_CODES` in PLBartTokenizer [here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/plbart/tokenization_plbart.py#L90) need to be as follows.

```
FAIRSEQ_LANGUAGE_CODES = {
    "base": ["__java__", "__python__", "__en_XX__"],
    "multi": ["__java__", "__python__", "__en_XX__", "__javascript__", "__php__", "__ruby__", "__go__"],
}
```

The current PLBartTokenizer treats `java` as a special token, and thus it removes the token when decoding is performed. An example is given below.

```
code = "public void METHOD_1 ( TYPE_1 VAR_1 ) throws java.lang.Exception { super . METHOD_1 ( VAR_1 ) ; METHOD_2 ( VAR_1 ) ; }"
tokenizer = model_tokenizer_class.from_pretrained("uclanlp/plbart-base", language_codes="base")
model_inputs = tokenizer([code])
print(tokenizer.decode(model_inputs['input_ids'][0], skip_special_tokens=True, clean_up_tokenization_spaces=False))
# The code output is: "public void METHOD_1 ( TYPE_1 VAR_1 ) throws .lang.Exception { super . METHOD_1 ( VAR_1 ) ; METHOD_2 ( VAR_1 ) ; }"
```

### Who can help?

@gunjan

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
code = "public void METHOD_1 ( TYPE_1 VAR_1 ) throws java.lang.Exception { super . METHOD_1 ( VAR_1 ) ; METHOD_2 ( VAR_1 ) ; }"
tokenizer = model_tokenizer_class.from_pretrained("uclanlp/plbart-base", language_codes="base")
model_inputs = tokenizer([code])
print(tokenizer.decode(model_inputs['input_ids'][0], skip_special_tokens=True, clean_up_tokenization_spaces=False))
# public void METHOD_1 ( TYPE_1 VAR_1 ) throws .lang.Exception { super . METHOD_1 ( VAR_1 ) ; METHOD_2 ( VAR_1 ) ; }
```

### Expected behavior

```
code = "public void METHOD_1 ( TYPE_1 VAR_1 ) throws java.lang.Exception { super . METHOD_1 ( VAR_1 ) ; METHOD_2 ( VAR_1 ) ; }"
tokenizer = model_tokenizer_class.from_pretrained("uclanlp/plbart-base", language_codes="base")
model_inputs = tokenizer([code])
print(tokenizer.decode(model_inputs['input_ids'][0], skip_special_tokens=True, clean_up_tokenization_spaces=False))
# public void METHOD_1 ( TYPE_1 VAR_1 ) throws java.lang.Exception { super . METHOD_1 ( VAR_1 ) ; METHOD_2 ( VAR_1 ) ; }
```

Comments:
cc @gchhablani, could you take a look at this?

@LysandreJik I can have a look at this if it isn't being looked at.

Please go ahead, thank you @jordiclive!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
cannot load bart encoder
Author: icedpanda
Pull request: no

### System Info

- `transformers` version: 4.23.1
- Platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
- Python version: 3.10.0
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.12.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@patil-suraj

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Failed to load BartEncoder, BartDecoder

```python
from transformers.models.bart.modeling_bart import BartEncoder, BartDecoder
```

Error
```
ImportError: cannot import name 'SAFE_WEIGHTS_INDEX_NAME' from 'transformers.utils' (/home/usr/miniconda3/envs/mkg/lib/python3.10/site-packages/transformers/utils/__init__.py)
```

### Expected behavior

Able to load BartEncoder and BartDecoder

Comments:
@icedpanda  I am not able to reproduce this error, Can you try to reproduce this in a colab notebook and share the notebook here ? 

was working on the terminal but not in the notebook. Works now after reinstalling the notebook.
fix device issue
Author: ydshieh
Pull request: yes

# What does this PR do?

When this block is run
```
        if isinstance(target_sizes, List):
            img_h = torch.Tensor([i[0] for i in target_sizes])
            img_w = torch.Tensor([i[1] for i in target_sizes])
```
`scale_fct` (defined a few line below) is always on `cpu`. We need to put it on the proper device.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@alaradirik Only for those post-processing with this block
```python
        if isinstance(target_sizes, List):
            img_h = torch.Tensor([i[0] for i in target_sizes])
            img_w = torch.Tensor([i[1] for i in target_sizes])
```
have to be fixed.

If there is only `img_h, img_w = target_sizes.unbind(1)`, I think it is fine (will be on the correct device already).

Do you have a link to a place that you think need to be fixed but not done in this PR yet?

> @alaradirik Only for those post-processing with this block
> 
> ```python
>         if isinstance(target_sizes, List):
>             img_h = torch.Tensor([i[0] for i in target_sizes])
>             img_w = torch.Tensor([i[1] for i in target_sizes])
> ```
> 
> have to be fixed.


@ydshieh that makes sense, and we are assuming that the input `target_sizes` is on the correct device, right?


> @ydshieh that makes sense, and we are assuming that the input `target_sizes` is on the correct device, right?

Yes, as so far we don't have CI failure from that one, so we should be good.


> @ydshieh that makes sense, and we are assuming that the input `target_sizes` is on the correct device, right?

Yes, as so far we don't have CI failure from that one, so we should be good.
Bug in MobileViTForSemanticSegmentation output shape
Author: kevinpl07
Pull request: no

### System Info

Python 3.9.6
transformers==4.24.0


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import MobileViTFeatureExtractor, MobileViTForSemanticSegmentation
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

feature_extractor = MobileViTFeatureExtractor.from_pretrained("apple/deeplabv3-mobilevit-small")
model = MobileViTForSemanticSegmentation.from_pretrained("apple/deeplabv3-mobilevit-small")

inputs = feature_extractor(images=image, return_tensors="pt")

outputs = model(**inputs)
logits = outputs.logits
predicted_mask = logits.argmax(1).squeeze(0)
```

### Expected behavior

When running inference on pretrained MobileViTForSemanticSegmentation as in [here](https://huggingface.co/apple/deeplabv3-mobilevit-small), I get very bad segmentation mask in the huggingface online inference. When running locally I see that the output mask has a shape of (1, 21, 32, 32).

I don't think outputting a 32x32 mask is intended or am I wrong? 

Comments:
Hi,

MobileViT's semantic segmentation is indeed very low-resolution, as seen in this Space: https://huggingface.co/spaces/Matthijs/mobilevit-deeplab-demo. cc @hollance 

If you want to look into models that output more fine-grained segmentation results, I'd recommend checking out [SegFormer](https://huggingface.co/docs/transformers/model_doc/segformer) and [MaskFormer](https://huggingface.co/docs/transformers/model_doc/maskformer).

The original model will do a bilinear upsampling to 513x513 which makes the mask larger but also more blurry. In the Transformers library we don't do this upsampling in the model as we don't know ahead of time how you want to use the results, and we want to avoid doing multiple upsampling operations. That's why the Transformers model outputs the low resolution version.

Closing this issue as it has been resolved, feel free to reopen.
Generate: add generation config class
Author: gante
Pull request: yes

# What does this PR do?

Adds the class that handles generation configurations. From this class, we can then start working on the interaction with other parts of the Hugging Face 🤗 ecosystem. See https://github.com/huggingface/transformers/issues/18655 for the original plan, which matches this implementation.

Throughout the PR, there are a few very visible comments where your input would be appreciated to establish solid base functionality.

The expected follow-up tasks are:
1. We can load a Generation Config from a Model Config (to make it compatible with all existing repos)
2. `.generate()` inherits default values from generate config, instead of from model config. Add an argument to pass a generation configuration.
3. Add doctests for loading, storing, and using with `.generate()`
4. The top 100 models with generate compatibility get a PR with the corresponding generation confg file(s). At least 1 has multiple generation config files, to be used as an example.
5. (bonus) Generate config validates its attributes at initialization time
6. (bonus) Pipelines make automatic use of the appropriate generation config file
7. (bonus) Ensure that advanced functionality (e.g. models like RAG and Trainer) operates well with the generation config file

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20218). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20218). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20218). All of your documentation changes will be reflected on that endpoint.

@sgugger @patrickvonplaten all comments addressed. There is a doctest example that loads our first `generation_config.json`, [from gpt-2](https://huggingface.co/gpt2/blob/main/generation_config.json) 🙌 

I took the liberty to add a few goodies, like making it a public class (to allow `from transformers import GenerationConfig`), adding a `__repr__` (for a preview) and sections in the `__init__` docstring (I think users will appreciate).
[Switch Transformers] Fix failing slow test
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR should fix the currently failing slow test for `SwitchTransformers`.
Putting the model on GPU and running the inference on GPU should fix the test. I have tested the test on PyTorch 1.12 (same as the daily CI runner), but I need to test it on the daily CI runner first before merging

cc @ydshieh @ArthurZucker @sgugger 

EDIT: it passes on the CI daily runner, marking it as ready for review

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks very much for the explanation! This should have been fixed now 💪
longformer_content
Author: wesleygmorris
Pull request: no
Labels: name: New model

### Model description

This model will generate a content score for a summary given the context and the summary itself.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Which new model are you talking about? Please fill the template properly otherwise no one will be able to help.

Hey @wesleygmorris, sorry for the very slow reply here, we're catching up on the backlog.

A summary content-scorer like this doesn't need a new architecture: it's a fine-tuned Longformer with a regression head, which transformers already supports natively via `LongformerForSequenceClassification` (set `num_labels=1`). Once fine-tuned, just push the checkpoint to the Hub and load it with `AutoModelForSequenceClassification.from_pretrained(...)`
Add translation of docs to greek
Author: ArchontisKostis
Pull request: no
Labels: name: WIP

I thought it would be awsome to add greek translation to the documentation I currently have translated only the index.mdx


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Image Format (BGR/RGB) bug for lxmert example
Author: lvyiwei1
Pull request: no

### System Info

Repository code (accessed 10/13/2022) in examples/research_projects/lxmert

### Who can help?

@LysandreJik 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run demo.ipynb in examples/research_project/lxmert (or instead use [this colab notebook](https://colab.research.google.com/drive/1N0z-mplcu-20TZPr7-TNkZUCASmmP90i?usp=sharing))
but instead of using an URL, upload arbitrary jpg image and do local file
(i.e. change line `frcnn_visualizer = SingleImageViz(URL,id2obj=objids, id2attr=attrids)` 
to `frcnn_visualizer = SingleImageViz('pic.jpg',id2obj=objids, id2attr=attrids)` where `pic.jpg` is arbitrary jpg file you have)
Then the result will have flipped red and blue colors

### Expected behavior

The result have flipped red and blue colors.
Technically, the image inputed to frcnn is RGB instead of BGR (but frcnn uses BGR), so this is probably due to doing BGR2RGB one extra time in the image preprocessing step.

### Source

This bug was discovered through [MultiViz paper](https://arxiv.org/pdf/2207.00056.pdf)

Comments:
Note that we do not maintain the research project examples, so you will have better luck pinging the original author :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[WIP] Better Transformers  integrations for BERT
Author: HamidShojanazeri
Pull request: yes

# What does this PR do?
This PR, is the first PR from a series that add[ PyTorch Better Transformers ](https://pytorch.org/blog/a-better-transformer-for-fast-transformer-encoder-inference/)support to Bert model for inference speed ups.



# Usage

`model = AutoModelForSequenceClassification.from_pretrained("bert-large-cased").eval().to(device)
 model.bert.encoder.to_fast()
`


## Before submitting

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. -- Offline discussions
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? -- In progress


## Who can review?

 @LysandreJik @younesbelkada 




Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[WIP] support auto-compress for glue task
Author: ceci3
Pull request: yes

# What does this PR do?

support auto-compress for glue task

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19115). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
add `transformers-cli pt-to-flax`
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Following the addition of the `transformers-cli pt-to-tf` command, this uses the same scipt to convert to `FLAX`. [Since another PR](https://github.com/huggingface/transformers/pull/18419)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18420). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add MobileNetV1
Author: atturaioe
Pull request: no
Labels: name: New model

### Model description

MobileNets are small, low-latency, low-power models parameterized to meet the resource constraints of a variety of use cases. They can be built upon for classification, detection, embeddings and segmentation similar to how other popular large scale models, such as Inception, are used. MobileNets can be run efficiently on mobile devices [...] MobileNets trade off between latency, size and accuracy while comparing favorably with popular models from the literature.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

[Paper](https://arxiv.org/abs/1704.04861)

Hi there, I wonder whether MobileNetV1 is going to be added or not, I see there's already a dedicated [card](https://huggingface.co/google/mobilenet_v1_1.0_224) on model's hub.

@hollance 

Comments:
There's an open PR for it that is (almost) ready to merge. I just need to rebase it because it conflicts with MobileNetV2 that was recently added. I'll probably get around to this later this week.

See the PR: https://github.com/huggingface/transformers/pull/17799

Oh, thanks. Closing this issue.

Just a FYI: it has been merged now with the main branch of Transformers. :-)
Generate: `model_kwargs` can also be an input to `prepare_inputs_for_generation`
Author: gante
Pull request: yes

# What does this PR do?

Fixes #20347 

`model_kwargs` can also be a model input in `prepare_inputs_for_generation` -- in some models it is `kwargs`, in others `model_kwargs`. 

This PR updates the input validation function to reflect that.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger added a minor test.

I'd rather spend the energy removing the `**kwargs` and `**model_kwargs`, which are only used as a lazy pattern (as opposed to being forwarded to the model or similar). It would allow for much stricter checking :)