noamaanMulla-03/transformers-issues huggingface dataset:
`MinNewTokensLengthLogitsProcessor` for `.generate` method #20814
Author: kotikkonstantin
Pull request: yes

### **Approved** by [#20814 issue](https://github.com/huggingface/transformers/issues/20814)

### What does this PR do?
It implements `MinNewTokensLengthLogitsProcessor` class for enforcing a min-length of **NEW** tokens by setting EOS (end-of-sequence) token probability to 0.

Framework: `pytorch`

### Who can review?

@gante

Comments:
_The documentation is not available anymore as the PR was closed or merged._

There seems to be a problem with CI that we'll have to fix before merging. @kotikkonstantin, what fo you see when you click on "Details" next to "setup_and_quality" in the checks section below?

> There seems to be a problem with CI that we'll have to fix before merging. @kotikkonstantin, what fo you see when you click on "Details" next to "setup_and_quality" in the checks section below?

Skipped:
![image](https://user-images.githubusercontent.com/22777646/209811255-28367baa-66de-4d23-a315-a0f255972674.png)

I suppose it's skipped because it keeps previous successful parts of the CI pipeline if it's not unchanged in the last commit

> > There seems to be a problem with CI that we'll have to fix before merging. @kotikkonstantin, what fo you see when you click on "Details" next to "setup_and_quality" in the checks section below?
> 
> Skipped: 
> 
> I suppose it's skipped because it keeps previous successful parts of the CI pipeline if it's not unchanged in the last commit

I'm not right here. After failed CI-pipeline run, in the following successful CI-pipeline run, `setup and quality` just was stopped instead of launching

@kotikkonstantin CircleCI is complaining about terms of service -- are you based in one of the countries linked [here](https://support.circleci.com/hc/en-us/articles/360043679453-CircleCI-Terms-of-Service-Violation-Sanctioned-Country)?

It seems there is an issue with your CircleCI permissions, the tests won't run.
Could you try refreshing your permissions as shown [here](https://support.circleci.com/hc/en-us/articles/360048210711-How-to-Refresh-User-Permissions-)? You might need to push an empty commit afterward.

@gante @sgugger 
Thank you, guys, for your assistance in approaching it!

 I've filled out [Individual Appeal Form](https://docs.google.com/forms/d/e/1FAIpQLSeaVwzPnt2xREoZxe_ysnmNEJQUfBWrTI1TzkE7bq1h06eHqA/viewform). I hope I get access. If not, could you launch the CI pipeline on your own? 

@kotikkonstantin I think we can. Let's try it out:
1 - add me as a contributor to your fork of `transformers`
2 - I will push an empty commit there
3 - maybe CI gets triggered

> @kotikkonstantin I think we can. Let's try it out: 1 - add me as a contributor to your fork of `transformers` 2 - I will push an empty commit there 3 - maybe CI gets triggered

@gante done

@gante I can see CI-logs:
<img width="1663" alt="image" src="https://user-images.githubusercontent.com/22777646/210263144-92cc0aae-5e86-4547-a6b5-012cd2346a06.png">


@kotikkonstantin yup, I've took the liberty to run the `make fixup` shell command and push :) (which should fix it)
Fix post_process_object_detection method descriptions
Author: alaradirik
Pull request: yes

# What does this PR do?
Fixes the descriptions of all effected model methods (post_process_object_detection and deprecated post_process methods) that inaccurately state the methods returns bounding boxes in the format expected by the COCO API (x_center, y_center, w, h) format instead of the (x1, y1, x2, y2) format. 

I will open a separate PR to add an option to return the bounding boxes in the COCO API format. 


## Before submitting
- [ X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Improve OWL-ViT postprocessing
Author: alaradirik
Pull request: yes

# What does this PR do?
- Adds post_process_object_detection method to OWL-ViT with the same functionality as other object detection post-processing methods (thresholding, different target sizes for each image in the batch).
- Updates the zero-shot-object-detection pipeline to use the new method

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?




Comments:
Fix valid ratio for Deformable Detr
Author: long8v
Pull request: yes

# What does this PR do?

I encountered unexpected behavior that single-batch image and multi-batch image return significantly different output.
I found that its reason is from function `get_valid_ratio`, which returns ratio of image size for each example from batch (which is padded for longest width and height in batch). Since mask has opposite value(True for real pixel, False for padded) from original repo, mask should be opposite with [original repo](https://github.com/fundamentalvision/Deformable-DETR/blob/main/models/deformable_transformer.py#L117-L124) inside of function `get_valid_ratio`. Otherwise it will return image ratio for **pad width and height**, which is obviously erroneous.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@NielsRogge 



Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @NielsRogge and @amyeroberts
[WIP] Add MANTa-LM
Author: NathanGodey
Pull request: yes

# Implement the MANTa-LM model (upcoming paper)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@patrickvonplaten, @patil-suraj

Comments:
We're looking forward to it, @NathanGodey!

I'm pinging @ArthurZucker and @sgugger to make sure it's on their radar even if the implementation isn't ready to review. Let us know when you'd like for us to jump in and help!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Improve OWL-ViT postprocessing
Author: alaradirik
Pull request: yes

# What does this PR do?
- Adds post_process_object_detection method to OWL-ViT with the same functionality as other object detection post-processing methods (thresholding, different target sizes for each image in the batch).
- Updates the zero-shot-object-detection pipeline to use the new method

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Hugging Face Dies Silently when Memory insufficient for loading Model / Training Model 
Author: courtneysprouse
Pull request: no

Currently, when you load a model into memory that is too large or if you try to train a model with insufficient memory. The process gets killed without an error message. It's a bit tough to track down what is going on as a result. I'm wondering if you can add an error message similar to pytorch when you have insufficient memory to run a given process?

Comments:
If you have insufficient GPU memory, you will get the PyTorch error. For RAM issues, I don't think there is anything that exists to issue the same errors.

I was running on CPU. I know I've gotten the pytorch errors on GPU.  If nothing exists that's alright. Just thought it would be nice to get an error message so you could more easily see what was going on, particularly when you're just loading a model for inferencing, which is often done on cpu.
[Feature Request] Add 3D attention mask for T5Model
Author: yongyi-wu
Pull request: no
Labels: name: Good Second Issue

name: Feature request

## Environment info

- `transformers` version: 4.2.1
- Platform: Linux
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0+cuda11.0

### Who can help
T5: @patrickvonplaten

## Information
The `get_extended_attention_mask()` does not sufficiently address the case with 3D attention mask. The problem emerges for T5Model as `input_ids` and `decoder_input_ids` are of different length and the `attention_mask` is of shape [Batch_size, Seq_length, Seq_length]. The decoder uses `attention_mask` directly for `encoder_attention_mask` in cross-attention, which is of incorrect shape and the error message does give any information about why it happens. 

## To reproduce
As described above. I can add code later if needed. 

## Expected behavior
I propose to add a sanity check for attention masks or improve the `get_extended_attention_mask()` method. 


Comments:
Hello @yongyi-wu, 

yes you're right. T5 is not yet fully compatible with 3D attention_mask input. Currently, I won't find enough time to work on adding this feature, but I'll post it under "Community projects" in case someone from the community is interested in giving it a shot.

Also feel free to open a PR yourself, if you want to try :-) 

Hi, I am new here but would like to give this a shot. Because it is my first issue I could use some direction if on how to tackle this, if this is okay with you. 

- Would you prefer the sanity check or an improved `get_extended_attention_mask()` method? 
- Do you know of an already existing implementations with 3D attention mask as a reference. 
- Where would you like to see the solution implemented. 

Hi, I'm a newbie in transformers and trying to make customized 3D attention mask with T5ConditionalGeneration.
I was googling 3D attention mask for T5 model and found it in here.

From what I understood above is that, I should add one more `encoder_sequence_length` variable at the end of line `encoder_hidden_shape = (encoder_batch_size, encoder_sequence_length)` in T5Stack forward function if I want to build 3D attention?

Do I need to edit anything else?

@lexhuismans 

Thanks!
update template
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Adds a better template for Korean Readme and replaces the previous text.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The following substitution pattern was used : 
- match : `\((?:f|F)rom ([^\(]*)\)(?:,)? released (?:together )?with the paper (.*) by (.*).`
- sub : `($1 에서) $3 의 $2 논문과 함께 발표했습니다.`
The 54b MoE!
Author: alzyras
Pull request: no

### System Info

Hello,

I'm (And I believe many others) are intrigues by the Model of Experts model.

I've looked at the only documentation I could find here: https://github.com/facebookresearch/fairseq/blob/nllb/examples/nllb/modeling/README.md

However the arguments for generation / evaluation are unclear to me :)

I will be starting a data analysis job shortly and I see some possible applications for the 54b model.

Surely there are the other models, but I believe many enthusiasts are looking forward to trying translations with this model. I'm just looking for starting arguments for translating, i.e. eng to de.

I saw the model is coming to huggingface at some point so definitely looking forward to that.

I am also interested in running real person evaluations of 3.3b model, 54b MoE model, Deepl and others to see how far the models have come :)

Thank you so much for your work. 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

-

### Expected behavior

-

Comments:
Pinging @ArthurZucker and @younesbelkada who have been working on contributing the Switch Transformer, an MoE, to `transformers`. I agree adding the 54B NLLB model would be quite cool too!

also personally very interested in support for large MoEs

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Any news about MoE?

[Switch Transformers](https://huggingface.co/docs/transformers/model_doc/switch_transformers) has been added to the library.
Fix bug in segmentation postprocessing
Author: alaradirik
Pull request: yes

# What does this PR do?
- Fixes bug in `MaskFormerFeatureExtractor.compute_segments()` that causes an error when label_ids_to_fuse is set to None during instance and panoptic segmentation post-processing. 
- Adds test to check if target labels are fused correctly


Fixes #20132


## Before submitting
- [X ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? 
https://github.com/huggingface/transformers/issues/20132




Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20198). All of your documentation changes will be reflected on that endpoint.

> Could you add a corresponding test for this?

Added a test to `test_feature_extraction_maskformer.py` to test label fusing, I think I covered all cases but the test might still be a bit flaky due to using dummy model outputs.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20198). All of your documentation changes will be reflected on that endpoint.

@NielsRogge I thought this issue was fixed with the ImageProcessor PR but the MaskFormer instance segmentation post-processing issue still persists. I updated the PR and added a test for label fusing. Could you take another look and approve if everything looks good?

> PR looks good to me, except that modeling_vit.py should not be changed

The changes are reverted, could you take another look? 

> Thanks for fixing!
> 
> However I'm still wondering how MaskFormer would solve instance segmentation datasets which have overlapping instances, like COCO

We have MaskFormer models trained on the  COCO panoptic and ADE semantic datasets, other models don't have model cards yet. But I can test it if one of the other models is trained on an instance segmentation dataset.
Add Donut image processor
Author: amyeroberts
Pull request: yes

# What does this PR do?

Adds an image processor for Donut. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I'm currently unable to create a PR myself but please note that `DonutImageProcessor.preprocess()` still calls `.pad()` instead of `.pad_image()`, generating a *lot* of logger noise...

@pasky Thanks for raising! This should now be resolved with the merging of #20904
Don't call deprecated method
Author: amyeroberts
Pull request: yes

# What does this PR do?

Call `pad_image` instead of `pad` which has been deprecated in order to maintain consistent method naming across image processors. 

There's no difference in logic, as `pad` calls `pad_image`, it just reduces excessive logging raised in [this comment](https://github.com/huggingface/transformers/pull/20425#issuecomment-1364747167).


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Refactor the function get_results
Author: milyiyo
Pull request: yes

# What does this PR do?

A small refactor for the function `get_results`.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger and @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: TF uses `GenerationConfig` as the basis for `.generate()` parametrization
Author: gante
Pull request: yes

# What does this PR do?

Changes the TF side of `.generate()` such that it relies on the `GenerationConfig`. This is the TF equivalent of https://github.com/huggingface/transformers/pull/20388

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Convert assertions to exceptions in some examples
Author: aphedges
Pull request: yes

# What does this PR do?

For #12789.

This PR converts assertions to exceptions in some example files in `/examples/pytorch/language-modeling/`. I found this commit locally from over a year ago, so new scripts have been added since it was created.

## Who can review?

Maintained examples (not research project or legacy):

- PyTorch: @sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The issue was aimed at the modules in the library, not the examples. I'd keep the examples as they are for now.

@sgugger, I was not aware of that! Can you mention that prominently in the issue (#12789) so others know to avoid them?

Also, I modified the initial comment because I had forgotten to link the issue in the first place, even though I had made a note to before I created this PR.

Edited my comments on the issue to reflect this.

Thanks! I'm closing PR now, then.