noamaanMulla-03/transformers-issues huggingface dataset:
Adds GroupViT to models exportable with ONNX
Author: unography
Pull request: yes

Adds GroupViT to models exportable with ONNX

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@regisss hopefully you don't have to make any changes this time to make the tests pass!

Pinging @sgugger for final approval

Feel free to merge if you approve @lewtun

@lewtun Can you take a quick look at this PR and merge it when you approve? :slightly_smiling_face:
AdamW algorithm is not the same as in the referenced paper
Author: quantitative-technologies
Pull request: no
Labels: name: bug

### System Info

Not relevant

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The HuggingFace implementation of AdamW is not the same as the algorithm from the paper [Decoupled Weight Decay Regularization](https://arxiv.org/abs/1711.05101), as claimed in the documentation. 

It is easy to show this by mathematical proof, it it is not obvious by inspection. The HuggingFace AdamW has been deprecated in any case. 

But this is still a bug which "caught" me. The issue is referenced, but not as a bug, in the closed issue  [#3407](https://github.com/huggingface/transformers/issues/3407).

### Expected behavior

The PyTorch implementation matches the algorithm in the paper. 

Comments:
Hey @quantitative-technologies, we're in the process of removing the AdamW implementation, as you have seen it is deprecated. It will be removed in v5.
reproducing the performance of XLM-ROBERTA on MLQA dataset on the zh language
Author: nooralahzadeh
Pull request: no

# ❓ Questions & Help
## Details
<!-- Description of your issue -->

I have trouble in reproducing the result of XLM-ROBERTA on MLQA dataset for Chinese languages. The result of the rest of the language seems fine, however on the zh section, I have very low  {'exact_match': 4.263188631496982, 'f1': 17.451059178461946})! 

Comments:
Credit to people at Microsoft Asia: Ning Wu and Nan Dua
To achieve the best performance on the "zh" test sets, you just need to add 
"final_text = tok_text" after line 497 in squad_metrics.py (only for zh). Because there isn't space and subword in Chinese, so we don't need to execute the get_final_test() function.

This is very interesting! Thanks for letting us know @nooralahzadeh!

@LysandreJik  Do you think the training model  "RobertaForQuestionAnswering" also need to be updated for 'zh' lang.  Because when I try to fine-tune the xlm-r on 'zh' language and evaluate on its test set, the results became very lower than not fine-tuning.


Just found the same problem, thanks bro!

> Credit to people at Microsoft Asia: Ning Wu and Nan Dua
> To achieve the best performance on the "zh" test sets, you just need to add
> "final_text = tok_text" after line 497 in squad_metrics.py (only for zh). Because there isn't space and subword in Chinese, so we don't need to execute the get_final_test() function.



I used huggingface to train & predict then use https://github.com/facebookresearch/MLQA/blob/main/mlqa_evaluation_v1.py to calculate the scores. Seem to match the paper.
Fix cost condition in DetrHungarianMatcher and YolosHungarianMatcher to allow zero-cost
Author: kongzii
Pull request: yes

# What does this PR do?

Fixes costs condition in DetrHungarianMatcher and YolosHungarianMatcher. In https://github.com/huggingface/transformers/pull/16720 a bug was introduced while switching from asserts to conditions. Currently, any zero-cost will result in a ValueError:

```python
if class_cost == 0 or bbox_cost == 0 or giou_cost == 0:
    raise ValueError("All costs of the Matcher can't be 0")
```

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@LysandreJik @sgugger based on reviewers of previous PR https://github.com/huggingface/transformers/pull/16720.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

This looks reasonable given the error message! cc @NielsRogge
Warn on TPUs when the custom optimizer and model device are not the same
Author: muellerzr
Pull request: yes
Labels: name: Distributed Training / Models

name: bug

# What does this PR do?

This PR raises an error when a user creates a custom optimizer on a TPU and they did not move the model to the right TPU device beforehand. Not doing so will cause issues such as the ones described in https://github.com/pytorch/xla/issues/3675#issuecomment-1171702988 and https://github.com/huggingface/transformers/issues/18635. This check is performed similar to the one in Accelerate

Fixes # (issue)

https://github.com/huggingface/transformers/issues/18635

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@LysandreJik 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix add model like 
Author: LysandreJik
Pull request: yes

It seems that `pip show` does not return the same location as before for editable packages.

However, listing editable packages (`pip list -e`) returns the correct location in which it was installed. Doing it locally returns the following:

```
Package      Version     Editable project location
------------ ----------- ---------------------------------------------
transformers 4.22.0.dev0 /home/lysandre/Workspaces/python/transformers
```

I'm therefore updating the way to identify if we're looking at the correct install to use `pip list -e` instead of `pip show`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18834). All of your documentation changes will be reflected on that endpoint.
Define metric for save the best model
Author: dimka11
Pull request: no

### Feature request

I suggest to add a metric choice to save the best model

### Motivation

I use multiple metrics in process of fine tuning models through Trainer and don't know how the metric is chosen to save the best model (I suppose it's first metric in dictionaty?!).

### Your contribution

-

Comments:
If you are using HF trainer it provides two arguments for selecting your preferred metric for choosing the best model. The first one is
`metric_for_best_model`. It defaults to "loss". The second argument is `greater_is_better`. The default value is `False` but If metric for best model is set to any value other than "loss" then it will default to `True`


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Memory is not released when moving model to CUDA
Author: piEsposito
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.21.0
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): 2.8.2 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: GPU
- Using distributed or parallel set-up in script?: No

### Who can help?

@patil-suraj

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Steps to reproduce behavior:
1. Run Colab: https://colab.research.google.com/drive/1NWJPqwe7MOJIWd4w5LGYGaflkWXTkHGB?usp=sharing
2. Check results: 

```
model device = cuda:0
Filename: memory_leak.py

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     7    239.4 MiB    239.4 MiB           1   @profile
     8                                         def main():
     9    271.6 MiB     32.2 MiB           1     processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch16")
    10   1405.8 MiB   1134.2 MiB           1     model = CLIPModel.from_pretrained("openai/clip-vit-base-patch16")
    11                                           
    12   1405.8 MiB      0.0 MiB           1     device = torch.device("cuda")
    13   2644.0 MiB   1238.2 MiB           1     model = model.to(device)
    14                                         
    15   2644.5 MiB      0.5 MiB           1     print(f"model device = {model.device}")
    16   2644.5 MiB      0.0 MiB           1     gc.collect()
```

### Expected behavior

RAM should be released when a model is moved to GPU. 

This bug can be reproduced for lots of different models within the lib.

Comments:
Pinging the king of memory, @ydshieh :raised_hands: 

Hi @piEsposito 

As I have seen quite a few times `torch` has some of its own memory management, and related memory issues, it would be great if you can provide and example that creates a (big enough) PyTorch models (not from `transformers`) on CPU, send it to CUDA, and see if you get the memory been released.

The goal is to make sure this issue is not coming from `PyTorch`. Would you like to work on an example, please? Thanks in advance.

@ydshieh just added it to the notebook. It seems like this issue comes from PyTorch. Thanks!
Skip XNLI test
Author: LysandreJik
Pull request: yes

Skips an XNLI test that currently fails due to https://github.com/fsspec/filesystem_spec/issues/1034

Comments:
Preceded by https://github.com/huggingface/transformers/pull/18837

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18838). All of your documentation changes will be reflected on that endpoint.
Improve GPT2 doc
Author: ekagra-ranjan
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes the typos and dimensions of args in doc of GPT2.


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add an option to `HfArgumentParser.parse_{dict,json_file}` to raise an Exception when there extra keys
Author: FelixSchneiderZoom
Pull request: yes

I added an option to `HfArgumentParser.parse_{dict,json_file}` to raise an Exception when there extra keys. The option is off by default, for backward compatibility.

For users of these functions, misspelled or incorrect keys in the parsed files/dicts could lead to hard-to-find errors, like putting `batch_size` in your config and wondering why the Trainer is using a different one, because the actual name is `per_device_train_batch_size`. The option will output a very similar error message to the normal argparse.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

@sgugger 




Comments:
_The documentation is not available anymore as the PR was closed or merged._
Improve Text Generation doc
Author: ekagra-ranjan
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

* Add relevant args explicitly in beam search decoding example in generation utils
* PAD token is absent in GPT2 instead of EOS token


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


@sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you @sgugger for the review. I have applied your suggestions.
Add SegFormer ONNX support
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds support for the first "semantic-segmentation" feature, SegFormer.

Related to #16308

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@lewtun I can confirm that all tests pass!

@lewtun ok for me to merge?
Cannot import OPTForSequenceClassification on Kaggle notebooks (transformers 4.20.1, huggingface_hub 0.8.1)
Author: navinelahi
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.20.1
- Platform: Linux-5.10.133+-x86_64-with-debian-bullseye-sid
- Python version: 3.7.12
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): 2.6.4 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.0 (gpu)
- Jax version: 0.3.16
- JaxLib version: 0.3.15
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Below is the code snippet and the error it caused. It can import OPTForCausalLM but not OPTForSequenceClassification
```
from transformers import OPTForCausalLM
from transformers import OPTForSequenceClassification, Trainer, TrainingArguments
```
**ImportError: cannot import name 'OPTForSequenceClassification' from 'transformers' (/opt/conda/lib/python3.7/site-packages/transformers/__init__.py)**

Any pointers to what might be wrong? Thank you so much. 

### Who can help?

@LysandreJik

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import OPTForCausalLM
from transformers import OPTForSequenceClassification, Trainer, TrainingArguments

### Expected behavior

I would expect OPTForSequenceClassification to be imported normally.

Comments:
Hey @navinelahi, this class was released in v4.21; could you upgrade your `transformers` library`?

Yes I upgraded with `pip install transformers==4.21` and now it's working. Thank you so much. The issue is solved
Unpin fsspec
Author: albertvillanova
Pull request: yes

# What does this PR do?

The `fsspec` team has made a patch release (2022.8.2) to fix their issue:
- fsspec/filesystem_spec#1032

They yanked both 2022.8.0 and 2022.8.1, so no need to pin to exclude them.

Follows:
- #18837


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Delete `state_dict` to release memory as early as possible
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix #18782.

Note that this is not a real memory issue. A call to `gc.collect()` at the end of `from_pretrained()` works well too.

However this PR finds and simply `del state_dict` at the end of `_load_state_dict_into_model()`, and `GC` is able to perform housekeeping on its own at a earlier time.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The change regarding having a new argument `state_dict` in the nested function `load` is to pass black check, otherwise we get 

```bash
src/transformers/modeling_utils.py:422:17: F821 undefined name 'state_dict'
```
with the new line `del state_dict`. (It's quite strange though)

Ready for review.

The failing test is `test_encodings_from_xnli_dataset` which is irrelevant to this PR.
Generate: smaller TF serving test
Author: gante
Pull request: yes

# What does this PR do?

`tests/test_modeling_tf_common.py::UtilsFunctionsTest::test_generate_tf_function_export` is often failing because it times out (>60s).

The previous version took ~35s in my machine. This PR's takes ~19s, which may avoid the time out issue.

If it still fails, a custom config must be added :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
add a script to get time info. from GA workflow jobs
Author: ydshieh
Pull request: yes

# What does this PR do?

As we might need to get the running time for workflow jobs again in the future, here is a simple script. It's probably better to move to the directory `utils`. I put it under `.github/scripts/` to emphasize this is really for GitHub Actions only.

The output looks like

```bash
(py39) λ python get_github_job_time.py --workflow_run_id 2945609517
Model tests (onnx, multi-gpu): 337
Model tests (onnx, single-gpu): 334
Torch CUDA extension tests (multi-gpu): 44
Torch CUDA extension tests (single-gpu): 43
TensorFlow pipelines (multi-gpu): 20
TensorFlow pipelines (single-gpu): 19
...
```

P.S. I will add another simple script to get test failures and their counts in another PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yes, I will move it to `utils`.