noamaanMulla-03/transformers-issues huggingface dataset:
Fix None loss in docstring for Wav2Vec2ForPretraining
Author: abdouaziz
Pull request: yes

- [ ]  This PR Fix None loss in docstring for Wav2Vec2ForPreTraining

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[Conditional, Deformable DETR] Add postprocessing methods
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds `post_process_object_detection` to Conditional DETR and Deformable DETR's feature extractors. They both use the same postprocessing (and it's different from original DETR + YOLOS).

It also removes methods which aren't necessary: YOLOS and Deformable DETR don't have a segmentation head model, so no `post_process_segmentation` methods are needed for those models.

To do:

- [x] add corresponding tests
- [x] remove legacy segmentation postprocessing methods of conditional DETR, add new ones

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Could you add Conditional DETR to MODEL_FOR_XXX_SEGMENTATION_MAPPING_NAMES?

Do you mean I need to add the model to all 3 segmentation mapping names? Cause that isn't the case with DETR either



> > Could you add Conditional DETR to MODEL_FOR_XXX_SEGMENTATION_MAPPING_NAMES?
> 
> Do you mean I need to add the model to all 3 segmentation mapping names? Cause that isn't the case with DETR either

`MODEL_FOR_IMAGE_SEGMENTATION_MAPPING_NAMES` is marked for deprecation and there is no `MODEL_FOR_PANOPTIC_SEGMENTATION_MAPPING_NAMES`,  I meant we can create one and add DETR and MaskFormer to panoptic mapping, along with Conditional DETR.


Ok, let's do that in a separate PR.
Fix ONNX tests for ONNX Runtime v1.13.1
Author: lewtun
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes the slow tests for ONNX in `test_onnx.py` which were failing due `input_qType` being renamed by `activation_qType` in `onnxruntime` v1.13.1

With this fix, the follow passes:

```
RUN_SLOW=1 pytest tests/onnx/test_onnx.py
```


Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Does the new name work with the minimum version of ONNX pinned?

Are you referring to the version we list in `setup.py` or somewhere else? In `setup.py` we have `onnxruntime>=1.4.0` so users with a pre-existing installation of ONNX Runtime would have to upgrade to `v1.13.1`.

An alternative is to have an if/else statement that checks the `onnxruntime` version to guarantee backwards compatibility - I'll implement that instead :)

> An alternative is to have an if/else statement that checks the onnxruntime version to guarantee backwards compatibility - I'll implement that instead :)

Yes please!

Backwards compatibility added in https://github.com/huggingface/transformers/pull/19950/commits/01ccbea13800e6912c1e34cbece4311cd2b6b420 :)
Tranformers documentation translation to Italian #17459
Author: draperkm
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # #17459


Italian translation of transformes/docs/source/en/perf_training_tpu.mdx


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
Output scores in TranslationPipeline
Author: d-miketa
Pull request: no

### Feature request

Please consider adding scores to the output dict of TranslationPipeline.

https://huggingface.co/docs/transformers/v4.21.3/en/main_classes/pipelines#transformers.TranslationPipeline.__call__

### Motivation

It'd be nice to see the scores/probabilities of translated sentences rather than just an ordered list of the top k beam search outputs. In many cases the scores can be interpreted as a measure of confidence in the prediction and this is valuable, especially in production.

Pipelines are natively supported by Seldon Deploy so this would also improve that integration.

### Your contribution

I don't currently have capacity to submit a PR.

Comments:
cc @Narsil 

This seems a nice addition ! 

Same here, I have  limited bandwidth at the moment.

Notes for anyone wanting to implement this.

The goal is NOT to support every single feature `generate` supports in terms of return, only the one that make sense for users not knowing about ML, and not being power users (anyone that knows enough, should be able to drop down from pipelines and using lower level objects to get full control, or override the pipeline by subclassing). 
1 score per proposed translation fits that model.

A counter-example would be:  `score` per token was asked by users on `bloom` (it's `text-generation` not `translation`, but since it works similarly under the hood I'm leaving breadcrumbs. This for instance is outside of the scope of pipelines, since tokens are a ML construct, and users without any ML background will have trouble understanding what they are. In addition, returning such things change the return type which is never ideal when function/classes return types depend on arguments. (and subclassing should be easy enough to add support for anyone that so desires).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[EncoderDecoderModel] Add support for gradient checkpointing
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds gradient checkpointing support for EncoderDecoderModel in PyTorch.

As requested on the forum: https://discuss.huggingface.co/t/feature-request-gradient-checkpointing-for-encoderdecodermodel/25278

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Using GPT2 tokenizer with DataCollatorForLanguageModeling
Author: martinez-zacharya
Pull request: no

### System Info

- `transformers` version: 4.21.3
- Platform: Linux-5.10.0-18-amd64-x86_64-with-glibc2.31
- Python version: 3.9.12
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.12.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed

### Who can help?

@SaulLu @patil-suraj @patrickvonplaten 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer, AutoModelForCausalLM, DataCollatorForLanguageModeling

model = AutoModelForCausalLM.from_pretrained("nferruz/ProtGPT2")
tokenizer = AutoTokenizer.from_pretrained("nferruz/ProtGPT2")
tokenizer.pad_token = tokenizer.eos_token

data_collator = DataCollatorForLanguageModeling(tokenizer = tokenizer, mlm=False)

list_of_seqs = ['GLWSKIKEVGKEAAKAAAKAAGKAALGAVSEAV', 'DGVKLCDVPSGTWSGHCGSSSKCSQQCKDREHFAYGGACHYQFPSVKCFCKRQC']

tokens = tokenizer(list_of_seqs, padding = True, return_tensors='pt', return_special_tokens_mask=True)
collated = data_collator(tokens)
```

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/zachmartinez/miniconda3/lib/python3.9/site-packages/transformers/data/data_collator.py", line 42, in __call__
    return self.torch_call(features)
  File "/home/zachmartinez/miniconda3/lib/python3.9/site-packages/transformers/data/data_collator.py", line 732, in torch_call
    "input_ids": _torch_collate_batch(examples, self.tokenizer, pad_to_multiple_of=self.pad_to_multiple_of)
  File "/home/zachmartinez/miniconda3/lib/python3.9/site-packages/transformers/data/data_collator.py", line 404, in _torch_collate_batch
    length_of_first = examples[0].size(0)
AttributeError: 'tokenizers.Encoding' object has no attribute 'size'
```





### Expected behavior

I would expect that the collator would accept the tokenizer output and perform the collating. Any help would be appreciated.

Comments:
The data collator will expect a list of samples from a torch Dataset, but you are passing a single dictionary to it (the output of the tokenizer is one dictionary, with the keys being the arguments expected by the models like `input_ids`, and the values being tensors).

You can actually do directly `model(**tokens)`, you don't need the data collator on this example.

Thank you very much for your quick response! I'm simply trying to fine-tune the model with CLM, so I figured I needed to use the datacollator.
Trainer RuntimeError CUDA error
Author: devozs
Pull request: no

### System Info

### Versions
i`ve tried using
- transformers==4.15.0, 4.8.0 and latest
- python 3.8, 3.9 and 3.10

### nvidia-smi
NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7

Thanks!

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

### Actual Error:
Class: `transformers/trainer.py`
Failing at: `tr_loss = torch.tensor(0.0).to(args.device)`
Returns: `RuntimeError: CUDA error: invalid argument`

### Code
I am following several examples and getting the same above error and the same line.
As a reference you can reproduce it according to this [code sample](https://github.com/dredwardhyde/gpt-neo-fine-tuning-example/blob/main/gpt_neo.py)
The exact same error occur with other huggingface training examples.

### Also tried
- if i am running the line `tr_loss = torch.tensor(0.0).to(args.device)` as a standalone its working fine
- Also tried to run this line as part of gpt_neo.py in the above example and it worked fine but later failed as part of `transformers/trainer.py`
- I made sure the CUDA is running fine:
 `torch.cuda.is_available()`
- Running only `torch.tensor(0.0)` works fine, only when adding `.to(device)` its failing

### Expected behavior

No errors at `torch.tensor(0.0).to(device)`

Comments:
This looks linked to your particular setup. Can you add a print of `args.device` in the script you are running and copy-paste the result of `transformers-cli env` (as was requested in the template)?

thanks for the prompt reply and sorry for missing the `transformers-cli env`

`args.device` cuda:0

Env:
- transformers version: 4.23.1
- Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.35
- Python version: 3.8.15
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.12.1+cu116 (True)
- Tensorflow version (GPU?): 2.10.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.1 (cpu)
- Jax version: 0.3.23
- JaxLib version: 0.3.22
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

(i`ve also tried with other PyTorch & transformers versions)

i hope its ok that i putting a link to different ML library (in case its not - i`ll delete it)
[this issue seems to be similar](https://github.com/Lightning-AI/lightning/issues/11818#issuecomment-1033541994)

It doesn't look exactly similar in the sense that it is in an environment without a GPU, whereas yours shows one. Unless you are not executing the script within the exact same env as the results of the commands passed above of course.
Add ESMFold
Author: Rocketknight1
Pull request: yes

cc @sgugger @LysandreJik @tomsercu @rmrao @nikitos9000

Opening a draft PR because deadlines are getting tight and I'd like to get everyone on the same page!

What's done:
- [X] Create a minimal port of `openfold`
- [X] Port ESMFold as `EsmForProteinFolding`
- [X] Update weight conversion scripts to port ESMFold weights from original repo
- [X] Update config formats to support ESMFold models

TODO:
- [x] Resolve small output discrepancies in ESM-2 stem that cause differences in final protein predictions
- [x] Add documentation
- [x] Add testing
- [x] Ensure everything is importable from the `transformers` root 
- [x] ~Add an auto class for protein folding?~
- [x] Ensure non-folding ESM classes can be loaded with AutoModel
- [x] Remove some `openfold` functions/methods that aren't being called
- [x] Clean up the `openfold` port into a single dir/file
- [x] Ensure all `openfold` code is correctly licenced
- [x] ~Add auxiliary method(s) to convert the outputs into bio file formats like `pdb`~
- [ ] Reupload ESM checkpoints with the new formats
- [x] Upload ESMFold_v1 checkpoint

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Merging for now, there are still a few improvements needed (example in a docstring for instance) but they can go in their own PRs :-)
Pin torch to < 1.13 temporarily
Author: ydshieh
Pull request: yes

# What does this PR do?

Pin torch to < 1.13 temporarily, as it causes strange failures (segmentation fault) on CircleCI.

Evidence could be found

with torch 1.12.1 
https://app.circleci.com/pipelines/github/huggingface/transformers/50619/workflows/5b665ba2-5f45-4b61-9e08-de6c8a2349cd

with torch 1.13.0
https://app.circleci.com/pipelines/github/huggingface/transformers/50621/workflows/8a137f60-2e66-48fd-aeb6-1a8d49369d4c

Comments:
Failed test is irrelevant. Merge now.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19989). All of your documentation changes will be reflected on that endpoint.
Unpin PyTorch to test if doc builds
Author: sgugger
Pull request: yes

# What does this PR do?

The doc building seg-faults since we pinned PyTorch. Using this PR to experiment.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The tests fail from the beginning with `Fatal Python error: Segmentation fault`, and no failed tests being collected/reported.
**We still need to pin torch.** 

Doc build jobs pass now after the docker image being re-built last night (my best guess for the reason)
Some weights of BertForPreTraining were not initialized from the model checkpoint 
Author: dsaban
Pull request: no

### System Info

Some weights of BertForPreTraining were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['cls.predictions.decoder.bias']

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

run script.


### Expected behavior

load BertForPreTraining

Comments:
Please use the [forums](https://discuss.huggingface.co/) to ask questions or fill the template to report a bug. Getting a warning because the checkpoint you selected does not contain all weights for your architecture is not, by itself, a bug.

Yes, `bert-base-uncased` only includes the weights and bias of the language modeling head, but not the next sentence prediction task, which is what the warning is telling you. In other words, it corresponds to the `BertForMaskedLM` model.

Therefore closing this issue, feel free to reopen.
Fix dataset in quicktour
Author: stevhliu
Pull request: yes
Labels: name: Documentation

This PR adds a dataset in the `Trainer` section of the Quicktour so users can successfully run the code samples (from forum feedback [here](https://discuss.huggingface.co/t/trainer-a-pytorch-optimized-training-loop-example-code/25163)).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add BERT resources
Author: stevhliu
Pull request: yes

This PR kicks off #19848 and adds official resources for BERT to the BERT model doc page. The resources are grouped by the types of tasks you can use the model for as well as using it for some other applications like inference and deployment.

If possible, I think it'd also be cool to use the task icons we use on the Hub and Tasks page (see below for example). What do you think?

![Screen Shot 2022-10-24 at 1 26 25 PM](https://user-images.githubusercontent.com/59462357/197622917-a9978857-e0b4-4d42-b066-a4b44574c792.png)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I really like this!

Awesome, I'll start on some of the other models then! In the meantime, I'll check with @mishig25 if we can use the icons in the docs :)

We don't have an icon for multiple choice, so would it be ok to add it under question answering (since its kind of a variant of question answering)?

It's more a variant of sequence classification technically.

Ah ok! Would it be too confusing to include it under sequence classification then? We can also just leave it as is 😄
Generate: contrastive search with full optional outputs
Author: gante
Pull request: yes

# What does this PR do?

This PR further massages PT's `contrastive_search` in advance of the conversion to TF. It does the following modifications:
1. Pipes additional outputs that were missing (e.g. when `output_attentions` is `True`)
2. Rewrites part of the input replication to share logic with beam search -- replicating the input for `top_k` candidates is the same as replicating the input for `num_beams`
3. Removes additional redundant/unused operations
4. Because we now have all outputs (see 1), adds the standard suite of tests for a generation method
5. Moves integration tests to the corresponding model folder

All tests passing locally (`RUN_SLOW=1 py.test tests/* -k contrastive -vv`)

After this PR, we can start with the TF conversion.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update glossary
Author: stevhliu
Pull request: yes
Labels: name: Documentation

This PR adds some more computer vision and speech terms (feel free to suggest more!) and reorganizes it alphabetically (and each term can be linked to) so it's more like an actual glossary. I also edited some of the terms, like `attention mask` for length, since a glossary typically just provides a brief definition. If we want to keep the explanations, maybe I can link to the course instead.

Comments:
I cannot review this PR as it is. You have changed the text for the existing entries and removed the links to the YouTube videos and since you did it with the reorganization, the diff makes it impossible to properly comment.
You have also removed existing entries like autoencoder model or autoregressive model.

Please focus on one thing at a time per PR (I have already told you this multiple times). For instance here, focus first on the reorg **with no other changes in existing content** and then in a second PR we can discuss text changes.

Sorry, I'll try to slow down and just focus on one thing at a time!

I'll close this PR and open two separate ones to address the reorganization and new terms.
Update object detection pipeline to use post_process_object_detection methods
Author: alaradirik
Pull request: yes

# What does this PR do?
Updates the `ObjectDetectionPipeline` to use the `XXXFeatureExtractor.post_process_object_detection` methods instead of the deprecated `XXXFeatureExtractor.post_process` methods.

Postprocessing methods have been updated recently with this [PR](https://github.com/huggingface/transformers/pull/19709).

Partially fixes the hardcoded threshold issue with the inference widgets, which requires adding a threshold button to the widgets.
Fixes # 414


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?



Comments:
_The documentation is not available anymore as the PR was closed or merged._