noamaanMulla-03/transformers-issues huggingface dataset:
Run Vit-MAE script key error
Author: RobertHua96
Pull request: no

### System Info

- `transformers` version: 4.23.1
- Platform: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.15
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): 2.9.2 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@NielsRogge @sg

### Information

- [x] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I've put down the main parts of the run MAE script [https://github.com/huggingface/transformers/blob/main/examples/pytorch/image-pretraining/run_mae.py]
into a colab notebook: 
[https://colab.research.google.com/drive/1WtOTp-ocbBTgVXFiEXuY8MWhz_ex9OBy?usp=sharing]
When I reach the trainer.train() step I get a key error from the data collator function - is the script perhaps outdated, or am I doing something wrong?

### Expected behavior

Trainer starts training

Comments:
You'll need to add `remove_unused_columns=False` to the `TrainingArguments`.

The reason is because of the use of `set_transform` when preparing the datasets, which does things on-the-fly. Hence we still need to `image` column in the datasets to turn them into `pixel_values`.

Amazing thank you!
Enable ONNX export for `VisionDecoderEncoderModel`
Author: lewtun
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

As discussed on the [forums](https://discuss.huggingface.co/t/can-i-export-a-visionencoderdecoder-checkpoint-to-onnx/12885), it would be nice if one could export `VisionDecoderEncoderModel` classes using the `transformers.onnx` package.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

It is currently not possible to export `VisionDecoderEncoderModel` classes (or vision models more generally), unless the end user is willing to write their own `OnnxConfig`. It might make sense to first explore what is involved in exporting ViT before tackling this more complex example. 

Looking at this a bit more closely, I can see that `transformers.onnx` currently has a tight integration with tokenizers (e.g. to generate dummy inputs), so some refactoring will be necessary to support other modalities.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I believe progress on this ticket will be welcome by the communitiy.

Hey @albertoandreottiATgmail thanks for the interest! This is next up on my list now that we've just merged the ViT export in #15658 :)



> Hey @albertoandreottiATgmail thanks for the interest! This is next up on my list now that we've just merged the ViT export in #15658 :)

Hi @lewtun is there any way (or script) to export trocr to onnx ?

Hi all, any updates on this? I've also tried to convert TrOCR to ONNX without success. Thanks in advance :) cc @lewtun 

Re-opening this issue as it hasn't been resolved yet

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Adding another feature request for this: https://github.com/NielsRogge/Transformers-Tutorials/issues/183

> Adding another feature request for this: [NielsRogge/Transformers-Tutorials#183](https://github.com/NielsRogge/Transformers-Tutorials/issues/183)

Any updates on this one?

Thank you

Hi folks, @mht-sharma will be tackling this issue - stay tuned :)

@mht-sharma Any updates?

Thank you.

@BakingBrains you can see the PR from @mht-sharma linked above - we should be able to get this merged pretty soon!

@lewtun and @mht-sharma Thank you.
Padding in GPT-2  
Author: Oxi84
Pull request: no
Labels: name: wontfix

How do I add padding in GTP2?
 I get something like this when I add zero in front to pad the sequences, but then I found out that 0 is actually not "[PAD]" but "!".
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 1639, 481]

The zeros change the result quite a lot, not that it totally ruins it, but it makes the results less precise and usually order of the most frequent predicted words is altered.
So how to we add padding there.


I have tried to follow docs,  but I didn't find a way to add  something analog to attention mask BERT.


 





Comments:
No padding implemented in GPT-2, you have to add implement your-self if you want e.g. by adding a special token but note that:
- GPT-2 doesn't like left side padding (doesn't mix well with a causal transformer having absolute positions)
- right-side padding is often not necessary (the causal mask means that right context is ignored anyway).

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


> * GPT-2 doesn't like left side padding (doesn't mix well with a causal transformer having absolute positions)

To those stumbling on this issue, this doesn't seem to be a problem anymore. [#3021](https://github.com/huggingface/transformers/issues/3021#issuecomment-1232149031)
Generate: minor docstring fix
Author: gante
Pull request: yes

# What does this PR do?

Fixes this:
<img width="816" alt="Screenshot 2022-10-21 at 22 12 40" src="https://user-images.githubusercontent.com/12240844/197289701-0092c00a-2aec-4c72-96ba-fb2df4ce14b1.png">


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Fixed
<img width="834" alt="Screenshot 2022-10-21 at 22 28 30" src="https://user-images.githubusercontent.com/12240844/197291473-5c5c8984-d553-4e7c-85a3-94a254e452f8.png">
Increased Memory Consumption In Containers
Author: ai-john
Pull request: no
Labels: name: bug

### System Info

Transformers version: 4.16.2
Python version: 3.8
Pytorch version: 1.11.0
CUDA version: 11.7
Docker version: 20.10.17

### Reproduction

Hello!

We noticed an interesting issue.

Currently, we have a monolithic app with 2 PyTorch models - model A and model B(both GPT based models).

If we run the app with only Model A enabled - it consumes 2.5 GB GPU.
If we run the app with only Model B enabled - it consumes 2.2 GB GPU.
If we run the app with Model A and Model B together - memory consumption is less than model A + model B launched separately.

At the same time, if we divide the monolithic app into 2 smaller apps (Model A in container A, model B in container B) and run it via Docker, we see that GPU memory consumption is higher than if we run it as a monolithic single app. It literally becomes 4.7 GB (2.5 GB + 2.2 GB).

It seems that PyTorch reserves some GPU for itself. So, GPU memory consumption in a multi-services setup is higher than in a monolithic.
Is is a bug or is it a planned behaviour?

Could you, please, give us some hints if there are any ways to optimize memory consumption for PyTorch and if it is possible to share PyTorch reserved memory between different Docker containers?

Thanks!

### Expected behavior

Memory consumption in containers and in monolithic app should be the same

Comments:
Hi @ai-john  👋 As per our [issues guidelines](https://github.com/huggingface/transformers/blob/main/ISSUES.md), we reserve GitHub issues for bugs in the repository and/or feature requests. For any other matters, we'd like to invite you to use our [forum](https://discuss.huggingface.co/) 🤗

As you wrote, ML frameworks do allocate some GPU memory for themselves :) Hugging Face has deployment optimization solutions, see [Optimum](https://huggingface.co/docs/optimum/index)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
AttributeError: 'TrainingArguments' object has no attribute 'main_process_first'
Author: ghost
Pull request: no
Labels: name: bug

### System Info

AttributeError: 'TrainingArguments' object has no attribute 'main_process_first'

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

AttributeError: 'TrainingArguments' object has no attribute 'main_process_first'

### Expected behavior

AttributeError: 'TrainingArguments' object has no attribute 'main_process_first'

Comments:
Hey @xueyongfu, could you provide a minimal reproducible code example for the issue you're facing? Thank you.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Small typo in docs/README.md
Author: ankrgyl
Pull request: no
Labels: name: bug

### System Info

Working on main (commit 9129fd0377e4d46cb2d0ea28dc1eb91a15f65b77).

The suggested command:

```
doc-builder build transformers docs/source/ --build_dir ~/tmp/test-build
```

fails with:

```
FileNotFoundError: [Errno 2] No such file or directory: 'docs/source/_toctree.yml'
```

I think is because you have to specify a language (e.g. `en`) while building the docs, e.g.

```
doc-builder build transformers docs/source/en/ --build_dir ~/tmp/test-build
```

I'm happy to contribute a fix.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the following command:

```
doc-builder build transformers docs/source/ --build_dir ~/tmp/test-build
```

### Expected behavior

The command should _not_ error, and instead should generate the expected MDX files.

Comments:
Hey @ankrgyl! It seems that is correct! Would you like to open a PR to fix the problem? Happy to guide you through it if you need pointers; please ping @sgugger and myself on the PR. Thanks!

Hi, `CONTRIBUTING.md` also requires a language such as "en" to be specified in the doc-builder. @ankrgyl @LysandreJik 

> Hi, `CONTRIBUTING.md` also requires a language such as "en" to be specified in the doc-builder. @ankrgyl @LysandreJik

This typo has been fixed, sorry I missed the latest version of `CONTRIBUTING.md`.
[Doctest] Add `configuration_electra.py`
Author: ayaka14732
Pull request: yes

# What does this PR do?

Add `configuration_electra.py` to `utils/documentation_tests.txt` for doctest.

Based on issue https://github.com/huggingface/transformers/issues/19487

@sgugger could you please take a look at it?

Thanks =)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctest] Add `configuration_poolformer.py`
Author: ayaka14732
Pull request: yes

# What does this PR do?

Add `configuration_poolformer.py` to `utils/documentation_tests.txt` for doctest.

Based on issue https://github.com/huggingface/transformers/issues/19487

@sgugger could you please take a look at it?

Thanks =)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctest] Add `configuration_plbart.py`
Author: ayaka14732
Pull request: yes

# What does this PR do?

Add `configuration_plbart.py` to `utils/documentation_tests.txt` for doctest.

Additionally, I updated its doctest format to make it consistent with BERT.

Based on issue https://github.com/huggingface/transformers/issues/19487

@sgugger could you please take a look at it?

Thanks =)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctest] Add `configuration_nezha.py`
Author: ayaka14732
Pull request: yes

# What does this PR do?

Add `configuration_nezha.py` to `utils/documentation_tests.txt` for doctest.

Additionally, I updated its doctest format to make it consistent with BERT.

Based on issue https://github.com/huggingface/transformers/issues/19487

@sgugger could you please take a look at it?

Thanks =)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
transformers.data.metrics: replace mention of 🤗 Datasets in DEPRECATION_WARNING with 🤗 Evaluate
Author: angus-lherrou
Pull request: yes

# What does this PR do?

Changes DEPRECATION_WARNING in `src/transformers/data/metrics/__init__.py` to point to 🤗 Evaluate for metrics functionality instead of 🤗 Datasets, whose metrics functionality has been deprecated and moved to 🤗 Evaluate since the metrics in this file were deprecated in 🤗 Transformers


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 
@albertvillanova 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Not sure what's causing that test failure.
Fix OOM in config doctest
Author: ydshieh
Pull request: yes

# What does this PR do?

This model is too large to be tested (OOM).
```python
    >>> # Initializing a model (with random weights) from the gpt-neox-20b style configuration
    >>> model = GPTNeoXModel(configuration)  # doctest: +SKIP
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
fix broken links in testing.mdx
Author: xffxff
Pull request: yes

# What does this PR do?

fix a broken link in https://huggingface.co/docs/transformers/main/en/testing

I also found the link I marked out in the picture below is also broken, but I don't know how to fix it. You can go to "How transformers are tested" from [here](https://huggingface.co/docs/transformers/main/en/testing#how-transformers-are-tested)

![image](https://user-images.githubusercontent.com/30254428/197367017-d0aa691d-54cf-48f9-a4f4-89ec693396d6.png)



## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix nightly CircleCI
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix a few nightly CircleCI issues

The effect could be found [here](https://app.circleci.com/pipelines/github/huggingface/transformers/50144/workflows/aaa65460-bbb5-4048-af3e-9450af06e231).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger I accidentally push `GitPython` to `main`

https://github.com/huggingface/transformers/commit/6f8064da6b0c8f003731f292acc64f281a2aea65

😨 

Ah so this is done, perfect!
Update expected values
Author: ydshieh
Pull request: yes

# What does this PR do?

PR #19654 changed some string literals in `LEDModelIntegrationTests.test_seq_to_seq_generation` (Use of `r"""`), which gives different outputs. I think the inputs are different with/without `r"""`, so I just update the expect values in this PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
How to find the accuracy of the generated questions from the text ?
Author: Roshni3499
Pull request: no

I am generating a 4 types of questions. 1. WH (what, where, who, which etc.) 2. Boolean type 3. Fill in blanks 4. MCQ.

I am able to find the accuracy of the WH & MCQ type question through the code given below.

```
question= list of question
context = paragraph

from transformers import pipeline

question_answerer = pipeline("question-answering")
question_score = []
for i in question:
      final_score_list = question_answerer(question=i, context=context)
      final_score = final_score_list.get("score")
      final_score ="%.2f" % round(final_score*100, 2)+"%"
      question_score.append(final_score)

Output = 98.87%
```

As this model find the accuracy based on question and context, I am getting accuracy for WH & MCQ. 
but there is no question in Fill in blanks type question , I am not able to find the accuracy for the fill in blanks type question. and also it not working for the Boolean questions. 

So, is there any other model or any other way for find the accuracy of the FIB & Boolean type questions ? 

Here is the reference link for the question generation code.
1. WH = [https://medium.com/featurepreneur/question-generator-d21265c0648f](https://medium.com/featurepreneur/question-generator-d21265c0648f)
2. MCQ = [https://github.com/AMontgomerie/question_generator/blob/master/examples/question_generation_example.ipynb](https://github.com/AMontgomerie/question_generator/blob/master/examples/question_generation_example.ipynb)
3. FIB = [https://github.com/sudheernaidu53/Machine-learning-Deep-learning-projects](https://github.com/sudheernaidu53/Machine-learning-Deep-learning-projects)
4. Boolean = [https://github.com/ramsrigouthamg/generate_boolean_questions_using_T5_transformer/blob/master/t5_inference.py](https://github.com/ramsrigouthamg/generate_boolean_questions_using_T5_transformer/blob/master/t5_inference.py)

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix doctest for MarkupLM
Author: ydshieh
Pull request: yes

# What does this PR do?

The CI complains
```
UNEXPECTED EXCEPTION: SyntaxError('EOF while scanning triple-quoted string literal', ('<doctest markuplm.mdx[2]>', 7, 96, 'html_string = """\n <!DOCTYPE html>\n <html>\n <head>\n <title>Hello world</title>\n </head>\n <body>\n'))
```
due to the lack of `...` in some empty lines. But `make style` will remove those `...` after I add it to those lines.

So I just removed those empty lines to pass the doctest.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Small update to model addition guide
Author: stevhliu
Pull request: yes
Labels: name: Documentation

This PR is a smaller version of #19778 (shelved for now) which includes only the more important fixes for maintaining accuracy:

* remove *call-for-model-addition* program
* `cookiecutter` adds a `mdx` instead of a `rst` file
* fix to do list so it doesn't have numbers and bullets

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Refactor conversion function
Author: sgugger
Pull request: yes

# What does this PR do?

This PR introduces a refactor of the conversion functions to be able to re-use with safetensors. It comes with zero change of code inside but the main takeaway is to build two functions that take a PyTorch-formatted (resp. TF-formatted) state dict and load it in a TF (resp. PyTorch) model. The state dict is just a dictionary name to tensor, and the tensor car be a NumPy array (as it was) or a torch/tf tensor (which it will be with safetensors).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
would huggingface  like  support cpp env  libtorch or rewrite the core code for cpp ?
Author: mullerhai
Pull request: no
Labels: name: bug

### System Info

libtorch 1.21
macos 
Hi:  
      I  want to  use huggingface ,but ,I find the model  api  almost python api, I use the cpp env for  libtorch , would you like support cpp ,or rewrite the code with cpp?
thanks 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

no

### Expected behavior

no

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add TF VideoMAE
Author: sayakpaul
Pull request: no
Labels: name: WIP

### Feature request

Add the [VideoMAE](https://huggingface.co/docs/transformers/main/en/model_doc/videomae) model in TensorFlow.

### Motivation

There's an evident scarcity of SoTA and easy-to-use video models in TensorFlow. I believe having VideoMAE in TensorFlow would greatly benefit the community. 

### Your contribution

I am willing to contribute the model. Please assign it to me. 

@amyeroberts possible to assign this to me?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Please reopen the issue as I am working on it. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

It's taking more time as I am in a job switch. Please reopen it. I apologise for the inconvenience.