noamaanMulla-03/transformers-issues huggingface dataset:
Map model_type and doc pages names
Author: sgugger
Pull request: yes

# What does this PR do?

As discussed on [moon-landing](https://github.com/huggingface/moon-landing/pull/1692#discussion_r774957604), it's not possible to automatically generate the link to the documentation from the configuration of a model, due to multiple inconsistencies between model type and doc page name.

This PR fixes that and adds a script to make sure we have a model type for every documented model (for instance currently we have a mising model type for models having a tokenizer only, like Wav2Vec2-Phoeneme and they don't appear on the AutoTokenizer doc as a result) and that the doc page name matches the model type.

It breaks some links to the doc, but we (I and Julien) think this is acceptable.

Comments:
very neat!

Sounds good to me. Will resolve conflicts and merge.
Make OpenAIGPTTokenizer work with SpaCy 2.x and 3.x
Author: cody-moveworks
Pull request: yes

SpaCy 3.x introduced an API change to creating the tokenizer that
breaks OpenAIGPTTokenizer. The old API for creating the tokenizer in
SpaCy 2.x no longer works under SpaCy 3.x, but the new API for creating
the tokenizer in SpaCy 3.x DOES work under SpaCy 2.x. Switching to the
new API should allow OpenAIGPTTokenizer to work under both SpaCy 2.x and
SpaCy 3.x versions.

Fixes https://github.com/huggingface/transformers/issues/14449



Comments:
I'm not able to test the changes locally. When I run `pip install -e ".[dev]"`, I encounter a build error and the install fails. It doesn't look like pytest coverage is sufficient here either since SpaCy and ftfy are not required by transformers, and the code can execute correctly without them being installed.

@patrickvonplaten @LysandreJik Not sure why but I can't tag you as reviewers on GitHub, so tagging you in comments here.

Yes, why not! GPT-2 is among our most used models, so I think testing that the tokenization behaves correctly across all possibilities is important. Would you like to take a stab at it @cody-moveworks?

In order to do so, you could start by adding an `is_spacy_available` method in `file_utils.py`, analog to other methods such as `is_vision_available` here: https://github.com/huggingface/transformers/blob/2e9af294940083915ccb2740a7c8d5b154194f15/src/transformers/file_utils.py#L507-L508

Then it would require defining a `require_spacy` unittest decorator in `testing_utils.py`, such as the `require_vision` here: https://github.com/huggingface/transformers/blob/2e9af294940083915ccb2740a7c8d5b154194f15/src/transformers/testing_utils.py#L404-L412

Thirdly, you can add a test in the `tests/test_tokenization_gpt2.py` file with the `@require_spacy` decorator, which will only run when SpaCy is installed.

And finally, we can modify this CircleCI run: https://github.com/huggingface/transformers/blob/2e9af294940083915ccb2740a7c8d5b154194f15/.circleci/config.yml#L538-L565

So that it:
1. Installs SpaCy
2. Runs the tokenization GPT-2 test file!

@LysandreJik Thanks for the detailed walkthrough of the changes needed to add testing. I'll take a stab at it.

@LysandreJik I made the changes and all checks are passing. Can you take a look?
[Fix doc example] RagModel
Author: ydshieh
Pull request: yes

# What does this PR do?

This part fails
https://github.com/huggingface/transformers/blob/623b4f7c63f60cce917677ee704d6c93ee960b4b/src/transformers/models/rag/modeling_rag.py#L308-L310

Change to `from_pretrained_question_encoder_generator` instead in this PR.

## Who can review?

@patrickvonplaten 

Comments:
Bigger batch size decrease the throughput.
Author: dancingpipi
Pull request: no

I'm training bert-base-uncased from scratch, single node, 8xA100, if increase the batch size, the throughput decrease, detail is:

- set batch size to 256, the throughput is 7000 sample/s, GPU Utilization rate 80%
- set batch size to 384(while full gpu memory), the throughput is 4000 sample/s, GPU Utilization rate 30%

It look like the IO problem(or maybe not, you can correct me)?
Any help will be appreciated~ txs

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Got extra_id_%d when generating texts, and %d is negative.
Author: Cthericy
Pull request: no

Mission: Translate Graph to English.
In generation_utils.py   _generate_beam_search()
After getting next_scores and next_tokens, i was tring to generate texts using the next_tokens with the second highest next_scores.
But i got ids which converts to be <extra_id_%d>, and %d is a negative number.
Anyone got some ideas?
Thanks.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[doc] normalize HF Transformers string
Author: stas00
Pull request: yes

I noticed we use  🤗 Transformer instead of 🤗 Transformers in a couple of places, this PR fixes it.

That was a tricky one with UTF32 emoji. This worked.

```
LC_ALL=C find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's|(\xF0\x9F\xA4\x97 Transformer) |$1s |' {} \;
```
To grep I had to do:

```
LC_ALL=C grep -Ir -P '\xF0\x9F\xA4\x97 Transformer ' .
```

@sgugger 


Comments:
Unable to match GPT-2 reported perplexity results on CBT, Wikitext-103, and 1BW datasets and doubt on LAMBADA accuracy
Author: samzabdiel
Pull request: no

I am having the following discrepancies on trying to reproduce the results of GPT-2 small (117M parameters).

On using this huggingface [script for perplexity](https://huggingface.co/transformers/perplexity.html). I am able to match reported perplexity on Wikitext-2 (reported - 29.41, reproduced 29.94). But it also gives the same perplexity on Wikitext-103. Likewise [EleutherAI's language model evaluation harness](https://github.com/EleutherAI/lm-evaluation-harness) gives perplexity that matches Wikitext-103's reported perplexity (reported 37.50, reproduced - 37.37) and gives the same result for Wikitext-2. The methods give same perplexity to Wikitext-2 as well as Wikitext-103 as the test set for the datasets are the same. The huggingface method uses the actual token count and the EleutherAI one uses word count to calculate perplexity. Is the assumption correct that the different perplexities are simply due to the different ways they are computed?

I am also unable to match Children's Book Test (CBT) dataset tasks CBT-CN and CBT-NE accuracies of 87.65 and 83.4. I am able to get up to 84.35 and 73.65 only.

Getting accuracy of 46.67 on LAMBADA dataset for predicting the last token instead of the last word. Is this way correct?

Getting perplexity of 45.88 using token count and 89.35 using word count on one billion words (1BW) dataset instead of the reported 75.20.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

hi @samzabdiel, did you ever find what was going on? 

Not completely, but I was able to closely match accuracies reported on CBT datasets using the appropriate preprocessing. The results seem to be tightly coupled with the preprocessing done.
Take gradient accumulation into account when defining samplers
Author: sgugger
Pull request: yes

# What does this PR do?

This PR takes the gradient accumulation steps into account when defining samplers that use the batch size (like the `LengthGroupedSampler`) so that training with large batch size or training with smaller batch size and gradient accumulation (e.g. batch size 64 or batch size 8 and gradient accumulation steps of 8) yield the same results.

Fixes #14638 

Comments:
[Fix doc example] Speech2TextForConditionalGeneration
Author: ydshieh
Pull request: yes

# What does this PR do?

This fails for `Speech2TextForConditionalGeneration`
```
>>> generated_ids = model.generate(input_ids=input_features)
```
I changed it to `model.generate(inputs=input_features)`.

## Who can review?

@patrickvonplaten 

Comments:
Adds IBERT to models exportable with ONNX
Author: MaximovaIrina
Pull request: yes

Adds IBERT to models exportable with ONNX

Comments:
Hey @MaximovaIrina we recently switched all of our documentation from RST to MDX, so you can rebase on `master` to eliminate the conflict with `docs/source/serialization.rst`

Checking slow tests for ibert 
![image](https://user-images.githubusercontent.com/36035908/147254715-5affc916-e2c8-4984-acdb-67c5732f1a53.png)


Hey @MaximovaIrina would you mind rebasing your branch on `master` and resolving the merge conflicts?

Gently pinging @LysandreJik for his blessing on this PR too :)

Could you just run the code quality tool to ensure that the code quality passes? You can install them with the following, from the root of your clone:
```
pip install -e ".[quality]"
```
And then run them with:
```
make fixup
```

Merging this since all the CI checks now pass and we have approval from a core maintainer. Thank you for your contribution @MaximovaIrina 🤗 !
[Quick poll] Give your opinion on the future of the Hugging Face Open Source ecosystem!
Author: LysandreJik
Pull request: no

Thanks to all of you, Transformers just passed 55k :star2:  this week!

Since the last survey, we've expanded our efforts into vision and speech, and the community now actively contributes to the [model hub](https://hf.co/models) and [datasets hub](https://hf.co/datasets); so this survey isn't solely focused around `transformers`, but around the entire HF ecosystem!

If you have a couple of minutes and want to participate in shaping the future of the ecosystem, please share your thoughts: 

[**hf.co/oss-survey**](https://hf.co/oss-survey)

(please reply in the above feedback form rather than to this thread)

Thank you all on behalf of the HuggingFace team! 🤗

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
change metric_key_prefix in seq2seq_trainer.py
Author: JejuWayfarer
Pull request: yes


# What does this PR do?

In the trainer, the metric_key_prefix of the predict is set as "test", and in the seq2seq trainer, it is set as "eval", so the same result does not come out.
To solve this problem, the metric_key_prefix for the seq2seq trainer was modified to "test".

## Who can review?

 --> trainer: @sgugger


Comments:
GPT-J Tokenizer model_max_length=1024 despite n_positions=2048
Author: oborchers
Pull request: no

On  the hub the `tokenizer_config.json` states, that GPT-J has a `"model_max_length": 1024`, despite the fact that it can generate up to `2048` tokens, as per `"n_positions": 2048` according to the config.

Is this intended? 

Tagging due to knowledge of the original issue:
@StellaAthena @EricHallahan @patrickvonplaten 


Comments:
This sounds like its a typo. If you change it to 2048, does the model run without issue?

Yes, that’s why I raised the issue 👍🏻

Hi, good catch! This is because GPT-J uses `GPT2Tokenizer`which has `model_max_length` set to 1024.  Here we could directly update`tokenizer_config.json` file on the hub.

@StellaAthena would it be okay if I update the file in the `tokenizer_config` repo with this change?

@patil-suraj I made the change :)

Thanks a lot!
Add Nystromformer
Author: novice03
Pull request: yes

# What does this PR do?

This PR adds the Nystromformer transformer model to the repository.

Paper: [https://arxiv.org/abs/2102.03902](https://arxiv.org/abs/2102.03902)
Code: [https://github.com/mlpen/Nystromformer](https://github.com/mlpen/Nystromformer)
Checkpoint: [Nystromformer sequence length 512](https://www.dropbox.com/s/8uv4f6q52oaqwkh/Nystromformer.model?dl=0)


## Who can review?

@NielsRogge 


Comments:
Hey @novice03 , really cool PR, can't wait to fine-tune models with it :)

I have some questions about the tokenization part: it seems that the ALBERT tokenizer (and an albert spm model) is used and uploaded to the model hub, so I think in `config.json` the entry `"tokenizer_class": "AlbertTokenizer"` should be added so that `AutoTokenizer` is working.

But I've looked at the original pre-processing and [pre-training code](https://github.com/mlpen/Nystromformer/blob/main/code/run_pretrain.py#L42) and it seems that the RoBERTa tokenizer is used for pre-training a Nyströmformer model, why is an albert spm model used here :thinking: 



Hello, thanks for taking a look at this PR. I've added the tokenizer entry to the config.json file. It is true that the code seems to use RoBERTa tokenizer. However, the model checkpoint released is of a model with `vocab_size=30,000`. Furthermore, they've also used albert tokenizer in their [code](https://github.com/mlpen/Nystromformer/blob/6539b895fa5f798ea0509d19f336d4be787b5708/reorganized_code/BERT/dataset.py#L25). I've tried using `BertTokenizer` and `RobertaTokenizer`, but they give errors or incorrect results. The checkpoint released by the author only works with this specific albert tokenizer configuration.



The test_masked_lm_end_to_end fails since AutoTokenizer doesn't encode the mask token correctly. The test passed when I use AlbertTokenizer instead. The other 2 slow tests pass.

Very cool to have this new model @novice03 :hugs: .

Let me share with you a little information about the tokenizers of `uw-madison/nystromformer-512` on this PR.  

@NielsRogge showed me that there is currently an inconsistency in the treatment of special tokens with the slow and fast versions of `uw-madison/nystromformer-512` tokenizers. 

I think that to generate into the `slow_and_fast` folder the files for the fast tokenizer, we should do:
```python
from transformers import AlbertTokenizer

tokenizer = AlbertTokenizer.from_pretrained("uw-madison/nystromformer-512")
tokenizer.save_pretrained("slow_tokenizer")

fast_tokenizer = AlbertTokenizerFast.from_pretrained(
        "slow_tokenizer", 
        bos_token="[CLS]",
        eos_token="[SEP]",
        unk_token="<unk>",
        sep_token="[SEP]",
        pad_token="<pad>",
        cls_token="[CLS]",
        mask_token="[MASK]"
        )

fast_tokenizer.save_pretrained("slow_and_fast")
```

I hope this is helpful!
bert onnx to tensorrt wrong
Author: WallE-Chang
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: ubuntu 18
- Python version: 4.11.3
- PyTorch version (GPU?):1.8.0 gpu
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:

I get bert onnx ckpt by
`python -m transformers.onnx --model=bert-base-cased onnx/bert-base-cased/`

Then, I want to convert onnx to tensorrt 
`TensorRT-8.2.0.6/bin/trtexec --verbose \
                             --workspace=20000 \
                             --minShapes="input_ids":1x1,'attention_mask':1x1,'token_type_ids':1x1 \
                             --optShapes="input_ids":4x64,"attention_mask":4x64,"token_type_ids":4x64 \
                             --maxShapes="input_ids":8x128,"attention_mask":8x128,"token_type_ids":8x128 \
                             --onnx=./onnx/bert-base-cased/model.onnx \
                             --saveEngine=./trt/bert-base-cased/model.trt \
                             --fp16`

However, I get error

![image](https://user-images.githubusercontent.com/22662737/138287788-8ff5d519-1ef9-40e1-9bd8-6411ff415a14.png)

Can you give me some help ？


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I am currently working on that there: https://github.com/ELS-RD/transformer-deploy
TBH, it's not a simple problem, but it brings lots of performance improvement compared to vanilla pytorch or even CUDA provider on ONNX Runtime

Hey @WallE-Chang this issue would be better suited for the `optimum` [library](https://github.com/huggingface/optimum), which is more focused on these framework-specific aspects. Would you mind opening your issue there?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Issues with FNet
Author: gchhablani
Pull request: no

This is a tracking issue for problems/enhancements needed with FNet.

So far, @stefan-it @ontocord have mentioned that:
* [ ] `--fp16` precision throws an error with the model.
* [ ] It is not possible to keep variable sequence length during training because of the way DFT matrices are initialized (via a config variable).
    - For this @ontocord suggested that we keep different buffers ready. A major drawback to this is that we can't possibly keep dft matrices for all sequence lengths. However, the values of the DFT matrices vary according to the total sequence length expected. 
    - So, one efficient way of handling this is that we can try to create a generic matrix (not DFT) and then modify it on the fly accordingly based on the sequence length.
        - For example, the DFT matrix is defined as:
        ![image](https://user-images.githubusercontent.com/29076344/135800441-2d5f69e8-9231-4ccc-a971-52f60c4059e5.png)
            - We can create a matrix without the multiplier, and then on the fly take the portion of the matrix that is needed for the batch, and then multiply with the correct multiplier. Wdyt @patrickvonplaten @sgugger?
            - Or multiply the matrix with `sqrt(N)/sqrt(seq_length)` and take `mat[:seq_length, :seq_length]` while multiplying.
 
* [ ] Need to verify if pushing to GPU pushes everything to the device (including buffers).

I will be adding more issues/problems here as and when they arise.

Comments:
I think if PyTorch doesn't support --fp16 for the fourier transform, we sadly can't do much here.
Regarding the second point, I think keeping a buffer would indeed be a good solution here!

I think you can keep N (maybe 5) fixed buffer, like 64, 128, 256, 512, 1024. You would need to pad for those buffers that are too long.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Unstale

Should we implement a buffer solution here? @gchhablani - do you think this makes sense? How big would the buffer be in terms of size? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Doc styler tip
Author: sgugger
Pull request: yes

# What does this PR do?

This PR makes sure the doc styler considers the start and end of Tip blocks as paragraph-breakers. For instance
```py
"""
<Tip>
Short tip with no new line
</Tip>
"""
```
is currently restyled on one line (which may or may not work with the frontend).

With this PR, it's restyled like this:
```
"""
<Tip>

Short tip with no new line

</Tip>
"""
```

I can remove the part that adds new lines if you think it's too much.

Comments:
Update ONNX docs
Author: lewtun
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR gives the ONNX part of the documentation a much needed update to:

* Explain why we have three ONNX configuration objects (`OnnxConfig`, `OnnxConfigWithPast`, and `OnnxSeq2SeqConfigWithPast`) and how to interpret them.
* Explain what we mean by the `--features` argument of the export CLI.
* Provide an end-to-end guide on how to export a custom model, using the DistilBERT implementation as an example. I chose this approach so readers can run actual code instead of relying on a hypothetical unsupported architecture.

I also removed the deprecated section that was based on the `convert_graph_to_onnx.py` script as it seems to confuse people on which API to use. I couldn't find clear instructions on how the deprecation cycle works for the documentation of scripts like `convert_graph_to_onnx.py`, so please let me know if I should re-instate this part of the docs.

I also tweaked the help messages in the `transformers.onnx` CLI to make things a bit clearer to the end user.

The diff on the docs is quite huge, so I recommend reading the file directly.

Comments:
This is great, thanks for working on it @lewtun! Pinging @stevhliu and @sgugger for knowledge.

Thank you for the feedback @sgugger 🙏 !

I've now added the doctest `>>>` syntax to all the Python code blocks and removed the permalinks to the class references. OK to merge once the CI tests pass?