noamaanMulla-03/transformers-issues huggingface dataset:
add a test checking the format of `convert_tokens_to_string`'s output
Author: SaulLu
Pull request: yes

# What does this PR do?

For context, the tokenizers 0.12.0 release was breaking in the sense that it changed the output format of the `decode` method of the `backend_tokenizer` 's `decoder` which was used in `convert_tokens_to_string` (cf #16537, #16520, #16525). The 0.12.0 version has since been yanked but I think it would be beneficial to add a common test to the transformers tokenizers to verify that the `convert_tokens_to_string` method output format is respected.

This PR therefore proposes adding such tests with some overridden tests for tokenizers with constraints on possible tokens or with a slightly different output format.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Would love to have your opinion @LysandreJik , @sgugger and/or @Narsil 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for the review @Narsil :slightly_smiling_face: 

> Why is tokenization_common test not enough ? Don't the other tests inherit the same base/meta class ?

They also inherit from `TokenizerTesterMixin`, but unfortunately there are 1) tokenizers that have constraints on acceptable tokens - this is the case of `ByT5`  and `perceiver` - and 2) tokenizers whose `convert_ids_to_string` method returns not a string but a dictionary - this is the case of `wav2vec2` and `wav2vec2_phoneme`.

Thanks for the explanation, definitely not obvious !

That a good point: I've added a comment for each overridden test in my last commit to reflect that last explanation :slightly_smiling_face:
TPU slow finetuning T5-base
Author: GenVr
Pull request: no

Hi,
I am trying to train a T5-base on Colab with the TPU. I am using the official code to perform a fine-tuning on the T5-base (with my dataset), but the training with TPU is extremely slow! I'm using the offical [code](https://github.com/huggingface/transformers/blob/master/examples/pytorch/translation/run_translation.py).

I am attaching the colab code with the various libraries I have installed: [notebook](https://colab.research.google.com/drive/1g4PYEOa0zhJnMaThPZVSQepn_wr9md4f#scrollTo=XNVao-sd4fbH).

Also, if I try to increase the batch size as >= 64, I get a memory error, as there seems to be only about 8 Gb available.

Can someone help me? Thank you!

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

@LysandreJik Done days ago, but no answer yet

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Documentation of DataCollatorForLanguageModeling
Author: yarongef
Pull request: no

# 🚀 Feature request

Would you consider adding to the following tip:

"For best performance, this data collator should be used with a dataset having items that are dictionaries or BatchEncoding, with the "special_tokens_mask" key, ...."

The reason for doing so, for example: "So that special mask tokens won't be masked".


Comments:
Hey! Would you like to open a PR?

Yes, I'll do that

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix CI: test_inference_for_pretraining in ViTMAEModelTest
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix CI (scheduled): `test_inference_for_pretraining` in `ViTMAEModelTest`: incorrect device

https://github.com/huggingface/transformers/runs/5809883645?check_suite_focus=true

Comments:
_The documentation is not available anymore as the PR was closed or merged._
add a template to add missing tokenization test
Author: SaulLu
Pull request: yes

# What does this PR do?

In this PR I propose to add a cookie cutter template for tokenization tests. 

It is a first version and could be useful to propose in a good first issue to users to add the missing tests to the tokenizers of the following models:
- Flaubert #15137
- LED
- RemBert
- Splinter

 and eventually for these tokenizers too (currently they just inherit from `BertTokenizer` and just re-define the attributes):
- MobileBert
- ConvBert
- Electra
- Longformer
- RetriBert

I plan to give a little more information on how to add the test to the good first issue ticket than is currently shown in the readme. But don't hesitate if you think it is better to say as much as possible in the current readme

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. I would love to have your input on this @LysandreJik , @sgugger , @patrickvonplaten or @patil-suraj  :hugs: 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
about attention value
Author: wonjunchoi-arc
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->



Hi I'am a beginner of ML programming

I'm studying transformer, but there's something I don't understand.

The first is about the attention value. I can understand the correlation between each word through self-attention and multiply the value by softmax. But I don't understand why the value added to all those values contains information about the context.

Secondly, I don't know why Position-Wise Feed-Foward Networks are used.

Comments:
Hi @wonjunchoi-arc 👋  We have some information about the questions you asked in our course -- see [here](https://huggingface.co/course/chapter2/5?fw=pt#attention-masks), for instance. The course and other documentation are searchable, and they are a great source of information.

We also reserve these issues for bugs in the repository and/or feature requests. For any other requests, we'd like to invite you to use our [forum](https://discuss.huggingface.co/) 🤗 I'm closing this issue, but feel free to reopen with queries that fit the criteria I described.

> Hi @wonjunchoi-arc 👋 We have some information about the questions you asked in our course -- see [here](https://huggingface.co/course/chapter2/5?fw=pt#attention-masks), for instance. The course and other documentation is searchable, and it is a great source of information.
> 
> We also reserve these issues for bugs in the repository and/or feature requests. For any other requests, we'd like to invite you to use our [forum](https://discuss.huggingface.co/) 🤗 I'm closing this issue, but feel free to reopen with queries that fit the criteria I described.

thank you!!
I'll study harder!!
PretrainedModel: made `_load_pretrained_model_low_mem` static + bug fix
Author: FrancescoSaverioZuppichini
Pull request: yes

# What does this PR do?
This PR makes `PretrainedModel._load_pretrained_model_low_mem` a `staticmethod` since it's not returning an instance of `PretrainedModel`. I've also fixed a bug inside the method itself, the main for loop was wrong.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Merging this, since `low_cpu_mem_usage` is not working on master right now and is required for to test the addition of some new models.

@stas feel free to still leave a comment about the `classmethod` change :)
Why does BART generate additional EOS token at the beginning?
Author: ayaka14732
Pull request: no

## Environment info

- `transformers` version: 4.18.0.dev0
- Platform: Linux-5.11.0-1018-gcp-x86_64-with-glibc2.31
- Python version: 3.10.4
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.11.0+cu102 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.4.1 (cpu)
- Jax version: 0.3.4
- JaxLib version: 0.3.2
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

@patil-suraj @patrickvonplaten @sgugger

## Information

Model I am using: BART

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import BartTokenizer, BartForConditionalGeneration

model_name = 'facebook/bart-base'
tokenizer = BartTokenizer.from_pretrained(model_name)
model = BartForConditionalGeneration.from_pretrained(model_name)

sentences = ('A flower.', 'Some good sentences.')

inputs = tokenizer(sentences, return_tensors='pt', max_length=8, padding='max_length', truncation=True)
output = model.generate(inputs.input_ids)

print('Input:', tokenizer.batch_decode(inputs.input_ids))
print('Output:', tokenizer.batch_decode(output))

print('Input IDs:', inputs.input_ids.tolist())
print('Output IDs:', output.tolist())
```

Output:

```
Input: ['
A flower.
<pad><pad><pad>', '
Some good sentences.
<pad><pad>']
Output: ['

A flower.
<pad>', '

Some good sentences.
']
Input IDs: [[0, 250, 14214, 4, 2, 1, 1, 1], [0, 6323, 205, 11305, 4, 2, 1, 1]]
Output IDs: [[2, 0, 250, 14214, 4, 2, 1], [2, 0, 6323, 205, 11305, 4, 2]]
```

## Expected behavior

From my understanding, the model should be generating in this way:

![](https://user-images.githubusercontent.com/68557794/161550447-dbad465f-ac7d-4530-9812-e112c8109ac7.png)

```
250, 14214,     4, 2
  ↑      ↑      ↑  ↑
  0,   250, 14214, 4
```

So the first item of the output should be:

```
[250, 14214, 4, 2, 1, 1, 1, ...]
```

But actually it is:

```
[2, 0, 250, 14214, 4, 2, 1]
```

Therefore, I would like to know where the 2 (i.e. the EOS token) comes from?

Comments:
This is because the way the model are trained. In BART `eos` (`
`) token is used as the `decoder_start_token_id`. So the format for the `decoder_input_ids` during training is `
 
 tokens ...`. So the model always needs to generate `

` as the first two tokens.

@patil-suraj Thank you! I am still a little confused. If the `decoder_input_ids` during training is `
 
 a b c ...`, according to my understanding, the model output should be `
 a b c ...` (`
 -> 
`, `
 -> a`, `a -> b`, `b -> c`, ...). But the model output is actually `
 
 a b c ...`.

I think I understand it now. We always add the decoder output to the input, and in the end we take the input as the final result.
Adding missing type hints for BigBird model  
Author: reichenbch
Pull request: yes

PyTorch Implementation model added with missing type hints

## What does this PR do?

Added type hints for BigBird PyTorch as described in https://github.com/huggingface/transformers/issues/16059


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Rocketknight1 
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

This looks great, thank you! And sorry for the delay in reviewing it.
Issue running run_glue.py at test time
Author: md975
Pull request: no

Hi,

I'm using [this script](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py) to train a model on my own dataset.
(transformers  4.15.0, datasets  1.17.0, python3.7)
I'm doing single sentence classification, and my CSV files have 2 cols: label, sentence1
I can run the script with no issue when --do_train --do_eval --do_predict are true.
However, later, when I use the same test set used when training, but this time with only "--do_predict" on one of the checkpoints of the models I previously trained, I get the following error:

> File "/lib/python3.7/site-packages/datasets/arrow_writer.py", line 473, in <listcomp>
    pa_table = pa.Table.from_arrays([pa_table[name] for name in self._schema.names], schema=self._schema)
  File "pyarrow/table.pxi", line 1339, in pyarrow.lib.Table.__getitem__
  File "pyarrow/table.pxi", line 1900, in pyarrow.lib.Table.column
  File "pyarrow/table.pxi", line 1875, in pyarrow.lib.Table._ensure_integer_index
KeyError: 'Field "sentence2" does not exist in table schema'

I would appreciate any tips on how to fix this, thanks!


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
TypeError: expected str, bytes or os.PathLike object, not NoneType
Author: skpig
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:4.17.0
- Platform:Linux-4.15.0-142-generic-x86_64-with-debian-buster-sid
- Python version:3.6.6
- PyTorch version (GPU?): 1.7.1 (True)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
@patrickvonplaten This issue is similar to https://github.com/huggingface/transformers/issues/9328, but I'm not so sure. If adding a merge.txt is able to fix the error, maybe the model hub should update many models / notify the creators.

## Information

Model I am using (Bert, XLNet ...): Bart

The problem arises when using:
* [x] the official example scripts: (give details below)
```
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("fnlp/bart-base-chinese")

model = AutoModel.from_pretrained("fnlp/bart-base-chinese")
```

## To reproduce
Just run the official script above. The error is reported as below
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/huangbz/.conda/envs/NLP/lib/python3.6/site-packages/transformers/models/auto/tokenization_auto.py", line 546, in from_pretrained
    return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/home/huangbz/.conda/envs/NLP/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 1795, in from_pretrained
    **kwargs,
  File "/home/huangbz/.conda/envs/NLP/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 1819, in _from_pretrained
    **(copy.deepcopy(kwargs)),
  File "/home/huangbz/.conda/envs/NLP/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 1923, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/huangbz/.conda/envs/NLP/lib/python3.6/site-packages/transformers/models/bart/tokenization_bart.py", line 220, in __init__
    with open(vocab_file, encoding="utf-8") as vocab_handle:
TypeError: expected str, bytes or os.PathLike object, not NoneType
```

Comments:
Hey @skpig, I think the model `fnlp/bart-base-chinese` is broken sadly. We would need to ping the author here to notify her/him. Soon this will be able on the Hub :heart: cc @julien-c 

Thanks for your reply @patrickvonplaten .
Add implementation of typical sampling
Author: cimeister
Pull request: yes

# What does this PR do?

Adds an implementation of typical sampling (https://arxiv.org/abs/2202.00666) to the `generate` function

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Looks cool! Could you add one test? :-)

> Looks cool! Could you add one test? :-)

Added a test and fixed the style issues :)

Of course! Are we waiting on @thomwolf at this point? Or is there something else that has to happen before merging?

I confirm, very clean! Good job @cimeister 

@cimeister You are using `nansum` here, which is available in torch>=1.7.0. @patrickvonplaten Does approving this for main branch means `transformers` drop support for torch<1.7.0 ?

Ah nice catch @LSinev 
@patrickvonplaten should we change to something like:
```
plogp = p * normalized 
ent = - plogp[~torch.isnan(plogp)].sum(-1, keepdim=True)
```

Same answer here is given by @sgugger here: https://github.com/huggingface/transformers/pull/13292#discussion_r837529073 . Would be great if someone could open a PR to fix it :-)
change num_attention_heads=[1, 2, 5, 8] to [1,2,4,8]
Author: JunMa11
Pull request: no

Hi @NielsRogge ,

Is this a typo in segformer? The default number of attention heads should be `[1,2,4,8]`.

https://github.com/huggingface/transformers/blob/4354005291dacd5c8264f0936a33678df4f4bb71/src/transformers/models/segformer/configuration_segformer.py#L107

Comments:
@NielsRogge I'll work on this.

Hi,

Thanks for pointing out, actually that piece of code is correct, we should just update the docstring.

As seen below (table taken from the [paper](https://arxiv.org/abs/2105.15203)), (N1, N2, N3, N4) refer to the number of heads in each stage:

<img width="832" alt="Screenshot 2022-04-05 at 16 50 13" src="https://user-images.githubusercontent.com/48327001/161781658-53578f3d-462c-470c-846f-9f07ecae4872.png">

Can you open a PR to fix the docstring?





Hi @NielsRogge ,

Thanks for your quick reply very much.

This is my misunderstanding. I'm sorry to waste your time.

Best regards,
Jun

Can you still open a PR to fix it in the docs?
added type hints to CTRL pytorch 
Author: anmolsjoshi
Pull request: yes

# What does this PR do?

I added type annotations for CTRL (PT) as described in #16059

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Moved functions and classes to pytorch_utils.py
Author: anmolsjoshi
Pull request: yes

# What does this PR do?

Moved all pruning stuff, apply_chunking_to_forward, get_parameter_device, get_parameter_dtype to pytorch_utils.py

Fixes #15543

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten @sgugger 

Comments: