noamaanMulla-03/transformers-issues huggingface dataset:
EncoderDecoderModel's decoder gets unexpected use_cache argument
Author: ghost
Pull request: no

## Environment info

- `transformers` version: 4.5.1
- Platform: Linux-4.15.0-123-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.8.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

@patrickvonplaten, @patil-suraj I think you might have an idea on what went wrong.

## Information

EncoderDecoderModel internally calls decoder with use_cache parameter, which seems to be not define for a Bert decoder.

## To reproduce
I have bigger configs but the bug is still here with minimalistic configs:
```python
encoder_config = BertConfig()
encoder = BertModel(config=encoder_config)
decoder_config = BertConfig(is_decoder=True)
decoder = BertForMaskedLM(config=decoder_config)
model = EncoderDecoderModel(encoder=encoder, decoder=decoder)
input_ids = torch.ones(5, dtype=torch.long).unsqueeze(0)
model(input_ids=input_ids, decoder_input_ids=input_ids, labels=input_ids)
```
Outputs error from `/opt/conda/lib/python3.8/site-packages/transformers/models/encoder_decoder/modeling_encoder_decoder.py` at line 416 :
```
forward() got an unexpected keyword argument 'use_cache'
```

## Expected behavior

- Default Bert encoder and decoder can be stacked in an EncoderDecoderModel



Comments:
Hi @biggoron 

`BertForMaskedLM` can't be used as a decoder, it's intended for masked LM. The `BertLMHeadModel` model should be used if you wan to use bert as a decoder. Also the right way to initialize the bert as decoder is as follows

```python
dec_config = BertConfig.from_pretrained("bert-base-uncased")
dec_config.add_cross_attentions = True # add cross attention if you want to use it in EncoderDecoderModel
dec_config.is_decoder = True
dec = BertLMHeadModel.from_pretrained("bert-base-uncased", config=dec_config)
```

or simply use the `from_encoder_decoder_pretrained` method which takes care of this.

Thanks a lot for your help, it is much clearer!
Pegasus-Large Question
Author: karrtikiyer
Pull request: no

Is [Pegasus Large model checkpoint](https://huggingface.co/google/pegasus-large) trained on any downstream task? Or is it only trained on pre-training task of gap sentence?

Comments:
`pegasus-large` is a pre-trained checkpoint. It's not fine-tuned on downstream task. The finetuned checkpoint name will have the name of the datsets in them. 

Also please the [forum](https://discuss.huggingface.co/) to ask such questions.

Thanks.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add `cross_attentions` to the output of TensorFlow encoder-decoder models
Author: stancld
Pull request: no

# 🚀 Feature request

TensorFlow encoder-decoder models cannot return `cross_attentions` as do their PyTorch counterparts.

## Motivation
It would be nice to narrow the gap between PyTorch and Tensorflow implementations.

## Your contribution
I've been working on PR fixing this issue.

## Reviewers
@jplu and whoever else within the community

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Space token cannot be add when is_split_into_words = True
Author: Boltzmachine
Pull request: no

for example, 
```python
>>> tokenizer = BertTokenizer.from_pretrained('bert-base-chinese')
>>> tokenizer.add_tokens(' ')
1
```

```python
>>> tokenizer.encode('你好 世界', add_special_tokens=False)
[872, 1962, 21128, 686, 4518]
>>> tokenizer.encode(['你','好',' ', '世', '界'], is_split_into_words=True, add_special_tokens=False)
 [872, 1962, 686, 4518]
```
Obviously, the blank token is ignored. But if you change it to another token like ‘[balabala]’, it works.
So what is the proper way to do this?



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
I get different results everytime I run run_squad.py
Author: nrjvarshney
Pull request: no

Is it possible to have deterministic results using the run_squad.py script?
It has the set_seed() method but still, it gives different results every time I run it.

How can I get same results across all runs?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Request: Ignore Dataset transforms when iterating to the most recent checkpoint when resuming training
Author: jncasey
Pull request: no

# 🚀 Feature request

It'd be great if, when resuming training from a checkpoint and using a Dataset with a format/transform function applied, the dataset's format/transform function could be ignored while iterating up to the last checkpoint step.

@lhoestq @sgugger 

## Motivation

I doubt it's much of an issue most of the time, but I've started playing with `dataset.set_transform()` for doing some heavy preprocessing, and just iterating through samples to the current checkpoint step can take a ridiculously long time compared to a dataset without a transform applied. And I don't think there's any case where the transformed sample would be used, right?

See [this conversation in the forum](https://discuss.huggingface.co/t/understanding-set-transform/3740/6?u=jncasey) for more backstory and my rudimentary thoughts on how I'd accomplish it.

## Your contribution

I'm hesitant to try updating any of the trainer code myself since it's so complicated, and needs to cover so many edge cases I'm not familiar with. 


Comments:
This is already there :-) Just pass along `--ignore_data_skip` in your script or `ignore_data_skip=True` in your `TrainingArguments`.

Wow, that was fast! :) 

That loads the model from the checkpoint and advances the dataset to the next sample that would have been trained in the original run?

From my reading of the code I assumed that it reloaded the model and started the training over at the first sample of the dataset.


Ah sorry I misunderstood your feature request. Indeed it starts from the first sample instead of iterating. What you ask is a bit more complicated and will require a minimum version of datasets. It's possible but will take some time.

Sorry for the confusion from my writing! And thanks, as always, for your work on this amazing project.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
changing ".view()" to ".reshape()" for pytorch
Author: KaiQiangSong
Pull request: no

New Version of PyTorch uses ".reshape()" instead of ".view()".
There might be some issue if still using ".view()".

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Not able to convert T5 tf checkpoints
Author: RachitBansal
Pull request: no

Hi,
I was trying to convert some tf checkpoints for T5 into PyTorch using ```transformers-cli convert```, and am getting the following error:

> Traceback (most recent call last):
  File "/home/william18026/miniconda3/bin/transformers-cli", line 32, in <module>
    service.run()
  File "/home/william18026/miniconda3/lib/python3.7/site-packages/transformers/commands/convert.py", line 158, in run
    raise ValueError("--model_type should be selected in the list [bert, gpt, gpt2, transfo_xl, xlnet, xlm]")
ValueError: --model_type should be selected in the list [bert, gpt, gpt2, transfo_xl, xlnet, xlm]

My initial attempt was with transformers==4.3.3, but then I also tried using the source version (4.4.0dev) and the editable clone, but got the same error with all. T5 seems to have been added to the pipeline in #9654, but for some reason, it's not working at the user (/my) end.

What could I be doing wrong?

Comments:
It seems that T5 was added to the conversion scripts two months ago in https://github.com/huggingface/transformers/pull/9654 as you've mentioned.

However, in your error there is no mention of "t5":
```
ValueError: --model_type should be selected in the list [bert, gpt, gpt2, transfo_xl, xlnet, xlm]
```

But on master it clearly shows there should be one:
```
"--model_type should be selected in the list [bert, gpt, gpt2, t5, transfo_xl, xlnet, xlm, lxmert]"
```

Are you certain you're launching the command in the correct environment? Could you share the result of `transformers-cli env`?

---

Also, this conversion command is far from being complete, I think it could use some of your templating magic @sgugger if you ever feel like it :) 
It isn't high priority as we already have the model-specific conversion scripts.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
request about deepspeed tutorial 
Author: dorost1234
Pull request: no
Labels: name: DeepSpeed

Dear @stas00 

You have created this great tutorial here that without you it was really very hard near impossible to be able to train these large models, thank you so much 

https://github.com/huggingface/transformers/issues/8771

Do you mind updating your comment, including how numbers would change if you add distributed training and deepspeed and how the command should be in this case? so to further speedup deepspeed on multiple GPUs 

I also get this message with deepspeed with transformer 4.3.3 running the tutorial:
[deepscale] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 131072.0, reducing to 65536.0

could you tell me if this shows any issue in training? sorry I am very unfamiliar with deepspeed.

One more question, in tutorial you did not use --fp16, could you add a few comments if we can use it with deepspeed?
this is such a great toturial and would be great if we could have all info in one place.

thank you so much for the hard work and a lot of help



Comments:
@dorost1234, thank you for the kind words. I'm glad to hear it was useful.

In general to answer the bulk of your questions - you will find the full documentation here:
https://huggingface.co/transformers/master/main_classes/trainer.html#deepspeed

Please let me know if you still have any question after reading it.

> [deepscale] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 131072.0, reducing to 65536.0

I shall document this. This is just a warning that DeepSpeed prints when it delays the optimizer stepping due to fp16 dynamic scaling. I definitely want to document that since it's alarming and hard to understand. and how to get the optimizer kick on the first step.

Also I started working on a notebook, which you can try - should work on  jupyter or colab:
https://github.com/stas00/porting/blob/master/transformers/deepspeed/DeepSpeed_on_colab_CLI.ipynb
Please let me know what you'd like to be added to it.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
The dimension of Feature extraction
Author: LemonQC
Pull request: no


![image](https://user-images.githubusercontent.com/30914380/110243654-4dadf480-7f96-11eb-8a2a-1bff9407c786.png)
![image](https://user-images.githubusercontent.com/30914380/110243672-5999b680-7f96-11eb-91e4-2728099b2418.png)
why this happened?


Comments:
How did you create the `nlp_features` function?

The sequence length is different due to different tokenization. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Dear developer, does transformers have the support to translate Chinese text into English?
Author: j2538318409
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
Hi @j2538318409,

I think Mbart model from facebook can do that for you [mbart](https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt), 

You need to specify `zh_CN` as the source language and `en_XX` as the target language.

 This [colab notebook](https://github.com/bhadreshpsavani/UnderstandingNLP/blob/master/MultilingualMBart.ipynb) does translation from English to hindi. You can use the same for doing translation from Chinese to English by modifying `src_lang` and `trg_lang`.

There are other translation models also available in huggingface like [Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en).  You can find more details on [huggingface models section](https://huggingface.co/models?pipeline_tag=translation)

I hope this will be helpful to you!

https://huggingface.co/transformers/master/model_doc/m2m_100.html

M2M is in master since today, is this what you are looking for ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Similar issue like #1091 in Blenderbot
Author: karthikgrama
Pull request: no

Tokenizer and model are not in sync. I am using "facebook/blenderbot-400M-distill" 

Tokenizer has 8009 base tokens where as model has 8008. 

Could you please help me with this?

`from transformers import BlenderbotTokenizer, BlenderbotForConditionalGeneration

mname = "facebook/blenderbot-400M-distill"

model = BlenderbotForConditionalGeneration.from_pretrained(mname)
tokenizer = BlenderbotTokenizer.from_pretrained(mname, local_files_only=True)

print(len(tokenizer))
print(model.config.to_dict()['vocab_size'])`

Here is the output that I get.
8009
8008


## Environment info

- `transformers` version: 4.3.2
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.8.6
- PyTorch version (GPU?): 1.7.1+cpu (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj




Comments:
Do you encounter any errors because of the mismatch in length?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Trainer._remove_unused_columns() returns None
Author: guyrosin
Pull request: no

## Environment info
- `transformers` version: 4.6.0.dev0
- Platform: Linux-4.15.0-134-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.1 (False)
- Tensorflow version (GPU?): 2.4.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@sgugger, @LysandreJik

## Information

`Trainer._remove_unused_columns()` returns None in case `args.remove_unused_columns` is `False`, instead of returning the given dataset.
Related to #11343.

Model I am using (Bert, XLNet ...): BERT

The problem arises when using:
* [x] the official example scripts: (give details below) run_mlm/glue/...
* [ ] my own modified scripts: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Set `TrainingArguments.remove_unused_columns=False`
2. Train/eval/test your model using `Trainer`
3. The dataset would be None, and so the following exception would raise:

```
Traceback (most recent call last):
  ...
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/home/guyrosin/src/transformers/src/transformers/trainer.py", line 814, in num_examples
    self.num_examples(train_dataloader) if train_dataset_is_sized else total_train_batch_size * args.max_steps
  File "/home/guyrosin/src/transformers/src/transformers/trainer.py", line 814, in num_examples
self.num_examples(train_dataloader) if train_dataset_is_sized else total_train_batch_size * args.max_steps
  File "/home/guyrosin/src/transformers/src/transformers/trainer.py", line 814, in num_examples
    return len(dataloader.dataset)
TypeError: object of type 'NoneType' has no len()
```

## Expected behavior

`Trainer._remove_unused_columns()` should always return a dataset.

Comments:
Thank you so much for flagging! This should be fixed by the PR above.
Potential bug in PLM training
Author: HarshTrivedi
Pull request: no
Labels: name: wontfix

There seems to be a bug in `mask_tokens` method of `DataCollatorForPermutationLanguageModeling`. Based on the comment, [this line](https://github.com/huggingface/transformers/blob/master/src/transformers/data/data_collator.py#L294) is supposed to compute mask for non-functional tokens, ie. anything but padding and special tokens. So there should be an OR between the `padding_mask` and `special_tokens_mask`, and not AND. For reference, the [corresponding line](https://github.com/zihangdai/xlnet/blob/master/data_utils.py#L602) in the original XLNet code also has an OR.

I should acknowledge that I haven't understood the permutation masking code properly yet. But raising an issue, because it seems wrong to me.

-----

Besides the above problem, I'm also getting a very bad perplexity (**296.0**) on evaluating (w/o finetuning) `xlnet-base-cased` PLM model on plain wikitext2 dataset (`wiki.test.raw`). I've used XLNet example from [here](https://github.com/huggingface/transformers/tree/master/examples/language-modeling) (without `--do-train` flag) to get the perplexity.

The PLM code only works if the sequence lengths are even. To workaround this, I append a padding token when sequence length is odd. Concretely, I replaced the [error here](https://github.com/huggingface/transformers/blob/master/src/transformers/data/data_collator.py#L255) with:

```
padding = inputs.new_ones((inputs.size(0), 1))*self.tokenizer.pad_token_id
inputs = torch.cat([inputs, padding], dim=1)
```

For comparison, the perplexity of BERT in this dataset is around 10.

Transformer Version: from master.

@patrickvonplaten @TevenLeScao @LysandreJik @shngt 

Comments:
I think this is correct. It should be replaced by an `|`. Do you get a better perplexity if you change this line?

Thanks a lot for opening this issue @HarshTrivedi ! I also agree that the logic should be OR and not AND. @shngt - can you maybe comment here as well?

Thank you for confirming this!

If I remember correctly, changing `&` to `|` didn't fix the high zero-shot perplexity for me. I'll try it again later today or tomorrow and report back the numbers with `&` vs `|`.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I agree - the logic should be OR and not AND. Could you please confirm if the numbers change @HarshTrivedi?

Sorry for the delay - I missed the notification at the time. I'll submit a PR for AND -> OR fix asap, and try to do some more stringent testing to catch the reason for the perplexity difference. How can I proceed with the latter @patrickvonplaten @LysandreJik ?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


(Resolved by https://github.com/huggingface/transformers/pull/8409 I believe)
RuntimeError: Error(s) in loading state_dict for BertModel:
Author: zyc1310517843
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Hello, I need to use "py torch_model.bin" in a model. I used your "convert_bert_original_tf_checkpoint_to_pytorch.py" to generate bin file, but I used "model_bert.load_state_dict (torch.load (init_checkpoint, map_location='cpu') to load it in my code, except for the following mistakes:

Traceback (most recent call last):
File "train.py", line 579, in <module>
Model, model_bert, tokenizer, bert_config = get_models (args, BERT_PT_PATH)
File "train.py", line 157, in get_models
Args.no_pretraining)
File "train.py", line 125, in get_bert
Model_bert. load_state_dict (torch. load (init_checkpoint, map_location='cpu'))
File "/ home/ubuntu/anaconda3/lib/python 3.7/site-packages/torch/nn/modules/module.py", line 777, in load_state_dict
Self. class. Name, "n T. join (error_msgs)"
Runtime Error: Error (s) in loading state_dict for BertModel:
Missing key (s) in state_dict: "embeddings. word_embeddings. weight", "embeddings. position_embeddings. weight".
thanks

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


try:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model_path = None
model = my_model.load_state_dict(torch.load(model_path, map_location=device))
The exact English pretraining data and Chinese pretraining data that are exact same to the BERT paper's pretraining data.
Author: guotong1988
Pull request: no

(Sorry I can not visit the forum.)

Any one know where to get them?
Thank you and thank you.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.