noamaanMulla-03/transformers-issues huggingface dataset:
when I overwrite a model,I meet some bugs like this
Author: yang-collect
Pull request: no

ok,when I firstly overwrite a bert-bigru-crf model,i meet som bugs.
this is the model
``` python3
from torchcrf import CRF
from torch import nn
from transformers import BertModel, PreTrainedModel

class Bert_BiGru_Crf(PreTrainedModel):
      ''' I want define a  bert-bigru-crf model end-to-end,and i know how to define this model by torch.nn.Module
      '''
    def __init__(self, config):
        super().__init__(config)
        self.bert = BertModel(config)
        self.gru = nn.GRU(768,
                          768,
                          num_layers=2,
                          bidirectional=True)
        self.fc = nn.Linear(768 * 2, 15)  # BOS EOS
        self.crf = CRF(13, batch_first=True)

    def froward(self, text, label):
        out = self.bert(**text) # this text is above input_ids and attention mask
        output, _ = self.gru(out.last_hidden_state)
        output = self.fc(output)
        loss = self.crf(output, label)
        pred = self.crf .decode(output, text['attention_mask']) # decoder need mask
        return output, pred, loss
```
and i use `Bert_BiGru_Crf.from_pretrained(model_path)`，`model_path is a bert model path`
i got some bugs like those:
```
Traceback (most recent call last):
  File "C:/Users/wie/Documents/项目/ner/bert-crf/train.py", line 103, in <module>
    train()
  File "C:/Users/wie/Documents/项目/ner/bert-crf/train.py", line 42, in train
    embed_model = model.Bert_BiGru_Crf.from_pretrained(model_path)
  File "C:\Users\wie\Miniconda3\lib\site-packages\transformers\modeling_utils.py", line 1067, in from_pretrained
    config, model_kwargs = cls.config_class.from_pretrained(
AttributeError: 'NoneType' object has no attribute 'from_pretrained'
```
``` python3
# Name                    Version                   Build  Channel
pytorch-transformers      1.2.0                    pypi_0    pypi 
transformers              4.6.1                    pypi_0    pypi
pytorch-crf               0.7.2                    pypi_0    pypi
pytorch-lightning         1.4.4                    pypi_0    pypi
torch                     1.8.1+cpu                pypi_0    pypi
torchaudio                0.8.1                    pypi_0    pypi
torchmetrics              0.5.0                    pypi_0    pypi
torchtext                 0.9.1                    pypi_0    pypi
torchvision               0.9.1+cpu                pypi_0    pypi
```
my python vesion is 3.7.9
would you mind to help me slove this problem?

Comments:
GPTNeo Error Attempting to Generate Text
Author: ncoop57
Pull request: no
Labels: name: WIP

## Environment info

- `transformers` version: 4.9.0.dev0
- Platform: Linux-5.4.0-1043-gcp-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version (GPU?): 1.9.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.3.4 (tpu)
- Jax version: 0.2.16
- JaxLib version: 0.1.68
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no
- Using TPU: Yes

### Who can help
@patil-suraj and @patrickvonplaten

Models:
GPTNeo

Library:

- flax transformers

 -->

## Information

Model I am using (Bert, XLNet ...): GPTNeo


## To reproduce

Steps to reproduce the behavior:

Here is a Google Colab for reproducing: https://colab.research.google.com/drive/1tba52h5t-BP3g13FMdPXVjKqpoLTlGvP?usp=sharing

For convenience here is the error msg:
```
TypeError: dynamic_update_slice update shape must be smaller than operand shape, got update shape (1, 45) for operand shape (1, 20).
```

I was originally getting the same error as #12081. However, when I attempted to implement the same fix as in that issue, I got the above error. The error might be because I am using the "ForCausalLM" version of GPTNeo. However, there is no LMHead version

## Expected behavior
Generate the output sequence


Comments:
Hey @ncoop57,

The reason for this error is that `input_ids.shape[1]` (the length of the input length) is larger then `max_length`. By default `max_length` of generate is 20 and in your case `input_ids.shape[1]` is > 20 which will error out. `max_length` define the number of total output tokens (not just the number of generated tokens). So if number of input tokens (`input_ids.shape[1]` is already > `max_length`) the model is told to not generate anything and will error out (we should put better error messages here - which is why I'm leaving this issue open).

In short to solve your problem, simply pass a higher `max_length` parameter:

```
output_seq = model.generate(input_ids=inputs.input_ids, max_length=100)
```

@patrickvonplaten we should probably raise an error if cur_length is greater than `max_length`, otherwise, it seems it's hard to figure out.
Making it raise real errors on ByT5.
Author: Narsil
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

Comments:
Optimized bad word ids
Author: guillaume-be
Pull request: yes

# What does this PR do?

This PR optimizes the generation routines when `bad_word_ids` are provided by the user. Currently, two inefficiencies significantly slow down the text generation when these are passed:
- single-token bad words are looped over at each generation iteration: this is not required as these single word tokens are always banned, regardless of the input generated so far.
- The token_ids are queried in nested for loops, causing unnecessary and inefficient cross-device communication if these are on the GPU.

The issue was raised in https://discuss.huggingface.co/t/gpt2-many-bad-words-ids-leading-to-slow-text-generation/9721
I could reproduce the issue and observed a severe slowdown of the generation when ~2000 bad word ids are provided (see https://gist.github.com/guillaume-be/2a3e91951869414b6f1f8ab8c2cd642f gist). I observed a ~20x slowdown of the generation when using the bad words with a GPU, from ~1.7s to >25s per generation.

This PR fixes the issue by:
- Moving all of the current token ids to a Python list before multiple iteration through that list, leading to a 20x speedup
- Splitting the bad word ids into 1-element bad words, and words that are made of multiple sub-tokens. For the 1-element bad words, a static bad word pas is pre-computed and re-used for each generation step. This accelerates the generation by a further ~10%.


Fixes https://discuss.huggingface.co/t/gpt2-many-bad-words-ids-leading-to-slow-text-generation/9721


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case: https://discuss.huggingface.co/t/gpt2-many-bad-words-ids-leading-to-slow-text-generation/9721
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten - maybe you would like to have a look?

Comments:
Error while loading pretrained model with "return_dict=True"
Author: tuner007
Pull request: no

# ❓ Questions & Help

torch: 1.6.0+cu101
Transformers: 3.0.2

**Error with "return_dict=True"** 

```
from transformers import BertTokenizer, BertForPreTraining
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForPreTraining.from_pretrained('bert-base-uncased', return_dict=True)
```

```
TypeError                                 Traceback (most recent call last)
<ipython-input-3-5eca8cb45c88> in <module>()
      2 import torch
      3 tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
----> 4 model = BertForPreTraining.from_pretrained('bert-base-uncased', return_dict=True)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    670 
    671         # Instantiate model.
--> 672         model = cls(config, *model_args, **model_kwargs)
    673 
    674         if state_dict is None and not from_tf:

TypeError: __init__() got an unexpected keyword argument 'return_dict'
```



Comments:
Hi! I believe that parameter is only available on `master` right now, so you should install `transformers` from the `master` branch to use it (`pip install git+https://github.com/huggingface/transformers`). It'll be available in version `3.1.0` which will be released in a couple of days.

Working for me after the upgrade to 3.1.0 - thanks @LysandreJik 

> pip install git+https://github.com/huggingface/transformers

Thanks, this helped me too.
Seems like ```transformers``` latest version must be important , especially stable and master branch one
A dead link in GPT2 description
Author: shabie
Pull request: no

I see that there is a dead link in the GPT2 description for some time referring to an argument on how pre-computed values can be reused.

The link to the description with the dead link (the 3rd tip): https://huggingface.co/transformers/model_doc/gpt2.html

The link that is dead: https://huggingface.co/transformers/quickstart.html#using-the-past

My suggestion is that if this quickstart page does not exist anymore than just linking it to the `forward` function docs on the same page (that has the parameter `past_key_values`) should do the job.

As you are taking care of the docs @sgugger, I can make a PR to do this if it makes sense to you.

Comments:
Thanks for reporting, feel free to open a PR with your change indeed!

So I have made a PR. One problem I did find however was the inconsistency between the TF and PT arguments.

In PT it is called `past_key_values` and in TF it is simply called `past`. I have made this clear in the PR for the docs but it would be nice if they hadn't diverged.

However, fixing this is problematic since this would be a breaking change (probably the reason it is still there :D)

The way you did it is great! I think the divergence comes from when we unified that argument on the PyTorch side a while ago (it was different names for different models), but forgot to do the same on the TF side.
[docs] update dead quickstart link on resuing past for GPT2
Author: shabie
Pull request: yes

Thed dead link have been replaced by two links of forward and call methods of the GPT2 class for torch and tensorflow respectively.

# What does this PR do?

This PR replaces the dead link with two new links on showing how to reuse the precomputed attention values.

Fixes #13434

## Before submitting
- [ X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

Comments:
> Thanks a lot for fixing!
> 
> Could you just run `make style` on your branch to fix the code quality issue?

Done.

Thanks again!
TFRobertaModel warning  - `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated
Author: eyalshafran
Pull request: no

I'm using:
google colab
transformers  4.1.1
tensorflow 2.4.0 (with gpu)
model TFRobertaModel

I keep getting a warning when calling TFRobertaModel. For example:
```
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
config = RobertaConfig.from_pretrained('roberta-base')
roberta_layer = TFRobertaModel(config)

max_seq_len = 64
ids = tf.keras.layers.Input((max_seq_len,), dtype=tf.int32)
att = tf.keras.layers.Input((max_seq_len,), dtype=tf.int32)
tok = tf.keras.layers.Input((max_seq_len,), dtype=tf.int32)

roberta_inputs = [ids, att, tok]

sequence_output = roberta_layer(ids,attention_mask=att,token_type_ids=tok) # this produces the message
```
Produces the following message:

The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model. They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.

I tried to set the variables in the config object but there is no change in the message:
```
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
config = RobertaConfig.from_pretrained('roberta-base',output_attentions=False,output_hidden_states=False,return_dict =True)
roberta_layer = TFRobertaModel(config)
```

@jplu, @LysandreJik 



Comments:
Hello!

These two messages are just a warning, you can ignore them if you are not concerned. Basically, these messages will always be displayed everytime the graph node is executed, and only in graph mode.

Is there a way to suppress these warnings? They overwhelm the logs with useless messages...

@jplu the issue still persists, what is the purpose of these warnings? Why are they displayed to the user of the library?

@jklaise @Gilthans The logging happens through a TF Logger (See [this line](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_utils.py#L50)) .  

You can suppress them by using something like `tf.get_logger().setLevel('ERROR')`
cannot find model in model name list
Author: zxr19980213
Pull request: no
Labels: name: Core: Tokenization

name: Usage

name: Version mismatch

Hi , thank you for developing well-made pytorch version of BERT !

I am new to NLP area and have problem while coding like this:
```python
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
```

The error discription is below:
```
INFO:pytorch_pretrained_bert.file_utils:https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt not found in cache, downloading to C:\Users\zxr\AppData\Local\Temp\tmpb3lgzjlo
ERROR:pytorch_pretrained_bert.tokenization:Model name 'bert-base-uncased' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese). We assumed 'https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt' was a path or url but couldn't find any file associated to this path or url.
```

I searched it and thought it might be poor internet connect.

I downloaded the model i want but donot know how to load it with code.

Thank you very much for your attention!

Comments:
Hi, could you please provide all the information required in the template so that we may help you? Namely which version of `transformers`, python and PyTorch are you using?

You seem to be using `pytorch-pretrained-BERT`, which is a very old version of this repository. Have you tried using the newer `transformers`, which has much more functionalities and is more robust than `pytorch-pretrained-BERT`?

I am using `Python 3.6.9` , `torch 1.3.1` and `pytorch-pretrained-bert 0.6.2` .

I am following the tutorial on https://pypi.org/project/pytorch-pretrained-bert/ and meet with this problem.

I changed to a better Internet and this problem solved .
Sorry for your time and thank you for your attention !

can u tell me how to change a better internet? i met thie question too

@zxr19980213
Difference between `logit_scale` initialisation in Transformers CLIP and the original OpenAI implementation.
Author: lycfight
Pull request: no

I tried another training code based on the OpenAI'CLIP version: I found a difference at logit_scale between them. Does it mean temperature parameter? Is it the reason for loss rising?
huggingface transformers' CLIP:

```
self.logit_scale = nn.Parameter(torch.ones([]))
```
OpenAI CLIP:

```
self.logit_scale = nn.Parameter(torch.ones([]) * np.log(1 / 0.07))
```

Comments:
Hi @lycfight 

Please provide a proper title when opening an issue, I've updated the title and issue description a bit.

> I found a difference at logit_scale between them. Does it mean temperature parameter? Is it the reason for loss rising?

Great catch!  Yes `logits_scale`  is also called `temperature`. And indeed not initializing the `logits_scale` to the right value could cause issues during training.

Also, training CLIP is a bit complicated, for contrastive learning try to use as big a batch size as possible. Initializing `logits_scale` also matters a lot. You could find some info in this [report](https://huggingface.co/spaces/clip-italian/clip-italian-demo) as well (it's CLIP for Italian language), this [issue](https://github.com/openai/CLIP/issues/83)  might also help.

> Hi @lycfight
> 
> Please provide a proper title when opening an issue, I've updated the title and issue description a bit.
> 
> > I found a difference at logit_scale between them. Does it mean temperature parameter? Is it the reason for loss rising?
> 
> Great catch! Yes `logits_scale` is also called `temperature`. And indeed not initializing the `logits_scale` to the right value could cause issues during training.
> 
> Also, training CLIP is a bit complicated, for contrastive learning try to use as big a batch size as possible. Initializing `logits_scale` also matters a lot. You could find some info in this [report](https://huggingface.co/spaces/clip-italian/clip-italian-demo) as well (it's CLIP for Italian language), this [issue](https://github.com/openai/CLIP/issues/83) might also help.

Furthermore, the loss rising happens when I fine-tuned CLIP pretrained model in COCO dataset: 
`model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")`
Then, in order to find the difference, I also try to train CLIP from scratch, and the loss descents normally:
`from transformers import CLIPConfig, CLIPModel, CLIPProcessor
    config = CLIPConfig()
    model = CLIPModel(config)`
That confused me extremely.
[CLIP] fix logit_scale init
Author: patil-suraj
Pull request: yes

# What does this PR do?

This PR fixes the initialization of `logit_scale` parameter as per the [original CLIP implementation ](https://github.com/openai/CLIP/blob/3b473b0e682c091a9e53623eebc1ca1657385717/clip/model.py#L291).

Also, the initial value of `logit_scale` needs to be carefully tuned for CLIP training, so this PR, adds a `logit_scale_init_value` as a `config` parameter, so users could pass different init values as per their training.


Fixes #13430

Comments:
Don't modify labels inplace in `LabelSmoother`
Author: sgugger
Pull request: yes

# What does this PR do?

Currently, the `LabelSmoother` makes an inplace modification in the labels, but those labels are re-used afterward to compute metrics (see [the forums](https://discuss.huggingface.co/t/label-smoothing-and-compute-metrics-in-trainer/9778) for instance). This PR fixes that issue

Comments:
additional global attended token in bigbird-roberta
Author: calderma
Pull request: no

Hello, based on my understanding of the bigbird style attention mechanism, the only tokens in the ITC construction supported in HuggingFace that are global are the first and last tokens. Is there an easy way to add an additional global token position that should always be attended to? For example, if I want to make sure the second token in the sequence is always globally attended to, where in the modeling_bigbird.py (or elsewhere) do I need to add that, or is there another easier way to pass additional globally attended token positions? Any help would be greatly appreciated. Thank you.

Comments:
cc @vasudevgupta7 

Hey @calderma, sorry for late reply. I somehow missed your comment.

Yes, only ITC code is supported for now and hence only 1st & last **blocks** (i.e collection of tokens) are global. So, you can control which tokens should be global by playing around with block_size a bit, though this would increase compute a bit as random & sliding tokens are also dependent on it.

But in your case, as you want to make 2nd token as global, it will be a global token if block size is > 2 (Note: default block size is 64).

Ah ok got it. I thought it was just the first token, not the first block. My mistake. Thanks!