noamaanMulla-03/transformers-issues huggingface dataset:
"Write With Transformer" not generating text (502 Bad Gateway)
Author: flarn2006
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): (Distil-)GPT2 on WriteWithTransformer

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [X] the official example scripts: Write With Transformer
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: Not sure what the task is called but it's WWT
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Attempt to use the autocomplete on Write With Transformer
2. Notice it appears to be loading forever
3. Open the browser console, go to the Network tab, and try again
4. Observe the "502 Bad Gateway" error

```
HTTP/1.1 502 Bad Gateway
Server: nginx/1.14.2
Date: Wed, 01 Jul 2020 12:14:42 GMT
Content-Type: text/html
Content-Length: 173
Connection: keep-alive
X-JeanClaude: True
Access-Control-Allow-Headers: Content-Type

<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.14.2</center>
</body>
</html>
```

## Expected behavior

The autocomplete should appear as normal.

## Environment info
You'd know that better than I do.


Comments:
@LysandreJik is rebooting the Raspberry Pi right now

Joking, I mean the Hugging Face data center

It's back up!

Thanks! It does work now, but it seems slower to respond and sometimes it times out. This is to the point where it's close to unusable. Do you happen to know whether this is on my end or yours?

Yes, there seems to be an issue. I'm fixing restarting the server.

So I guess that would be why I just started getting 502 Bad Gateway again? :)
Thanks for the help btw.

Everything should be back to normal now. Thanks for letting us know!

Yep, I was using it earlier and it appears so. Glad I could help!
[Benchmark] Parameter settings of XLM-R on NER tasks
Author: lixin4ever
Pull request: no
Labels: name: wontfix

# 🖥 Benchmarking `transformers`

## Benchmark

`XLM-RoBerta` on CoNLL 03 English NER dataset

## Set-up
`model_name_path`: xlm-roberta-base
`max_steps`: 3000 (roughly 6 epochs)
`warmup_steps`: 300
`save_steps`: 200
`learning_rate`: 5e-5
`per_gpu_train_batch_size`: 8
`n_gpu`: 3 (Titan X)

## Results
91.017 (averaged results of five runs)

## Issues
As reported in original [paper](https://arxiv.org/pdf/1911.02116.pdf), XLM-R can achieve 92.25 on this English NER dataset. Since they do not share any parameter settings for fine-tuning, I have to adopt the above ones and the F1 score is about 91.017. Has anybody ever obtained similar results to those in this paper with appropriate settings?


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Decoding predictions for masked language modeling task using custom BPE
Author: singhay
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 
https://stackoverflow.com/questions/61232399/decoding-predictions-for-masked-language-modeling-task-using-custom-bpe

Comments:
Maybe @mfuntowicz ? :-) 

Bump

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
🐛 [TF] `create_optimizer` wrong superposition of learning rate schedules
Author: astariul
Pull request: no

# 🐛 Bug

When using `create_optimizer`, 2 learning rate schedulers are placed on top of each other (WarmUp and keras Polynomial Decay) :

https://github.com/huggingface/transformers/blob/efeb75b8054cc299698cf8bc09f395ada2660745/src/transformers/optimization_tf.py#L70-L80

But the step in Warmup scheduler is not updated to take into account the warmup steps, which lead to a wrong learning rate :

![image](https://user-images.githubusercontent.com/43774355/84980216-69edd500-b16c-11ea-97f6-44642bdc64ce.png)

---

Expected learning rate shape :

![Dessin sans titre](https://user-images.githubusercontent.com/43774355/84981152-d2d64c80-b16e-11ea-9092-ccc50217fc66.png)





Comments:
why the accuracy is very low when we test on sentences one by one of the words input?
Author: sqkika
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help
hi, friends:
there is a problem , that we trained a good lm model on gpt2 and sequence len is 30,when we batch test in training with 30 seqlen, the accuracy can reach 90%. But when we test like this: 
[1]->[2],[1,2]->[3],[1,2,3]-->[4], left is the input, and we hope it can predict the right, we found the accurcy is very low which is only 6%.



Comments:
when we were training, the accuracy count method is like this: 
input: [0,1,2,3,4,5,...,29]
label:  [1,2,3,4,5,6,...,30]
so we count the right hits in 30 seqs.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Unable to use run_squad with xla_spawn.py on TPU
Author: dhruvluci
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
Electra

Language I am using the model on (English, Chinese ...):
ENG

The problem arises when using:
* [ ] the official example scripts: RUN_squad.py + xla_spawn.py


The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name): official SQUaD task


## To reproduce

Steps to reproduce the behavior:

1. Install pytorch-xla on colab using:
```
VERSION = "20200325"  #@param ["1.5" , "20200325", "nightly"]
!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py
!python pytorch-xla-env-setup.py --version $VERSION
```
2. Trying to run_squad on colab TPUs using xla_spawn.py
```
python examples/xla_spawn.py --num_cores 8 \
   examples/question-answering/run_squad.py \
  --model_type electra \
  --model_name_or_path google/electra-base-discriminator \
  --do_train \
  --do_eval \
  --do_lower_case \
  --train_file "/content/drive/My Drive/bert/train.json" \
  --predict_file "/content/drive/My Drive/bert/val.json" \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir "/content/drive/My Drive/bert/newdir6"

```

2. Error is thrown up
```
Traceback (most recent call last):
  File "examples/xla_spawn.py", line 72, in <module>
    main()
  File "examples/xla_spawn.py", line 68, in main
    xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
AttributeError: module 'run_squad' has no attribute '_mp_fn'

```
## Expected behavior
Training should run properly using xla_spawn.py, which it does for GLUE tasks using:
```
python examples/xla_spawn.py --num_cores 8 \
	examples/text-classification/run_glue.py
```


## Environment info

     
- `transformers` version: 2nd July 2020 clone.
- Platform: Colab
- Python version: 3.8
- PyTorch version (GPU?): 20200325 (pytorch-xla)
- Tensorflow version (GPU?):N
- Using GPU in script?:N
- Using distributed or parallel set-up in script?:N


Comments:
Hi! The SQuAD example doesn't have trainer support yet. We're in the process of adding it. You can see the supported tasks [here](https://github.com/huggingface/transformers/tree/master/examples#the-big-table-of-tasks), only the tasks with Trainer, TFTrainer or pytorch-lightning support can run on TPU.
Tokenizer summary
Author: sgugger
Pull request: yes

This PR introduces a mid/high-level summary of the different tokenizer types used in the library (a bit like the model summary).

Preview is [here](https://56179-155220641-gh.circle-artifacts.com/0/docs/_build/html/tokenizer_summary.html).

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5467?src=pr&el=h1) Report
> Merging [#5467](https://codecov.io/gh/huggingface/transformers/pull/5467?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/35befd9ce31c23a774fd34f57bc44033ce70141d&el=desc) will **decrease** coverage by `0.08%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5467/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5467?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5467      +/-   ##
==========================================
- Coverage   77.57%   77.48%   -0.09%     
==========================================
  Files         141      141              
  Lines       24581    24581              
==========================================
- Hits        19068    19046      -22     
- Misses       5513     5535      +22     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5467?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_xlm\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/5467/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxtX3JvYmVydGEucHk=) | `61.90% <0.00%> (-33.34%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5467/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `85.42% <0.00%> (+1.50%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5467?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5467?src=pr&el=footer). Last update [35befd9...80529fa](https://codecov.io/gh/huggingface/transformers/pull/5467?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
TFAutoModelForSequenceClassification: ValueError: Layer #1 (named "classifier") expects 2 weight(s), but the saved weights have 4 element(s).
Author: TheophileBlard
Pull request: no

# 🐛 Bug

## Information

`TFAutoModelForSequenceClassification` does not work on v3.0.0 / can't load a model that was working on v2.11.0

## To reproduce

Steps to reproduce the behavior:

- This work:

```python
!pip install transformers==2.11.0
from transformers import TFAutoModelForSequenceClassification
model = TFAutoModelForSequenceClassification.from_pretrained("tblard/tf-allocine")
```

but not this:
```python
!pip install transformers>=3.0.0
from transformers import TFAutoModelForSequenceClassification
model = TFAutoModelForSequenceClassification.from_pretrained("tblard/tf-allocine")
```

as it outputs:
```shell
ValueError: Layer #1 (named "classifier") expects 2 weight(s), but the saved weights have 4 element(s).
```

- Using `TFCamembertForSequenceClassification` instead of `TFAutoModelForSequenceClassification` also works.

- I couldn't find any other model using `TFAutoModelForSequenceClassification` on the model zoo to verify the issue does not come from the model itself. 

## Expected behavior

No errors.

## Environment info

Standard Google colab environment.

- `transformers` version: 3.0.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Comments:
Hi! Thanks for opening this issue. This should have been fixed by https://github.com/huggingface/transformers/pull/5414.
How to find a correct place of original word from the list of predicted words from GPT-2 model?
Author: states786
Pull request: no
Labels: name: wontfix

Hi,

I would like to calculate at which place correct word option lies in the top 5 predicted words from  GPT-2 model? 

For this purpose, I am using following code snippet:
```

        subseq = "The car moves very" #sample sequence
        orignal_word="fast"
        sequence = tokenizer.encode(subseq, return_tensors="pt")
        next_word_id = tokenizer.encode(orignal_word, return_tensors="pt")
        next_word = tokenizer.decode(next_word_id[0])
        next_word_logits = model(sequence)[0][0, -1].detach()
        probabilities, word_ids = next_word_logits.topk(5) #Getting top 5 next word options
        
        rank=1.0 
        for word_id in word_ids:
            word = tokenizer.decode([word_id])
            if word == next_word:
                   break;
            rank=rank+1.0

       print("Rank of Correct option is "+ str(rank))
```
I am not sure whether it is done perfectly or not  as GPT-2 model uses BPE tokenizer. Am I doing it in a right way? Kindly share your thoughts, and correct me if I am doing something wrong in it.
        

Comments:
It won't be that easy since some words will  be split into multiple tokens so you have to make two forward passes. 

If you limit your `original_word` to just one token words (you can check that simply with `len(tokenizer.encode(original_word))==1`. Then your idea here should work. 

If not it's gonna be trickier. Also this issue might be helpful: 
https://github.com/huggingface/transformers/issues/2311

Thanks @patrickvonplaten  for your response. 
Yes, the code works for `len(tokenizer.encode(original_word))==1`, but not for those `original_word` , which consist of more than one tokens.

I look at the shared issue, but I am confused, which selected word id, should I pass to the model again, as `next_word_logits.topk(5)` gives me 5 token ids?

Can you please share any code snippet, which will work for the second part?

Hi @patrickvonplaten,

can u plz let me know about any update?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Albert language model fine tuning not running run_lm_finetuning.py 
Author: abdallah197
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using (Albert(all types)):

Language I am using the model on (English):

The problem arises when using:
* [ ] the official example scripts: (give details below)
the code returns memory allocation problems when run with any version from albert. i tried to reduce the sequence length and batch size to a minum setting, but the issue still arises. my setting and the minimized setting both run normally with bert or roberta, the issue arises only when i change the model to Albert.
an example:
`tcmalloc: large alloc 1951195136 bytes == 0x7f750f664000 @  0x7f76efbf8887 0x7f764c2a1b79 0x7f764c29fb0f 0x7f764c29fc33 0x7f764c26a155 0x7f764c26837e 0x7f764c26bbb1 0x7f764c2606df 0x50a8af 0x50c5b9 0x509d48 0x50aa7d 0x50c5b9 0x508245 0x509642 0x595311 0x5a067e 0x50d966 0x58efc9 0x4c9546 0x5886f4 0x58892e 0x551b81 0x5aa6ec 0x50abb3 0x50c5b9 0x508245 0x50a080 0x50aa7d 0x50c5b9 0x508245`

The tasks I am working on is:
* [ ] my own task or dataset: (give details below)
language model finetuning for albert
## To reproduce

Steps to reproduce the behavior:

1. in run_lm_finetuning add:
` from transformers import (AlbertConfig,
    AlbertForMaskedLM,
    AlbertTokenizer,
)`
2.add to MODEL_CLASSES dictionary:
`    "albert": (AlbertConfig, AlbertForMaskedLM, AlbertTokenizer),`
3. add file text.txt, a similar txt file to the wiki dataset that's mentioned in the docs. 
4.run the finetuning script:
`python transformers/examples/run_lm_finetuning.py \
    --output_dir=output \
    --model_type=albert \
    --model_name_or_path=albert-base-v1 \
    --do_train \
    --train_data_file test.txt \
    --block_size 50 \
    --per_gpu_train_batch_size 2 \
    --max_steps 520000 \
    --weight_decay 0.01 \
    --logging_steps 5000 \
    --mlm`



## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment

* OS: Google colab
* Python version: 3.7
* PyTorch version: 1.3.1
* `transformers` version (or branch): latest
* Using GPU ? yes
* Distributed or parallel setup ? no
* Any other relevant information:


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I am facing the same problem with BERT fine tuning for a masked language modeling fine tuning task. Can someone please help? I am exactly following https://github.com/huggingface/transformers/tree/master/examples/language-modeling
Question about the output linear weight  
Author: snakeztc
Pull request: no
Labels: name: wontfix

I notice that the output linear weight for BertForMaskedLM is the same as input embedding. I don't read anywhere in the original paper that the output projection is tied with word embedding? Can you please explain the design choice here? 

To reproduce:
```
m = transformers.BertForMaskedLM.from_pretrained('bert-base-uncased')
m.get_input_embeddings().weight - m.get_output_embeddings().weight
```


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
🌟 Transformer Lite
Author: astariul
Pull request: no
Labels: name: wontfix

# 🌟 New model addition

## Model description

Part of the abstract :

> In this paper, we present an efficient mobile NLP architecture, Lite Transformer to
facilitate deploying mobile NLP applications on edge devices. [...] brings consistent improvement over the vanilla transformer on three well-established language
tasks: machine translation, abstractive summarization, and language modeling. [...] Notably, Lite Transformer outperforms the AutoML-based Evolved
Transformer by 0.5 higher BLEU for the mobile NLP setting without the costly
architecture search that requires more than 250 GPU years

## Open source status

* [x] the model implementation is available: [Repository](https://github.com/mit-han-lab/lite-transformer)
* [x] the model weights are available: [In the README](https://github.com/mit-han-lab/lite-transformer#models)
* [x] who are the authors: @Michaelvll


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
More flexible wandb support for Trainer
Author: Guitaricet
Pull request: no
Labels: name: trainer

# 🚀 Feature request

**A.** Make it possible to initialize wandb outside Trainer class.
**B.** Add `use_wandb` argument to the Trainer arguments.

## Motivation

**A.** Currently, wandb inside Trainer configuration if very limited. There are only three environment variables `WANDB_WATCH`, `WANDB_PROJECT`, and `WANDB_DISABLED`. (And `WANDB_DISABLED` does not work properly in some cases).

Making it possible to initialize wandb outside Trainer will allow us to:
1. Add custom fields to wandb.config
1. Add tags, notes and just to make configuration as flexible as possible
1. Upload files
1. Use `wandb.log` more safely outside the Transformers code

It will also make the interaction with wandb more clear.

**B.** It is a much more clear interface then an env variable. The question here is which option should have the priority?

## Your contribution

There's a picture in my mind about how to do this and not to destroy backward compatibility. I can make a PR, but maybe need some minor help on writing tests.

Comments:
Yeah I agree with the motivations here. From my experience I like to initialize wandb as soon as the main script starts which has benefits like 
- Capturing all the console logs, so if something failed before the execution reached the trainer class one could debug the issue through Wandb. This is specially useful for doing training in a kubernetes environment where the logs are not very easily available after a crash.
- Instantiating at the beginning also allows us to capture all the CLI arguments properly before they might have been modified. This is a problem right now because Wandb only captures the training_args leaving out model_args or data args. This is important for replicating a training run in the future.

Doesn't it work already when initializing wandb outside?

I believe that `wandb.init` does not create a new run if one is already running so all the functions should be available anywhere in the script.

When looking over the code, I was pretty sure it will initialize a new run. However, I checked and everything (mentioned in the issue) works smoothly. 

Ok, thanks for checking @Guitaricet
The inference speed of gpt2-xl has a gap between pytorch and tensorflow.
Author: zhm9484
Pull request: no

**Environment:**
- OS: Ubuntu 18.04
- Python: 3.7.6
- Transformers: 3.0.0
- PyTorch: 1.4.0
- Tensorflow: 2.2.0
- CUDA: 10.1
- CUDNN: 7.6
- GPU: V100

**My code:**
```import time
import torch
import tensorflow as tf
from transformers import AutoTokenizer, AutoModelWithLMHead, TFAutoModelWithLMHead

TIMES = 100

tokenizer = AutoTokenizer.from_pretrained("./gpt2-xl")

# pytorch
model = AutoModelWithLMHead.from_pretrained("./gpt2-xl")
model = model.to("cuda")
input = tokenizer.encode("This is the benchmark of gpt2-xl.", return_tensors="pt").to("cuda")
total = 0
cnt = 0
start = torch.cuda.Event(enable_timing=True)
end = torch.cuda.Event(enable_timing=True)
for i in range(TIMES):
    start.record()
    o = model(input)
    end.record()
    torch.cuda.synchronize()
    if not i:
        continue
    total += start.elapsed_time(end)/1000
    cnt += 1
print("Pytorch version --- cnt: {}, avg_time_cost: {}s".format(cnt, total/cnt))

# tensorflow
gpus = tf.config.experimental.list_logical_devices('GPU')
gpu = gpus[0].name
with tf.device(gpu):
    model = TFAutoModelWithLMHead.from_pretrained("./gpt2-xl")
input = tokenizer.encode("This is the benchmark of gpt2-xl.", return_tensors="tf")
total = 0
cnt = 0
with tf.device(gpu):
    for i in range(TIMES):
        start = time.time()
        o = model(input)
        end = time.time()
        if not i:
            continue
        total += (end-start)
        cnt += 1
print("Tensorflow version --- cnt: {}, avg_time_cost: {}s".format(cnt, total/cnt))
```

**Output:**
```
Pytorch version --- cnt: 99, avg_time_cost: 0.05521844493981564s
Tensorflow version --- cnt: 99, avg_time_cost: 0.2912752628326416s
```

**The utilization of gpu:**
- PyTorch 33%
- Tensorflow 8%

I'm new to Transformers. Did i use it in a wrong way, or any mistakes in my code?
Thanks!

Comments:
Turning off the eager mode of tensorflow makes inference much faster.