noamaanMulla-03/transformers-issues huggingface dataset:
[Reformer, Longformer, Roberta, GPT2, CTRL] attention_mask should be at second argument
Author: patrickvonplaten
Pull request: no
Labels: name: wontfix

Reformer, Longformer and Roberta have some models where the `attention_mask` is not at the second position. IMO, this was not done on purpose, but sloppy implementation. In order to use `torchscript` with `attention_mask`, the `forward()` args should be refactored. This is a breaking change however.

Additionally, GPT2 and CTRL also don't have `attention_mask` at their 2nd position in the forward pass. One can argue that it's more intuitive to have `past_key_values` at the second position, but leaving it there would also mean that torchscript + `attention_mask` can never really be used with GPT2. I think we should re-order the position_ids here as well, even though this is a big breaking change.



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[XLNet] attention_mask / input_mask - Why two `attention_mask` inputs?
Author: patrickvonplaten
Pull request: no
Labels: name: wontfix

For whatever reason XLNet accepts both an `attention_mask` and a `input_mask`. As far as I understand `attention_mask` = 1 - `input_mask`. I don't think having the same input twice (one is the inverse of the other) has any advantage. We should remove `input_mask` IMO (first depreciate, then remove). Also, `attention_mask` should be put in the 2nd positino to make model compatible with torchscript.

Comments:
I agree! Deprecation until v4.0.0 seems like a reasonable solution,

(We'll need at least one release with it before 4.0.0 if we want to remove the deprecation at 4.0.0)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Setting up transformers/examples/seq2seq
Author: jsrozner
Pull request: no
Labels: name: wontfix

 @sshleifer 

I'm following the instructions in the README for seq2seq. In particular, I forked, cloned, and then ran "pip install -e ."

But then when I tried to run finetune.sh, a number of libraries had not been installed. I had to just manually install them with pip (e.g. rouge_score, git, sacrebleu). Should these all have been included automatically? Or was there a better way to get them?

Is this still the best way to go about finetuning a seq2seq model on a custom task (in my case I am doing T5)?


Comments:
You can install those libraries by doing `pip install -r requirements.txt` in the `transformers/examples` folder!

Aha, thank you! 

Could I submit a pull request with updates to the README as I go through and encounter these little snags?

Also, is there a reason that the README in the examples/seq2seq directory doesn't show up here? https://huggingface.co/transformers/model_doc/t5.html#t5forconditionalgeneration 

I couldn't find the (quite helpful) README (in the examples/seq2seq dir) and example scripts for training a seq2seq model until I started exploring the github repository!

I think all we are missing is some pointer to examples/README.md. Would that have helped you?

I would add
- mention to both pip install -e ., pip instal -r requirements.txt
- reference the seq2seq readme from the transformers T5 conditional generation page

The only other bug I've encountered so far, #7426, was fixed already!

finetune.py ended up being a great script for me. But I wasn't sure if I needed to do any other special things. Is there a place where it would be good to write up the couple of steps I did need? For example:
- load tokenizer /model and modify the vocab (special tokens), then resave them locally (then use load_from_pretrained() on these files)
- prep data files ({val|test|train}.{source|target}) and put into directory
- mod the finetune.sh command

And I do have one other question:
It seems sort of weird that these models (e.g. SummarizationModel) are hidden in the examples directory. Is it because you guys don't generally expect them to be subclassed? I would have expected them to be in, say, transformers/language_generation or similar. 

`SummarizationModule` is a pytorch_lightning.Module. The models under src/ are `nn.Module`.

The main package under `src/` does not depend on pytorch_lightning, or have scripts to train things. Everything that does is under examples/

You could write a forums post with your steps needed: would be super helpful!
https://discuss.huggingface.co/

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
quick questions about the `BertModelLMHeadModel`.
Author: h56cho
Pull request: no
Labels: name: wontfix

Hello,
I have a few questions about the `BertModelLMHeadModel`:

1. Is `BertModelLMHeadModel`  used to conduct the causal language modeling (next token prediction), as it is the case for the `GPT2LMHeadModel`?

2. For `GPT2LMHeadModel`, I can just specify `labels = input_ids` for convenience. I just specify the `labels` in this way for the `BertModelLMHeadModel` as well?

Thanks,

Comments:
Bert is a Masked Lanuage Model. 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Problem with converting XLM checkpoint to pytorch (missing merges.txt)
Author: sebastian-nehrdich
Pull request: no
Labels: name: wontfix

Hello, 

I would like to use a checkpoint that I trained with XLM before (https://github.com/facebookresearch/XLM). 
For that I am using the following script: https://github.com/huggingface/transformers/blob/master/src/transformers/convert_xlm_original_pytorch_checkpoint_to_pytorch.py
The conversion works fine but in the end I only get a vocab.json but merges.txt is missing (which is no surprise looking at the script above). Is there any way to reconstruct the merges.txt from the vocab? I also still have the original training data used for the model and the bpe codes etc. used with XLM if that is of any help.



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
XLNet finetuning
Author: alshahrani2030
Pull request: no
Labels: name: wontfix

I am trying to fine-tune XLNet model and it used work fine but I think huggingface update some classes and I ran through this error:

RuntimeError: Trying to create tensor with negative dimension -1: [-1, 768]

![xlnet](https://user-images.githubusercontent.com/55197626/94844162-d030f780-03eb-11eb-84e4-e72deaaa71ec.PNG)


Comments:
Hello! Could you run `transformers-cli env` in your environment and paste the result here? Thank you!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Finetuning T5: Keyword arguments not recognized.
Author: MichaelJanz
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

## Details
Hi, I want to finetune the T5-small model for summarization purposes, to finetune the T5-large later. I prepared my data as as shown in the examples but during training i receive the message: 
`Keyword arguments {'src_lang': None, 'tgt_lang': None, 'add_prefix_space': False} not recognized.`
which indicates to me, that somehow my data preparation process is wrong. However, I was not able to find out how the data for T5 has to be prepared properly (as the T5 is a multi-ability model, the data has to be marked somehow, but I dont know how).
Currently my train data looks as the follow:
train.source: 
Line 1: A long text
Line 2: Another long Text

train.target:
Line 1: 'target':'Target for the text' 
Line 2: 'target':'Another target for the text' 

This question is in my view to specific to the transformer huggingface-architecture as I use the `finetune_sh` script in the seq2seq example folder. A small example, of how the data has to be structured for T5 will be very helpful, thank you! 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Is it possible to finetune reformer model for summarization task? 
Author: banunitte
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
There are no pre-trained reformer weights yet -> so that's a no sadly

Following this issue for updates.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
GPT2LMHeadModel forward input
Author: manzar96
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help
Hello,
I would like to fine-tune the GPT2 model on EmpatheticDialogues doing kind of conditional generation as like in this paper: https://arxiv.org/pdf/1911.11161.pdf
What concerns me is the format of the input_ids and labels in the forward function.
I think that concatenating the input with the target is a good solution separating them with a special token 
(e.g. "hi! how are you? <endofinput> I am fine!)
However I am not sure what to do with the labels. Shall I mask all the input part and the padded tokens with -100 index and leave only the target part as is? or shall I mask with -100 only the padded tokens?

Thank you in advance :)

Comments:
Hello! Your question would get more answers if you asked it over at https://discuss.huggingface.co, which are the forums for broad questions like this one. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Changing learning rate for BertModelforTokenClassification
Author: YojanaGadiya
Pull request: no
Labels: name: wontfix

Dear all,

I wanted to set a different learning rate for the linear layer and the Bert model for a BertModelforTokenClassification. How can I do so? This change should be done after loading a locally saved BertModelforTokenClassification model.

Thank You.

Comments:
You can specify the learning rate in the optimizer, see the example using SGD below (ref: https://discuss.pytorch.org/t/different-learning-rate-for-a-specific-layer/33670)

```
optim.SGD([
                {'params': model.base.parameters()},
                {'params': model.classifier.parameters(), 'lr': 1e-3}
            ], lr=1e-2)
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[Feature Request] Extract Predictions from Trainer
Author: geblanco
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Add the possibility to return predictions along with its example id in the new Trainer class.

## Motivation

When working with extractive QA (i.e.: SQuAD), you get back the best predictions, but, the current example for running squad uses the old, plain training/eval script, without the new Trainer class.

Additionally, there are other tasks where predictions can be extremely useful (i.e.: Multiple Choice).

Adding such functionality in the Trainer class could solve this and unify both question answering and multiple choice examples.

## Your contribution

I am familiarized with the code (both the Trainer class and the old train/eval script), so I could submit a PR with the new functionality.

Comments:
Unless I'm mistaken, Question-answering does support Trainer. Is a specific feature missing for your use case?

Hi @julien-c!

At the time of writing the PR was not merged yet (and I was not aware of it, my bad). It is great!

In any case, the previous example (the one without trainer) outputted both `predictions` and `nbest_predictions`, which are crucial for error analysis.

The new example with trainer looses this ability (because the new dataset/trainer API does not support it). Adding this feature could benefit not only SQuAD, but other datasets too (i.e.: RACE).

What do you think on this regard?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Any updates on this matter? I recently started using `transformers` and running the multiple-choice example I realized there's an output of the overall accuracy but I can't find the predictions.  Currently going through the code to obtain them...

Hi @dianags, 

Might be a little late, but tired of waiting, I programmed my own solution that output predictions and some more data, the code can be found [here](https://github.com/geblanco/mc_transformers)

Best,

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Missing keys when loading weights in TF are not useful
Author: patrickvonplaten
Pull request: no
Labels: name: wontfix

## This concerns all TF models 

If one loads weights of a tensorflow model these lines are run:
https://github.com/huggingface/transformers/blob/9e68d075a4100906509170498480823e7e61874a/src/transformers/modeling_tf_utils.py#L627
to check which layers are in the model weights file and which layer names of the model are actually loaded.

The problem is that these layer names consists only of the "highest" layer names of a model. *E.g.* for *TFBertForMaskedLM*, these layer names are just:
"bert" and "mlm",

but a name for each weight as it should be.

See:
https://github.com/huggingface/transformers/blob/3c6bf8998fb6ca5aca063fed2543b7176883b004/src/transformers/modeling_tf_bert.py#L865

So the missing keys argument for tensorflow will only capture the most "high" level missing weights.


Comments:
Fixed in #7422 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Problem while using tokenizer.encode_plus for sentence pairs
Author: rxlian
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
Hi, thanks for this great work.
I was trying to use tokenizer.encode_plus to encode sentence pairs. The code looks like

```py
training_encoded_dict = tokenizer.encode_plus(
            seq0,
            seq1,
            add_spicial_tokens = True,
            max_length = 256,
            truncation_strategy = 'only_second',
            pad_to_max_length = True,
            return_attention_mask = True,
            return_token_type_ids = True,
            return_tensors = 'pt',

        )
```

However, the problem looks like
```
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-17-ae3ce93d62ba> in <module>()
     24             return_attention_mask = True,
     25             return_token_type_ids = True,
---> 26             return_tensors = 'pt',
     27 
     28         )

2 frames
/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py in truncate_sequences(self, ids, pair_ids, num_tokens_to_remove, truncation_strategy, stride)
   2068             ids = ids[:-num_tokens_to_remove]
   2069         elif truncation_strategy == "only_second":
-> 2070             assert pair_ids is not None and len(pair_ids) > num_tokens_to_remove
   2071             window_len = min(len(pair_ids), stride + num_tokens_to_remove)
   2072             overflowing_tokens = pair_ids[-window_len:]

AssertionError: 
```

This issue doesn't occur while using truncation_strategy = 'longest_first', and it happens with other truncation_strategy such as 'only_second' and 'only_first'.
I was wondering does anyone have the same issue or have any idea about how to fix it? Thanks a lot in advance.
**A link to original question on the forum/Stack Overflow**:

Comments:
Could you provide some `seq0` and `seq1` values so that we may investigate further?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Distilbart's summaries start with an empty space?
Author: songwangnlp
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.11.0
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?): 1.51, gpu
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help


 

 @sshleifer 



Steps to reproduce the behavior:

```
from transformers import BartTokenizer, BartForConditionalGeneration, BartConfig
model = BartForConditionalGeneration.from_pretrained('sshleifer/distilbart-xsum-12-3')
tokenizer = BartTokenizer.from_pretrained('sshleifer/distilbart-xsum-12-3')
ARTICLE_TO_SUMMARIZE = "\"The accident meant the motorway was closed, making travel to Mourneview Park impossible for the team and fans travelling from Belfast, \" said the Irish Football Association . A new date for the match has yet to be confirmed by Uefa . Northern Ireland have three points from their first two Group Six qualifiers."
inputs = tokenizer.batch_encode_plus([ARTICLE_TO_SUMMARIZE], max_length=512, return_tensors='pt')
summary_ids = model.generate(inputs['input_ids'], num_beams=5, max_length=62, min_length=10, early_stopping=True)
print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in summary_ids])
```
[" The Republic of Ireland's Euro 2016 qualifier against Northern Ireland has been postponed because of a motorway crash."]

facebook/large-bart-xsum will give a correct summary. Why this summary starts with an empty space? What shall we pay attention to these inconsitency



Comments:
I think i figure it out now. in BART, " The", " Republic" are individual words.

I'm having a similar issue as @songwanguw where each line of my output summaries start with a blank space when using Distilbart. This does not happen when I use facebook/bart-large-cnn.

Unfortunately, I don't know any way to fix this besides postprocessing or retraining.
You're using the model correctly. I probably trained on targets with an extra prefixed space. My bad!


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Error while loading albert for token classification
Author: nirajkale
Pull request: no
Labels: name: wontfix

## Environment info
- `transformers` version: 3.0.2
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.6.10
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik
@jplu 
 
## Information

Model I am using albert-base-v2 or albert-base-v1:

The tasks I am working on is:
token classification using albert-base-v2 or v1

## To reproduce
```

>>> from transformers import AlbertTokenizer, TFAlbertForTokenClassification
>>> import tensorflow as tf

>>> tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2', cache_dir = 'cache')
>>> model = TFAlbertForTokenClassification.from_pretrained('albert-base-v2', cache_dir = 'cache')
```

When I run above script I get error:

```

Traceback (most recent call last):
  File "C:\Users\703235761\AppData\Local\Continuum\anaconda3\envs\slot\lib\site-packages\transformers\modeling_tf_utils.py", line 581, in from_pretrained
    raise EnvironmentError
OSError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "model_utils.py", line 7, in <module>
    model = TFAlbertForTokenClassification.from_pretrained('albert-base-v1', cache_dir = 'cache')
  File "C:\Users\703235761\AppData\Local\Continuum\anaconda3\envs\slot\lib\site-packages\transformers\modeling_tf_utils.py", line 588, in from_pretrained
    raise EnvironmentError(msg)
OSError: Can't load weights for 'albert-base-v1'. Make sure that:

- 'albert-base-v1' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'albert-base-v1' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.
```

## Expected behavior

I think this model was supposed to work with TFAlbertModel as well.
Thanks in advance! :-) 


Comments:
Quick update:
Above code works just fine in Ubuntu environment with below specs:
- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-1034-azure-x86_64-with-debian-buster-sid
- Python version: 3.6.10
- PyTorch version (GPU?): 1.5.1 (True)
- Tensorflow version (GPU?): 2.0.0 (True)
- Using GPU in script?: yes (Tesla P40)
- Using distributed or parallel set-up in script?: No

I think this issue only occurs in Windows.


Hi! This may be related to a network error, can you download other models on your Windows machine?

set REQUESTS_CA_BUNDLE env var to ca-certificates.crt

In my case I am using Ubuntu, so running the following command on terminal solves the issue:

` export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt`

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.