noamaanMulla-03/transformers-issues huggingface dataset:
wrong variable name used
Author: elyesmanai
Pull request: yes
Labels: name: model card

Turned 
`nlp.tokenizer.mask_token` to `fill_mask.tokenizer.mask_token` 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4328?src=pr&el=h1) Report
> Merging [#4328](https://codecov.io/gh/huggingface/transformers/pull/4328?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/241759101e7104192d01a07fc70432fa02ae8cb7&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4328/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4328?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4328   +/-   ##
=======================================
  Coverage   78.18%   78.18%           
=======================================
  Files         120      120           
  Lines       20020    20020           
=======================================
+ Hits        15652    15653    +1     
+ Misses       4368     4367    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4328?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4328/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.08% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4328?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4328?src=pr&el=footer). Last update [2417591...f0bb172](https://codecov.io/gh/huggingface/transformers/pull/4328?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
wrong configuration of ALBERT xlarge
Author: zheyuye
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information
If you are **reproducing** the  [ALBERT](https://github.com/google-research/albert), the `num_attention_heads` of albert xlarge should be 32 instead of 16 as
https://github.com/huggingface/transformers/blob/db9279dedbb9c5e7d24569a1ac3f74f9d5c3eb18/src/transformers/configuration_albert.py#L27


Comments:
Where did you get 32 from?

The [official file ](https://tfhub.dev/google/albert_xlarge/2)says 16.

Well, the model configuration in tar file downloaded from TF Hub shows 32, which conflicts with the official definition.

Indeed, that's right. I'll follow the issue you opened https://github.com/google-research/ALBERT/issues/180 and act accordingly.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Add image and link to model paper
Author: ViktorAlm
Pull request: yes
Labels: name: model card

Now i've had my fun. I thought i could do better but it will do :)

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4331?src=pr&el=h1) Report
> Merging [#4331](https://codecov.io/gh/huggingface/transformers/pull/4331?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/241759101e7104192d01a07fc70432fa02ae8cb7&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4331/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4331?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4331   +/-   ##
=======================================
  Coverage   78.18%   78.18%           
=======================================
  Files         120      120           
  Lines       20020    20020           
=======================================
  Hits        15652    15652           
  Misses       4368     4368           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4331?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4331/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.44% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4331/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.93% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4331?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4331?src=pr&el=footer). Last update [2417591...65588fe](https://codecov.io/gh/huggingface/transformers/pull/4331?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


lol, it's great

You can also add it as a metadata for it to display on social media:

```
---
language: norwegian
thumbnail: https://i.imgur.com/yxnE5GC.png
---
```
[Docs, Notebook] Include generation pipeline
Author: patrickvonplaten
Pull request: yes

- adds the generation pipeline to usage docs with a couple of examples
- adds the generation pipeline to the notebook
- Also, I updated the config of all models that can be used with generation with model specific params: GPT2, OpenAI-GPT, XLNet, TransfoXL, Reformer, so that they now include a `text-generation` special params dict.

Comments:
Get the CNN/Daily Mail Data for BART
Author: andr-ec
Pull request: no
Labels: name: wontfix

@sshleifer
In the README for the BART summarization it says: 

> download both CNN and Daily Mail datasets from Kyunghyun Cho's website
`tar -xvf cnn_stories.tgz && tar -xvf dailymail_stories.tgz`
> this should make a directory called cnn_dm/ with files like test.source. To use your own data, copy that files format. Each article to be summarized is on its own line.

This doesn't produce a cnn_dm directory, it produces two different folders. The contents of the folders are `text.story` files, not `test.source`. 

Did you use [this repo](https://github.com/artmatsak/cnn-dailymail)? 
Or did you get the data from somewhere else? 

Happy to submit a PR either way!
Thanks!



Comments:
Its a typo in the docs. I just used cnn.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
error load bert model ：not found model file
Author: iambyd
Pull request: no
Labels: name: wontfix

Error content:
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] found in directory ./uncased_L-12_H-768_A-12_transformers or `from_tf` set to False
but file exists
![image](https://user-images.githubusercontent.com/11927058/67140873-4d259880-f291-11e9-8e27-25ae7fb661f7.png)


Comments:
I don't understand when you get this error?

In order to understand when you've encountered this bug, as suggested by @iedmrc , you've to write down the source code that generates the bug! And please show your environment (Python, Transformers, PyTorch, TensorFlow versions) too! 

> Error content:
> OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] found in directory ./uncased_L-12_H-768_A-12_transformers or `from_tf` set to False
> but file exists
> ![image](https://user-images.githubusercontent.com/11927058/67140873-4d259880-f291-11e9-8e27-25ae7fb661f7.png)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I got the same error when loading a TF BERT model: 
```
dir = "/Users/danielk/ideaProjects/farsi-language-models/src/models/perbert_L-12_H-768_A-12/"
tokenizer = BertTokenizer.from_pretrained(dir)
config = BertConfig.from_json_file(dir + '/bert_config.json')
model = TFBertForMaskedLM.from_pretrained(dir, config=config)
```
The error happens in the last line. 
```
Traceback (most recent call last):
  File "6.2.try_tf_bert_transformers.py", line 8, in <module>
    model = TFBertForMaskedLM.from_pretrained(dir, config=config)
  File "/Users/danielk/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_tf_utils.py", line 353, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /Users/danielk/ideaProjects/farsi-language-models/src/models/perbert_L-12_H-768_A-12/ or `from_pt` set to False
```
Align sentiment-analysis' tokenizer (currently cased) to the model (uncased)
Author: mfuntowicz
Pull request: yes

`sentiment-analysis` pipeline has a tokenizer using a cased vocab while the model is uncased. 

This PR makes the tokenizer uncased to match the actual model's vocab.

Comments:
LGTM, or even better we could duplicate those tokenizer files into `distilbert-base-uncased-finetuned-sst-2-english` as now we kind of assume that a model == model + tokenizer.

Merging this for now and can update to @julien-c's proposal in a future PR

Yeah, just wanted to ping @mfuntowicz on this model/tokenizer subject 

Hi guys, I pulled from master after the merge of the PR but the behaviour is still the same. It doesn't look like aligning the tokenizer resolved this issue.
TPU needs a rendezvous
Author: LysandreJik
Pull request: yes

When running on TPU, each ordinal needs to call `save_pretrained`. See [original implementation by @jysohn23](https://github.com/huggingface/transformers/pull/3702/files#diff-28e1baa470caa8e6f23b78a356b6bbdfR162).

If we don't do it this way, the trainer hangs when saving the model.

Comments:
is pytorch-pretrained-bert still being maintained in the future?
Author: yuhujia
Pull request: no
Labels: name: wontfix

## 📚 Migration

<!-- Important information -->

Model I am using (Bert, XLNet....):

Language I am using the model on (English, Chinese....):

The problem arise when using:
* [ ] the official example scripts: (give details)
* [ ] my own modified scripts: (give details)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details)

Details of the issue:

<!-- A clear and concise description of the migration issue. If you have code snippets, please provide it here as well. -->

## Environment

* OS:
* Python version:
* PyTorch version:
* PyTorch Transformers version (or branch):
* Using GPU ?
* Distributed or parallel setup ?
* Any other relevant information:

## Checklist

- [ ] I have read the migration guide in the readme.
- [ ] I checked if a related official extension example runs on my machine.

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
Hi, `pytorch-pretrained-BERT` is the name of this library as it was a year ago. It has since evolved into `pytorch-transformers` and now `transformers`. It is the same library.

Hi thank you for your reply. But my question really is that I'm now using apis from pytorch-pretrained-BERT directly and will this library be maintained under new release (new python release, bug fixed, etc)? 

The reason is that I found some discrepancies between apis from pytorch-pertrained-BERT library and transformers library and the old one (from pytorch-pretrained-BERT) gave better results so I'm sticking with that library.

No updates will be done to the `pytorch-pretrained-BERT`, no bug fixes either. It is deprecated. It will remain on pip however.

Would you mind sharing where the `pytorch-pretrained-BERT` package gave better results so that we may investigate this? Thank you.

yes i'm encountering performance drop with sequence classification tasks, same as issues described in this thread: https://github.com/huggingface/transformers/issues/938.

@LysandreJik There seem to be quite a few posts that highlight this difference in performance. It is quite alarming but I'm not sure if it is worth investigating because it might be impossible or improbable to solve.

https://github.com/huggingface/transformers/issues/938
https://github.com/huggingface/transformers/issues/931
https://github.com/UKPLab/sentence-transformers/issues/50
https://github.com/huggingface/transformers/issues/2441

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Encoder - Decoder loading wrong weights and missing decoders
Author: flozi00
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Roberta as encoder and Bert as decoder

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. https://github.com/ThilinaRajapakse/simpletransformers/pull/360

## Expected behavior
Predicting the same string before saving and after loading the saved model are completely different.
Expected that i get the same results both times

Furthermore your tutorial on medium is not working, caused missing decoder models.
Link to medium: https://medium.com/huggingface/encoder-decoders-in-transformers-a-hybrid-pre-trained-architecture-for-seq2seq-af4d7bf14bb8
There is missing an argument in the forward function:
`TypeError: forward() got an unexpected keyword argument 'encoder_hidden_states'`

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform: Linux
- Python version: 3.7
- PyTorch version (GPU?): GPU 1.4.0
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Comments:
Pretraining QA corpora from scratch with sentence pairs
Author: yuanbit
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I would like to pretrain a non-factoid QA corpora (question and answer passages) from scratch using BERT. I have looked at both:

https://huggingface.co/blog/how-to-train and 
https://gist.github.com/aditya-malte/2d4f896f471be9c38eb4d723a710768b#file-smallberta_pretraining-ipynb

I would like to confirm that what I am doing is correct: 

1. I concatenated the question and answers with a [SEP] token, so each line in my input data file looks like:
question [SEP] answer
2. I am running the script with --line_by_line

I am uncertain if this is correct because how would the script know which is sentence A and which is sentence B, or is this not necessary? 

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[docs] bad RST in PretrainedModel.generate docstring
Author: sshleifer
Pull request: no

![image](https://user-images.githubusercontent.com/6045025/80641266-96218b00-8a32-11ea-93d6-36484b3aad92.png)
https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_utils.py#L857

Comments:
fixed in Marian PR
Tokenizer.batch_decode convenience method
Author: sshleifer
Pull request: yes

convenience method that turns the list returned by `model.generate` into a list of sentences.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4159?src=pr&el=h1) Report
> Merging [#4159](https://codecov.io/gh/huggingface/transformers/pull/4159?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/7822cd38a0e18004ab1a55bfe85e8b3bc0d8857a&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4159/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4159?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4159   +/-   ##
=======================================
  Coverage   78.14%   78.15%           
=======================================
  Files         120      120           
  Lines       20053    20053           
=======================================
+ Hits        15670    15672    +2     
+ Misses       4383     4381    -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4159?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/4159/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWFyaWFuLnB5) | `87.32% <ø> (-0.35%)` | :arrow_down: |
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4159/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `89.51% <100.00%> (+0.02%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4159/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.60% <0.00%> (+0.12%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4159/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.85% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4159?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4159?src=pr&el=footer). Last update [7822cd3...cc21682](https://codecov.io/gh/huggingface/transformers/pull/4159?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I'm in favor of this since we also have a `batch_encode_plus` method. 

One small thing: the `encode_plus` method uses the `batch_encode_plus` whereas here it's the other way around. But this does not bother me too much.

I don't have a strong knowledge of the API design for tokenizers so will let others chime in instead.
Trainer doesn't calculate eval loss for models with lm_labels parameter
Author: patil-suraj
Pull request: no

# 🐛 Bug
The trainer doesn't calculate evaluation loss for models with `lm_labels` parameters in the `forward` method. This line https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L682 is the source of the bug
`has_labels = any(inputs.get(k) is not None for k in ["labels", "masked_lm_labels"]) `
loss is only calculated when `has_labels` is `True` and its only `True` for models with `labels` and `masked_lm_labels` parameters and not for `lm_labels`

## Information

Model I am using (Bert, XLNet ...): T5

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
Use any model with `lm_labels` parameter in the `forward` method, ex T5 or BART and evaluate using trainer

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Evaluation loss should be calculated for models with `lm_labels` parameter as well.

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.9.1
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: Python 3.6.9
- PyTorch version (GPU?):  1.5.0+cu101
- Tensorflow version (GPU?): Not applicable 
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Comments:
p_mask in SQuAD pre-processing
Author: LysandreJik
Pull request: yes

In question answering, the `p_mask` is a mask with 1 for tokens than cannot be in the answer (0 for token which can be in an answer).

Currently the p_mask construction fails for RoBERTa, and is overall not very robust as it depends directly on the `token_type_ids`, while some models do not make use of them and therefore do not generate them (best example being RoBERTa).

This PR changes the way the p_mask is built, and slightly changes it as well. Up to now, here's what was done with the `p_mask`, I believe with the original implementation being google-research BERT's SQuAD script: 

- Consider every token that is not the question to be a possible answer. This includes special tokens, and padding tokens.

This PR changes this by removing those special tokens and padding tokens from the possible answer pool. It does still keep the classification token as a possible answer, given that it's sometimes used as the `impossible` token.

closes https://github.com/huggingface/transformers/issues/2788

Comments: