noamaanMulla-03/transformers-issues huggingface dataset:
How to use Huggingface pytorch bert to generate the prediction TSV file from the test set of a GLUE task?
Author: hrheru20
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Hello folks! Can you provide some simple example of using pytorch bert to generate the prediction TSV file from the test set of a GLUE task (such as MRPC) based on the fine-tuned model so that I can submit the prediction TSV file for each GLUE task to GLUE leaderboard? Thank you very much.



<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This was implemented ~1 month ago so closing this issue.
[Generate] beam search should generate without replacement
Author: patrickvonplaten
Pull request: yes

When doing beam search decoding and sampling instead of argmax (edge case, probably very rarely used), we need to sample **without** replacement. This is implemented by  default in torch, but not in TF, see https://pytorch.org/docs/master/generated/torch.multinomial.html#torch.multinomial (torch). An easy solution is to use the Gumbel max trick instead: https://github.com/tensorflow/tensorflow/issues/9260

This will fix the sometimes flaky TFBeamSearchGenerate Tests as well: #4447 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4845?src=pr&el=h1) Report
> Merging [#4845](https://codecov.io/gh/huggingface/transformers/pull/4845?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f9414f7553d3f1872b372990ef03205c0d1141df&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4845/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4845?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4845      +/-   ##
==========================================
- Coverage   76.06%   76.05%   -0.01%     
==========================================
  Files         128      128              
  Lines       21498    21502       +4     
==========================================
+ Hits        16352    16354       +2     
- Misses       5146     5148       +2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4845?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4845/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.12% <100.00%> (-0.24%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4845?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4845?src=pr&el=footer). Last update [f9414f7...d2201fe](https://codecov.io/gh/huggingface/transformers/pull/4845?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
TF Beam Search generation seems to be flaky sometimes
Author: patrickvonplaten
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): ALL TF generate models 

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)
* [ ] all generate beam search tests in TF

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Some commits are failing due to `Beam size should alway be full` in circle ci - this should actually never happen. See a failed circle ci here: https://circleci.com/gh/huggingface/transformers/39780?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link .

## Expected behavior

Circle ci should not fail with this message.

Comments:
BUG while calculate LM loss in AlbertForMaskedLM
Author: slczgwh
Pull request: no

# 🐛 Bug

Error Code Here: https://github.com/huggingface/transformers/blob/e33fdc93b4ecb571dd7a8002a74789ec8bfffc09/src/transformers/modeling_albert.py#L822


Here calculate the loss of all tokens in a seq. While MLM actually only calculate loss on [MASK]

Comments:
That's because the labels of all tokens *except* the mask tokens should be set to -100, as it's written in the [documentation](https://huggingface.co/transformers/model_doc/albert.html#albertformaskedlm).

Setting these tokens to -100 will result in the cross entropy ignoring them.
TFXLMRobertaForSequenceClassification: call() got an unexpected keyword argument 'labels'
Author: QixinLi
Pull request: no

# 🐛 Bug

## Information

Model I am using : TFXLMRoberta

Language I am using the model on : cross-lingual

The problem arises when using:
* [ ] the official example scripts: 
* [√] my own modified scripts: 
```python
import tensorflow as tf
from transformers import TFXLMRobertaForSequenceClassification,XLMRobertaTokenizer,XLMRobertaConfig

tokenizer = XLMRobertaTokenizer.from_pretrained('jplu/tf-xlm-roberta-base')
model = TFXLMRobertaForSequenceClassification.from_pretrained('jplu/tf-xlm-roberta-base')
input_ids = tf.constant(tokenizer.encode("Hello, my dog is cute"))[None, :]  # Batch size 1
labels = tf.reshape(tf.constant(1), (-1, 1)) # Batch size 1
outputs = model(input_ids, labels=labels)
loss, logits = outputs[:2]
```
and I got error here:
```
  File "run_classifier.py", line 180, in train
    outputs = self.model(input_ids,attention_mask = input_mask, token_type_ids = token_type_ids, labels=labels, training = True)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 891, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/modeling_tf_roberta.py", line 379, in call
    outputs = self.roberta(inputs, **kwargs)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 891, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
TypeError: call() got an unexpected keyword argument 'labels'
```

I read the notes, it says TFXLMRobertaForSequenceClassification class **overrides** TFRobertaForSequenceClassification.

And [TFRobertaForSequenceClassification](https://huggingface.co/transformers/model_doc/roberta.html#tfrobertaforsequenceclassification) class's call() method accepts 'labels' argument.

For the TFRobertaForSequenceClassification model's example code, I just change the model and tokenizer to XLM, and it got an error.

## Environment info

- `transformers` version:2.5.1
- Platform:MacOS Catalina
- Python version: 3.5
- PyTorch version (GPU?): 
- Tensorflow version (GPU?): 2.0
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


Comments:
Hi! You're on `transformers` version `v2.5.1`, but the TensorFlow models can only accept labels since this PR https://github.com/huggingface/transformers/pull/4530 was merged, 4 days ago.

This currently isn't available in any release, so you will have to install from source to use that feature:

```
pip install git+https://github.com/huggingface/transformers
```

@LysandreJik thanks for your reminder！
Remove double bias in TF Albert
Author: LysandreJik
Pull request: yes

closes #3386

Comments:
Also remove the now-unused bias variable on https://github.com/huggingface/transformers/blob/50c9c3e98235d447a7a2d0efebc9ff4e426fce2d/src/transformers/modeling_tf_albert.py#L467

You're absolutely right, thanks @jarednielsen !
Top-k sampling and top-p sampling for generating phrases on batches with GPT-2?
Author: Barbara931120
Pull request: no

How can I generate on batches with GPT-2 if I want to make use of these awesome sampling techniques: [top-k sampling and top-p sampling](https://huggingface.co/blog/how-to-generate)?. 

There is this implementation for generating phrases on batches already in issue [#3021](https://github.com/huggingface/transformers/issues/3021). 

Any advice? thanks!

@patrickvonplaten 


Comments:
It's on our ToDo-List :-) Currently batch generation with GPT2 is not possible, so you will have to rely on the code in https://github.com/huggingface/transformers/issues/3021

Can top-k and top-p sampling be implemented in batches?

Sure, the provided `top-k-top-p sampling` function provided that :-)
Argument “never_split” not working on bert tokenizer
Author: acmilannesta
Pull request: no
Labels: name: Core: Tokenization

I used the ```never_split``` option and tried to retain some tokens. But the tokenizer still divide them into wordpieces. 

```
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', never_split=['lol'])
tokenizer.tokenize("lol That's funny")
['lo', '##l', 'that', "'", 's', 'funny']
```
**A link to original question on Stack Overflow**: 
https://stackoverflow.com/posts/60914793/edit

Comments:
Does this problem arise with the fast tokenizer too? Can you try both:

```python
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', never_split=['lol'], use_fast=True)
# and
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', never_split=['lol'], use_fast=False)
```

> Does this problem arise with the fast tokenizer too? Can you try both:
> 
> ```python
> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', never_split=['lol'], use_fast=True)
> # and
> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', never_split=['lol'], use_fast=False)
> ```

Tried. Neither works...
However, if I tried to load customzied vocab which replace "[unused]" toakens with the ones I don't want to split. The tokenizer works.

But the default vocab only allows around 1k new tokens. If I add more, the embedding size will change. But the TF models will raise implementation error if I called this:
```
bert = TFBertModel.from_pretrained('bert-base-uncased')
bert.resize_token_embeddings(36000)
```


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This problem still exists where the "never_split"

```python
from transformers import BertTokenizer
USE_FAST = False

tokenizer = BertTokenizer.from_pretrained("bert-base-cased",
                                          use_fast=USE_FAST,
                                          never_split=['lol'],
                                          do_basic_tokenize=True)
print(tokenizer.tokenize(" lol That's funny"))
```
I started going through the code, but it's a bit of a rabbit hole since you have "never_split" as an init argument of the basic tokenizer as well as the pretrained tokenizer, but also as part of the `tokenize` method. It isn't clear to me exactly where it is used.

Perhaps some doc changes are needed as well, since it is typed as boolean:

https://github.com/huggingface/transformers/blob/35df91148545e09cd199d89e707043eba5434f59/src/transformers/tokenization_bert.py#L133

cc @n1t0 @mfuntowicz 

I just tested the last example provided by @BramVanroy and it seems to work after #4723. Do not hesitate to reopen if needed!
Question: Where do I find the Transformer model from the paper "Attention is all you need" ?
Author: abhisheksgumadi
Pull request: no

Hello

  Firstly, thanks for supporting all questions here.

I read the paper "Attention is all you need" and wondering which class should I use in the HuggingFace library to use the Transformer architecture used in the paper.

Can you please advise?

Thanks
Abhishek 

Comments:
You don't need this library if you only want the transformer module specifically.

PyTorch: https://pytorch.org/docs/master/generated/torch.nn.Transformer.html
TensorFlow: https://www.tensorflow.org/tutorials/text/transformer#create_the_transformer

Thanks @BramVanroy
Add `masked_lm_labels` argument to `TFAlbertForMaskedLM`
Author: jarednielsen
Pull request: no
Labels: name: wontfix

name: TensorFlow

# 🚀 Feature request

The PyTorch `AlbertForMaskedLM` model has support for the `masked_lm_labels` parameter, while `TFAlbertForMaskedLM` does not. I'd like to bring feature parity.

It looks like a similar feature is also missing for `TFBertForMaskedLM`, `TFRobertaForMaskedLM`, `TFDistilBertForMaskedLM`. I'd be happy to add support for those models as well.

## Motivation

I'm pretraining TF NLP models, and this would simplify the training script by encapsulating the loss function.

## Your contribution

I'm happy to contribute the code. I'll follow CONTRIBUTING.md, any gotchas I should be aware of?


Comments:
Hi! This feature would be great to have.

I'm curious how `TFBertMaskedLM` (and the like) are supposed to be used with the keras `fit()` functionality?

It seems like one is supposed to loop through the training data and calculate the cross-entropy loss for each batch (#2926). I see there was related discussion also here #1999.

Happy for any input!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Closing because this feature was added in #4530
Memory issues in Transformers
Author: AishwaryaVerma
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
Does the BART model support Chinese? Having the pre-trained Chinese model?
Author: mtfelix
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
Have the same problem and do you have any url to download the bart-large-cnn or and other pretrained model

No chinese support, yet. 
Download:
```bash
wget https://s3.amazonaws.com/models.huggingface.co/bert/facebook/bart-large-cnn/pytorch_model.bin
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How can I use masked_lm_labels correctly?
Author: smelly-dog
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Hi, I have a dataset like :

From Monday to Friday most people are busy working or studying, but in the evenings and weekends they are free and _ themselves.

And there are four candidates for the missing blank area:

["love", "work", "enjoy", "play"], here "enjoy" is the correct answer, it is a cloze-style task, and it looks like the maskLM in the BERT.

I want to train the model so that it can work better. I notice that there is a parameter called masked_lm_labels, and it can computing the masked language modeling loss. What should I do to train the BertForMaskedLM model with it.

Do you have any Example? Or can you teach me how to do that?
Thanks!
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
Can any one help me
QAQ

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Evaluation - Output False Positive and False Negative Sentences
Author: sunyangfu
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help
Hi, regarding the sequential classification task, after the evaluation on test data, how could I output the actual false positive and false negative sentences? Basically convert BERT embedding back to the actual sentences for error analysis?


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to load BertforSequenceClassification  models weights into BertforTokenClassification model?
Author: ghost
Pull request: no
Labels: name: wontfix

name: Usage

Initially, I have a fine-tuned BERT base cased model using a text classification dataset and I have used BertforSequenceClassification class for this. Now I want to use this fine-tuned BERT model weights for Named Entity Recognition and I have to use BertforTokenClassification class for this. I'm unable to figure out how to load the fine-tuned BERT model weights into the new model created using BertforTokenClassification.  @thomwolf 

Thanks in advance....................... 

Comments:
Have you tried loading your checkpoint in the model using `from_pretrained`:

```py
model = BertForTokenClassification.from_pretrained("checkpoint)
```

? It should work out of the box. Please be aware that the NER head will be randomly initialized as it is not in the sequence classification checkpoint.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Why isn't there a SequenceClassificationModel for GPT-2 (and some other models)?
Author: c-flaherty
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

<!-- Description of your issue -->
Why isn't there a SequenceClassificationModel (like there is for BERT) for GPT-2? I was able to implement this pretty easily by adding a "[CLS]" token to the vocabulary (like in the GPT2DoubleHeadsModel), appending sequences with "[CLS]", and then adding a linear layer that maps from the embedding of "[CLS]" to a vector of logits corresponding to the classes. After training, this model worked comparably to BertSequenceClassificationModel for my use-case. It would be nice to have this model in the transformers library and not have to code it up from scratch. 

If this sounds like a good idea, I can make a pull request with a GPT2SequenceClassificationModel added. If not, why is it not a good idea? 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Issue loading custom tokenizer for fine-tuning gpt2
Author: modern-online
Pull request: no
Labels: name: wontfix

name: Core: Tokenization

I'm trying to fine-tune gpt2 with a custom tokenizer. It was working fine just over 10 days ago, with  --tokenizer_name=/path to vocab and merges folder/  and now it cannot load, asking to check if it's a correct model identifier or contains a config.json file. As if instead of a tokenizer it is now trying to load a model? It also asked for an extra model identifier in the config file of my model, which before was not required.

I suppose there was a library update? What would be the workaround? Thanks in advance. 


Comments:
Can you post a sample code?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.