noamaanMulla-03/transformers-issues huggingface dataset:
[WIP] deleted special tokens as attributes from model config
Author: patrickvonplaten
Pull request: yes

Given our conversation in #3011 , I thought about deleting all model special config attributes to make sure that no data is duplicated between the tokenizer of a model and the model. 

There are two instances where config special attributes were used (e.g. self.config.pad_token_id)

1. the generate() function. But the generate function also takes all those tokens as attributes, so it should not at all rely on self.config.pad_token_id. This one is trivial to fix.

2. the bart model. It seems like the pad_token_id is actually an integral part of the bart model itself, so to me it seems very hard to disentangle the pad_token_id from the bart model. 

I see three options:

1) Leave the code as it is and leave default attributes self.config.pad_token_id, self.config.bos_token_id, self.config.eos_token_id = None in the PretrainedConfig class.

2) Remove the self.config.pad_token_id, ... from the PretrainedConfig class, make the generate function independent of those variables, but add those variables to the BartConfig only.

3) Make the models completely independent from all special tokens. This probably would mean that the bart model class needs quite a lot of changes. 

I tend to option 2) or 3). I like the idea of separation token ids from internal model behavior completely, but I cannot really estimate whether this is feasible for Bart (@sshleifer you probably have a better opinion on this).

What do you think? @LysandreJik , @julien-c , @thomwolf , @sshleifer  

Comments:
Not feasible for bart at the moment, sadly.

As mentioned in #3011, I think #3011 is the way to go.
[WIP] add generate tests to more models
Author: patrickvonplaten
Pull request: yes

- [ ] add prepare_input_for_generation() to all bert masked lm models 
- [ ] add slow integration tests to check results (also include camembert for this one)

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3046?src=pr&el=h1) Report
> Merging [#3046](https://codecov.io/gh/huggingface/transformers/pull/3046?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/298bed16a841fae3608d334441ccae4d9043611f?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3046/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3046?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3046      +/-   ##
==========================================
+ Coverage   77.18%   77.19%   +<.01%     
==========================================
  Files          98       98              
  Lines       16063    16063              
==========================================
+ Hits        12399    12400       +1     
+ Misses       3664     3663       -1
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3046/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `92.38% <0%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3046?src=pr&el=footer). Last update [298bed1...e295138](https://codecov.io/gh/huggingface/transformers/pull/3046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Refactoring and bug fixing beam search generate
Author: patrickvonplaten
Pull request: yes

This PR cleanes the beam_search decoding part of language generation. It simplifies the code and fixes a small bug for do_sample=True (see comments in code). 
It was also tested on all language generation slow tests.

### Future PR

- [x] Do the same change for TF 2.0 if ok -> #3148 

Comments:
Good to merge for me
fix missed BartForMaskedLM renaming
Author: patrickvonplaten
Pull request: yes

Quick fix @sshleifer 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3149?src=pr&el=h1) Report
> Merging [#3149](https://codecov.io/gh/huggingface/transformers/pull/3149?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/857e0a0d3ba39be6259961524a730d3f106cec9c?src=pr&el=desc) will **decrease** coverage by `0.05%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3149/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3149?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3149      +/-   ##
==========================================
- Coverage   78.03%   77.97%   -0.06%     
==========================================
  Files          98       98              
  Lines       16588    16588              
==========================================
- Hits        12944    12935       -9     
- Misses       3644     3653       +9
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3149?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3149/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `91.08% <0%> (-2.13%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3149/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `94.56% <0%> (+0.15%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3149?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3149?src=pr&el=footer). Last update [857e0a0...58fc8f9](https://codecov.io/gh/huggingface/transformers/pull/3149?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks!
Finetuning on several tasks 
Author: paul-you
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

Hello, 

Is it possible to finetune a Transformer on some dataset, and then finetune the model again on another dataset with a different number of output labels ? 

I tried this and got the following error: 
```
RuntimeError: Error(s) in loading state_dict for BertForSequenceClassification:
        size mismatch for classifier.weight: copying a param with shape torch.Size([2, 768]) from checkpoint, the shape in current model is torch.Size([3, 768]).
        size mismatch for classifier.bias: copying a param with shape torch.Size([2]) from checkpoint, the shape in current model is torch.Size([3]).
```
Thanks in advance



Comments:
The `BertForSequenceClassification` model has a classifier head transforming Bert output into `num_labels` output. So if you change the classification output, it can't be loaded as you could see.
The only hack you could do is to load the fine-tuned model with previous `num_labels`, then remove classifier head and replace it with a new classifier with your new `num_labels` (re-initialized) and fine-tune again.
Yet, your Bert model was fine-tuned on a first classification task that is not the same as the second classification task. So if both tasks are too different on too different datasets, it's not sure it will learn anything... or not sure your previous head will give anything decent after that...


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How Can I load the google bert model(ckpt)? 
Author: zwqjoy
Pull request: no
Labels: name: wontfix

TF2.0
## ❓ Questions & Help
import os
pretrained_path = 'chinese_L-12_H-768_A-12'
config_path = os.path.join(pretrained_path, 'bert_config.json')
checkpoint_path = os.path.join(pretrained_path, 'bert_model.ckpt')
vocab_path = os.path.join(pretrained_path, 'vocab.txt')

config = BertConfig.from_json_file(config_path)
model = TFBertForPreTraining.from_pretrained(checkpoint_path, config=config)

![image](https://user-images.githubusercontent.com/12653212/71568255-d3b3ed00-2b00-11ea-836b-5dc5ad7b9d86.png)

 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Load the google bert model(ckpt) from TFBertForPreTraining error
Author: zwqjoy
Pull request: no
Labels: name: wontfix

I want use the google chinese bert ckpt model in transforms, and env use tf2.
the transforms can load the ckpt  into pytorch model
But I want load the ckpt into tf.keras model, How can I do this ?


![image](https://user-images.githubusercontent.com/12653212/71570221-ce5d9f00-2b0e-11ea-8595-7b8b22db87f2.png)

model = BertForPreTraining.from_pretrained(checkpoint_path, from_tf=True, config=config)

Success.


But:
![image](https://user-images.githubusercontent.com/12653212/71570227-f0572180-2b0e-11ea-9978-f9c679a3e37b.png)


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
i want to browse and store image in data base in tkinter can u give me suggessions?
Author: vrajesh16
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
![image](https://user-images.githubusercontent.com/60382059/76066795-4aafab00-5fb4-11ea-8afe-6630aea361ab.png)
BART FP16
Author: astariul
Pull request: no

# 🚀 Feature request

I would like to use BART in FP16 mode, but it seems impossible for now :

```
config = BartConfig(vocab_size=50264, output_past=True)
model = AutoModelWithLMHead.from_pretrained('bart-large-cnn', config=config).cuda().half()
tokenizer = AutoTokenizer.from_pretrained('bart-large-cnn')
ARTICLE_TO_SUMMARIZE = "My friends are cool but they eat too many carbs."
inputs = tokenizer.batch_encode_plus([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors='pt')
generated_ids = model.generate(inputs['input_ids'].cuda(), attention_mask=inputs['attention_mask'].cuda(), num_beams=4, max_length=5)
```

>   File "/data/user/.venv/bartqg/lib/python3.6/site-packages/transformers/modeling_bart.py", line 647, in forward
    attn_output = torch.bmm(attn_probs, v)
RuntimeError: Expected object of scalar type Float but got scalar type Half for argument #2 'mat2' in call to _th_bmm


@sshleifer Do you plan to implement a FP16-friendly version of BART ?

Comments:
Not on my roadmap just yet, but I would definitely consider it if there were lots of demand. Since we only have inference code right now, the benefit seems marginal. 

@BramVanroy Should this issue be closed ?

FP16 is not implemented yet. And the `wontfix` label is clear.

Keeping the issue open may make it easier for people to find it and show their potential interest in FP16.

This should not be closed indeed.

@sshleifer, we intend all the models to be compatible with FP16, this is the direction the field is going and with the Volta-level GPU being widespread now, there is less and less reason not to use mixed-precision fine-tuning (half memory and significantly faster).

This can probably be fixed by changing the `torch.float32` casting [here](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bart.py#L643) to a cast to the type of `attn_weights` like it's done in the original fairseq code [here](https://github.com/pytorch/fairseq/blob/cb2dc414c692d7de283bec4e4f9c923a66205792/fairseq/modules/multihead_attention.py#L335).

Do you mind fixing this and testing the failing script posted in the issue @sshleifer?

Yep, on it!

Hi, @sshleifer. Thank you so much for your effort on BART. I encountered the same fp16 issues today. The current BART code can be trained (without fp16) using the run_glue script in: https://github.com/huggingface/transformers/blob/master/examples/run_glue.py 
So, it will be really nice if the fp16 training can also work out.

My bad, I thought @sshleifer's labeling was a note that he isn't planning to change anything `wontfix`, so no future updates would be possible and then I closed it. Will keep that in mind for the future.

No bad

@sshleifer for the moment, please ping me with DM before adding "wontfix" labels to issues, thanks.
[hf_api] Get the public list of all the models on huggingface
Author: julien-c
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3132?src=pr&el=h1) Report
> Merging [#3132](https://codecov.io/gh/huggingface/transformers/pull/3132?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ff9e79ba3a3dd35c1a7edbd669cf78e082b2f7dc?src=pr&el=desc) will **decrease** coverage by `0.03%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3132/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3132?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3132      +/-   ##
==========================================
- Coverage      78%   77.97%   -0.04%     
==========================================
  Files          98       98              
  Lines       16561    16581      +20     
==========================================
+ Hits        12919    12929      +10     
- Misses       3642     3652      +10
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3132?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/hf\_api.py](https://codecov.io/gh/huggingface/transformers/pull/3132/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9oZl9hcGkucHk=) | `98% <100%> (+0.5%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3132/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `91.29% <0%> (-2.34%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3132/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `94.45% <0%> (+0.15%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3132?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3132?src=pr&el=footer). Last update [ff9e79b...3f067f4](https://codecov.io/gh/huggingface/transformers/pull/3132?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
BART: move boilerplate code inside encoder/decoder
Author: tomhosking
Pull request: no

# 🚀 Feature request

Could we move the boilerplate code in the `BartModel` `forward()` method inside the `forward()` methods for the encoder and decoder? That way, the encoder and decoder can be called separately as independent modules more easily.


## Motivation

Currently, there's some cleanup work that is done before calling `BartEncoder.forward()` and `BartDecoder.forward()` (looks like attention mask flipping and preparing dummy outputs). If we want to call the encoder and decoder separately from our own code (eg to use Bart as an encoder, or limit which parts are fine tuned) we currently have to re-implement this code.

If we move this logic inside the encoder and decoder, such that `BartModel.forward()` is a thin wrapper around the encoder+decoder, this kind of work would be much easier.

Example usage:
```
model = BartModel.from_pretrained(...)
inputs = tokenizer.encode('I want to classify this text.')
encoding = model.encoder(inputs)
predictions = my_classifier(encoding)
```

## Your contribution

I could put together a PR for this if you agree?

(cc @sshleifer )


Comments:
Sounds good to me
Correct missing keys
Author: LysandreJik
Pull request: yes

closes #3142 

The problem came from the fact that `from_pretrained` set the model on which to load the weights to the base model: it detected that the state dict was made for the base, therefore loading only onto the base.

It didn't look for the weights it didn't load. Here, both state dicts are analyzed and the difference (keys present in the state dict with the head and not in the base state dict) are added to the missing keys.

Added a test.

Comments:
Very good catch
[Bart] add imports to examples
Author: sshleifer
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3160?src=pr&el=h1) Report
> Merging [#3160](https://codecov.io/gh/huggingface/transformers/pull/3160?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6ffe03a0a1d472a4e5941793fd361d2b82c8be3f?src=pr&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3160/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3160?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3160      +/-   ##
==========================================
+ Coverage   78.11%   78.12%   +0.01%     
==========================================
  Files          98       98              
  Lines       16651    16651              
==========================================
+ Hits        13007    13009       +2     
+ Misses       3644     3642       -2
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3160?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/3160/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `93.41% <ø> (ø)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3160/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `94.72% <0%> (+0.31%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3160?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3160?src=pr&el=footer). Last update [6ffe03a...0f206be](https://codecov.io/gh/huggingface/transformers/pull/3160?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
seed parameter for model generate()
Author: minimaxir
Pull request: no

# 🚀 Feature request

There should be a `seed` parameter for the `generate()` function of a model.

Although a seed can be manually set before calling `generate()` (as tested in #3063), using it as a parameter is more intuitive (and covers all the bases)

## Motivation

Generation reproducibility (also good for CI tests)

## Your contribution

The implementation `set_seed()` functions around the repo (e.g. https://github.com/huggingface/transformers/blob/6b1ff250842f52136d5159bb67a26b50ba01485d/examples/run_generation.py#L74) should be sufficient.

Comments:
Hi @minimaxir - thanks for the Feature request! 

Not too sure about this though. Two arguments against it from my side:

1. I feel like it's very easy to set the seed parameter before calling `generate()` without any real drawback. 
2. Also we want all our `generate()` arguments to have default values with a lot of them defined in `configuration_utils.py`. Adding a `seed` argument would either break this logic or set a default seed value in the  `PretrainedConfig` class in `configuration_utils.py` which I definitely don't want to do.

Same here, thanks for the proposition @minimaxir but I feel like there are many ways you can/should assigne seeds (e.g. [here](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py#L104-L109)) and I don't think we would be comfortable with having this inside the model it-self.

That's fair; for my purposes, it's not too difficult to wrap. (and I inadvertently realized it's better to wrap it for batch generation too)

Thanks!